Agents on Call, Part 7. Sizing: Token Math Nobody Does Upfront

A single deep incident investigation on this platform, eight rounds of tool calls before the triage agent has enough evidence to propose a diagnosis, costs 32,950 tokens, and 31,600 of them are input, because Bedrock's Converse API resends the whole growing transcript on every round. Multiply that by real alarm volume and the platform's entire monthly on-demand model bill comes out to about $14, one of the smaller line items in a total monthly cost near $21. Buying Provisioned Throughput on day one, the thing Part 1 explicitly deferred, would have cost $15,768 a month for a workload running at roughly a four-hundredth of the utilization it needs to break even. None of that is obvious from a diagram. It only shows up once someone does the arithmetic, which is what this part does, end to end, with every number traceable to either a Bedrock price sheet or a formula shown in full.

Part 1 set the scenario, 40 pages a week across 30 accounts, and picked AgentCore plus Strands over the alternatives. Part 2 built the account boundary and the inference profile wiring, and named the exact question this part answers: the point where Provisioned Throughput's hourly rate beats on-demand's per-token rate. Part 3 shipped the triage agent's tool-calling loop, the thing whose rounds get costed below. Part 4 moved those tools behind AgentCore Gateway. Part 5 added the supervisor, runbook, and cost agents this part's monthly bill now has to cover. Part 6 put Guardrails on every invocation, a decision that turns out to matter more for this part's bill than expected. Every number below lives in docs/sizing-model.md in the companion repository at github.com/flightlesstux/agents-on-call, alongside a checklist for re-running it against real numbers instead of this fictional company's.

The token math worked example

The Converse API keeps no server-side conversation state. Every tool-calling round resends the full transcript so far: system prompt, tool schemas, every prior tool call and its result. That means input tokens do not grow linearly with rounds, they grow with the sum of everything already sent, and a longer investigation pays for its own history over and over. The building blocks, declared explicitly rather than assumed: a 1,800-token static prefix (system prompt, tool schemas, guardrail config) identical on every call, a 400-token initial alarm payload, 150 tokens of assistant output per round, and 350 tokens of tool result appended per round, 500 tokens of growth each round. Round r's input size is S + 400 + 500*(r-1).

A shallow investigation that resolves in 3 rounds looks like this:

A deep investigation that needs 8 rounds, more accounts, more tools, more back-and-forth before the evidence adds up, looks like this:

32,950 tokens total, against a shallow investigation's 8,700, a 3.8x cost for a 2.7x round count. That gap is the resend penalty made visible: total_input(N) = N*(S+400) + 500*N*(N-1)/2 grows quadratically in the round count, not linearly, because round 8 is paying for rounds 1 through 7 all over again.

Part 1's on-call data puts real volume against this: 40 pages a week, split here 70/30 into 28 shallow and 12 deep investigations. That is 243,600 shallow tokens and 395,400 deep tokens a week, 639,000 total, about 2.77 million tokens a month for the triage agent alone. The runbook, cost-sweep, and supervisor agents add roughly another 1.1 million, for a platform-wide total near 3.87 million tokens a month. The full per-agent breakdown, including the runbook agent's Knowledge Base-augmented rounds and the cost agent's daily sweep, is in the companion doc; the number worth carrying into the next section is that none of this, at this volume, is large.

Burst incidents against a steady sweep

The cost agent's volume is smooth on purpose: one scheduled EventBridge run a day, off-hours, never sharing a minute with an incident. The triage agent's volume is not smooth, and averages hide that. Spread evenly, 606,000 weekly input tokens is about 60 tokens a minute. Real incidents correlate: one upstream failure fanning out to eight alarms inside the same five minutes spins up eight triage sessions whose first rounds land in the same TPM window, 8 x 2,200 = 17,600 input tokens in that one minute from triage alone, before the supervisor's own dispatch calls are counted. That is roughly 290 times the smoothed average, in exactly the ten minutes an outage is underway and the platform's ability to keep responding matters most. Size against the peak, not the mean; the mean is what shows up on next month's invoice, the peak is what determines whether an alarm storm gets an answer or a `ThrottlingException`.

Quotas, throttling, and alarming before the pain

Bedrock's two invocation endpoints enforce quotas differently, and the difference changes what a burst actually hits. The bedrock-runtime endpoint counts input and output tokens together against a single per-model TPM quota, and RPM is model-specific: some top-tier models carry no RPM quota at all, governed by tokens alone. The bedrock-mantle endpoint splits input and output into separate token quotas and drops RPM entirely. Either way, the quota is account- and region-specific, shared across every agent this platform runs, not allocated per agent, so the supervisor's dispatch calls and the triage agent's eight-round investigation are drawing from the same pool during a correlated alarm storm. Self-service quota increase requests cover both endpoints as of a change earlier this year, previously bedrock-mantle needed a support ticket; that closes a real gap but does not remove the need to know the ceiling before an incident finds it.

The Bedrock SDK's own retry logic backs off on a throttling response, but that only smooths a burst, it does not make quota headroom appear. The more useful move is upstream of any retry: alarm on quota utilization itself, not on the first ThrottlingException a retry already absorbed. A quota-utilization alarm that fires at 70% of a TPM ceiling gives capacity planning a page days before an alarm storm gives the on-call engineer a degraded triage agent. Waiting for the throttling error to be the first signal means the incident that most needs this platform's help is the one where the platform's own error budget gets spent on retries instead of tool calls.

Cross-region profiles as burst headroom

Part 2 put this platform on cross-region inference profiles everywhere, for reliability and no added per-token cost. The sizing angle on that decision: a TPM quota is scoped to a region's endpoint, and cross-region routing means a burst that would exhaust one region's ceiling can spill into a sibling region's separate ceiling instead of queuing behind a single region's limit. It is not unlimited headroom, and it does not substitute for knowing the real ceiling, but it turns one region's quota into a geography's worth of quota for the cost of nothing, which is a better trade than most capacity headroom this platform buys.

Provisioned Throughput economics: the break-even, and why buying it now is wrong

A Model Unit buys a flat, always-billed hourly rate in exchange for a fixed input and output tokens/minute ceiling, whatever the model's price sheet and MU throughput happen to be for the chosen model, neither of which AWS publishes as a fixed number independent of the model, so the formula below uses declared placeholder values, not AWS facts, and needs the live console's numbers before anyone spends real money on it:

max_monthly_input  = T_in  * 60 * 730   # 730 hours in an average month
max_monthly_output = T_out * 60 * 730

on_demand_equiv_cost = (max_monthly_input  / 1000) * I
                      + (max_monthly_output / 1000) * O

pt_monthly_cost        = P * 730
break_even_utilization = pt_monthly_cost / on_demand_equiv_cost

With illustrative placeholders (200,000 input and 100,000 output tokens/minute per MU, $0.003 and $0.015 per 1,000 tokens on-demand, $21.60/MU-hour no-commit): 100% utilization of that MU's ceiling is worth $91,980 a month on-demand, PT itself costs $15,768 a month, and break-even lands at 17.14% sustained utilization. Past that fraction, PT is cheaper. Below it, on-demand wins, and the word doing the work is sustained: this is a monthly average, and the burst example two sections up shows exactly why hitting a ceiling for one minute is not the same thing as sustaining it.

This platform's actual monthly input volume, about 3.65 million tokens, is 0.042% of that MU's monthly input ceiling, roughly 411x below the 17.14% break-even point. Buying one MU on day one, the exact move Part 1 deferred and Part 2 flagged as needing this math first, would spend $15,768 a month protecting a workload whose entire on-demand model bill is about $14. The mistake is not the arithmetic, it is doing none, and committing to a monthly rate before the request-rate distribution across four agents is even measured. On-demand until measured, provisioned only after, is not caution for its own sake, it is what the break-even calculation actually says.

What prompt caching does to the same math

Caching turns the resend penalty from the token math section into a mostly-solved problem. A cached prefix reads at a 90% discount; new content written into the cache costs 1.25x the normal input price, no separate fee beyond that multiplier. Applied round by round, round 1 writes its full input to cache, and every later round reads the entire previous round's input at 10% of normal price, paying full-plus-25% only on the newly appended 500 tokens:

billing_input(1) = input(1) * 1.25
billing_input(r) = input(r-1) * 0.10 + 500 * 1.25   # r >= 2

For the shallow investigation, cached billing-equivalent input drops from 8,100 to 4,490 tokens, a 44.6% reduction. For the deep investigation, it drops from 31,600 to 9,715, a 69.3% reduction, larger because the resend penalty it is cancelling out compounds with every extra round: caching helps more exactly where the uncached cost hurts more. In dollar terms, a deep incident's total cost (input plus output, output unaffected by caching) falls from $0.1151 to $0.0494, a 57.1% reduction. Applied across the triage agent's full weekly mix, monthly model cost for that agent alone falls from $10.02 to $5.29, a $4.73 saving, 47.2%. The same mechanism applies to the runbook and cost-sweep agents' own multi-round loops at a smaller absolute scale, since they run far less often; the companion doc carries the full recomputation only for triage, the highest-volume, highest-round-count agent where the effect is largest.

The monthly cost model for the whole platform

Putting every category together, using the sourced AgentCore and Guardrails prices against the token volumes above, with the model-token line shown both with and without triage-agent caching:

Two things worth sitting with. Runtime, Gateway, and Memory are rounding errors at this volume: per-second billing and no charge for idle time waiting on the model mean the managed infrastructure barely registers until concurrency and session duration climb by orders of magnitude. Guardrails is not a rounding error: at roughly 30% of the no-caching total, it costs more than Runtime, Gateway, and Memory combined by about 15x, and after caching shrinks the model line, it is nearly two-thirds the size of the model bill it is scanning. A sizing exercise that stops at token count for the model misses a third of the actual bill, sitting in a service most people size for compliance, not cost.

Model right-sizing per agent

The token math above assumes one model for every agent, which is the easy default and not the right end state. The four agents ask genuinely different things of a model. Triage does structured diagnosis from tool output, evidence in, a bounded set of plausible root causes out, work a smaller, cheaper model handles well once its system prompt and tool schemas are tight, exactly the workload Part 3's low-temperature setting was already tuned for. The cost agent's daily sweep is mostly aggregation and threshold comparison against numbers a tool already fetched, an even better fit for the smallest model that reliably follows the output format. The runbook agent is the one place a stronger model earns its higher per-token price: it is drafting the exact SSM document and parameters a human is about to approve behind Part 4's approval gate, and a mistake there is a mistake with real blast radius, not a mistake in a diagnosis someone else double-checks anyway. Matching model size to what a mistake actually costs, not defaulting every agent to the same model, is the same instinct behind prompt caching and PT: do not spend on capability the task does not need, and do spend it where a wrong answer is expensive. Part 8's eval harness is what turns "handles it well" from a guess into a number worth trusting.

Failure modes to watch

Four things worth knowing before this model runs against a real account. First, every dollar figure above depends on two placeholder inputs, the on-demand per-token price and the Provisioned Throughput MU throughput ceiling, that this series deliberately never pins to one named Bedrock model; swap in the live pricing page before trusting a total, since a stale placeholder produces a confidently wrong bill, not an obviously wrong one. Second, the 350-token tool-result assumption is almost certainly conservative: a real CloudWatch metric dump or a log excerpt can run 5 to 10x that, and since it is the term that compounds quadratically across rounds, an underestimate there understates every downstream number proportionally, not by a fixed offset. Third, the break-even calculation is a monthly average; a platform whose real traffic looks like the burst section, long quiet stretches with sharp correlated spikes, can be nowhere near break-even on paper while still throttling during the ten minutes that matter, which argues for cross-region headroom and quota alarms over a PT purchase either way. Fourth, prompt caching's savings depend on cache hits actually landing, a system prompt or tool schema edited mid-month invalidates the cached prefix and briefly reintroduces the full uncached cost until the cache repopulates, worth watching for after any prompt change, not assuming away.

Read this next

• Part 6, Guardrails: The Part Everyone Skips, for the per-unit Guardrails pricing this part's monthly cost model relies on, and the invocation pattern that decides how much text actually gets scanned.
• Part 8, Production: Observability, Evals, and the Day It Lies, where the model right-sizing decisions above get tested against real traces instead of assumed correct.
• The AWS Well-Architected Framework: A Key to Cloud Success on ercan.cloud, for the Cost Optimization pillar this part is really one long worked example of: measure before you commit.

The full docs/sizing-model.md, including the per-agent breakdown for the runbook and cost-sweep agents trimmed from the tables above and a checklist for re-running every number against a real account, lives in the companion repository at github.com/flightlesstux/agents-on-call. For the database and infrastructure side of running platforms like this one at scale, the field notes are at ercan.cloud, and the hub is at ercanermis.com.