The chargeback design is decided by an Azure Monitor limit, not by an accounting preference: a custom metric allows 10 dimension keys, API Management already spends 5 of them on defaults, and the active time series a policy produces is the product of the unique values of every dimension you add. Three dimensions with ten values each is a thousand time series, against a cap of 50,000 active time series per region per subscription in a twelve hour window. Put a user ID or a request ID on that metric and the platform's own telemetry becomes the outage. Part 4 finished the three request paths. This part answers the question the whole series exists for: which team spent what, and how the gateway stops one of them from spending everyone else's quota.

Two credentials, two different jobs

API Management gives you two mechanisms that people routinely treat as alternatives when they are complements.

A subscription key identifies a consumer for the purposes of products, quotas, and analytics. It arrives in the Ocp-Apim-Subscription-Key header, or in a subscription-key query parameter that is only checked when the header is absent, and both names are configurable per API. Keys are issued in pairs so an application can move from key A to key B and regenerate A with minimal disruption.

Two things about keys need to be designed around rather than assumed away. First, API Management has no built-in lifecycle for subscription keys: no expiration dates, no automatic rotation. Rotation is a workflow you build, with Azure PowerShell or the SDKs, and if you do not build it the keys live forever. Second, the key is passed to the backend by default, where it can end up in backend monitoring logs and anywhere those logs are shipped. A set-header at the end of the inbound section that strips it is a two-line fix for a credential-in-logs finding that is otherwise discovered by an auditor.

A JWT from Microsoft Entra ID answers a different question: is this caller currently authorized. It expires natively, it can be revoked centrally, and it carries claims. Applications obtain one through the OAuth2 client credentials flow, and the gateway validates it before anything else runs:

<validate-jwt header-name="Authorization" require-scheme="Bearer"
              require-expiration-time="true" require-signed-tokens="true"
              clock-skew="60"
              output-token-variable-name="jwt"
              failed-validation-httpcode="401"
              failed-validation-error-message="Invalid or missing token.">
  <openid-config url="https://login.microsoftonline.com/{{tenant-id}}/v2.0/.well-known/openid-configuration" />
  <audiences>
    <audience>api://genai-gateway</audience>
  </audiences>
  <required-claims>
    <claim name="roles" match="any">
      <value>model.invoke</value>
    </claim>
  </required-claims>
</validate-jwt>

output-token-variable-name is the part that earns its place: the validated token lands in a policy variable, so later policies can read a claim from it instead of trusting a header the caller controls. That matters immediately, because the next decision is which value keys the quota.

The anonymous access trap

Worth stating plainly because the configuration that produces it looks harmless. Whether an API can be called without any subscription key depends on two independent settings: whether the API itself requires a subscription, and whether every product it is assigned to requires one. If the API does not require a subscription, calls without a key are allowed in the API context even when a product-scoped key would also work. The gateway is doing exactly what it was told; nobody told it the intent.

For a gateway in front of paid model capacity, the rule is that every API requires a subscription and every product requires a subscription, and the validate-jwt policy runs regardless, so an unauthenticated call fails twice. Key scopes matter too: a service-scoped or all-APIs-scoped key opens every API in the instance, which is right for the platform team's own tooling and wrong for a consumer. Consumers get product-scoped keys.

Quota, and what to count it against

The token limit policy from Part 1 takes a counter-key, and that choice is the entire tenancy model. Keying on the subscription ID gives one bucket per consumer, which is what the incident in Part 1 needed: the marketing batch job cannot drain the customer assistant's quota because they are different subscriptions with different counters.

<llm-token-limit counter-key="@(context.Subscription.Id)"
                 tokens-per-minute="{{tpm_tier}}"
                 estimate-prompt-tokens="true"
                 remaining-tokens-variable-name="remainingTokens"
                 remaining-tokens-header-name="x-remaining-tokens" />

Returning the remaining allowance in a response header is worth the two extra attributes. A client that can see its own headroom can back off before it gets a 429, and a support conversation about throttling starts from a number both sides can see.

Products become the tiers: an interactive product with a high per-minute limit and a modest daily quota, a batch product with the inverse, and a sandbox product with limits low enough that a runaway loop in a notebook is a nuisance rather than an incident. A tenant is an Entra ID application plus one or more product subscriptions, provisioned by the Terraform in Part 2's 30-tenants layer, never by a ticket.

The dimension budget

Now the constraint from the lead, stated as the design rule it produces. Azure Monitor caps custom metrics at 10 dimension keys, and API Management uses 5 of those for defaults including Region, Service ID, Service Name, and Service Type. That leaves a maximum of 5 custom dimensions per policy. The bigger limit is combinatorial: active time series equal the product of the unique values across your dimensions in the period, so three dimensions with ten values each contribute a thousand, and the regional ceiling is 50,000 active time series per subscription in twelve hours. Multiple API Management instances in the same region contribute to the same regional total.

The consequence is a clean split that is worth writing into the platform's own documentation:

  • Metrics carry low-cardinality dimensions only. Tenant, model alias, environment. Three dimensions, bounded value sets, dashboards that load quickly and alerts that fire on the right thing.
  • Logs carry everything else. Request ID, user ID if it is captured at all, prompt and completion sizes, latency, which backend served it. Application Insights and Log Analytics are where per-request attribution lives, and the monthly chargeback job queries them.
<llm-emit-token-metric namespace="llm-metrics">
  <dimension name="Tenant"      value="@(context.Subscription.Name)" />
  <dimension name="ModelAlias"  value="@(context.Request.MatchedParameters.GetValueOrDefault("model","unknown"))" />
  <dimension name="Environment" value="{{env}}" />
</llm-emit-token-metric>

Three dimensions, deliberately. The fourth one somebody will ask for is user ID, and the answer is no: it belongs in the log record, where cardinality costs storage rather than an entire region's metric budget.

Chargeback that survives an argument with finance

The monthly job is ordinary data engineering, and its credibility rests on being explicit about accuracy rather than on being precise. Parts 3 and 4 established three classes, and the chargeback document names them:

  • Non-streamed synchronous: metered from the model's own usage. Exact.
  • Streamed synchronous: prompt and completion tokens estimated at the gateway, because streaming forces estimation regardless of policy configuration. Approximate, with the drift tracked as its own metric.
  • Batch: exact, from the job's own request counts and the per-response usage in the output file, because the control plane submitted it.

The job produces a per-tenant figure and a reconciliation line: the sum of attributed spend against the actual resource cost for the period. A residual of a few percent, stable month over month, is the honest cost of estimation. A residual that grows is a bug, most often a caller that found a path around the gateway. Publishing the residual rather than hiding it is what makes the rest of the number believable, and it doubles as the detector for the bypass path Part 1 warned about.

When a team needs its own gateway

Occasionally a consumer's requirements do not fit shared infrastructure: a regulated workload that needs its own network isolation, or a team that must manage its own APIs without touching anyone else's. API Management workspaces exist for this, and the choice is between the service's default managed gateway, available in the v2 tiers with no extra gateway cost and access to built-in capabilities, and a separate workspace gateway, available on Basic v2, Standard v2, Premium, and Premium v2, which buys strong runtime isolation and independent scaling, hostname, and network configuration at the price of extra cost, longer deployment, and support in fewer regions.

One detail decides whether this is reversible: a workspace gateway's virtual network configuration can only be set when the gateway is created, and cannot be changed afterward. Its network configuration is also independent of the API Management instance's. A workspace gateway created without isolation, by a team that later needs it, is a rebuild.

Failure modes to watch

  • A high-cardinality dimension in a policy. It will not error. It will quietly consume the region's active time series budget, and the first symptom is other teams' custom metrics failing to appear.
  • Keys that never expire. There is no built-in expiration or rotation. Without a rotation workflow, the credential issued to a decommissioned application still works, and nothing reports that.
  • The subscription key in backend logs. Default behaviour forwards it. Strip it in inbound, and check the backend's logs once to confirm rather than assuming.
  • An API that does not require a subscription. Anonymous access in the API context, arrived at by a configuration that reads as permissive rather than as open.
  • Chargeback published without its residual. The first time a product owner finds a discrepancy you did not disclose, every future number is negotiable.

What Part 6 inherits

Tenants with real identities, quotas that hold, and a monthly figure per team with a stated accuracy. Which sets up the next question precisely: the cheapest token is the one never sent, and semantic caching is the first mechanism in this series that lowers the bill rather than merely attributing it. It also quietly changes what the numbers above mean, because a cache hit is a request that cost nothing and still has to appear in somebody's report.

Read this next

For the infrastructure and platform side of running this at scale, the field notes are at ercan.cloud, and the hub is at ercanermis.com.

References