The most consequential line in this platform's Terraform is one nobody writes: version_upgrade_option on an Azure model deployment defaults to OnceNewDefaultVersionAvailable, so the deployment moves itself to a newer model version whenever Azure decides a new default exists. A gateway whose entire justification is owning the model lifecycle cannot leave that on a default. This part builds the infrastructure layer under Part 1's architecture, picks between Terraform, Bicep, and ARM on grounds that are actually load-bearing, and turns off the two defaults that quietly take the lifecycle back.

Three languages, and only one real question

The comparison is usually argued on syntax, which is the least interesting axis. ARM JSON is the substrate: everything eventually becomes an ARM template, and nobody authors it by hand any more except when debugging what Bicep emitted. That leaves two candidates and one question that decides between them.

Bicep is a domain-specific language that transpiles to ARM, ships in the Azure CLI, needs no state file, and has a real answer to lifecycle management in deployment stacks. A stack tracks the resources it manages, and actionOnUnmanage decides what happens to a resource that leaves the template: detachAll leaves it running and untracked, deleteResources deletes the resources, deleteAll deletes resources and resource groups. Stacks also carry deny settings, so a stack can be created with denySettingsMode set to denyDelete or denyWriteAndDelete, with specific actions and principals excluded. That last capability has no direct Terraform equivalent: it is Azure itself refusing the delete, not a plan refusing to generate one.

Terraform brings a state file, a plan you can review in a pull request, and, decisively for this platform, more than one provider in a single dependency graph. That is the question that actually decides it: does the platform manage anything outside the Azure Resource Manager control plane?

It does. Part 1's tenant onboarding hands a team an Entra ID application, a service principal, and a client credential, and wires the resulting object ID into an API Management subscription and a quota. Entra ID objects are not ARM resources. Bicep cannot create them. A Bicep-first platform ends up with a second tool for identity and a manual step joining the two, which is exactly the seam where a decommissioned team keeps working credentials for another year. Terraform holds azurerm, azuread, and azapi in one graph, so the application registration and the API Management subscription that depends on it are created, changed, and destroyed together.

So: Terraform for this platform. The honest counterweight is that if your platform is pure ARM-plane infrastructure and your organization already runs Azure Policy and deployment stacks well, Bicep with denyWriteAndDelete is a stronger guarantee than a Terraform plan anyone can override with a targeted apply.

Two planes, two state files

The split from Part 1 (API Management as the data plane, a Python service on AKS as the control plane) has an infrastructure consequence that is worth making explicit before writing any HCL. These two layers change at different rates by two orders of magnitude. The API Management instance, the AKS cluster, and the network change a few times a year. Model deployments, backends, and policies change weekly, sometimes daily.

Putting both in one state file means a routine model deployment change runs a plan against the AKS cluster, and a failed apply halfway through leaves both layers partially applied. Split them:

infra/
  10-platform/        # resource group, network, APIM instance, AKS, Log Analytics
  20-models/          # Foundry account, model deployments, APIM backends
  30-tenants/         # Entra ID apps, APIM subscriptions, quotas

Each directory is its own root module with its own state, and the later ones read the earlier ones through terraform_remote_state or, better, through data sources that look resources up by name. Data sources are slower and more verbose, and they are worth it: they mean 20-models does not break when 10-platform's state is refactored.

The tier decision, which is not a cost decision

Part 1 said the gateway mediates Anthropic Claude natively. That single requirement selects the API Management tier, because the Anthropic Messages API schema is supported in the v2 tiers, not the classic ones. The v2 tiers also deploy in minutes rather than the long provisioning wait the classic tiers are known for, scale to 10 units on Basic v2 and Standard v2 and to 30 on Premium v2, and Standard v2 upward supports virtual network integration and inbound private endpoints.

What you give up is not small, and it is better known now than during an incident. The v2 tiers currently do not support multi-region deployment, backup and restore of the instance, sending events to Event Grid, Git-based service configuration, direct Management API access, self-hosted gateways, or Azure DDoS Protection. There is also no upgrade path from a classic tier to a v2 tier, and no resource move. A gateway that starts on Developer or Standard and later needs Anthropic support is a migration, not a scale operation.

The platform module therefore starts here:

resource "azurerm_api_management" "gw" {
  name                = "apim-genai-${var.env}"
  location            = azurerm_resource_group.platform.location
  resource_group_name = azurerm_resource_group.platform.name
  publisher_name      = "Platform Engineering"
  publisher_email     = "platform@example.net"

  # StandardV2 is required for the Anthropic Messages API schema.
  # Capacity is units, not tokens: 1 unit, scale to 10 without redeploying.
  sku_name = "StandardV2_1"

  identity {
    type = "SystemAssigned"
  }
}

Two provider constraints belong in a comment next to that block rather than in a postmortem. delegation blocks are rejected outright on any V2 SKU and on Consumption. And public_ip_address_id is only accepted for Developer and Premium instances deployed in a virtual network, so a V2 instance cannot pin its outbound IP that way, which matters if a model backend sits behind an IP allowlist.

Model deployments, and the default that moves production

The models are a azurerm_cognitive_account plus one azurerm_cognitive_deployment per logical model. This is where the lead of this post lives:

resource "azurerm_cognitive_deployment" "chat_default" {
  name                 = "chat-default"
  cognitive_account_id = azurerm_cognitive_account.foundry.id

  model {
    format  = "OpenAI"
    name    = "gpt-4.1"
    version = "2025-04-14"   # pinned, never omitted
  }

  sku {
    name     = "GlobalStandard"
    capacity = 300           # thousands of TPM: 300 = 300,000 TPM
  }

  # Default is OnceNewDefaultVersionAvailable, which upgrades this
  # deployment when Azure changes the default version. The whole point
  # of the catalog is that we decide when a model version changes.
  version_upgrade_option = "NoAutoUpgrade"
}

Three things in that block are easy to get wrong. capacity is not a unit count and not a raw TPM figure: it is tokens-per-minute in thousands, and it defaults to 1, meaning 1,000 TPM. A deployment created without an explicit capacity will throttle a single test script. The version field is optional, and omitting it assigns whatever the default version is at creation time, which makes the resource's behaviour a function of the day it was applied. And version_upgrade_option has three values, OnceNewDefaultVersionAvailable, OnceCurrentVersionExpired, and NoAutoUpgrade, of which only the last leaves the decision with you. OnceCurrentVersionExpired is the defensible middle: it holds the pinned version until Azure retires it, then moves rather than breaking. Pick one deliberately per model, and record which in the catalog Part 9 builds.

One more property of this resource shows up as a mysteriously slow apply rather than an error. The provider takes an account-level lock while creating a deployment, so eleven deployments on one Foundry account do not create in parallel no matter what -parallelism says. Splitting model deployments across accounts by workload is a throughput decision as much as a quota one.

Where azurerm stops and azapi starts

The AI surface of Azure moves faster than any Terraform provider release cycle, which is a structural fact rather than a complaint. Microsoft's own guidance names both providers: azurerm for stable resources, and azapi for driving the Azure Resource Manager APIs directly, which keeps up with the newest functionality without waiting for a provider update.

The practical rule that keeps this from becoming a mess: azapi is for resources, never for whole subsystems. A single azapi_resource for a preview capability, sitting next to twenty azurerm resources and referencing them by ID, is fine and reversible. A platform where half the resources are raw API bodies has given up the schema validation and the readable plan that were the reason to use Terraform at all. Every azapi block gets a comment naming the azurerm resource it is waiting for, so the migration back is a task rather than an archaeology project.

State, backend, and the boring part that saves you

Remote state in an Azure Storage account, one container per environment, with blob-lease locking, which Terraform handles natively for the azurerm backend. The storage account itself is not in Terraform: it is created once by a documented script, because a state backend that lives in the state it stores is a circular dependency waiting for a bad day.

terraform {
  required_version = "~> 1.9"

  backend "azurerm" {
    resource_group_name  = "rg-tfstate"
    storage_account_name = "sttfstategenai"
    container_name       = "prod"
    key                  = "10-platform.tfstate"
    use_azuread_auth     = true      # no storage keys in the pipeline
  }

  required_providers {
    azurerm = { source = "hashicorp/azurerm", version = "~> 4.36" }
    azuread = { source = "hashicorp/azuread", version = "~> 3.0" }
    azapi   = { source = "Azure/azapi",       version = "~> 2.0" }
  }
}

use_azuread_auth is the line worth arguing for. Without it the pipeline authenticates to the state backend with a storage account key, which is a long-lived secret with full access to every environment's state, sitting in a variable group. With it, the pipeline's workload identity is authorized by role assignment, and the state file inherits the same identity model as everything else.

Policy XML belongs in files

Part 1 named policy sprawl as a day-one failure mode. The mechanical defence is that policy XML is never authored in the portal. It lives in the repository, is templated, and is applied by the same pipeline as everything else:

resource "azurerm_api_management_api_policy" "chat" {
  api_name            = azurerm_api_management_api.chat.name
  api_management_name = azurerm_api_management.gw.name
  resource_group_name = azurerm_resource_group.platform.name

  xml_content = templatefile("${path.module}/policies/chat.xml", {
    tpm_default = var.tpm_default
    metric_ns   = "llm-metrics"
  })
}

The templated file is ordinary XML with the token limit and metric policies from Part 1 in it, reviewed as a diff like any other code:

<inbound>
  <base />
  <llm-token-limit counter-key="@(context.Subscription.Id)"
                   tokens-per-minute="${tpm_default}"
                   estimate-prompt-tokens="true"
                   remaining-tokens-variable-name="remainingTokens" />
  <llm-emit-token-metric namespace="${metric_ns}">
    <dimension name="Tenant" value="@(context.Subscription.Name)" />
  </llm-emit-token-metric>
</inbound>

Note estimate-prompt-tokens="true", which is the setting that makes an over-limit request fail at the gateway instead of consuming backend quota to find out. It costs a little accuracy on the estimate and saves the exact tokens you were trying to protect.

Four gotchas worth writing on the wall

  • Deleting a Foundry resource does not free its name for 48 hours. Soft delete keeps it recoverable, and a terraform destroy followed by an apply with the same name fails until the resource is purged explicitly. Worse, charges for provisioned deployments continue until the purge, so a destroyed environment can keep billing. Delete deployments before deleting the account, and purge deliberately.
  • PTU cannot be bought from Terraform. The provisioned SKUs (ProvisionedManaged, GlobalProvisionedManaged, DataZoneProvisionedManaged) are purchased on an hourly basis by deployed PTU count, with term discounts through Azure Reservations, and that purchase step is not something the provider can complete. The deployment is code; the commitment is a procurement action that happens beside it.
  • Quota is not infrastructure as code. A model deployment's capacity cannot exceed the subscription's quota for that model and region, and quota arrives through a support request. Plan output is not the constraint; the quota page is.
  • No upgrade from classic to v2. Worth repeating because it is the one mistake in this post that costs a migration rather than an apply.

What Part 3 inherits

At the end of this part there is an API Management instance on Standard v2 with a system-assigned identity, a Foundry account with pinned model deployments that will not move underneath anyone, an AKS cluster with nothing on it yet, Log Analytics and Application Insights receiving nothing yet, and three state files that can be applied independently. No application can call any of it, because no API exists yet and no tenant has a credential. That is the next two parts.

Read this next

For the infrastructure and platform side of running this at scale, the field notes are at ercan.cloud, and the hub is at ercanermis.com.

References