Azure AI Search does not size by how many vectors you have. It sizes by tier: the vector index is bounded by the memory reserved for vector search on your SKU, and on the Serverless model by a hard 300 MB per index that fails the indexing job outright when exceeded. That single property, not a feature comparison, decides whether the company's retrieval layer belongs in Azure AI Search or in a dedicated vector database. Part 6 made repeated questions cheap. This part handles the far more common case, where the model needs facts it never saw, and the retail knowledge search from Part 1 finally gets built.

Where retrieval sits relative to the gateway

A point worth settling before any comparison, because it is the one that keeps the series' premise intact: retrieval is not gateway traffic. The application queries the search index directly, assembles a prompt from what comes back, and sends that prompt through the gateway like any other completion. The gateway meters the tokens the retrieved context adds, which is exactly the visibility you want, since retrieval is usually where prompt size quietly triples.

The one part that does touch the gateway is embedding. Generating query and document embeddings is a model call, so it goes through the same door as everything else, with the same quota and the same attribution. That also means the embeddings deployment Part 6 created for semantic caching is likely the same deployment retrieval wants, and its capacity now serves two masters.

The limits that actually decide it

Feature checklists make Azure AI Search and a dedicated vector store look interchangeable. The service limits do not.

  • Vector index size is a function of the SKU, not of your data. In the Dedicated pricing model it is bounded by the memory reserved for vector search on the tier; in Serverless it is 300 MB per index, roughly 30 percent of total index storage, and a hard limit that makes indexing fail rather than degrade.
  • Vector limits depend on when the service was created. Higher vector quotas apply to services created from April 2024 onward in regions with the extra capacity. An older service in a supported region may need an explicit upgrade to get them, which is a migration question rather than a config toggle.
  • Capacity is replicas multiplied by partitions. Basic supports three partitions and three replicas, nine search units, on services created after 3 April 2024; older Basic services are limited to one partition. S1 through S3 go to twelve of each.
  • The SLA is a replica count. Two or more replicas for query workloads, three or more for query and indexing. Partition count is not an SLA consideration at all, which surprises people who scaled for storage and assumed availability came with it.
  • Index count is capped per tier. Five or fifteen on Basic, fifty on S1, two hundred on S2 and S3. A platform that gives every tenant its own index runs into that ceiling long before it runs into storage, and index-per-tenant is the default instinct.
  • Maximum 4096 dimensions per vector field, on every tier, which is generous but worth knowing before someone standardizes on a larger embedding model.

Read together, these say something specific: Azure AI Search is priced and scaled as a search service that also does vectors. If the vector corpus is the dominant cost driver and it grows independently of query volume, you are paying for search units to hold memory. That is the condition under which a dedicated vector database wins, and it is a quantitative condition you can check rather than an architectural preference.

What Azure AI Search gives you that a vector store does not

The counterweight is real and often decisive for a corporate knowledge base.

Hybrid search in one request. A single query carries both a search parameter and vectorQueries, runs full-text and vector retrieval in parallel, and merges them with Reciprocal Rank Fusion into one ranked result set. Text ranking uses BM25; vector ranking uses HNSW or exhaustive KNN. That matters because product documentation is full of exact tokens, model numbers, error codes, part numbers, that lexical search finds reliably and embeddings blur.

{
  "search": "device not connecting after the firmware update",
  "vectorQueries": [
    { "kind": "vector", "vector": [ ... ], "k": 50,
      "fields": "contentVector", "exhaustive": false, "oversampling": 10 }
  ],
  "queryType": "semantic",
  "semanticConfiguration": "kb-semantic",
  "top": 10
}

Filtering, faceting, sorting, and scoring profiles apply to the same request, so document-level authorization (this tenant, this product line, this language) is a filter rather than a separate index. Integrated vectorization through indexers pulls from Blob Storage, Cosmos DB, Azure SQL, or OneLake and embeds on ingest, which removes an entire pipeline the platform team would otherwise own. And the index can be attached as a knowledge source for agent scenarios in Microsoft Foundry.

One capacity detail hides inside the ranking quality story. Semantic ranker is throttled by a queue, with a maximum of concurrent requests per search unit that is 2 on Basic, 3 on S1, and 4 on S2 and above, plus a bounded queue behind it. Exceed the queue and requests are rejected and must be retried. The way to raise it is more search units, or a support request. A retrieval layer that turns semantic ranking on for every query has therefore just made its concurrency ceiling a function of the search tier, and that ceiling is small on Basic.

The decision, stated as a test

Not a preference. Three checks, in order:

  • Does the corpus need lexical and vector retrieval together? If exact identifiers matter, and for product documentation they always do, hybrid plus semantic ranking in one service is worth a lot, and reimplementing it over a pure vector store means running an inverted index too.
  • Does the vector footprint grow independently of query volume? If yes, model the search units needed to hold it and compare against a vector database sized on storage. This is arithmetic, and it is usually where the answer flips.
  • How many logical corpora are there? Index-per-tenant hits the per-tier index ceiling quickly. The alternative is one index with a tenant filter, which is a security control that has to be enforced in the query layer, not in the client.

For this platform the answer is Azure AI Search, because the retail knowledge base is a few tens of gigabytes of product documentation where exact model numbers matter, and because the operational cost of one more managed service is lower than the cost of a self-run vector store on the same AKS cluster. The answer for an agent memory store with hundreds of millions of vectors and no lexical requirement would be the opposite, and the series would be wrong to pretend otherwise.

Tenancy, and the filter that is a security control

One index with a tenant filter is the right default under the index-count ceiling, and it puts an authorization decision in the query. The rule is the same as vary-by in Part 6: the filter value comes from the validated token, never from a request parameter. An application that passes its own tenant ID is one bug away from passing someone else's.

Because retrieval does not pass through API Management, the enforcement point is the application or the control plane, not a gateway policy. That is a genuine gap in the one-door model, and it is better named than glossed: the gateway governs model access, the search service governs document access, and they are two different control planes that must agree on who the tenant is. Deriving both from the same Entra ID claim is what keeps them from drifting.

Failure modes to watch

  • Indexing that fails at a size limit rather than degrading. Exceeding the per-index vector cap on Serverless fails the operation. Capacity for the vector index is a planning input, not something to discover during a bulk load.
  • An old search service with old vector quotas. Created before April 2024, in a region that now offers more, and silently limited until someone upgrades it.
  • Scaling partitions and expecting availability. Partitions are storage. The SLA counts replicas, two for queries and three when indexing runs alongside.
  • Semantic ranker throttling under load. Concurrency is per search unit and small on lower tiers. The symptom is rejected queries at peak, not slow ones.
  • Index-per-tenant against a tier ceiling. It works beautifully for the first dozen tenants and then stops, at which point the migration to a filtered index is a data project.
  • Retrieved context blowing up prompt size. The gateway meters it, so it shows up in Part 5's numbers as a token increase with no code change to explain it. Alert on tokens per request by tenant, not just on total spend.

What Part 8 inherits

A retrieval layer the applications call directly, an embeddings deployment shared with the cache, and prompts that are now considerably larger than they were. Which raises the question the next part is about: once an application needs retrieval, tool calling, and multiple model hops, does an orchestration framework belong on top of this gateway, or does the framework start fighting it.

Read this next

For the infrastructure and platform side of running this at scale, the field notes are at ercan.cloud, and the hub is at ercanermis.com.

References