Stop Fine-Tuning. You Need RAG, a Cache, and Better Prompts

For most teams reaching for fine-tuning on Amazon Bedrock, the correct answer is retrieval, a prompt cache, and better prompts, in that order. Fine-tuning is the tool you consider after those three are exhausted, not before. The reason is not ideology. It is the bill. A custom fine-tuned model on Bedrock has to be served through Provisioned Throughput, and that pricing model changes the economics of your whole application.

Fine-tuning feels like the serious move. You have proprietary data, you want the model to "know" it, so you train. But most of what people fine-tune for is not knowledge the weights need to absorb. It is context the model needs at inference time, formatting it can be told to follow, and instructions that were never written clearly. All three have cheaper solutions.

The cost you sign up for when you fine-tune

On Bedrock, you cannot call a custom fine-tuned model with pay-per-token on-demand pricing. To run inference against it you buy Provisioned Throughput, which reserves model units by the hour, with the cheapest commitments running monthly or longer. You are now paying for reserved capacity whether or not traffic is flowing.

That flips your cost structure. On-demand pricing scales with use: no traffic, no charge. Provisioned Throughput is a fixed floor: an idle model unit at 3am costs the same as a busy one at peak. For a workload that is spiky, low-volume, or still finding product-market fit, you are paying rent on capacity you are not using. And you have added an MLOps burden, retraining as your data drifts, versioning models, evaluating each new checkpoint, that someone now owns forever.

What the cheaper stack actually solves

RAG handles knowledge

If the goal is for the model to answer from your documents, that is retrieval, not training. A Bedrock Knowledge Base embeds your content and pulls the relevant passages into context at query time. New document, no retraining: you index it and it is retrievable in minutes. Fine-tuning bakes knowledge into weights that are stale the moment your data changes. Retrieval keeps knowledge in a store you can update continuously, on pay-per-token inference against a base model.

Prompt caching handles the repeated context

The usual objection to RAG and few-shot prompting is token cost: you are resending a long system prompt, tool definitions, and retrieved context on every call. Prompt caching removes most of that. Bedrock caches the stable prefix of your prompt so repeated tokens are billed at a large discount and served faster, and the cache duration now extends to an hour, which comfortably spans a user session or a batch job. The thing that made a fat prompt expensive is the thing caching is built to fix.

Better prompts handle behavior and format

A surprising share of fine-tuning projects are really asking the model to output consistent JSON, adopt a tone, or follow a procedure. That is a prompt problem. A clear system prompt with a few well-chosen examples gets you most of the way, at zero training cost and with a change cycle measured in seconds instead of a retraining run. Exhaust structured prompting and few-shot examples before you conclude the weights need to change.

The cost math, in plain terms

Put the two paths side by side for a typical low-to-medium volume application:

Fine-tune path:
  training run (one-off)
  + Provisioned Throughput (fixed monthly floor, idle or not)
  + retraining + eval + versioning (ongoing engineering)

RAG + cache + prompts path:
  on-demand tokens (scales to zero when idle)
  + Knowledge Base storage + embedding (small, usage-based)
  + prompt cache (discounts the repeated prefix)

For anything short of high, steady, predictable volume, the second column is cheaper in dollars and far cheaper in engineering time. The fixed floor of Provisioned Throughput only pays off when you are running enough constant traffic to keep those reserved units busy.

When fine-tuning is genuinely the answer

It is a real tool with a real niche. Fine-tune when you need a behavior no prompt reliably produces: a specialized output style, a domain vocabulary the base model handles poorly, or a latency and token budget that a long prompt cannot meet at your volume. Those cases exist. They are the exception, and you should be able to point at a measured failure of the cheaper stack before you commit to the fixed cost of Provisioned Throughput.

The takeaway

Fine-tuning on Bedrock means Provisioned Throughput, which means a fixed cost floor and an MLOps commitment you carry indefinitely. Most of what teams fine-tune for is knowledge, repeated context, or unclear instructions, and those are solved more cheaply by retrieval, prompt caching, and better prompts. Reach for the cheap stack first, measure where it fails, and only then pay for training.

Read this next

• Knowledge Base Chunking Is Where Your RAG Quality Dies, because a RAG stack is only as good as how you cut the documents.
• Cutting Amazon Bedrock Knowledge Base Costs by ~90%, on squeezing the same cost discipline out of the vector store.

For the wider cost-optimization playbook across AWS, the field notes live at ercan.cloud, and the hub is at ercanermis.com.