Amazon Bedrock
Filed under
Per-App Bedrock Cost Tracking with Inference Profiles
Application inference profiles put cost allocation tags on Bedrock calls, turning one shared bill into per-team lines. The tag design is the hard part.
EU AI Act, August 2: The Deadline That Didn't Move
The Digital Omnibus deferred the high-risk deadlines to 2027 and 2028. Article 50 transparency still lands on 2 August 2026, and that is the one most teams hit.
SageMaker vs Bedrock: An Org Decision, Not a Technical One
The SageMaker or Bedrock question is really about whether your org has a team that owns models. Pick for the team topology you have, not the one on the slide.
Prompt Injection via Your Own Docs: The RAG Attack Surface
Your knowledge base is untrusted input. Retrieval hands attacker-authored text to the model, so the control that pays off is scanning at ingestion time.
AWS Monthly (June '26): Agents Get a Feedback Loop
June 2026 on AWS: the Summit in New York turned production traces into agent improvement, shipped Continuum for security, and took the AgentCore harness to GA.
Your Multi-Agent System's Real Limit Is Tokens Per Minute
Amazon Bedrock now exposes per-model tokens-per-minute quotas in Service Quotas. For agents, TPM is the real scaling ceiling. Plan for it before the 429s start.
The Context Window Is Not Your Friend
A huge context window is not a replacement for retrieval. Recall degrades as the prompt grows, cost scales with every token, and the middle gets skimmed.
AWS Monthly (May '26): Agents Get a Wallet
May 2026 on AWS: AgentCore Payments lets agents transact, and the Agent Toolkit for AWS plus a GA managed MCP server harden the toolchain that builds them.
When Haiku Beats Opus: Model Right-Sizing on Bedrock
Defaulting every call to Opus is how LLM bills balloon. Route by task class: Haiku for the mechanical majority, Opus as the escalation path for hard cases.
Agentic RAG Is Mostly Latency You Don't Need
Agentic RAG loops through retrieval hops, each a model round trip. For most questions one good query wins. Reach for the loop only when it earns the latency.
Evals Before Agents: You Can't Ship What You Can't Score
Without an eval harness, every agent change is a vibe check. Build the scoreboard before the agent, and treat the LLM-as-judge as a component that can be wrong.
Semantic Caching: Two Different Questions, One Answer
Semantic caching returns one stored answer for two differently worded questions. It cuts cost and latency, but a false hit serves a confidently wrong reply.
LLM Gateways: Why Every Platform Team Builds One Eventually
A second team calling a model means ungoverned fan-out. An LLM gateway centralizes auth, quota, routing, and audit. Build vs LiteLLM vs API Gateway.
AWS Monthly (Apr '26): OpenAI Lands on Bedrock
April 2026 on AWS: OpenAI models, Codex, and Managed Agents arrive on Bedrock, and AgentCore adds a managed harness and CLI to shorten the path to an agent.
Cross-Region Inference: Cheap Resilience or Residency Trap?
Bedrock cross-region inference smooths throughput and throttling. But a global profile can route your prompt out of its geography. Read residency first.
Your LLM Bill Is an Observability Problem
A surprising Bedrock bill is not a pricing problem, it is a visibility one. If you cannot attribute tokens to a feature, tenant, or agent, you cannot manage it.
Batch Inference on Bedrock: Half Price If You Can Wait
Amazon Bedrock batch inference runs at 50 percent of on-demand pricing. The only cost is latency. For any job where nobody is waiting, that trade is free money.
Multi-Tenant LLM Apps: Isolating Customers on a Shared Model
One shared Bedrock model, many customers. The model is stateless, so isolation is your job: scope the retrieval, cap the quota, carry identity per tenant.
Structured Output Beats Clever Parsing
Still regex-parsing JSON out of model text? Stop. Bedrock structured outputs enforce a JSON Schema during decoding, so the response is valid by construction.
Prompt Caching on Bedrock: The 90% Discount Most Teams Ignore
Bedrock prompt caching reads a repeated prefix at 90 percent off, but a cache write costs more than not caching. The breakpoint decides which you get.
AWS Monthly (Mar '26): Governance Comes for the Agents
March 2026 on AWS: AgentCore Policy and Evaluations reach GA, Elemental Inference ships, and agent governance moves from demo to a production control plane.
Streaming Responses Are a UX Decision, Not a Performance One
Streaming model responses is a user-experience choice about time to first token, not a speed fix. Sometimes it makes structured output and tool use worse.
Bedrock Agents vs Rolling Your Own Loop
Amazon Bedrock Agents handle orchestration, memory, and tool calls for you. Here is when the managed framework saves you real work and when it quietly owns you.
Stop Fine-Tuning. You Need RAG, a Cache, and Better Prompts
Fine-tuning plus provisioned throughput is the expensive answer to most LLM problems. The cheaper path is retrieval, prompt caching, and better prompts.
Knowledge Base Chunking Is Where Your RAG Quality Dies
Most bad RAG answers are a retrieval problem, not a model problem. How fixed, semantic, and hierarchical chunking in Bedrock Knowledge Bases set your quality.
Bedrock Guardrails Won't Save You From Prompt Injection
Amazon Bedrock Guardrails filter content, they do not authorize actions. Real prompt injection defense is input isolation, tool allowlists, and IAM scoping.