Four tools now sit behind one AgentCore Gateway instead of being wired one Python function at a time into each agent: cloudwatch-read, logs-read, and cost-read assume a read-only role in a spoke account and can never mutate anything, and ssm-execute, the platform's only mutating tool, cannot reach a spoke directly either, it can only start a Step Functions execution that pauses for a human's Slack approval. That pause is not a UI nicety. It is the one place in the whole platform where an AWS credential capable of changing something in a spoke account gets minted, and it only gets minted after a person clicks approve.

Part 1 set the scenario and picked AgentCore plus Strands. Part 2 built the account boundary: the ops-readonly and ops-mutate spoke roles this part assumes exist and reuses without modification. Part 3 shipped the first agent with two tools wired directly, cloudwatch_read and logs_read as plain @tool-decorated Python functions the triage agent called in-process, and said outright that Gateway would have been indirection with nothing yet to route between. This part is where that indirection earns its keep: a second read tool (cost-read) and the platform's first mutating tool (ssm-execute) both exist now, and four tools each agent might call independently is exactly the number where a shared, centrally-scoped tool plane stops being premature. The companion code lives at github.com/flightlesstux/agents-on-call, in terraform/20-gateway-tools/ and agents/tools/, and every snippet below is excerpted from those files, not simplified for the post.

Why inline tools stop scaling at agent number two

Part 3's two tools lived inside agents/triage/agent.py: plain functions, a cross-account assume-role call each, called directly by the one agent that needed them. Nothing wrong with that at one agent. The trouble starts at the second: Part 5's cost agent needs cloudwatch-read and cost-read, its runbook agent needs cloudwatch-read, logs-read, and eventually ssm-execute. Wired in-process the way Part 3 did it, that is the same Python function pasted into three agent codebases, three IAM roles independently granting the same sts:AssumeRole permission, and three places to fix a bug in how the cross-account session gets built, a typo in one copy's external ID handling that nobody notices until that one agent's calls start failing in a way the other two don't reproduce.

AgentCore Gateway removes the copy-paste, not by making the tools smarter but by making them exist exactly once. One Lambda per tool, one execution role per Lambda, one Gateway routing every agent's tool calls to the right target. An agent's own IAM role no longer needs sts:AssumeRole into any spoke at all (Part 3's triage runtime role still has it, since that agent predates this part); a future agent needs only permission to invoke the Gateway's MCP endpoint, and every cross-account read credential the platform mints comes from exactly four Lambda execution roles this module owns, not from however many agents exist by the time the series ends.

AgentCore Gateway as the MCP tool plane

AgentCore Gateway has converted APIs, Lambda functions, and existing services into MCP-compatible tools since AgentCore's general availability on October 13, 2025, with IAM-based authorization available at that same GA. MCP itself, the protocol every Gateway target speaks, is the open standard Anthropic published in November 2024 for connecting AI applications to data and tool sources; Gateway's contribution is turning four separate Lambda functions into one MCP server an agent's client library can discover tools from, instead of the agent's code needing each Lambda's ARN and invocation shape directly. Pricing follows the rest of AgentCore's shape: $0.005 per 1,000 API invocations (ListTools, InvokeTool, Ping) and $0.02 per 100 tools indexed per month, small enough that four tools cost nothing next to a single model invocation's token cost.

The Gateway resource itself is short. authorizer_type = "AWS_IAM" means the same IAM boundary this whole series has leaned on decides who can call the Gateway at all, no separate JWT issuer to stand up for a platform that already has an identity system:

resource "aws_bedrockagentcore_gateway" "tools" {
  name            = "${var.platform_name}-tools"
  description     = "MCP tool plane: cloudwatch-read, logs-read, cost-read, ssm-execute. AWS_IAM authorizer, IAM does the access control both for who can call the Gateway and what each tool Lambda can touch."
  role_arn        = aws_iam_role.gateway.arn
  protocol_type   = "MCP"
  authorizer_type = "AWS_IAM"

  # Optional Cedar policy layer, see variables.tf's cedar_policy_engine_arn
  # comment for why the engine itself is a variable, not a resource, at this
  # part's date.
  dynamic "policy_engine_configuration" {
    for_each = var.cedar_policy_engine_arn == null ? [] : [var.cedar_policy_engine_arn]
    content {
      arn  = policy_engine_configuration.value
      mode = var.cedar_policy_engine_mode
    }
  }

  tags = merge(var.tags, { Component = "gateway" })
}

That policy_engine_configuration block is the newest piece of Terraform in this series so far, worth a date check. Cedar-based policy evaluation for Gateway shipped in the aws provider as a list-only resource on May 27, 2026, then got this write-path block on June 10, provider version 6.50.0, eight days before this part's own date. Safe under the series' backdating rule, but only just, and there is still no Terraform resource to create the Cedar policy engine itself, only to attach one that already exists. var.cedar_policy_engine_arn defaults to null and the block is wrapped in a dynamic, the same pattern Part 2 used for Bedrock model access: a real AWS capability Terraform can reference but not yet fully provision.

One Lambda, one job: the tool pattern

Every tool follows the same shape: a Lambda function, an execution role scoped to exactly what that one tool needs, and a Gateway target describing the tool's input schema so an agent's MCP client can call it without reading the Lambda's source. cloudwatch-read is the clearest example, because it is the same logic Part 3 already shipped, just moved. The execution role trusts only the Lambda service and grants only two things: assume ops-readonly in the configured spokes, and write its own CloudWatch Logs:

data "aws_iam_policy_document" "cloudwatch_read_permissions" {
  statement {
    sid       = "AssumeOpsReadonlyInSpokes"
    effect    = "Allow"
    actions   = ["sts:AssumeRole"]
    resources = local.ops_readonly_role_arns
  }

  statement {
    sid    = "WriteOwnLogs"
    effect = "Allow"
    actions = [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents",
    ]
    resources = [local.lambda_log_arn]
  }
}

resource "aws_lambda_function" "cloudwatch_read" {
  function_name    = "${var.platform_name}-cloudwatch-read"
  description      = "AgentCore Gateway MCP tool: reads a CloudWatch metric from a spoke account via ops-readonly. Read-only."
  role             = aws_iam_role.cloudwatch_read.arn
  handler          = "cloudwatch_read.handler"
  runtime          = "python3.12"
  timeout          = 30
  memory_size      = 256
  filename         = data.archive_file.cloudwatch_read_zip.output_path
  source_code_hash = data.archive_file.cloudwatch_read_zip.output_base64sha256
  ...
}

No wildcard resource anywhere in that policy document: local.ops_readonly_role_arns builds one ARN per configured spoke account ID, and nothing else this role can touch exists outside that list plus its own log group. logs-read and cost-read reuse the identical trust and permission documents; only their handler code and Gateway-visible tool schema differ. The handler itself is Part 3's logic almost unchanged, reading its arguments from a Gateway-supplied event instead of Python keyword arguments:

def handler(event: dict, context) -> dict:
    """Lambda entrypoint for the cloudwatch-read Gateway target.
    ...
    """
    spoke_account_id = event["spoke_account_id"]
    namespace = event["namespace"]
    metric_name = event["metric_name"]
    dimensions = event.get("dimensions", {})
    lookback_minutes = int(event.get("lookback_minutes", 60))
    stat = event.get("stat", "Average")
    period_seconds = int(event.get("period_seconds", 60))

    session = assume_role(spoke_account_id, OPS_READONLY_ROLE_NAME, "cloudwatch-read")
    cloudwatch = session.client("cloudwatch", config=_boto_config)

The Gateway target is what turns that Lambda into something an agent's MCP client can discover: a name, a description, and a JSON-Schema-shaped input contract. The full block runs to seven parameters; the shape below is the same for every tool:

resource "aws_bedrockagentcore_gateway_target" "cloudwatch_read" {
  gateway_identifier = aws_bedrockagentcore_gateway.tools.gateway_id
  name               = "cloudwatch-read"
  description        = "Reads a CloudWatch metric's recent datapoints from a spoke account."

  credential_provider_configuration {
    gateway_iam_role {}
  }

  target_configuration {
    mcp {
      lambda {
        lambda_arn = aws_lambda_function.cloudwatch_read.arn

        tool_schema {
          inline_payload {
            name        = "cloudwatch_read"
            description = "Read a CloudWatch metric's recent datapoints from a spoke account. An empty datapoint list is a real finding, not an error."

            input_schema {
              type = "object"

              property {
                name        = "spoke_account_id"
                type        = "string"
                description = "12-digit account ID of the spoke to query."
                required    = true
              }
              ...
            }
          }
        }
      }
    }
  }
}

credential_provider_configuration { gateway_iam_role {} } is the detail worth pausing on: it tells Gateway to invoke the target Lambda using the Gateway's own execution role over plain IAM, not an OAuth flow or an API key pulled from a credential provider. That is the right choice for a same-account, same-platform Lambda; the Gateway's role_arn is granted lambda:InvokeFunction on exactly these four Lambda ARNs and nothing else, the same one-purpose-one-permission discipline as every role in this series so far.

Read-only is not a slogan, it is three separate IAM facts

"Read-only by default" only means something if it survives contact with a bug. This wiring survives three specific ones. Blast radius: the three read roles hold no permission that can change state in a spoke, only sts:AssumeRole into ops-readonly, whose policy (Part 2) layers an explicit deny over its read-only managed policies, so a prompt injection that convinces an agent to ask a read tool to delete something has nowhere to go. Audit: every read still crosses an account boundary via sts:AssumeRole, so CloudTrail in the security account sees exactly which tool Lambda touched which spoke and when. Trust with the human team: an on-call engineer who sees ssm-execute behind an approval gate, and three tools named read whose roles hold no mutating action anywhere, has a reason to believe the "it only reads" claim instead of taking it on faith, checkable in five minutes with aws iam get-role-policy, not a sentence in a system prompt someone has to trust the model followed.

The one mutating tool, and the gate behind it

ssm-execute is the only tool on the platform whose IAM role includes no sts:AssumeRole permission into a spoke at all. Its entire permission set is states:StartExecution, scoped to exactly one state machine ARN:

data "aws_iam_policy_document" "ssm_execute_permissions" {
  statement {
    sid       = "StartApprovalStateMachineOnly"
    effect    = "Allow"
    actions   = ["states:StartExecution"]
    resources = [aws_sfn_state_machine.mutation_approval.arn]
  }
  ...
}

Calling ssm-execute does not run anything, it proposes running something. The handler checks the proposed document against a local allowlist before it even creates a Step Functions execution, a redundant, fail-fast check on top of the real one, ops-mutate's own IAM policy from Part 2, which is what actually holds if this Lambda's code has a bug:

if ssm_document_arn not in ALLOWED_SSM_DOCUMENT_ARNS:
    return {
        "status": "rejected",
        "reason": f"{ssm_document_arn} is not on the allowlist; no approval request was created.",
    }

execution = _sfn.start_execution(
    stateMachineArn=APPROVAL_STATE_MACHINE_ARN,
    input=json.dumps({
        "spoke_account_id": spoke_account_id,
        "ssm_document_arn": ssm_document_arn,
        "ssm_parameters": ssm_parameters,
        "diagnosis": diagnosis,
        "blast_radius": blast_radius,
    }),
)

The state machine is where .waitForTaskToken earns its name. Step Functions' own tutorial on this pattern is explicit that a Task state can pause indefinitely, generate a token, and resume only when something outside the state machine calls SendTaskSuccess or SendTaskFailure with that exact token, and that both the wait state and the whole execution need their own TimeoutSeconds or a stuck execution never ends. NotifySlackAndWaitForApproval invokes slack-post with the task token in its payload, then Step Functions itself pauses, not the Lambda:

definition = jsonencode({
  Comment        = "Human approval gate for ops-mutate SSM Automation execution."
  StartAt        = "NotifySlackAndWaitForApproval"
  TimeoutSeconds = var.state_machine_timeout_seconds
  States = {
    NotifySlackAndWaitForApproval = {
      Type     = "Task"
      Resource = "arn:aws:states:::lambda:invoke.waitForTaskToken"
      Parameters = {
        FunctionName = aws_lambda_function.slack_post.arn
        Payload = {
          "TaskToken.$"      = "$$.Task.Token"
          "Diagnosis.$"      = "$.diagnosis"
          "BlastRadius.$"    = "$.blast_radius"
          "SsmDocument.$"    = "$.ssm_document_arn"
          "SsmParameters.$"  = "$.ssm_parameters"
          "SpokeAccountId.$" = "$.spoke_account_id"
        }
      }
      TimeoutSeconds = var.approval_wait_timeout_seconds
      Next           = "RunApprovedAutomation"
      Catch = [
        {
          ErrorEquals = ["States.ALL"]
          Next        = "ApprovalDeniedOrTimedOut"
        }
      ]
    }
    ...
  }
})

Two states follow, not shown above: RunApprovedAutomation invokes executor.py with the wait state's own output as its payload, and ApprovalDeniedOrTimedOut is a plain Fail state the Catch above routes into. slack-post posts a card with the diagnosis, blast radius, exact SSM document, and parameters, plus approve/deny links carrying the task token, each pointing at an API Gateway route (/succeed, /fail) backed by approval_callback.py. That Lambda's whole job is one API call, send_task_success or send_task_failure against the clicked link's token, and its IAM policy grants both on Resource = "*" for a real reason: neither API addresses a specific execution by ARN, only the opaque token, the same shape Part 2's InspectAndStopOwnExecutions statement hit for SSM's own inspection APIs. Worth naming plainly: encoding that token in a GET link's query string is a real weakness, a forwarded or logged link is a forwarded or logged approval, documented in slack_post.py rather than hidden, to be replaced with a signed, single-use, datastore-backed token before this runs against anything that matters. These routes carry no authorizer at all: token possession is the entire access control, so anyone who obtains the link, not just the intended Slack approver, can fire it.

Only on approval does anything with permission to change a spoke account get created. executor.py is the sole Lambda whose execution role can assume ops-mutate: Part 2's trust policy names exactly this Lambda's role ARN as its only principal, and its own credential lifetime is deliberately short:

session = assume_role(
    spoke_account_id,
    OPS_MUTATE_ROLE_NAME,
    "post-approval-executor",
    duration_seconds=300,
)
ssm = session.client("ssm", config=_boto_config)

response = ssm.start_automation_execution(
    DocumentName=ssm_document_arn,
    Parameters={k: [v] if not isinstance(v, list) else v for k, v in ssm_parameters.items()},
)

Five minutes, shorter than the read tools' fifteen-minute default: this credential exists to make exactly one API call, not to sit around reusable for a follow-up the function never intended. Two Lambdas, two IAM roles, one narrow handoff through a state machine execution, is what makes "the only mutating tool cannot reach a spoke directly" a property of the wiring, not a promise this code keeps by choosing not to import boto3's STS client.

What the agent sees when a tool is denied

Three distinct kinds of "no" exist here, worth telling apart. An application-level rejection: ssm-execute's allowlist check returns {"status": "rejected", "reason": ...} as a normal tool result, no exception, no execution created; the agent reports it plainly. An IAM-level denial: a missing lambda:InvokeFunction grant or assume-role grant fails the call with an AWS AccessDeniedException surfaced as a tool error, not a tool result, the hard-stop distinction Part 3's system prompt already trained the triage agent to respect. And, once cedar_policy_engine_arn is set with mode = "ENFORCE", a policy-layer denial: Cedar rejects a call before it reaches the Lambda at all, on rules outside any one tool's own code; in LOG_ONLY the same call is logged but allowed through, which is why staging a new policy there against real traffic matters before flipping to ENFORCE, where an over-broad rule fails calls that were never a problem with no tool-level log to explain why.

Failure modes to watch

Five things worth knowing before this pipeline runs against a real proposal. The two SSM document allowlists, ssm-execute's local copy and ops-mutate's own IAM policy, live in two independently-applied Terraform modules (this one and 00-foundation), and nothing enforces they stay identical; a document missing from the local list is rejected before an approval request exists, a document missing from the IAM list gets a request created for something ops-mutate then refuses to run, and the second case is worse because a human already clicked approve before discovering it. Forgetting either TimeoutSeconds value, the wait state's or the execution's, turns an unanswered Slack proposal into a paused execution that ages forever instead of failing loud into ApprovalDeniedOrTimedOut, exactly what the Step Functions human-approval tutorial flags as the most common way this pattern breaks. And the approval link's own weakness stands until it is replaced with a signed, single-use, datastore-backed token: treat every Slack channel this posts to as one where forwarding a message is equivalent to forwarding an approval. A GET request that mutates state invites accidental approval too: Slack's own link unfurler, corporate link-security scanners, and browser prefetch can all fetch that URL before a human ever clicks it, and no code path here distinguishes that fetch from a deliberate approval. And approval_callback.py passes the proposal JSON straight from the clicked link's query string to send_task_success without checking it against whatever Step Functions actually recorded for that token, so a tampered link can swap in a different ssm_document_arn or ssm_parameters than what the human saw in Slack; ops-mutate's own IAM allowlist bounds the damage to something already permitted, but the specific document-and-parameters combination a human approved is not the one wired to what actually runs.

Read this next

The full terraform/20-gateway-tools/ module and agents/tools/, including the IAM policy documents and the three other Gateway targets trimmed from the snippets above, live in the companion repository at github.com/flightlesstux/agents-on-call. For the database and infrastructure side of running platforms like this one at scale, the field notes are at ercan.cloud, and the hub is at ercanermis.com.