Skip to content

Control token spend and context window growth in chained agents

Cap runaway LLM bills in multi-agent workflows with prompt caching, rolling context compression, and per-session token ceilings.

Copy the prompt and paste it into the AI assistant you already use.

Four levers to slash multi-agent token burn

  1. 01

    Filter verbose JSON tool payloads

    Project external API responses down to the five or six fields the agent actually needs instead of injecting 12KB raw objects.

  2. 02

    Structure prompts for prefix caching

    Place static system rules and tool definitions first and dynamic user state last so provider prompt caches cut input costs up to 80%.

  3. 03

    Compress older turns into structured summaries

    Keep the last four verbatim turns plus a compact state summary rather than re-sending entire multi-topic histories on turn 15.

  4. 04

    Monitor cost per resolved conversation

    Alert on anomalous sessions exceeding 99th-percentile token budgets to catch runaway tool retries or prompt-stuffing abuse early.

Questions and answers

What usually wastes the most tokens in production multi-agent chatbots?

Unfiltered JSON payloads from order, CRM, or search APIs re-sent across every subsequent conversation turn typically account for over half of token spend.

Does summarizing conversation history cause the chatbot to forget important details?

Not if you store exact identifiers—order IDs, emails, error codes—in dedicated JSON state keys outside prose summarization.

How do we protect public website chatbots from token-draining abuse?

Combine IP/session rate limiting, max input character caps, and per-visitor daily token budgets before calling frontier reasoning models.

Learn more

Related guides and background articles for this task