Prompt caching

Prompt caching cuts the cost of text that is sent again and again — a system prompt, tool definitions, a long document, or the full history of a multi-turn conversation. The first request writes the prefix to the vendor’s cache at the normal input price; later requests that reuse the same prefix read it at the model’s cache-read price, a fraction of the input price.

MirAPI relays your requests to the model vendor unchanged, so caching behaves exactly as it does when you call the vendor directly — nothing needs configuring on the gateway side.

Cache writes vs cache reads

Action	When	Price
Cache write	The first request that contains the prefix	Normal input price; some vendors add a small write premium
Cache read	Later requests that reuse the prefix unchanged	The model’s cache-read price — a fraction of input

Two limits apply, and both are set by the vendor, not by MirAPI:

Minimum length — prefixes shorter than the vendor’s minimum are not cached. For example, Claude models require 1,024–4,096 tokens depending on the model, and OpenAI models require 1,024 tokens.
TTL — a cache entry expires after inactivity, typically around 5 minutes. Anthropic accepts "ttl": "1h" to extend this to one hour.

Enabling caching

OpenAI protocol — automatic

OpenAI-compatible models that support caching enable it automatically — no request fields are needed. This covers OpenAI, DeepSeek, and most other vendors. Keep the prompt prefix stable (see below) and cache reads happen on their own.

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.mirapi.ai/v1", api_key=os.environ["MIRAPI_API_KEY"])

# First call: the long system prompt is written to the cache at input price.
r1 = client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": "You are a support agent. Reference manual: " + MANUAL},
        {"role": "user", "content": "How do I reset my password?"},
    ],
)

# Second call with the same prefix: the manual is read from the cache at
# the cache-read price.
r2 = client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": "You are a support agent. Reference manual: " + MANUAL},
        {"role": "user", "content": "How do I cancel my subscription?"},
    ],
)

Anthropic protocol — cache_control

On /v1/messages, caching is opt-in: add a cache_control object to the content blocks you want to cache. You can set up to four breakpoints, so reserve them for the large, stable parts — a system prompt, a character card, CSV data, or a RAG document:

{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "You are a support agent. Reference manual: HUGE TEXT BODY",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "How do I reset my password?" }
  ]
}

Breakpoints can also sit on message content blocks, for fine-grained control over exactly what gets cached:

{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Use the reference below when answering." },
        {
          "type": "text",
          "text": "HUGE TEXT BODY",
          "cache_control": { "type": "ephemeral" }
        },
        { "type": "text", "text": "Summarize the main implementation details." }
      ]
    }
  ]
}

The cache_control fields:

Field	Type	Required	Description
`type`	string	yes	Must be `"ephemeral"`
`ttl`	string	no	`"1h"` extends the entry from the default ~5 minutes to one hour

By default a cache entry expires after about 5 minutes. Add "ttl": "1h" to keep it for an hour — useful for long sessions, at a higher write price:

{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "You are a support agent. Reference manual: HUGE TEXT BODY",
      "cache_control": { "type": "ephemeral", "ttl": "1h" }
    }
  ],
  "messages": [
    { "role": "user", "content": "First question in a long session" }
  ]
}

Structure prompts for cache hits

Caching only helps when the prefix is byte-for-byte identical between requests:

Put the stable part first — system prompt, tool definitions, long documents — and the changing part last.
Do not reorder or edit the prefix between calls; a change invalidates the cache from that point onward.
In multi-turn conversations, send the full history every turn so the shared prefix reads from the cache.
Keep per-request content out of the cached prefix — anything that changes must sit after the breakpoint.

Verify cache hits

When the vendor reports cache usage, the response usage object carries it:

{
  "usage": {
    "prompt_tokens": 10339,
    "completion_tokens": 60,
    "total_tokens": 10399,
    "prompt_tokens_details": {
      "cached_tokens": 10318,
      "cache_write_tokens": 0
    }
  }
}

Field	Meaning
`cached_tokens`	Tokens read from the cache. Greater than zero means the request benefited.
`cache_write_tokens`	Tokens written to the cache on this request.

The console billing log shows the cache-read line separately, so you can see the saving per request next to GET /api/log/token.

Billing

Cache reads are billed at the model’s cache-read price, shown on the model’s catalogue entry next to input and output.
Cache writes are billed at the normal input price; a few vendors add a write premium, and Anthropic’s 1-hour TTL costs more to write.
Reasoning and thinking tokens are billed as output regardless of caching.

Troubleshooting

cached_tokens stays at 0. The prefix changed between requests, is shorter than the vendor’s minimum, or its TTL expired. Keep the prefix byte-identical and stable.
Cache hits vary between identical requests. The cache lives on the vendor side, and MirAPI routes each request to a healthy channel; if a later request lands on a different channel, it may miss the earlier write. This is expected, not an error.
429 means rate limited — retry with jittered exponential backoff (1s → 2s → 4s, capped around 30s). The gateway does not retry for you and sends no Retry-After header.
403 with a valid key means the balance is exhausted or another policy rejected the request; top up and retry.

Models & pricing — where the cache-read price lives
Billing & top-ups — how the three token tiers are charged
Reasoning models
Configuration