Prompt caching

Prompt caching cuts the cost of text you send repeatedly: a system prompt, tool definitions, a long document, or the full history of a multi-turn conversation. The first request writes the prefix to the vendor’s cache at the normal input price; later requests that reuse the same prefix read it at the model’s cache-read price, a fraction of the input price.

MirAPI relays your requests to the model vendor unchanged, so caching behaves exactly as it does when you call the vendor directly. There is nothing to configure on the gateway side.

Cache writes vs cache reads

Action	When	Price
Cache write	The first request that contains the prefix	Normal input price; some vendors add a small write premium
Cache read	Later requests that reuse the prefix unchanged	The model’s cache-read price, a fraction of input

Two limits apply, and both are set by the vendor, not by MirAPI:

Minimum length: prefixes shorter than the vendor’s minimum are not cached. For example, Claude models require 1,024–4,096 tokens depending on the model, and OpenAI models require 1,024 tokens.
TTL: a cache entry expires after inactivity, typically around 5 minutes. Anthropic accepts "ttl": "1h" to extend this to one hour.

Enabling caching

OpenAI protocol — automatic

OpenAI-compatible models that support caching enable it automatically; you don’t need any request fields. This covers OpenAI, DeepSeek, and most other vendors. Keep the prompt prefix stable (see below) and cache reads happen on their own.

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.mirapi.ai/v1", api_key=os.environ["MIRAPI_API_KEY"])

# First call: the long system prompt is written to the cache at input price.
r1 = client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": "You are a support agent. Reference manual: " + MANUAL},
        {"role": "user", "content": "How do I reset my password?"},
    ],
)

# Second call with the same prefix: the manual is read from the cache at
# the cache-read price.
r2 = client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": "You are a support agent. Reference manual: " + MANUAL},
        {"role": "user", "content": "How do I cancel my subscription?"},
    ],
)

Anthropic protocol — cache_control

On /v1/messages, caching is opt-in: add a cache_control object to the content blocks you want to cache. You can set up to four breakpoints, so reserve them for the large, stable parts, such as a system prompt, a character card, CSV data, or a RAG document:

{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "You are a support agent. Reference manual: HUGE TEXT BODY",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "How do I reset my password?" }
  ]
}

Breakpoints can also sit on message content blocks, for fine-grained control over exactly what gets cached:

{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Use the reference below when answering." },
        {
          "type": "text",
          "text": "HUGE TEXT BODY",
          "cache_control": { "type": "ephemeral" }
        },
        { "type": "text", "text": "Summarize the main implementation details." }
      ]
    }
  ]
}

The cache_control fields:

Field	Type	Required	Description
`type`	string	yes	Must be `"ephemeral"`
`ttl`	string	no	`"1h"` extends the entry from the default ~5 minutes to one hour

For long sessions, add "ttl": "1h" to keep the entry for an hour instead of the default of about 5 minutes (writes cost more with the longer TTL):

{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "You are a support agent. Reference manual: HUGE TEXT BODY",
      "cache_control": { "type": "ephemeral", "ttl": "1h" }
    }
  ],
  "messages": [
    { "role": "user", "content": "First question in a long session" }
  ]
}

Structure prompts for cache hits

Caching only helps when the prefix is byte-for-byte identical between requests:

Put the stable part first (system prompt, tool definitions, long documents) and the changing part last.
Do not reorder or edit the prefix between calls; a change invalidates the cache from that point onward.
In multi-turn conversations, send the full history every turn so the shared prefix is read from the cache.
Keep per-request content out of the cached prefix: anything that changes must sit after the breakpoint.

Verify cache hits

When the vendor reports cache usage, the response usage object includes it:

{
  "usage": {
    "prompt_tokens": 10339,
    "completion_tokens": 60,
    "total_tokens": 10399,
    "prompt_tokens_details": {
      "cached_tokens": 10318,
      "cache_write_tokens": 0
    }
  }
}

Field	Meaning
`cached_tokens`	Tokens read from the cache. A value above zero means the request hit the cache.
`cache_write_tokens`	Tokens written to the cache on this request.

The Platform billing log lists cache reads as a separate line, so you can see the savings on each request next to the per-request data from GET /api/log/token.

Billing

Cache reads are billed at the model’s cache-read price, shown on the model’s catalog entry next to input and output.
Cache writes are billed at the normal input price; a few vendors add a write premium, and Anthropic’s 1-hour TTL costs more to write.
Reasoning and thinking tokens are billed as output regardless of caching.

Troubleshooting

cached_tokens stays at 0. The prefix changed between requests, is shorter than the vendor’s minimum, or its TTL expired. Keep the prefix byte-identical and stable.
Cache hits vary between identical requests. The cache lives on the vendor side and is tied to one upstream. MirAPI keeps consecutive requests on the same upstream wherever it can, but when that upstream is unavailable, or your primary channel fails over to your backup, a later request may miss the earlier write. This is expected behavior, not an error.
429 means rate limited. Retry with jittered exponential backoff (1s → 2s → 4s, capped around 30s). The gateway does not retry for you and sends no Retry-After header.
403 with a valid key means the balance is exhausted or another policy rejected the request; top up and retry.

Models & pricing: where the cache-read price lives
Billing & top-ups: how the three token tiers are charged
Reasoning models
Configuration