# Prompt caching

> How prompt caching works on MirAPI, how to enable it over the OpenAI and Anthropic protocols, and how to verify cache hits in the usage fields.

Prompt caching cuts the cost of text that is sent again and again — a system prompt, tool definitions, a long document, or the full history of a multi-turn conversation. The first request **writes** the prefix to the vendor's cache at the normal input price; later requests that reuse the same prefix **read** it at the model's cache-read price, a fraction of the input price.

MirAPI relays your requests to the model vendor unchanged, so caching behaves exactly as it does when you call the vendor directly — nothing needs configuring on the gateway side.

## Cache writes vs cache reads

| Action | When | Price |
|---|---|---|
| Cache write | The first request that contains the prefix | Normal input price; some vendors add a small write premium |
| Cache read | Later requests that reuse the prefix unchanged | The model's **cache-read** price — a fraction of input |

Two limits apply, and both are set by the vendor, not by MirAPI:

- **Minimum length** — prefixes shorter than the vendor's minimum are not cached. For example, Claude models require 1,024–4,096 tokens depending on the model, and OpenAI models require 1,024 tokens.
- **TTL** — a cache entry expires after inactivity, typically around 5 minutes. Anthropic accepts `"ttl": "1h"` to extend this to one hour.

## Enabling caching

### OpenAI protocol — automatic

OpenAI-compatible models that support caching enable it **automatically** — no request fields are needed. This covers OpenAI, DeepSeek, and most other vendors. Keep the prompt prefix stable (see below) and cache reads happen on their own.

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.mirapi.ai/v1", api_key=os.environ["MIRAPI_API_KEY"])

# First call: the long system prompt is written to the cache at input price.
r1 = client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": "You are a support agent. Reference manual: " + MANUAL},
        {"role": "user", "content": "How do I reset my password?"},
    ],
)

# Second call with the same prefix: the manual is read from the cache at
# the cache-read price.
r2 = client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": "You are a support agent. Reference manual: " + MANUAL},
        {"role": "user", "content": "How do I cancel my subscription?"},
    ],
)
```

### Anthropic protocol — cache_control

On `/v1/messages`, caching is opt-in: add a `cache_control` object to the content blocks you want to cache. You can set up to **four** breakpoints, so reserve them for the large, stable parts — a system prompt, a character card, CSV data, or a RAG document:

```json
{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "You are a support agent. Reference manual: HUGE TEXT BODY",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "How do I reset my password?" }
  ]
}
```

Breakpoints can also sit on message content blocks, for fine-grained control over exactly what gets cached:

```json
{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Use the reference below when answering." },
        {
          "type": "text",
          "text": "HUGE TEXT BODY",
          "cache_control": { "type": "ephemeral" }
        },
        { "type": "text", "text": "Summarize the main implementation details." }
      ]
    }
  ]
}
```

The `cache_control` fields:

| Field | Type | Required | Description |
|---|---|---|---|
| `type` | string | yes | Must be `"ephemeral"` |
| `ttl` | string | no | `"1h"` extends the entry from the default ~5 minutes to one hour |

By default a cache entry expires after about 5 minutes. Add `"ttl": "1h"` to keep it for an hour — useful for long sessions, at a higher write price:

```json
{
  "model": "deepseek-chat",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "You are a support agent. Reference manual: HUGE TEXT BODY",
      "cache_control": { "type": "ephemeral", "ttl": "1h" }
    }
  ],
  "messages": [
    { "role": "user", "content": "First question in a long session" }
  ]
}
```

## Structure prompts for cache hits

Caching only helps when the prefix is **byte-for-byte identical** between requests:

1. Put the stable part first — system prompt, tool definitions, long documents — and the changing part last.
2. Do not reorder or edit the prefix between calls; a change invalidates the cache from that point onward.
3. In multi-turn conversations, send the full history every turn so the shared prefix reads from the cache.
4. Keep per-request content out of the cached prefix — anything that changes must sit after the breakpoint.

## Verify cache hits

When the vendor reports cache usage, the response `usage` object carries it:

```json
{
  "usage": {
    "prompt_tokens": 10339,
    "completion_tokens": 60,
    "total_tokens": 10399,
    "prompt_tokens_details": {
      "cached_tokens": 10318,
      "cache_write_tokens": 0
    }
  }
}
```

| Field | Meaning |
|---|---|
| `cached_tokens` | Tokens read from the cache. Greater than zero means the request benefited. |
| `cache_write_tokens` | Tokens written to the cache on this request. |

The console billing log shows the cache-read line separately, so you can see the saving per request next to `GET /api/log/token`.

## Billing

- Cache reads are billed at the model's **cache-read** price, shown on the model's catalogue entry next to input and output.
- Cache writes are billed at the normal input price; a few vendors add a write premium, and Anthropic's 1-hour TTL costs more to write.
- Reasoning and thinking tokens are billed as output regardless of caching.

## Troubleshooting

- **`cached_tokens` stays at 0.** The prefix changed between requests, is shorter than the vendor's minimum, or its TTL expired. Keep the prefix byte-identical and stable.
- **Cache hits vary between identical requests.** The cache lives on the vendor side, and MirAPI routes each request to a healthy channel; if a later request lands on a different channel, it may miss the earlier write. This is expected, not an error.
- **429** means rate limited — retry with jittered exponential backoff (1s → 2s → 4s, capped around 30s). The gateway does not retry for you and sends no `Retry-After` header.
- **403** with a valid key means the balance is exhausted or another policy rejected the request; top up and retry.

## Related links

- [Models & pricing](/docs/models) — where the cache-read price lives
- [Billing & top-ups](/docs/billing) — how the three token tiers are charged
- [Reasoning models](/docs/guides/reasoning)
- [Configuration](/docs/configuration)