# Batch processing

> Build a resilient client-side batch translator for CSVs with asyncio concurrency, jittered 429 backoff, checkpoint resume, and cost reconciliation.

Batch processing turns a list of inputs into a list of outputs without you babysitting every call. This page builds one Python script that translates each row of `input.csv` (`id,text`) into `output.csv` (`id,text,translation`) concurrently, survives a mid-run restart, and stays under your key's rate limit.

## Prerequisites

- Python 3.10+
- `pip install aiohttp` — the async HTTP client the script uses
- A `MIRAPI_API_KEY` created in the console: https://console.mirapi.ai → **API Keys**
- A chat model enabled for that key — the model whitelist applies, and a model outside it returns 403

```bash
pip install aiohttp
export MIRAPI_API_KEY=sk-...
```

## The script

Save this as `batch.py`:

```python
import asyncio
import csv
import os
import random

import aiohttp

BASE = "https://api.mirapi.ai/v1"
API_KEY = os.environ["MIRAPI_API_KEY"]
MODEL = "deepseek-chat"
CONCURRENCY = 5      # at most 5 requests in flight (client-side rate limit)
MAX_RETRIES = 5
INPUT_CSV = "input.csv"
OUTPUT_CSV = "output.csv"


def load_done() -> set[str]:
    """Resume: ids already written to the output file."""
    done: set[str] = set()
    try:
        with open(OUTPUT_CSV, newline="", encoding="utf-8") as f:
            for row in csv.DictReader(f):
                done.add(row["id"])
    except FileNotFoundError:
        pass
    return done


async def translate(session: aiohttp.ClientSession, text: str) -> str:
    payload = {
        "model": MODEL,
        "messages": [
            {"role": "system", "content": "Translate to English. Reply with the translation only."},
            {"role": "user", "content": text},
        ],
    }
    for attempt in range(MAX_RETRIES):
        try:
            async with session.post(
                f"{BASE}/chat/completions",
                json=payload,
                headers={"Authorization": f"Bearer {API_KEY}"},
            ) as resp:
                if resp.status == 429:
                    # MirAPI sends no Retry-After header; back off manually.
                    wait = min(2**attempt + random.uniform(0, 1), 30)
                    await asyncio.sleep(wait)
                    continue
                resp.raise_for_status()
                data = await resp.json()
                return data["choices"][0]["message"]["content"].strip()
        except (aiohttp.ClientError, KeyError):
            # ClientResponseError (4xx/5xx) subclasses ClientError, so
            # balance, whitelist, and quota failures retry like the rest.
            await asyncio.sleep(1)
    raise RuntimeError(f"failed after {MAX_RETRIES} attempts")


async def main() -> None:
    done = load_done()
    sem = asyncio.Semaphore(CONCURRENCY)

    with open(INPUT_CSV, newline="", encoding="utf-8") as f:
        rows = list(csv.DictReader(f))

    write_mode = "a" if done else "w"
    with open(OUTPUT_CSV, write_mode, newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=["id", "text", "translation"])
        if not done:
            writer.writeheader()

        async with aiohttp.ClientSession() as session:
            for row in rows:
                if row["id"] in done:
                    continue  # checkpoint: skip already-translated rows
                async with sem:
                    translation = await translate(session, row["text"])
                writer.writerow({"id": row["id"], "text": row["text"], "translation": translation})
                f.flush()  # durable checkpoint after every row


asyncio.run(main())
```

The knobs you'll adjust:

| Constant | Default | What it controls |
|---|---|---|
| `CONCURRENCY` | `5` | Max requests in flight — your client-side rate limiter |
| `MAX_RETRIES` | `5` | Retries per row before the script fails that row |
| `MODEL` | `deepseek-chat` | The exact catalogue `id` to call |
| `BASE` | `https://api.mirapi.ai/v1` | OpenAI-compatible base URL |

The script speaks the OpenAI protocol. MirAPI exposes the same catalogue over three front-ends — the Anthropic (`/v1/messages`) and Gemini (`...:generateContent`) variants are shown in [First integration](/docs/tutorials/first-integration).

## How the script works

- **Rate limiting:** the semaphore caps in-flight requests at `CONCURRENCY`. Raise it carefully — once you exceed your key's limit, the gateway returns 429.
- **429 backoff:** an exponential wait with jitter (`2**attempt`), capped at 30 seconds. MirAPI sends no `Retry-After` header and never retries on your behalf, so the backoff has to live in your code.
- **Resume:** ids already in `output.csv` are skipped on restart, and each row is flushed immediately — a crash loses at most the row in flight.

## Handle errors

| Status | Meaning | What to do |
|---|---|---|
| 401 | Key missing, invalid, or unknown | Fix the credential |
| 403 | Key valid but rejected — balance, whitelist, quota, or IP | Read the message, fix the cause, retry |
| 413 | Request body too large | Reduce the size |
| 429 | Rate limited | Back off with jitter — there is no `Retry-After` header |
| 500 | Gateway or upstream failure | Retry idempotent requests |

An empty balance returns **403**, not 401. OpenAI-protocol errors use the envelope `{"error": {"message", "type", "param", "code"}}`, where `type` is `new_api_error` (gateway-side) or `upstream_error` (a sanitized upstream failure). Every error message ends with a request ID — include it in a support or billing ticket.

The script retries `aiohttp.ClientError` (which covers 4xx/5xx responses) for `MAX_RETRIES` attempts, then gives up on that row. A persistent 403 from an empty balance will not clear itself by retrying — top up instead of spinning.

## Cost and reconciliation

Text is billed per 1M tokens in three tiers — input, output, and cache read — with cache reads cheaper than ordinary input. A short sentence runs a few hundred tokens, so the three-row test below costs well under a cent. Reasoning tokens, when a reasoning model is used, are billed as output.

:::tip
The system prompt is identical on every row, and OpenAI-compatible models cache prompts automatically. When a row hits the cache, `usage.prompt_tokens_details.cached_tokens` is nonzero and that portion is billed at the cheaper cache-read rate. See [Prompt caching](/docs/guides/prompt-caching).
:::

The non-streaming `usage` object reports token counts only — no dollar amount. Reconcile spend in three places: the console billing log, `GET /api/usage/token` (account-level summary), and `GET /api/log/token` (per-request detail). Billing runs on a prepaid USD balance that never expires; when it empties, requests return 403 and the same key works again immediately after a top-up. See [Billing & top-ups](/docs/billing).

## Troubleshooting

- **401 on a fresh key** — the environment variable isn't set or the key was copied incompletely. The `Bearer` scheme is case-insensitive, and a bare key without the scheme also works.
- **403 with a valid key** — usually an empty balance, a model outside the key's whitelist, a quota cap, or an IP restriction. Top up or adjust the key.
- **Model not found** — send the exact `id` the catalogue returns. The `vendor/` prefix is for browsing only and is not part of a request.
- **A "coming soon" model errors** — catalogue entries marked *Coming soon* can't be called yet; pick one marked *Available*.
- **Rows stall on 429** — that's the backoff doing its job. Lower `CONCURRENCY` if it happens constantly, rather than raising `MAX_RETRIES`.
- **A row fails after `MAX_RETRIES`** — the error message ends with a request ID; grab it before the script moves on.

## Verify

```bash
printf 'id,text\n1,Hello world\n2,Bonjour le monde\n3,Hola mundo\n' > input.csv
python batch.py
cat output.csv
```

Kill the script mid-run with Ctrl-C and rerun it — completed rows are not re-translated.

## Related links

- [Quickstart](/docs/quickstart)
- [Models & pricing](/docs/models)
- [OpenAI compatibility](/docs/openai-compat)
- [Prompt caching](/docs/guides/prompt-caching)
- [Billing & top-ups](/docs/billing)
- [Errors](/docs/api-reference/errors)
- [First integration](/docs/tutorials/first-integration)