# RAG

> Build a RAG pipeline on MirAPI: chunk and embed a knowledge base with /v1/embeddings, retrieve with cosine similarity, rerank with /v1/rerank, and answer with a chat model.

Retrieval-augmented generation (RAG) grounds a chat model's answers in your own documents by retrieving the relevant passages before it responds, cutting down hallucination without fine-tuning. This tutorial builds one complete pipeline on three MirAPI endpoints: `/v1/embeddings`, `/v1/rerank`, and `/v1/chat/completions`.

**Deliverable:** one Python script that reads a `kb.md` knowledge-base file and answers questions about it, citing the source lines.

**Prerequisites:** `pip install requests numpy`, a `MIRAPI_API_KEY`, and embedding / rerank / chat models enabled for your key. Pick the embedding and rerank model names from `GET /v1/models` — they are separate model families, not chat models.

## How it works

A RAG pipeline has four steps:

1. **Index** — split the document into chunks and embed each chunk.
2. **Retrieve** — embed the question and find the most similar chunks by cosine similarity.
3. **Rerank** — re-score the candidates against the question for precision.
4. **Answer** — hand the best chunks to a chat model as context and let it answer with citations.

## The knowledge base

Create `kb.md` — replace this sample with your own product docs, FAQ, or manual:

```text
# MirAPI knowledge base

MirAPI is a unified API gateway: one key gives access to many models through
OpenAI-, Anthropic- and Gemini-compatible endpoints.

Balance is prepaid in USD and does not expire; you top up from the console.

Streaming is standard SSE: pass stream=true to the chat completions endpoint.

Per-request usage is visible in the console and via GET /api/log/token.

Image generation is billed per image; video generation is billed per second
of output.
```

## The complete script

```python
import os
import sys
import requests
import numpy as np

BASE = "https://api.mirapi.ai/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['MIRAPI_API_KEY']}"}

# Choose these from GET /v1/models — embeddings and rerank are separate
# model families, not chat models.
EMBED_MODEL = "<embedding-model-from-catalogue>"
RERANK_MODEL = "<rerank-model-from-catalogue>"
CHAT_MODEL = "deepseek-chat"


def chunk_document(path: str) -> list[str]:
    """Split a Markdown file into paragraphs, dropping empty lines."""
    text = open(path, encoding="utf-8").read()
    return [p.strip() for p in text.split("\n\n") if p.strip()]


def embed(texts: list[str]) -> list[list[float]]:
    """Batch-embed a list of texts in one request."""
    resp = requests.post(
        f"{BASE}/embeddings",
        headers=HEADERS,
        json={"model": EMBED_MODEL, "input": texts},
        timeout=60,
    )
    resp.raise_for_status()
    items = sorted(resp.json()["data"], key=lambda d: d["index"])
    return [d["embedding"] for d in items]


def cosine(a: np.ndarray, b: np.ndarray) -> float:
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-12))


def retrieve(query_vec: np.ndarray, chunk_vecs: list[np.ndarray], k: int = 5):
    scores = [(i, cosine(query_vec, c)) for i, c in enumerate(chunk_vecs)]
    scores.sort(key=lambda x: x[1], reverse=True)
    return [i for i, _ in scores[:k]]


def rerank(query: str, documents: list[str], top_n: int = 3):
    resp = requests.post(
        f"{BASE}/rerank",
        headers=HEADERS,
        json={
            "model": RERANK_MODEL,
            "query": query,
            "documents": documents,
            "top_n": top_n,
            "return_documents": True,
        },
        timeout=60,
    )
    resp.raise_for_status()
    return resp.json()["results"]


def answer(query: str, results: list[dict]) -> str:
    context = "\n\n".join(
        f"[{i + 1}] {r['document']['text']}" for i, r in enumerate(results)
    )
    resp = requests.post(
        f"{BASE}/chat/completions",
        headers=HEADERS,
        json={
            "model": CHAT_MODEL,
            "messages": [
                {
                    "role": "system",
                    "content": (
                        "Answer the question using only the provided context. "
                        "Cite sources as [n]. If the context is not enough, say so."
                    ),
                },
                {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {query}"},
            ],
        },
        timeout=60,
    )
    resp.raise_for_status()
    return resp.json()["choices"][0]["message"]["content"]


def main(query: str) -> None:
    chunks = chunk_document("kb.md")
    chunk_vecs = [np.array(v) for v in embed(chunks)]

    query_vec = np.array(embed([query])[0])
    top = retrieve(query_vec, chunk_vecs, k=5)
    candidates = [chunks[i] for i in top]

    results = rerank(query, candidates, top_n=3)
    print(answer(query, results))


if __name__ == "__main__":
    main(sys.argv[1] if len(sys.argv) > 1 else "How is streaming done?")
```

Retrieval and rerank only have OpenAI-style endpoints, so the script uses one base URL, `https://api.mirapi.ai/v1`. For the final answer you can swap `/v1/chat/completions` for the Anthropic `/v1/messages` endpoint (base URL `https://api.mirapi.ai`, auth via `x-api-key`) or the Gemini `generateContent` endpoint (`https://api.mirapi.ai/v1beta`, auth via `x-goog-api-key`). This tutorial stays with the OpenAI-compatible chat endpoint so every call shares one base URL.

Run it:

```bash
export MIRAPI_API_KEY=sk-...
python rag_bot.py "How is streaming done?"
```

The answer cites the SSE paragraph from `kb.md` as `[1]`.

## Parameters

### Embeddings

`POST /v1/embeddings` turns text into vectors. Send a whole array in `input` to index every chunk in a single request.

| Parameter | Type | Description |
| --- | --- | --- |
| `model` | string | Embedding model id from `GET /v1/models`. |
| `input` | string or string[] | Text to embed; pass an array to batch. |

The response is `{"data": [{"index", "embedding"}]}`; reorder items by `index` to match your input order. A Gemini-style alias, `POST /v1/engines/{model}/embeddings`, accepts the same task.

### Rerank

`POST /v1/rerank` re-scores candidates against the query. Use it when you retrieve many candidates (say 20) and want the best few.

| Parameter | Type | Description |
| --- | --- | --- |
| `model` | string | Rerank model id from `GET /v1/models`. |
| `query` | string | The search query. |
| `documents` | string[] | Candidate texts to re-score. |
| `top_n` | integer | How many top results to return. |
| `return_documents` | boolean | Include the matched `document.text` in results. |

Each result carries a `relevance_score` and, with `return_documents`, the matched text.

### Chat completions

`POST /v1/chat/completions` produces the final grounded answer. Put the retrieved context in the user message and ask the model to cite sources as `[n]`.

| Parameter | Type | Description |
| --- | --- | --- |
| `model` | string | Chat model id, e.g. `deepseek-chat`. |
| `messages` | array | Conversation; embed the context in the user message. |

## Error handling

The three endpoints share one error model. Check the HTTP status first:

- **401** — the key is missing, invalid, or unknown; check `MIRAPI_API_KEY`.
- **403** — the key is valid but the request is refused: balance exhausted, model outside the key's whitelist, quota reached, or IP not allowed. Top up to restore the same key immediately.
- **413** — the payload is too large; split the batch or shrink the chunks.
- **429** — rate limited; retry with jittered exponential backoff (1s → 2s → 4s, capped around 30s). There is no `Retry-After` header.
- **500** — an upstream failure; the message is sanitized. Retry idempotent requests.

Each error message ends with a request ID — include it in any support ticket. OpenAI-style errors use the envelope `{"error":{"message","type","param","code"}}`; `type` is `new_api_error` (gateway) or `upstream_error` (upstream).

## Troubleshooting

- **Poor retrieval** — index and query with the same embedding model. Mixing models produces incompatible vector spaces.
- **Irrelevant answers** — keep chunks small (a paragraph or two) and attach metadata (source file, heading) so citations can link back.
- **Answers that miss context** — the retrieved context must fit the chat model's context window; rerank to the top 3 and drop low-scoring chunks.
- **Sudden 403 after a long run** — the balance ran out; top up in the console and rerun without changing the key.
- **Batch timeouts** — embed very large document sets in smaller batches instead of one giant request.

:::tip
For production, store the vectors in a vector database (pgvector, Qdrant, Weaviate, Pinecone, …) and search there instead of scanning in memory. The in-memory approach above is for prototyping.
:::

## Approximate cost

Embeddings and the final chat answer are billed per token; rerank pricing depends on the rerank model you choose. The exact per-request figures are in the console billing log and `GET /api/log/token`, and the account-level total is available from `GET /api/usage/token`. A small knowledge base like the sample typically costs a fraction of a cent per run.

## Related links

- [Embeddings & rerank](/docs/guides/embeddings-rerank) — full walkthrough of the two retrieval endpoints.
- [Models & pricing](/docs/models) — pick embedding, rerank, and chat model names from the catalogue.
- [OpenAI compatibility](/docs/openai-compat) — base URL, auth, and endpoint details for the OpenAI protocol.
- [Billing & top-ups](/docs/billing) — how the three pricing units and top-ups work.
- [Quickstart](/docs/quickstart) — send your first request in five minutes.