RAG

Retrieval-augmented generation (RAG) grounds a chat model’s answers in your own documents by retrieving the relevant passages before it responds, which reduces hallucination without fine-tuning. This tutorial builds one complete pipeline on three MirAPI endpoints: /v1/embeddings, /v1/rerank, and /v1/chat/completions.

Deliverable: one Python script that reads a kb.md knowledge-base file and answers questions about it, citing the source lines.

Prerequisites: pip install requests numpy, a MIRAPI_API_KEY, and embedding, rerank, and chat models enabled for your key. Pick the embedding and rerank model names from GET /v1/models. They are separate model families, not chat models.

How it works

A RAG pipeline has four steps:

1. Index: split the document into chunks and embed each chunk.
2. Retrieve: embed the question and find the most similar chunks by cosine similarity.
3. Rerank: re-score the candidates against the question for precision.
4. Answer: hand the best chunks to a chat model as context and let it answer with citations.

The knowledge base

Create kb.md from this sample, or replace it with your own product docs, FAQ, or manual:

# MirAPI knowledge base

MirAPI is a unified API gateway: one key gives access to many models through
OpenAI-, Anthropic- and Gemini-compatible endpoints.

Balance is prepaid in USD and does not expire; you top up from the Platform.

Streaming is standard SSE: pass stream=true to the chat completions endpoint.

Per-request usage is visible on the Platform and via GET /api/log/token.

Image generation is billed per image; video generation is billed per second
of output.

The complete script

import os
import sys
import requests
import numpy as np

BASE = "https://api.mirapi.ai/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['MIRAPI_API_KEY']}"}

# Choose these from GET /v1/models — embeddings and rerank are separate
# model families, not chat models.
EMBED_MODEL = "<embedding-model-from-catalogue>"
RERANK_MODEL = "<rerank-model-from-catalogue>"
CHAT_MODEL = "deepseek-chat"


def chunk_document(path: str) -> list[str]:
    """Split a Markdown file into paragraphs, dropping empty lines."""
    text = open(path, encoding="utf-8").read()
    return [p.strip() for p in text.split("\n\n") if p.strip()]


def embed(texts: list[str]) -> list[list[float]]:
    """Batch-embed a list of texts in one request."""
    resp = requests.post(
        f"{BASE}/embeddings",
        headers=HEADERS,
        json={"model": EMBED_MODEL, "input": texts},
        timeout=60,
    )
    resp.raise_for_status()
    items = sorted(resp.json()["data"], key=lambda d: d["index"])
    return [d["embedding"] for d in items]


def cosine(a: np.ndarray, b: np.ndarray) -> float:
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-12))


def retrieve(query_vec: np.ndarray, chunk_vecs: list[np.ndarray], k: int = 5):
    scores = [(i, cosine(query_vec, c)) for i, c in enumerate(chunk_vecs)]
    scores.sort(key=lambda x: x[1], reverse=True)
    return [i for i, _ in scores[:k]]


def rerank(query: str, documents: list[str], top_n: int = 3):
    resp = requests.post(
        f"{BASE}/rerank",
        headers=HEADERS,
        json={
            "model": RERANK_MODEL,
            "query": query,
            "documents": documents,
            "top_n": top_n,
            "return_documents": True,
        },
        timeout=60,
    )
    resp.raise_for_status()
    return resp.json()["results"]


def answer(query: str, results: list[dict]) -> str:
    context = "\n\n".join(
        f"[{i + 1}] {r['document']['text']}" for i, r in enumerate(results)
    )
    resp = requests.post(
        f"{BASE}/chat/completions",
        headers=HEADERS,
        json={
            "model": CHAT_MODEL,
            "messages": [
                {
                    "role": "system",
                    "content": (
                        "Answer the question using only the provided context. "
                        "Cite sources as [n]. If the context is not enough, say so."
                    ),
                },
                {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {query}"},
            ],
        },
        timeout=60,
    )
    resp.raise_for_status()
    return resp.json()["choices"][0]["message"]["content"]


def main(query: str) -> None:
    chunks = chunk_document("kb.md")
    chunk_vecs = [np.array(v) for v in embed(chunks)]

    query_vec = np.array(embed([query])[0])
    top = retrieve(query_vec, chunk_vecs, k=5)
    candidates = [chunks[i] for i in top]

    results = rerank(query, candidates, top_n=3)
    print(answer(query, results))


if __name__ == "__main__":
    main(sys.argv[1] if len(sys.argv) > 1 else "How is streaming done?")

Retrieval and rerank only have OpenAI-style endpoints, so the script also uses the OpenAI-compatible chat endpoint and sends every call to one base URL, https://api.mirapi.ai/v1. For the final answer you can instead swap /v1/chat/completions for the Anthropic /v1/messages endpoint (base URL https://api.mirapi.ai, auth via x-api-key) or the Gemini generateContent endpoint (https://api.mirapi.ai/v1beta, auth via x-goog-api-key).

Run it:

export MIRAPI_API_KEY=sk-...
python rag_bot.py "How is streaming done?"

The answer cites the SSE paragraph from kb.md as [1].

Parameters

Embeddings

POST /v1/embeddings turns text into vectors. Send a whole array in input to index every chunk in a single request.

Parameter	Type	Description
model	string	Embedding model ID from GET /v1/models.
input	string or string[]	Text to embed; pass an array to batch.

The response is {"data": [{"index", "embedding"}]}; reorder items by index to match your input order. A Gemini-style alias, POST /v1/engines/{model}/embeddings, does the same job.

Rerank

POST /v1/rerank re-scores candidates against the query. Use it when you retrieve many candidates (say 20) and want the best few.

Parameter	Type	Description
model	string	Rerank model ID from GET /v1/models.
query	string	The search query.
documents	string[]	Candidate texts to re-score.
top_n	integer	How many top results to return.
return_documents	boolean	Include the matched document.text in results.

Each result carries a relevance_score and, with return_documents, the matched text.

Chat completions

POST /v1/chat/completions produces the final grounded answer. Put the retrieved context in the user message and ask the model to cite sources as [n].

Parameter	Type	Description
model	string	Chat model ID, e.g. deepseek-chat.
messages	array	Conversation; embed the context in the user message.

Error handling

The three endpoints share one error model. Check the HTTP status first:

• 401: the key is missing, invalid, or unknown. Check MIRAPI_API_KEY.
• 403: the key is valid but the request is refused: balance exhausted, model outside the key’s whitelist, quota reached, or IP not allowed. If the balance ran out, top up and the same key works again immediately.
• 413: the payload is too large. Split the batch or shrink the chunks.
• 429: rate limited. Retry with jittered exponential backoff (1s → 2s → 4s, capped around 30s). There is no Retry-After header.
• 500: an upstream failure, with a sanitized message. Retry idempotent requests.

Each error message ends with a request ID. Include it in any support ticket. OpenAI-style errors use the envelope {"error":{"message","type","param","code"}}; type is new_api_error (gateway) or upstream_error (upstream).

Troubleshooting

• Poor retrieval: index and query with the same embedding model. Mixing models produces incompatible vector spaces.
• Irrelevant answers: keep chunks small (a paragraph or two) and attach metadata (source file, heading) so citations can link back to the source.
• Answers that miss context: the retrieved context must fit the chat model’s context window. Rerank to the top 3 and drop low-scoring chunks.
• Sudden 403 after a long run: the balance ran out. Top up in the Platform and rerun without changing the key.
• Batch timeouts: embed very large document sets in smaller batches instead of one giant request.

Tip

For production, store the vectors in a vector database (pgvector, Qdrant, Weaviate, Pinecone, …) and search there instead of scanning in memory. The in-memory approach above is for prototyping.

Approximate cost

Embeddings and the final chat answer are billed per token; rerank pricing depends on the rerank model you choose. The exact per-request figures are in the Platform billing log and GET /api/log/token, and GET /api/usage/token returns the account-level total. A small knowledge base like the sample typically costs a fraction of a cent per run.

Related links

• Embeddings & rerank: full walkthrough of the two retrieval endpoints.
• Models & pricing: pick embedding, rerank, and chat model names from the catalog.
• OpenAI compatibility: base URL, auth, and endpoint details for the OpenAI protocol.
• Billing & top-ups: how the three pricing units and top-ups work.
• Quickstart: send your first request in five minutes.