# LlamaIndex

> Connect LlamaIndex to the MirAPI gateway through its OpenAI-compatible endpoint via api_base and run indexing, querying, embeddings, and agents on your MirAPI key.

LlamaIndex's `OpenAI` LLM class accepts an `api_base` override. Point it at MirAPI and the whole indexing-and-querying stack — LLM, embeddings, and agents — runs on your key.

## What you need

| Element | Value |
|---|---|
| Base URL | `https://api.mirapi.ai/v1` (`api_base`) |
| API key | `MIRAPI_API_KEY` |
| Model | `deepseek-chat` |

`api_base` must end in `/v1`: LlamaIndex appends the path (`/chat/completions`, `/embeddings`) itself.

## Install

```bash
pip install llama-index llama-index-llms-openai
```

## Configure

```python
import os
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI

Settings.llm = OpenAI(
    model="deepseek-chat",
    api_key=os.environ["MIRAPI_API_KEY"],
    api_base="https://api.mirapi.ai/v1",
)
```

LlamaIndex passes `api_base` through to the OpenAI SDK's `base_url`, so `api_base` is the parameter to set. The `api_key` here is any MirAPI key from the console — MirAPI accepts the standard OpenAI `Authorization: Bearer` scheme, and the `OpenAI` class sends it automatically.

## Verify with a message

```python
response = Settings.llm.complete("Reply with exactly: connected to MirAPI")
print(response.text)
```

The output should contain `connected to MirAPI`. If it does not, start with the Error handling section below.

## Embeddings for RAG

Embeddings use a separate model class and a dedicated endpoint; a chat model will not work here.

```python
from llama_index.embeddings.openai import OpenAIEmbedding

Settings.embed_model = OpenAIEmbedding(
    model="<embedding-model-id>",
    api_key=os.environ["MIRAPI_API_KEY"],
    api_base="https://api.mirapi.ai/v1",
)
```

Pick an embedding model id from the [catalogue](/docs/models) and set `Settings.embed_model` once — every `VectorStoreIndex` you build will then embed through `/v1/embeddings`. Prompt caching applies to the chat model, not to embeddings: when the LLM reuses a large system prompt or index context, cache reads cost less than ordinary input and hits are visible in `usage.prompt_tokens_details.cached_tokens`.

## Tool calling and agents

MirAPI supports the OpenAI `tools` / `tool_choice` format, so LlamaIndex `FunctionTool` and agents that emit tool calls work without extra configuration; the model must advertise tool calling in the catalogue, and `deepseek-chat` qualifies. See the [tool-calling agent tutorial](/docs/tutorials/tool-calling-agent) for a full example.

## Streaming, reasoning, and structured outputs

- Streaming works through the standard `stream_complete` / `stream_chat` methods — MirAPI streams SSE `data:` lines and ends with `data: [DONE]`; reasoning tokens arrive before the final answer.
- For reasoning models, set `reasoning_effort` to control reasoning strength; reasoning tokens are billed as output.
- For typed extraction, LlamaIndex's `structured_predict` / `as_structured_llm` map to the OpenAI-compatible `response_format` (JSON mode or `json_schema`); the model must advertise JSON output in the catalogue. See [Structured outputs](/docs/guides/structured-outputs).

## Errors and troubleshooting

| Code | Meaning | What to do |
|---|---|---|
| 401 | Key missing, invalid, or unknown | Check `MIRAPI_API_KEY` and that it comes from the console. |
| 403 | Key valid but rejected — empty balance, model outside the whitelist, quota, or IP allowlist | Fix the cause (top up, add the model, raise the quota) and retry. |
| 413 | Payload too large | Reduce the input size. |
| 429 | Rate limited | Retry with jittered exponential backoff (1s → 2s → 4s, cap ~30s). There is no `Retry-After` header and the gateway does not retry for you. |
| 500 | Upstream failure | Retry idempotent requests safely. |

Every error message ends with a request ID; include it in any support ticket.

## Billing

MirAPI bills text and multimodal chat per token — per 1M USD, with separate input / output / cache-read prices — on a prepaid balance that never expires and has no subscription. Reasoning tokens count as output. When the balance runs out the API returns 403, and the same key recovers immediately after you top up. Reconcile spend in the console billing log, `GET /api/usage/token` (summary), or `GET /api/log/token` (per-request detail).

## Tool-specific pitfalls

- **`api_base` must include `/v1`** — LlamaIndex appends `/chat/completions` and `/embeddings`.
- **Embeddings are a separate class** — passing a chat model id to `OpenAIEmbedding` (or vice versa) fails.
- **A valid key that returns 403** usually means an empty balance or a model outside the key's whitelist; top up or fix the key and retry.
- **429 has no `Retry-After`** — implement your own backoff; the gateway will not retry on your behalf.

## Related links

- [Embeddings and rerank](/docs/guides/embeddings-rerank)
- [Prompt caching](/docs/guides/prompt-caching)
- [Tool-calling agent tutorial](/docs/tutorials/tool-calling-agent)
- [RAG tutorial](/docs/tutorials/rag)
- [Models](/docs/models)
- [Billing](/docs/billing)