oterm/docs/rag_example.md

RAG with haiku.rag

haiku.rag, also by oterm's author, is a RAG library. It stores documents in LanceDB, searches them with hybrid (vector and full-text) search, and serves that search over MCP. Its default configuration runs the embedding model through Ollama.

Build a database

The MCP server opens the database read-only, so documents go in through the haiku-rag command line:

uvx haiku-rag init --db /path/to/rag.lancedb
uvx haiku-rag add-src /path/to/report.pdf --db /path/to/rag.lancedb
uvx haiku-rag add-src https://example.com/article.html --db /path/to/rag.lancedb
uvx haiku-rag add-src /path/to/notes/ --db /path/to/rag.lancedb

A directory is added recursively. See the haiku.rag command line reference for everything else, including listing and deleting documents.

Connect oterm

Add the server to config.json (run oterm --data-dir to find it):

{
  "mcpServers": {
    "haiku-rag": {
      "command": "uvx",
      "args": ["haiku-rag", "mcp", "--stdio", "--db", "/path/to/rag.lancedb"]
    }
  }
}

Restart oterm, then select the haiku-rag tools when creating or editing a chat.

Tools

Tool	What it does
search_documents	Hybrid search over the database.
search_documents_by_image	Search by image. Only offered with a multimodal embedder.
list_documents	Titles, URIs and metadata of the stored documents.
get_document	A whole document, in reading order.
get_document_outline	A document's heading tree, with page numbers.
get_document_section	The text of one section of a document.
execute_code	Runs a Python program in a sandbox over the selected documents.

The model sees them as haiku-rag_search_documents and so on. The haiku.rag MCP reference documents their arguments.

Then ask questions such as:

What does the report say about revenue in the third quarter?

Which of my notes mention the migration, and what did we decide?

Configuration

haiku.rag reads haiku.rag.yaml from the path in HAIKU_RAG_CONFIG_PATH, then the current directory, then its own data directory. The MCP server runs in the directory oterm was started from, so either set cwd on the server or point it at the file:

{
  "mcpServers": {
    "haiku-rag": {
      "command": "uvx",
      "args": ["haiku-rag", "mcp", "--stdio", "--db", "/path/to/rag.lancedb"],
      "env": {
        "HAIKU_RAG_CONFIG_PATH": "/path/to/haiku.rag.yaml"
      }
    }
  }
}

The embedding model in that configuration must match the one the database was built with. See the haiku.rag configuration guide for providers, models and reranking.