Tune an ANN index against a recall target

HNSW and IVF in Faiss over about 20,000 embedded synthetic conveyancing chunks, swept against exact search with single-threaded, one-query-at-a-time timing, plotted as recall@10 against milliseconds. A 2% matter filter then compares post-filtering, a larger k, and exact search over the matter, which wins on both recall and latency.

Back to the exercise

Before you start

Before you start:Python with uv Get an OpenRouter key Local models with Ollama

This exercise calls a model. Run it either way

Any model through OpenRouterOne key for Claude, GPT, Gemini, Llama and more. Pay per token, with a credit limit.A local open-source modelFree and private with Ollama. Weaker on hard tasks.

Which should I pick? →

Everything is measured against IndexFlatIP on unit vectors, which is exact cosine search, so "recall" here is ANN recall: did the index find the true nearest neighbours, not did it find the right clause. Embeddings are computed once through the OpenAI SDK and cached; the sweeps and the filter experiment then run in seconds and can be repeated for free. The decision that matters most is timing honestly: one thread (faiss.omp_set_num_threads(1)) and one query at a time, because that is how an agent issues searches, and batched multi-threaded timings flatter every index.

Answers Lesson 3: Vector indexes and databases

Step-by-step answer

Core 60–90 min6 steps