Module 2 · Retrieval that reasons · Lesson 8

Long context versus RAG

When to put a whole case file in the prompt and when to retrieve, using current prices, prompt caching and the research evidence on accuracy, and how to route between the two.

65 min 1 video 2 exercises 6-question quiz Answers

Setup for the exercises:Install Claude Code Python with uv Get an OpenRouter key Local models with Ollama

By the end of this lesson you can

Compute the cost of answering a session of questions with cached long context versus RAG, and find the break-even point
Summarise what the research says about long-context versus RAG accuracy, and its limits
Choose between whole-case context, document-level retrieval, chunk RAG and agentic search for a given workload
Design a router that sends each question to the cheapest approach likely to answer it, with a fallback

A residential purchase file (the official copies, a lease, the local, drainage and environmental searches, the TA6 and TA7 forms, replies to enquiries and a management pack) comes to roughly 128,000 tokens. Claude Sonnet 5.5, Opus 5.5 and Haiku 5.5 all accept a million tokens, and for most current Claude models a 900,000-token request costs the same per token as a 9,000-token one (pricing). So why build any of the previous seven lessons? Why not put the whole matter in the prompt?

Sometimes you should. This lesson works out when, using current prices, prompt caching and the research evidence, and ends with a router that makes the choice per question.

Long context versus RAG

When to put a whole case file in the prompt and when to retrieve, using current prices, prompt caching and the research evidence on accuracy, and how to route between the two.

Long context versus RAG

Long context versus RAG

Sign in to keep reading.

Sign in to keep reading.