Step-by-step answer
Core 75–90 min8 stepsBuild a miniature knowledge-update benchmark
A LongMemEval-style benchmark in miniature. Six synthetic 12-week client histories with planted changes, 60 questions in five categories, three memory designs (full history, keyword retrieval, extract-then-store) answered with claude -p, exact scoring with a judge only for what string checks can't decide, and per-category accuracy with Wilson intervals and token costs. With 12 questions a category the intervals are wide, and that is the lesson.
Back to the exercise