Coders In Hoods

Courses
Answers
Setup
Bookmarks

Coders In Hoods Academy

Every course is free. Sign in to start, and your progress, quiz scores and bookmarks follow you.

Academy

Courses
Paths
Exercise answers
Setup guides
Bookmarks

Coders In Hoods

The agency
Coaching for teams

Social

LinkedIn
X (Twitter)

Contact

codersinhoods@gmail.com

© Coders In Hoods · since 2019

Free · no card

Coders In Hoods Academy

Sign in to start learning

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.

Course overview

Evaluating and Scoring Agents

Free · sign in to start and track progress

1 · Foundations of evaluation

1Why evals, and their vocabulary
2Error analysis from traces
3Code-based graders

2 · Scoring that you can trust

4LLM-as-judge
5Calibrating judges
6Rubrics per task
Exercise answers
- Answer: Build a per-task rubric grader and check it criterion by criterion
- Answer: Draft rubrics with a model and compare them with yours
7Statistics, trials and CI gates

CapstoneA full eval suite for your drafting agent Reference implementation

Answers Lesson 6: Rubrics per task

Step-by-step answer

Stretch 60 min6 steps

Draft rubrics with a model and compare them with yours

Back to the exercise

Previous answerBuild a per-task rubric grader and check it criterion by criterion Next answer Measure flakiness and compare two prompt versions

Free · sign in to start

Sign in to read the answer.

Free, no card. We use your account to save progress, quiz scores, bookmarks and occasionally send updates from our team.

6 more sections
Progress, quiz scores and bookmarks