LLM eval review types, off-the-shelf harnesses, and hallucination fixes
Mental model in one line: grade with code wherever you can, use an LLM judge where you must, and use humans to calibrate the judge. A harness is just dataset → task → scorers → report, run on every change.
Why this is in the vault
The founder is studying evals for the Claude cert track and for client work. He asked for one place to look these categories up later. These are the reference lists, with the distinctions most likely to be tested.
1. Three types of eval review (Anthropic course)
| Type | Best for | Cost and speed | Notes |
|---|---|---|---|
| Code-graded | Anything with a checkable right answer | Fastest, cheapest | Exact match, regex, string contains, "does it parse as valid JSON / match the schema", numeric tolerance. Push as much as possible into this tier. |
| LLM-as-judge | Subjective or fuzzy quality (tone, helpfulness, faithfulness) | Middle | Only as good as the rubric: specific criteria, a pass/fail or small fixed scale, reasoning before the score, a judge prompt separate from the one that produced the output. |
| Human review | Ground truth, high-stakes calls | Slowest, most expensive | Its best use is calibrating the LLM judge: label a sample, check that the judge agrees, then trust the judge at volume. |
2. Five ways to make unsupported answers (hallucinations) rarer
This is the five-item list from the founder's eval course. It draws on the Anthropic docs page linked above, which splits strategies into three "basic" ones (items 1-3) and four "advanced" ones (items 5 and 6 below, plus chain-of-thought verification and best-of-N verification). Item 4 is not on that page.
- Let it say "I don't know." Tell the model it may admit uncertainty. The docs' example is "I don't have enough information to confidently assess this." For structured output, the course example returns
null. - Quote before answering. Extract the supporting quotes first, then answer only from them. The docs scope this to long documents (>20k tokens). It works before the answer: it prevents bad claims.
- Cite back to the document. Answer, then find a supporting quote for each claim and retract any claim without one. It works after the answer: it audits claims and leaves a trail a human can check. (Separately, the API has a built-in Citations feature for document inputs that returns the cited passages. That comes from the API docs, not this page.)
- Add a verification pass. A second request checks the first answer and returns a verdict. The closest item on the docs page is "iterative refinement". It becomes the evaluator-optimizer pattern (a term from Anthropic's agent-pattern guidance, not this page) only when the verdict feeds a rewrite, in a loop.
- Restrict outside knowledge. Answer from the document alone, not from training data. The docs list this under advanced, as "external knowledge restriction".
The docs also list two more advanced techniques:
- Chain-of-thought verification: have the model reason step by step before answering, and review the reasoning when an answer looks wrong.
- Best-of-N verification: run the same prompt several times and compare. Disagreement signals the model is guessing.
Memory hook: IDK and restrict-knowledge set the rules. Quote-first grounds the answer. Citations audit it. Verification double-checks it.
3. Off-the-shelf eval harnesses
Every harness has the same shape:
- Dataset: test cases (inputs, plus expected outputs or reference answers where they exist).
- Task: the prompt + model (+ tools) under test.
- Scorers: code checks, LLM-judge rubrics, and hooks for human labels.
- Runner and report: run all cases, aggregate pass rates, compare against the last version, and ideally gate CI or a deploy.
Common options. These are from Ray's general knowledge, not checked against current vendor docs:
- Anthropic Console evaluation tool: test cases with prompt variables, side-by-side runs, human grading. The lowest-friction start for prompt work.
- promptfoo: open source; a YAML config of prompts × providers × test cases, with assertion types from
equals/contains/is-json/ JavaScript up tollm-rubric. A good default for teams. - Inspect AI (UK AI Security Institute): open-source Python; Task = dataset + solver + scorer. Strong for agent and tool-use evals.
- DeepEval: pytest-style LLM test assertions.
- Ragas: RAG-specific metrics (faithfulness, context precision and recall).
- Tracing and experiment platforms: traces, datasets, experiments and dashboards, for when you need production monitoring and not just pre-release testing. Braintrust and LangSmith are paid hosted products. Arize Phoenix is open source and can be self-hosted.
- OpenAI Evals: the early open-source registry pattern.
- Claude Code
claude plugin eval: an eval runner for plugin and skill suites.
Mapping against Ray Data Co
- Our own stack already uses all three review types. Code-graded: the leak grep and test suites. LLM-as-judge: the
verify-*fresh-eyes critics. Human review: the founder's approvals and corrections. The open gap is the calibration loop. [[2026-10-20-critic-layer-review-time-defect-escape]] found that the one critic with ground truth (design-critic) had 8 of 9 of its ACCEPTs overturned by the founder. That's 4 of 5 counted by distinct founder judgments, a small sample. This is the "humans calibrate the judge" step that we haven't run systematically yet. - For client work (phData, agents in production): the harness shape (dataset → task → scorers → report) is the plain-English explainer for clients. Start with promptfoo or the Console tool, push checks into code-graded first, and add an LLM judge only with a rubric calibrated against client-labeled examples. [[2026-07-25-evals-as-competitive-moat]] makes the case that the eval is the product.
- Hallucination fixes apply directly to any document-grounded client agent (contracts, policies, support knowledge bases). Default to options 1, 2 and 5 at build time, and add option 3 where a human must be able to audit the answer.
Open questions
- Should we run a promptfoo-style suite on our own skills, so critic verdicts get calibrated against founder labels instead of trusted as-is?
- Which harness would phData standardize on for Claude delivery, and is there an Anthropic-preferred one in the partner materials?
Related
- [[2026-07-25-evals-as-competitive-moat]]: why the eval is the product
- [[2026-10-20-critic-layer-review-time-defect-escape]]: our critic layer measured against ground truth
- [[2026-07-30-multi-llm-critic-ensembles-differentiated-rubrics]]: judge independence and rubric design
- [[2026-07-03-verify-skills-pass-disagreement-audit]]: where our LLM judges emit false passes
- [[2026-10-20-ccdv-f-study-sheet]]: cert study companion