A practical lab · about 18 minutes
Interactive learning lab
How AI works · guide 8 of 8
The LLM feature evaluation control room
0/5 core signals
Saved only on this device
Evaluation lab · BriefingScene 1 / 6
Fluent is not a metric
Measure the behaviours users need and the failures they cannot tolerate.
Fluent is not a metric ready.
Predict
Inspect
Verify
A fixed eval set makes changes comparable. Programmatic checks are strongest where rules are exact; reference answers support bounded tasks; LLM judges scale subjective rubrics; humans remain necessary for nuanced or high-stakes quality.
Cases before model comparison.
Critical failures can outweigh average score.
Production traffic reveals drift outside the fixed set.