ScorerTask successScores the candidate against the route's task tag.Judge modelA stronger model grades the candidate. The judge's tokens are counted as cost.Exact matchFor structured routes, the candidate must match the schema and the expected fields.

I confirm the estimated replay cost. The test stops at the replay cap you set.

Often opened from model comparison with the route and candidate already filled in.

Activity Savings Models Chat

← Quality tests

New quality test

Route

Candidate models

GPT-4.1GPT-4.1 miniClaude Sonnet 4.5Claude Haiku 4.5Gemini 2.5 ProGemini 2.5 FlashGrok 3Mistral LargeDeepSeek V3Llama 3.3 70B

Sample %Share of the route's traffic the replay uses. Capped at 20%.Pass thresholdThe customer's quality bar, from 0.50 to 1.

I confirm the estimated replay cost. The test stops at the replay cap you set.

Often opened from model comparison with the route and candidate already filled in.