Deprecated. Use /api/projects/{project_id}/ instead.

Name of the evaluation.

Optional description of what this evaluation checks.

Whether the evaluation runs automatically on new $ai_generation events.

Additional detail for the current system-disabled status. This is only populated when the detail is safe to show in the evaluation UI.

'llm_judge' uses an LLM to score outputs against a prompt; 'hog' runs deterministic Hog code; 'sentiment' classifies user-message sentiment. * `llm_judge` - LLM as a judge * `hog` - Hog * `sentiment` - Sentiment analysis

Configuration dict. For 'llm_judge': {prompt}; for 'hog': {source}; for 'sentiment': {source: 'user_messages'}.

Output format. Use 'boolean' for pass/fail evaluations and 'sentiment' for sentiment analysis. * `boolean` - Boolean (Pass/Fail) * `sentiment` - Sentiment

Output config. For 'boolean' output_type: {allows_na} to permit N/A results.

Trigger conditions that filter which events are evaluated. OR between condition sets, AND within each. Each set is {id, rollout_percentage, properties[]} — `rollout_percentage` (0-100, defaults to 100) is the sampling field the dispatcher reads.

What the evaluation runs on. 'generation' evaluates each matching $ai_generation event individually. 'trace' evaluates the whole trace once: the first matching generation schedules a run that waits for the trace to settle, then evaluates all of its events together. Condition filters still match individual generations — a trace is evaluated when any of its generations matches, and sampling applies per trace. * `generation` - Generation * `trace` - Trace

Target-specific config. For 'trace' target: {window_seconds}. Empty for 'generation'.

Set to true to soft-delete the evaluation.