symfonic.evals.rubric

rubric

Optional semantic judging that can never become a scenario's sole authority.

RubricJudge dataclass

RubricJudge(judge: RubricJudgePort | Callable[[RubricRequest], Awaitable[RubricVerdict]], rubric_id: str, rubric_version: str, criteria: tuple[str, ...], minimum_score: float = 1.0, name: str = 'rubric_judge', nondeterministic: bool = True)

Opt-in nondeterministic assertion, subordinate to deterministic evidence.

RubricRequest dataclass

RubricRequest(rubric_id: str, rubric_version: str, criteria: tuple[str, ...], response: str)

Pinned rubric plus the answer an adopter explicitly sends to a judge.

RubricVerdict dataclass

RubricVerdict(score: float, reason_code: str)

Provider-neutral structured judge response.