Skip to content

symfonic.evals.response_assertions

response_assertions

Deterministic assertions over response text and JSON.

CitationsSupported dataclass 

CitationsSupported(attribute: str = 'retrieved_source_ids', minimum: int = 1, name: str = 'citations_supported')

Require response citations to name only evidence retrieved for this turn.

ResponseContains dataclass 

ResponseContains(*fragments: str, case_sensitive: bool = False)

Require all fragments in the answer, optionally case-insensitively.

Source code in src/symfonic/evals/response_assertions.py 
75
76
77
78
79
80
def __init__(self, *fragments: str, case_sensitive: bool = False) -> None:
    if not fragments or any(not fragment for fragment in fragments):
        raise ValueError("ResponseContains requires non-empty fragments")
    object.__setattr__(self, "fragments", tuple(fragments))
    object.__setattr__(self, "case_sensitive", case_sensitive)
    object.__setattr__(self, "name", "response_contains")

ResponseEquals dataclass 

ResponseEquals(expected: str, name: str = 'response_equals')

Require an exact answer without copying it into a failed report.

ResponseJson dataclass 

ResponseJson(expected: Any = None, exact: bool = True, name: str = 'response_json')

Require valid JSON and optionally an exact object shape.

StructuredOutputMatches dataclass 

StructuredOutputMatches(output_type: type, expected: Mapping[str, Any] | None = None, name: str = 'structured_output_matches')

Require the target's validated structured value and optional exact fields.