symfonic.evals.assertions

assertions

Deterministic assertions over answers and kernel evidence.

AttributeCount dataclass

AttributeCount(attribute: str, minimum: int = 0, maximum: int | None = None, name: str = 'attribute_count')

Require an integer or collection-valued evidence attribute to be bounded.

AttributeEquals dataclass

AttributeEquals(attribute: str, expected: Any, name: str = 'attribute_equals')

Require target-supplied evidence without rendering its value.

CapabilitiesPresent dataclass

CapabilitiesPresent(*required: str)

Require capabilities derived from the target's compiled agent.

Source code in src/symfonic/evals/assertions.py

def __init__(self, *required: str) -> None:
    if not required or any(not item for item in required):
        raise ValueError("CapabilitiesPresent requires capability names")
    object.__setattr__(self, "required", tuple(required))
    object.__setattr__(self, "name", "capabilities_present")

EventsCorrelated dataclass

EventsCorrelated(name: str = 'events_correlated')

Require every observed event to belong to the response run.

ScopedRecords dataclass

ScopedRecords(attribute: str = 'memory_records', scope_path: str = '', minimum: int = 1, name: str = 'scoped_records')

Require store evidence to be visible only from an expected scope.

Targets place record-shaped values in an observation attribute. This check reads identity metadata only; it never examines or reports record content.

StageObserved dataclass

StageObserved(stage_id: str, *, outcome: str | None = None, minimum_counts: dict[str, int] | None = None, exact_counts: dict[str, int] | None = None)

Require a stage outcome and optional minimum integer counters.

Source code in src/symfonic/evals/assertions.py

def __init__(
    self,
    stage_id: str,
    *,
    outcome: str | None = None,
    minimum_counts: dict[str, int] | None = None,
    exact_counts: dict[str, int] | None = None,
) -> None:
    if not stage_id:
        raise ValueError("StageObserved requires a stage_id")
    counts = tuple(sorted((minimum_counts or {}).items()))
    exact = tuple(sorted((exact_counts or {}).items()))
    if any(
        not isinstance(value, int) or value < 0
        for _, value in (*counts, *exact)
    ):
        raise ValueError("stage counts must be non-negative integers")
    object.__setattr__(self, "stage_id", stage_id)
    object.__setattr__(self, "outcome", outcome)
    object.__setattr__(self, "minimum_counts", counts)
    object.__setattr__(self, "exact_counts", exact)
    object.__setattr__(self, "name", "stage_observed")