Skip to content

symfonic.evals.book_retrieval_assertions

book_retrieval_assertions

Retrieval-route and fresh-scope assertions for book evaluations.

FixtureScopeFresh dataclass 

FixtureScopeFresh(scope_attribute: str = 'physical_scope_id', count_attribute: str = 'fixture_record_count_before', name: str = 'fixture_scope_fresh')

Require a physical scope identity and zero fixture rows before ingestion.

SourceRouteAdmitted dataclass 

SourceRouteAdmitted(route: str, *source_ids: str, attribute: str = 'admitted_sources_by_route')

Require named sources to have reached the prompt through one route.

Source code in src/symfonic/evals/book_retrieval_assertions.py 
30
31
32
33
34
35
36
37
38
39
40
41
def __init__(
    self,
    route: str,
    *source_ids: str,
    attribute: str = "admitted_sources_by_route",
) -> None:
    if not route or not source_ids or any(not source_id for source_id in source_ids):
        raise ValueError("SourceRouteAdmitted requires a route and source ids")
    object.__setattr__(self, "route", route)
    object.__setattr__(self, "source_ids", tuple(source_ids))
    object.__setattr__(self, "attribute", attribute)
    object.__setattr__(self, "name", "source_route_admitted")