Skip to content

Documentation: 11.0 (current) · 9.x (previous)

symfonic-core

symfonic.evals.fixtures

Initializing search

symfonic-core on PyPI

Home
Start Simple
Capabilities
Runtime Services
Platform
Integrations
Low-Level Core
Migrating
Reference

symfonic-core

symfonic-core on PyPI

Home
Start Simple
Start Simple
- The Simple Agent
- Quickstart
- Scaffold a Project
- Agent Regression Evals
- Store Starter Walkthrough
- Scaffolded Projects Are Runtime Apps
- The Full Agent (SymfonicAgent)
- Examples
  Examples
Capabilities
Capabilities
- Tools
  Tools
- Prompt & Context
  Prompt & Context
- Memory (HMS)
  Memory (HMS)
- Knowledge & Documents
  Knowledge & Documents
  - Attachment Extraction
  - Authored Skills
- Delegation & Human Interaction
  Delegation & Human Interaction
  - Sub-agents (Delegation)
- Safety & Governance
  Safety & Governance
Runtime Services
Runtime Services
Platform
Platform
Integrations
Integrations
- Model Providers
  Model Providers
- Storage Backends
  Storage Backends
  - Mongo Atlas Vector Search
  - Mongo Vector Search
- Extensions & Observability
  Extensions & Observability
  - Plugins & Domains
  - Streaming & Callbacks
Low-Level Core
Low-Level Core
Migrating
Migrating
Reference
Reference
- Feature Catalog
- v8 Roadmap
- Changelog
- API Reference
  API Reference
  - symfonic
    
    symfonic
    
    agent
    
    agent
    
    attachments
    
    attachments
    
    errors
    
    extractors
    
    backend
    
    backend
    
    binding
    
    conversation
    
    messages
    
    model
    
    plan
    
    response
    
    structured_plan
    
    tool_adapters
    
    tool_ids
    
    tools
    
    transcript
    
    builder
    
    checkpointer
    
    checkpointer
    
    factory
    
    serde
    
    cli
    
    cli
    
    init_command
    
    main
    
    config
    
    configuration
    
    configuration
    
    base
    
    capabilities
    
    compat
    
    compat
    
    adapter
    
    fragments
    
    mapping
    
    models
    
    runtime_memory
    
    services
    
    state
    
    context
    
    integrations
    
    memory
    
    normalized
    
    normalizer
    
    prompt
    
    provenance
    
    runtime
    
    serialization
    
    services
    
    validation
    
    context
    
    context
    
    jit
    
    protocol
    
    stratified
    
    conversation
    
    cutover
    
    cutover
    
    admission_row
    
    admission_surfaces
    
    arguments
    
    authorised
    
    authority
    
    baseline
    
    budget
    
    bundle
    
    capability_labels
    
    completeness
    
    config_retirement
    
    container_semantics
    
    continuation
    
    continuation_admission
    
    continuation_guard_gap
    
    continuation_redemption_removed
    
    continuation_removal_refusals
    
    continuation_window
    
    continuation_window_admission
    
    continuation_window_line
    
    continuation_window_refusals
    
    criteria
    
    delegate
    
    delegation
    
    domain_fields
    
    envelope
    
    errors
    
    extension_tools
    
    extensions
    
    force_lever
    
    fragments
    
    governance_contract
    
    governance_surfaces
    
    guardrails
    
    injections
    
    kernel_resume
    
    legacy_pin
    
    lifecycle_contract
    
    lifecycle_refusals
    
    memory_admission
    
    memory_consolidation_rows
    
    memory_fold
    
    memory_orchestrator_rows
    
    memory_retrieval_rows
    
    memory_rows
    
    memory_surfaces
    
    messages
    
    observability
    
    orchestrator_policy
    
    orchestrator_settings
    
    pause_admission
    
    pause_policy
    
    pause_surfaces
    
    plugin_providers
    
    policy
    
    process
    
    projection
    
    prompt_assembly_policy
    
    prompt_block_contract
    
    prompting_rows
    
    prompting_surfaces
    
    registers
    
    rehydration
    
    retired_settings
    
    retirement
    
    rounds
    
    routes
    
    run_observability
    
    settings_contract
    
    switchboard
    
    tool_result_surfaces
    
    tools_surfaces
    
    turn_request
    
    typed_close_state
    
    typed_interrupts
    
    typed_projection
    
    typed_route
    
    domain
    
    engine
    
    errors
    
    facade
    
    facade_args
    
    facade_continuation
    
    facade_dispatch
    
    facade_model
    
    facade_prepare
    
    facade_turn
    
    facade_types
    
    factory
    
    fastapi
    
    fastapi
    
    authgate
    
    budget_dep
    
    dependencies
    
    dependencies_resolver
    
    graph_maintenance_audit
    
    graph_maintenance_router
    
    graph_router
    
    identity
    
    memory_create_handlers
    
    memory_create_helpers
    
    memory_create_models
    
    memory_create_router
    
    metrics_router
    
    router
    
    tenant_privacy_export
    
    tenant_privacy_router
    
    transport
    
    transport
    
    codec
    
    error_mapper
    
    frames
    
    http
    
    policies
    
    sse
    
    hygiene
    
    middleware
    
    middleware
    
    confidence
    
    critic_gate
    
    events
    
    fabrication
    
    fabrication_events
    
    fabrication_wiring
    
    interrupt_minter
    
    metacognitive
    
    pause_token
    
    pause_token_store
    
    pause_token_store_backends
    
    observability
    
    preflight_l1
    
    prompts
    
    prompts
    
    db_section
    
    extraction_prompt
    
    jit_manifest
    
    ports
    
    stratigraphic
    
    stratigraphic_layers
    
    system_prompt
    
    templates
    
    scheduler
    
    sessions
    
    stream_contract
    
    structured
    
    subagents
    
    subagents
    
    facade_child
    
    lockdown
    
    registry
    
    tool
    
    types
    
    transcript
    
    triage
    
    triage
    
    intent
    
    lexicon
    
    tool_routing
    
    wiring
    
    types
    
    types_attachment
    
    warnings
    
    capabilities
    
    capabilities
    
    delegation
    
    delegation
    
    capability
    
    compiler
    
    composition
    
    context
    
    contracts
    
    declarations
    
    depth
    
    errors
    
    lifecycle
    
    lockdown
    
    roster
    
    runner_adapter
    
    tools
    
    values
    
    extensions
    
    extensions
    
    admission
    
    bridge
    
    composition
    
    contracts
    
    declarations
    
    errors
    
    harvest
    
    legacy_prompts
    
    mcp
    
    mcp_reader
    
    ports
    
    trust
    
    values
    
    governance
    
    governance
    
    capability
    
    context
    
    contracts
    
    errors
    
    ordering
    
    pipeline
    
    records
    
    rulebook
    
    stages
    
    stages
    
    budget
    
    credentials
    
    fabrication
    
    intent
    
    metacognition
    
    metacognition_policy
    
    metacognition_validation
    
    objections
    
    preconditions
    
    steering
    
    values
    
    human
    
    human
    
    binding
    
    capability
    
    checkpoints
    
    consumption
    
    contracts
    
    contribution
    
    drain
    
    errors
    
    factory
    
    issuance
    
    ledger
    
    pausing
    
    refusals
    
    registration
    
    resume
    
    threads
    
    tokens
    
    turnstate
    
    values
    
    knowledge
    
    knowledge
    
    assembly
    
    attachments
    
    citations
    
    contracts
    
    documents
    
    errors
    
    factory
    
    labels
    
    multimodal
    
    outbound
    
    retrieval
    
    memory
    
    memory
    
    activation
    
    admin
    
    admission
    
    attribution
    
    background_cycles
    
    background_observation
    
    bridge
    
    budget
    
    cadence
    
    candidate_signals
    
    candidates
    
    capability
    
    commit
    
    compat
    
    composed
    
    consolidation
    
    contribution
    
    conversation
    
    coordinator
    
    counts
    
    cycle_state
    
    cycle_writes
    
    durability
    
    errors
    
    events
    
    extraction
    
    factory
    
    families
    
    fencing
    
    graph_admin
    
    graph_rows
    
    graph_store
    
    growth
    
    heartbeat
    
    hydration
    
    hydrator_assembly
    
    in_memory
    
    inventory
    
    journal
    
    layers
    
    leases
    
    legacy_port
    
    legacy_write
    
    linking
    
    mutations
    
    nap_stage
    
    napping
    
    operations
    
    overlay
    
    owned_cycles
    
    phase_context
    
    phase_savepoint
    
    phases
    
    phases
    
    adjacency
    
    deep
    
    drafts
    
    entities
    
    entity
    
    entity_helpers
    
    episodic
    
    evidence
    
    evidence_rules
    
    expiry
    
    maintenance
    
    merge
    
    merge_classification
    
    nightly
    
    phase_graph
    
    procedural
    
    procedural_llm
    
    profile
    
    pruning
    
    quick
    
    roster
    
    strengthen
    
    structural
    
    synthetic
    
    window
    
    policy
    
    ports
    
    procedural
    
    promotion
    
    provenance
    
    publish
    
    queries
    
    recall
    
    recall_stage
    
    record_access
    
    record_admission
    
    records
    
    relevance
    
    rendering
    
    reserved_keys
    
    rosters
    
    schedule
    
    scope
    
    scrubbing
    
    selection
    
    sources
    
    stage_descriptors
    
    transactions
    
    turn_records
    
    vector_recall
    
    work
    
    working
    
    workset_overlay
    
    write_stages
    
    writes
    
    prompting
    
    prompting
    
    admission
    
    assembly
    
    boundaries
    
    budget
    
    budgeting
    
    cache
    
    capability
    
    compiler
    
    contracts
    
    declarations
    
    errors
    
    gates
    
    guardrails
    
    hms
    
    identity
    
    layers
    
    onboarding
    
    persona
    
    ports
    
    procedural
    
    request
    
    sources
    
    strategies
    
    values
    
    tools
    
    tools
    
    capability
    
    catalog
    
    context
    
    execution
    
    execution
    
    binding
    
    executor
    
    preconditions
    
    progress
    
    retry
    
    skill_gate
    
    steering
    
    termination
    
    values
    
    keywords
    
    pipeline
    
    results
    
    results
    
    policy
    
    recall
    
    service
    
    values
    
    routing
    
    selection
    
    stages
    
    stages
    
    allowlist
    
    base
    
    forced
    
    intent
    
    lazy
    
    policy
    
    role
    
    values
    
    cli
    
    cli
    
    chat
    
    component_dependencies
    
    components
    
    example_journeys
    
    example_spec
    
    examples
    
    import_scan
    
    init
    
    init_template
    
    public_api
    
    public_api_surface
    
    core
    
    core
    
    agents
    
    agents
    
    in_memory
    
    loader
    
    schema
    
    callbacks
    
    callbacks
    
    base
    
    correction
    
    emit
    
    emit_support
    
    lifecycle_scopes
    
    manager
    
    noop
    
    protocol
    
    capabilities
    
    capabilities
    
    firecrawl
    
    openai_search
    
    protocols
    
    codex_backend
    
    codex_provider
    
    config
    
    contracts
    
    contracts
    
    callbacks
    
    elicitation
    
    errors
    
    interrupt
    
    model_config
    
    prompt_state
    
    resolvers
    
    skill_metadata
    
    tool_policy
    
    types
    
    deps
    
    documents
    
    documents
    
    in_memory
    
    loader
    
    schema
    
    truncation
    
    edges
    
    edges
    
    tool_condition
    
    events
    
    force_resolver
    
    graph
    
    kimi_credentials
    
    kimi_oauth_provider
    
    learning
    
    learning
    
    consolidation
    
    durability_gate
    
    entity_extractor
    
    entity_extractor_llm
    
    entity_extractor_spacy
    
    models
    
    phases
    
    phases_entity
    
    phases_episodic
    
    phases_maintenance
    
    phases_procedural_promotion
    
    phases_profile
    
    phases_pruning
    
    phases_semantic_expiry
    
    phases_semantic_merge
    
    phases_synthetic
    
    policy
    
    procedural_extractor
    
    procedural_extractor_llm
    
    promotion
    
    refiner
    
    logging
    
    logging
    
    console
    
    protocol
    
    models
    
    nodes
    
    nodes
    
    context_injection
    
    context_window
    
    context_window_memory
    
    elicitation
    
    interrupt
    
    precondition_gate
    
    react
    
    tool_results
    
    tool_selection
    
    oauth_provider
    
    observability
    
    observability
    
    audit
    
    budget
    
    budget_hydrate
    
    budget_models
    
    decorators
    
    hooks
    
    metrics
    
    metrics_execution
    
    metrics_query
    
    metrics_records
    
    metrics_store
    
    metrics_store_factory
    
    mongo_metrics_store
    
    node_names
    
    postgres_budget_store
    
    postgres_execution_sql
    
    postgres_metrics_store
    
    pricing
    
    protocol
    
    orchestration
    
    plugins
    
    plugins
    
    base
    
    contribution
    
    dispatcher
    
    section
    
    presets
    
    presets_react
    
    prompt
    
    prompt
    
    blocks
    
    blocks
    
    capabilities
    
    injection
    
    injection_parts
    
    memory_lane
    
    memory_lane_facts
    
    protocol
    
    protocol_conflict
    
    protocol_read
    
    render
    
    resolver
    
    scopeless
    
    snapshot
    
    sources
    
    sources
    
    computed
    
    computed_contract
    
    database
    
    database_schema
    
    database_sql
    
    file
    
    file_revision
    
    static
    
    spec
    
    taxonomy
    
    types
    
    validation
    
    vocabulary
    
    builder
    
    caching
    
    messages_cache
    
    messages_cache_rolling
    
    sections
    
    protocols
    
    providers
    
    resilience
    
    roles
    
    runtime
    
    scope
    
    skills
    
    skills
    
    filter
    
    formatter
    
    in_memory
    
    loader
    
    markdown_utils
    
    schema
    
    state
    
    streaming
    
    streaming
    
    adapters
    
    adapters
    
    anthropic_adapter
    
    openai_adapter
    
    citation_filter
    
    event_processor
    
    extraction_filter
    
    serialization
    
    transpiler
    
    testing
    
    testing
    
    deterministic
    
    scripted
    
    tool_interface
    
    tool_offload_gate
    
    tool_palette_resolver
    
    tool_result_ledger
    
    tools
    
    tools
    
    agent_tools
    
    ask_user
    
    chart_tools
    
    decorator
    
    helpers
    
    memory_tools
    
    registration
    
    registry
    
    skill_tools
    
    streaming
    
    web_tools
    
    usage
    
    utils
    
    utils
    
    compression
    
    devtools
    
    devtools
    
    archcheck
    
    archcheck
    
    boundaries
    
    budgets
    
    config
    
    cycles
    
    matrix
    
    model
    
    registry
    
    report
    
    runner
    
    scan
    
    archgate
    
    archgate
    
    baseline
    
    config
    
    dry
    
    expiry
    
    exports
    
    inventory
    
    processglobals
    
    ratchet
    
    reachability
    
    report
    
    sourcefacts
    
    suite
    
    integrations
    
    integrations
    
    catalog
    
    checks
    
    introspect
    
    model
    
    ownership
    
    ownership
    
    loader
    
    model
    
    overlap
    
    validate
    
    diagnostics
    
    diagnostics
    
    checks
    
    checks
    
    check_chat_model_resolution
    
    check_consolidate_schedule
    
    check_embedding_provider
    
    check_lazy_tooling_orphan
    
    check_orchestrator_layers
    
    check_postgres_dsn
    
    cli
    
    inspection
    
    inspection
    
    config_checks
    
    context
    
    extras
    
    legacy_imports
    
    model
    
    platform_checks
    
    probe
    
    report
    
    runner
    
    service_checks
    
    source_checks
    
    offline_cli
    
    result
    
    runner
    
    severity
    
    evals
    
    evals
    
    applicability
    
    assertions
    
    attachment_evidence
    
    book_assertions
    
    book_evidence
    
    book_journeys
    
    book_retrieval_assertions
    
    book_visible_truth
    
    cli
    
    concurrency_assertions
    
    concurrency_journeys
    
    consolidation_assertions
    
    consolidation_journeys
    
    consolidation_ledger
    
    consolidation_mutation
    
    content_packs
    
    delegation_journeys
    
    effects
    
    execution_packs
    
    fixture_ingestion
    
    fixtures
    
    fixtures
    
    loader
    
    model
    
    negative
    
    validation
    
    governance_assertions
    
    host_target
    
    http_target
    
    memory_assertions
    
    memory_journeys
    
    model
    
    observability_assertions
    
    observability_journeys
    
    observability_privacy
    
    pause_evidence
    
    procedural_evidence
    
    procedural_pack
    
    provider_evidence
    
    reporters
    
    response_assertions
    
    response_memory
    
    rubric
    
    runner
    
    security_assertions
    
    security_journeys
    
    selection
    
    target_evidence
    
    targets
    
    tool_assertions
    
    tool_journeys
    
    traits
    
    volume
    
    volume_assertions
    
    volume_corpus
    
    volume_journeys
    
    volume_lifecycle_assertions
    
    volume_sweeps
    
    infra
    
    infra
    
    protocols
    
    trace
    
    kernel
    
    kernel
    
    adapters
    
    apply
    
    background
    
    backpressure
    
    compiler
    
    context
    
    contracts
    
    contracts
    
    archive_limits
    
    checkpoints
    
    contributions
    
    diagnostics
    
    effects
    
    forcing
    
    freeze
    
    groups
    
    interrupts
    
    lifecycle
    
    palette
    
    payload
    
    plan
    
    ports
    
    resolved
    
    results
    
    run_identity
    
    snapshot
    
    stages
    
    tenancy
    
    values
    
    deadline
    
    dispatch
    
    fanout
    
    finalize
    
    finalizers
    
    invoker
    
    lifecycle
    
    observability_binding
    
    ordering
    
    palette
    
    policies
    
    post_model
    
    post_tool
    
    pre_model
    
    pre_tool
    
    prompt_assembly
    
    run_observation
    
    rungs
    
    runner
    
    streaming
    
    tool_round
    
    traces
    
    turn_guards
    
    validation
    
    knowledge
    
    knowledge
    
    adapters
    
    adapters
    
    chroma
    
    models
    
    protocols
    
    memory
    
    memory
    
    _internal
    
    _internal
    
    extraction
    
    extraction
    
    service
    
    llm
    
    llm
    
    response_parser
    
    backends
    
    backends
    
    chroma
    
    in_memory
    
    in_memory_candidates
    
    in_memory_deletion
    
    in_memory_queries
    
    in_memory_transaction
    
    in_memory_vector
    
    in_memory_worksets
    
    inventory
    
    mongo_candidates
    
    mongodb
    
    pool
    
    pool_pgvector
    
    pool_transaction
    
    postgres_candidates
    
    postgres_graph
    
    postgres_leases
    
    postgres_vector
    
    postgres_worksets
    
    workset_probe
    
    candidates
    
    classification
    
    dedup
    
    durability
    
    embeddings
    
    embeddings
    
    auto_embed
    
    factory
    
    graph
    
    graph
    
    scoring
    
    store
    
    store_queries
    
    traversal
    
    inventory
    
    janitor
    
    janitor_dedup
    
    layers
    
    layers
    
    episodic
    
    episodic_events
    
    procedural
    
    procedural
    
    catalog
    
    layer
    
    predicates
    
    router
    
    skill_render
    
    prospective
    
    semantic
    
    working
    
    models
    
    models
    
    context
    
    edge
    
    entry
    
    node
    
    operation
    
    orchestrator
    
    orchestrator
    
    builders
    
    config
    
    factory
    
    orchestrator
    
    orchestrator_writes
    
    pending
    
    prompts
    
    protocols
    
    protocols_graph
    
    reconcile
    
    render
    
    retraction
    
    retrieval
    
    retrieval
    
    discovery
    
    engine
    
    filters
    
    spreading
    
    scope_isolation
    
    subtree
    
    telemetry
    
    telemetry
    
    episodic_sink
    
    types
    
    warnings
    
    working
    
    working
    
    stm_summary
    
    stm_summary_llm
    
    worksets
    
    observability
    
    observability
    
    otel
    
    otel
    
    attributes
    
    callback_bridge
    
    exporter
    
    framework_bridge
    
    kernel_sink
    
    ports
    
    propagation
    
    tracer
    
    ports
    
    tool_span_seam
    
    platform
    
    platform
    
    admission
    
    agent_host
    
    audit
    
    budget
    
    continuation
    
    erasure
    
    errors
    
    extension_policy_stage
    
    extensions
    
    fence_audit
    
    governance
    
    governance_amendment
    
    governance_decisions
    
    governance_declaration
    
    governance_refusal
    
    governance_rungs
    
    governance_subject
    
    governance_verdict
    
    host
    
    human_continuation
    
    memory_curation
    
    observability
    
    ports
    
    posture
    
    privacy
    
    routes
    
    scope
    
    telemetry
    
    telemetry_sink
    
    token_composition
    
    transport
    
    transport_data
    
    transport_records
    
    values
    
    wire
    
    services
    
    services
    
    budget
    
    budget
    
    allocation
    
    contracts
    
    errors
    
    estimation
    
    overflow
    
    ports
    
    registry
    
    service
    
    truncation
    
    values
    
    conversation
    
    conversation
    
    capability
    
    checkpoint
    
    compat
    
    errors
    
    history
    
    horizon
    
    identity
    
    library
    
    migration
    
    notification
    
    records
    
    recovery
    
    refs
    
    registry
    
    session
    
    transcript
    
    values
    
    cost
    
    cost
    
    ports
    
    tracker_adapter
    
    values
    
    effects
    
    effects
    
    accounting
    
    admission
    
    compensation
    
    drain
    
    errors
    
    exposure
    
    fence
    
    incident
    
    ports
    
    quarantine
    
    revert
    
    store
    
    tracker
    
    values
    
    models
    
    models
    
    capabilities
    
    contract
    
    descriptors
    
    family
    
    ports
    
    resolution
    
    values
    
    observability
    
    observability
    
    bridge
    
    callbacks
    
    capture
    
    closing
    
    contracts
    
    cost
    
    metrics
    
    otel
    
    otel_handles
    
    otel_spans
    
    ports
    
    suite
    
    trace
    
    trace_policy
    
    values
    
    privacy
    
    privacy
    
    errors
    
    fence
    
    guard
    
    legacy
    
    lineage
    
    ports
    
    reference
    
    registry
    
    saga_store
    
    saga_values
    
    suite
    
    values
    
    shadow
    
    shadow
    
    admission
    
    classification
    
    comparator
    
    context
    
    crypto
    
    cutover
    
    digest
    
    effects
    
    errors
    
    gateway
    
    harness
    
    ledger
    
    ports
    
    privacy
    
    recorder
    
    recording
    
    redaction
    
    replay
    
    sentinel
    
    state
    
    store
    
    trust
    
    switching
    
    switching
    
    admission
    
    audit
    
    authz
    
    binding_stage
    
    constraints
    
    control_plane
    
    control_plane_source
    
    envelope
    
    errors
    
    evidence
    
    invocation_profile
    
    keys
    
    mutation
    
    ports
    
    registry
    
    release
    
    static_source
    
    store
    
    values
    
    tools
    
    tools
    
    corpus_scrub
    
    corpus_scrub
    
    jsonl_io
    
    patterns
    
    scrubber
    
    hydrate_context
    
    learning
    
    mcp
    
    mcp
    
    connection
    
    models
    
    protocol
    
    provider
    
    routing
    
    recall_tool_result

Table of contents

fixtures

`symfonic.evals.fixtures`¶

fixtures ¶

Committed evaluation fixture implementation; public exports live in symfonic.evals.

fixture_ingestion

Copyright © AmSiDev

Made with Material for MkDocs