USER-OWNED KNOWLEDGE EVAL EXAMPLE

From the directory containing custom_knowledge/:
  PYTHONPATH=. symfonic eval custom_knowledge.suite:SUITE --profile fast --json knowledge.json

Requires the candidate symfonic-core wheel with [cli,testing,openai] dependencies.
The published guide explains installation, manifest authoring and live mode:
  guides/23-agent-regression-evals/#test-your-own-markdown-or-database-records

data/*.md: input corpus, one stable filename ID per record.
manifest.json: evaluator-only questions, complete accepted answers and SHA-256 revisions.
records.py: bounded Markdown / read-only SQLite adapters and lexical retriever.
target.py: public Agent + knowledge/prompting composition and provider-boundary evidence.
checks.py: manifest validation and strict answer/citation/revision assertion.
suite.py: standard symfonic eval integration, fast and live targets.
inventory.py: print configured source IDs/checksums without content.
seed_sqlite.py: create a NEW demo database; refuses existing paths.
vector_search.py: bounded, scope-filtered pgvector and Atlas retrieval adapters.
connections.py: environment-configured database and real embedding connections.
existing_suite.py: integration = existing index retrieval; live = retrieval + LLM.

KNOWLEDGE_DIR selects a Markdown folder.
KNOWLEDGE_DB selects an existing SQLite file (takes precedence); KNOWLEDGE_SCOPE is required.
KNOWLEDGE_MANIFEST selects your reviewed JSON manifest.
Live mode requires KNOWLEDGE_MODEL and OpenAI-compatible provider configuration.

For an existing vector index, use custom_knowledge.existing_suite:SUITE instead.
Set KNOWLEDGE_MANIFEST, KNOWLEDGE_SCOPE, KNOWLEDGE_DIMENSIONS and the embedding
configuration matching the stored vectors. KNOWLEDGE_SEARCH_FACTORY selects
custom_knowledge.connections:pgvector or custom_knowledge.connections:atlas.
Install psycopg[binary] or pymongo respectively. See the guide's
#existing-pgvector-or-atlas-vector-search section for complete connection settings.
This separate suite's integration profile DOES use the network and embedding
service; it does not use a chat model. Its live profile also calls the chat model.
There is no fallback to the bundled Markdown corpus or fabricated embeddings.

No database writes occur during evals. No production credentials are included.
Do not pass the manifest to the model. Do not silently regenerate expected hashes
to make changed data pass. Do not mistake the offline echo model for an LLM eval.
The example neither exercises a running scaffold HTTP endpoint nor tests memory
extraction, durable writes, restart or your application's authorization layer.
Its database filter demonstrates one explicit scope; your production adapter
must enforce authorization and use bounded, server-side retrieval.
