Skip to content

symfonic.tools.corpus_scrub.patterns

patterns

v7.8.0 corpus-scrub regex patterns.

Pattern groups:

  • :data:DEFAULT_VALUE_CREDENTIAL_PATTERNS -- credential-shaped substrings in VALUE strings (Bearer tokens, API keys, JWTs, etc.). Distinct from symfonic.agent.hygiene which scans KEY NAMES.
  • :data:DEFAULT_PII_PATTERNS -- generic PII (email, phone, URL, street address fragments) mapped to opaque token labels.
  • :data:AMAZON_KEEP_PATTERNS -- structural identifiers retained AS-IS (ASIN, marketplace-id) because they carry no PII and the Phase 12 extractor needs them for tool-precondition learning.
  • :data:AMAZON_REDACT_PATTERNS -- Amazon-account identifiers that ARE PII (seller-id, merchant-id) and MUST redact.

Patterns are deliberately conservative: false negatives are recoverable (a missed pattern triggers an "unresolved" count toward the drop threshold), but false positives are not (a redacted ASIN breaks the extractor's pattern-key resolution).

Each pattern entry is (label, compiled_regex). The regex captures the entire match in group 0; substitution uses a counter- indexed token (<REDACTED-EMAIL-1>) so co-occurrence patterns within a corpus are preserved.