Selective Metacognition Gating (v8.2.0)¶
The metacognition critic (MetacognitiveMiddleware) runs a serial, blocking
LLM round-trip after every draft when its gate fires. On cold,
low-activation traffic (short conversations, sparse graph state) the legacy
gate fired on every turn -- compute_confidence returns 0.5 on sparse
activation, which is below the default 0.6 threshold, so the gate was
effectively always-on. A bare acknowledgment ("Got it, scheduling that") and a
risky numeric claim hit the identical gate result, because confidence is
computed from graph activation, not from what the model actually said.
Selective metacognition gating adds a deterministic (no-LLM) smart gate that can skip the round-trip on provably-trivial acknowledgment turns, while keeping the full critic on every turn that carries a risk signal. It is opt-in and, when off, byte-identical to v8.1.0.
The predicate¶
hard_fire = (
sensitive_hit is not None
or bool(fabrication_findings)
or took_tool_action # MUTATING only -- read_only_tools allowlist
or intent_label in {"action", "ambiguous"}
or draft_has_claims(draft) # numbers / entities / citations / IDs
)
trivial_skip = (
metacognition_selective_gate # opt-in flag, DEFAULT False
and not hard_fire
and draft_is_trivial_ack(draft)
)
return not trivial_skip # FIRE unless trivial_skip
confidence < threshold is deliberately NOT a hard_fire term. That was
the load-bearing correction: the cold-traffic acks are the low-confidence
turns, so keeping low confidence as an absolute floor would make every "Got it"
fire anyway -- the savings would be zero. In selective mode, low confidence is
relaxed only for short, claim-free, action-free, recognized-language
acknowledgments. The hard floors above always hold.
The conservative guarantee¶
Two distinct guarantees, kept separate:
-
Flag OFF (default
metacognition_selective_gate=False) -> byte-identical to v8.1.0.trivial_skipis unconditionallyFalse; the smart-gate code path is inert and the legacy gate (confidence < threshold OR sensitive_hit) runs unchanged. No adopter loses coverage on upgrade. -
Flag ON (opt-in) -> selective mode. Cold-confidence trivial acks now skip; the richer hard-fire signals (findings / mutating tool / claims / action-intent) activate together with selective mode. This is an intentional, documented trade -- it both adds fires (claims/findings/tools) and removes fires (cold trivial acks). Not a pure superset of old behaviour.
Invariant: in selective mode the gate may SKIP only when ALL hold -- flag
ON, no hard-fire signal, and draft_is_trivial_ack matches (short +
claim-free + recognized-language). Every ambiguity -- including an
unrecognized-language ack -- FIRES. The asymmetry is deliberate: the cost of
a false-skip (an unreviewed fabrication reaches the user) far exceeds the cost
of a false-fire (one wasted round-trip).
Config fields (FrameworkConfig)¶
| Field | Default | Purpose |
|---|---|---|
metacognition_selective_gate: bool |
False |
Master opt-in. False = byte-identical to v8.1.0. |
metacognition_gate: Any |
None |
Adopter override -- a Callable[[CriticGateContext], bool] (True=fire, False=skip). Validated at runtime via callable() (a non-callable is ignored with a warning). A raising gate degrades to FIRE. Supplying a gate implies opt-in. |
metacognition_trivial_ack_patterns: list[str] |
[] |
Adopter regex patterns marking a short draft as a trivial ack. Empty uses the built-in language-agnostic default set (length + no-digit + no-entity + common EN/ES/DE/FR/PT ack verb phrases). |
read_only_tools: list[str] |
[] |
Tool names that do NOT count as a mutating action. Empty = any tool call is treated as mutating. |
read_only_tools -- the claims interaction¶
A read tool whose result the model then asserts claims about STILL fires --
via draft_has_claims / fabrication_findings, not via took_tool_action.
The allowlist relaxes only the tool-action signal; it never suppresses the
claims signal. A pure read with a claim-free ack can skip; a read whose result
is quoted with numbers fires.
Multilingual acks¶
The default ack matcher is language-agnostic where possible (short length + no-digits + no-entity is itself largely language-neutral) and ships common EN/ES/DE/FR/PT ack verb phrases. An ack in an unrecognized language does not match -> is not treated as trivial -> FIRES (conservative -- far better to over-fire on an unmatched-language ack than to skip a non-English claim).
CriticGateContext¶
A frozen dataclass carrying the deterministic pre-critic signals passed to the
gate: draft, tool_calls_this_turn, fabrication_findings,
intent_verdict, confidence, sensitive_hit, read_only_tools,
selective_gate_enabled, trivial_ack_patterns, scope, run_id. Every
field is computed before the critic LLM call, so the gate adds ~zero marginal
cost. An adopter metacognition_gate reads this context to make its decision.
Savings caveat¶
The savings are realized only when selective mode is ON -- the default
config (metacognition_selective_gate=False) is byte-identical to today and
saves nothing. With the flag on, on high-turn production traffic the critic
firing rate drops from 20/20 to roughly 10-14/20: the ~6-10 short, claim-free,
mutating-action-free ack turns skip the serial Opus round-trip. Each removed
call is one full serial blocking round-trip off the request path -- precisely
on the fast "ack" turns where the extra round-trip was most disproportionate.
Tier-2 (cheap-model classifier) -- adopter pattern, not core¶
A two-tier design (deterministic gate emits skip/maybe/fire, a fast Haiku
classifier decides the maybe band) is intentionally not a core default:
it re-introduces a per-turn LLM round-trip -- the exact latency being removed.
An adopter who wants it implements it inside their own metacognition_gate
callable. The hook makes tier-2 possible without making it the default.
Capability policy and migration¶
The capability policy is not the inverse of the legacy predicate. Its default
SelectiveMetacognitionPolicy preserves legacy selective coverage: numeric or
entity claims, mutating tools, action/ambiguous intent, fabrication findings,
recalled sensitive tags, and configured sensitive terms all fire reflection.
Only a short, claim-free, recognized acknowledgment with no other evidence
skips. This intentional preservation can cost more than the earlier
confidence-only capability policy.
| Evidence | Supported source | Trigger |
|---|---|---|
| numeric/entity/ID claim | draft | draft_claim |
| mutating tool | egress tool calls (read_only_tools exempts named reads) |
mutating_tool |
| action or ambiguous intent | composed classifier | intent:action / intent:ambiguous |
| fabrication finding | composed detector | fabrication_finding |
| recalled tag | caller's factual turn state | recalled_sensitive_tag:<tag> |
trivial_ack_patterns and trigger_policy are supported migration seams. To
retain the earlier confidence-only cost behavior, compose the explicit
ConfidenceOnlyMetacognitionPolicy; it deliberately does not preserve the
hard floors above. A custom policy returns a stable trigger category instead
of claiming scalar confidence is equivalent to the full predicate.
Recall tags are evidence, not a draft substitution¶
sensitive_terms checks the draft text. sensitive_tags checks actual recall
evidence supplied by the adopter in Agent.run(..., state=...) or
Agent.stream(..., state=...) under metacognition_sensitive_tags. Native
retrieval does not populate this state automatically. An adopter that owns
retrieval must carry its real tags across the public turn boundary; it must not
replace them with a draft substring or an invented confidence value.
agent = Agent(
provider,
capabilities=[governance(reflector=critic, sensitive_tags=("legal",))],
)
await agent.run(
"answer the question",
state={"metacognition_sensitive_tags": ("legal",)},
)
Reporting confidence¶
Nothing upstream invents GovernanceSubject.confidence. A reporter may supply
one confidence signal, but it is supplemental to the evidence policy and never
stands in for it:
class RecallConfidence:
"""Whatever the adopter actually knows. Sync and offline."""
def confidence(self, subject, context) -> float | None:
recalled = subject.properties.get("recall_count")
if recalled is None:
return None # unknown; other evidence can still fire
return max(0.0, min(1.0, recalled / 5))
capability = GovernanceCapability.compose(
reflector=my_critic,
sensitive_terms=("jurisdiccion", "clausula"),
confidence_floor=0.7,
confidence=RecallConfidence(),
)
The port has four rules:
- The subject wins. A subject that already carries a confidence is not overridden, so there is one source per turn rather than a contest.
Nonewithholds. It supplies no confidence trigger; independent evidence can still require reflection.- A reporter that can never work is refused when you compose it. No
confidencemethod, wrong arity, or an async method/callable object raises at construction -- before any turn runs and outside containment. An async reporter is the mistake worth naming:Reflectorsits beside this port and is async, andruntime_checkablecannot tell them apart. - A bad value or reporter failure forces conservative review. A
non-number,
bool, NaN, infinity, value outside[0, 1], or reporter exception is neither coerced to zero nor hidden asNone: it produces an internalinvalid_confidence:<category>and attempts the reflector. A reflector outage remains fail-open and is separately marked as a degraded metacognition stage. Trigger-policy ports are also synchronous and shape-checked at composition; a dynamic exception, awaitable, or malformed result conservatively attempts reflection, while a deliberateNoneskips.
Direct GovernanceCapability.govern() callers inspect its outcome trace.
Agent.run() returns only the result. A fully drained Agent.stream() emits a
governance.egress stage event with a bounded metacognition status such as
metacognition[reviewed], metacognition[skipped],
metacognition[invalid-signal], metacognition[degraded], or
metacognition[invalid-signal-degraded]. The final category records that an
invalid configured signal did receive its conservative reflector attempt, but
the reflector was unavailable. The event never includes a raw trigger,
reflector exception, draft, recalled-tag payload, or finding text.
A note on the names in the refusal message¶
The RetiredSetting entries for these fields name a normalized target such as
cap.safety.metacognition.selective_gate. Those dotted names exist only in
agent/configuration/compat/mapping.py as legacy name normalisation; they
are not capability knobs and nothing reads them at runtime. The composed
arguments above are the real destination. Do not go looking for a setting
under that dotted path -- there isn't one, and there is not meant to be.
If you are migrating¶
Do not strip these fields to force a legacy config onto 11.0: with
metacognition_enabled=True and no way to switch the legacy selective gate
on, the critic reverts to firing on every turn. Compose the governance
capability instead. That is the supported destination, and it is available
now -- not something waiting on a decision.
Related¶
- Tool Call Policy -- the v8.1 additive opt-in / empty-default / byte-identity discipline this feature follows.