Skip to content

HMS budgets: warn vs strict vs hook

Status: Shipped in v7.25.0 Closes: HMS budget visibility ask Related: FrameworkConfig.hms_budget_mode, FrameworkConfig.on_hms_budget_exceeded, FrameworkConfig.hms_system_prompt_token_budget

What the budget is

FrameworkConfig.hms_system_prompt_token_budget caps the rendered size of the HMS L1 system-prompt section. Defaults are per-strategy:

Strategy Default budget
jit 1500 tokens
stratified 5500 tokens

When the rendered section exceeds the budget, the engine never drops or truncates it. Dropping mid-turn would invalidate the v7.24.1 descending-TTL invariant on the stratigraphic cache breakpoint, silently mismatch the cache_control marker, and ship a broken prompt to the wire. The architect-locked contract is: budgets are advisory; sections are sacred.

v7.25.0 adds three knobs that change what happens at the overflow site, not whether the section ships.

hms_budget_mode='warn' (default)

Preserves v7.24.x behaviour byte-for-byte.

  • The engine emits a single advisory WARNING log per agent instance (not per turn).
  • The emit-once gate latches after the first overflow so a long-running agent does not flood the log.
  • The section still ships in full.

Use this in development and for adopters who treat budget overflow as observability data, not a contract.

hms_budget_mode='strict'

from symfonic.agent.config import FrameworkConfig

config = FrameworkConfig(hms_budget_mode="strict")
  • The engine raises ConfigurationError at the overflow site.
  • The raise happens before the section is stamped with a cache_control marker downstream, so no half-cached prompt reaches the wire.
  • The emit-once gate still latches so subsequent turns continue to fail loudly without leaking partial state.

Use this in adopter CI and on production deployments where a budget regression is a hard error -- the test suite or the run fails the moment the prompt-assembly path exceeds budget.

on_hms_budget_exceeded hook

def record_overflow(layer_name: str, requested_tokens: int, budget: int) -> None:
    statsd.gauge("symfonic.hms.budget.requested", requested_tokens,
                 tags=[f"layer:{layer_name}"])
    statsd.gauge("symfonic.hms.budget.cap", budget,
                 tags=[f"layer:{layer_name}"])

config = FrameworkConfig(on_hms_budget_exceeded=record_overflow)

Contract:

  • The hook fires once per overflow -- same emit-once gate as the log.
  • The hook fires in both warn and strict modes.
  • In strict mode the hook fires before the ConfigurationError is raised, so the adopter observability dashboard records the signal even on a failing turn.
  • Hook exceptions are caught and logged at WARNING level. A buggy adopter sink MUST NOT crash the prompt-assembly path.

Wire it to whatever observability surface you own: statsd, OTEL, a custom dashboard, a Slack webhook on first overflow.

Choosing the right budget

The default stratified=5500 was sized for the v7.7 baseline HMS render: L0 brain kernel + L1 OS services + memory-extraction protocol cached prefix. Adopters who:

  • Inject contributions via inject_contributions(position='cached') -- add the contribution token count to the budget.
  • Run a richer domain template with a long domain.description -- the description rendering caps at domain_description_max_chars[0] (default 2000 chars / ~500 tokens); add that to the budget.
  • Use procedural_render_preflight_in_l1=True -- the imperative PRE-FLIGHT block adds ~200-500 tokens depending on the active skill count.

Measure once with hms_budget_mode='warn', take the observed peak from the log line (%d tokens (budget %d, ...)), add 20% headroom, then re-pin hms_system_prompt_token_budget to that value and flip to 'strict' in CI.

Cache invariant: why we don't drop sections

Anthropic's prompt-cache contract requires that any token before a cache_control breakpoint be byte-stable across turns. Dropping a portion of L1 mid-turn would change the bytes inside the breakpoint without changing the breakpoint position, the cache key would mismatch on the next turn, and Anthropic would charge a full cold-write on every subsequent turn.

The v7.24.1 descending-TTL invariant codifies this: TTL ladders must be monotonically descending from outermost (L0/L1) to innermost (L2). Any drop that bumps L1 content into L2 would break the ladder.

strict mode raises before the section is stamped because raising after would leave the framework in an undefined state -- it would either:

  1. Ship the half-stamped prompt (broken cache, broken cost), or
  2. Roll back the stamping (complex, error-prone, no clean state transition).

The architectural-line choice in v7.25.0: budget enforcement is a configuration check, not a runtime correction. The check runs at the right side of the breakpoint stamping decision, and it fails fast when configuration disagrees with reality.