Skip to content

symfonic.evals.recovery_budget

recovery_budget

Fail-closed evaluation guard; never discovers credentials or changes models.

BudgetConfigurationRefused

Bases: ValueError

The supplied model does not expose the required bounded configuration.

CallBudgetExceeded

Bases: RuntimeError

A fifth chat-model request was stopped at its start callback.

RecoveryBudgetProvider 

RecoveryBudgetProvider(provider: Any)

Require max_tokens<=256, retries off, and stop before request five.

Only ordinary LangChain models with shallow model_copy and installable list callbacks are accepted. Configuration introspection is not authority to run an external provider: the caller must separately approve its source and verify the provider honours these controls. Hidden SDK retries/cache bypasses remain outside this local callback guard's evidence.

Source code in src/symfonic/evals/recovery_budget.py 
40
41
42
def __init__(self, provider: Any) -> None:
    self.provider = provider
    self.guard = _CallGuard()