symfonic.evals.tool_assertions

tool_assertions

Deterministic assertions over tool-call and tool-result events.

ToolCallIdsUnique dataclass

ToolCallIdsUnique(minimum: int = 1, name: str = 'tool_call_ids_unique')

Require emitted tool calls to have non-empty, observation-unique IDs.

ToolCalled dataclass

ToolCalled(tool_name: str, times: int = 1, arguments: Mapping[str, Any] | None = None, name: str = 'tool_called')

Require an exact number of calls to a named tool.

ToolFailed dataclass

ToolFailed(tool_name: str, times: int = 1, name: str = 'tool_failed')

Require tool execution to return an error outcome, without exposing it.

ToolSucceeded dataclass

ToolSucceeded(tool_name: str, times: int = 1, arguments: Mapping[str, Any] | None = None, name: str = 'tool_succeeded')

Require an exact number of completed, error-free tool executions.