Constrained decoding¶
ConstraintViolation¶
class ConstraintViolation(message: str, output: str)
tau_llm.constraints.ConstraintViolation
A constrained generation did not satisfy its constraint.
Raised when verification (§4.3) finds the output outside the declared constraint — which, on a server where llguidance died mid-generation, means the generation ran unconstrained and the result is fabricated data. Carries the offending output.
Constructor parameters
message: str— (no description)output: str— (no description)
DecodeConstraints¶
tau_llm.constraints.DecodeConstraints
A decode constraint for one completion.
choices¶
tau_llm.constraints.DecodeConstraints.choices: list[str] | None
Sugar for the verdict pattern; compiled to a choice grammar.
describe¶
describe() -> dict[str, Any]
tau_llm.constraints.DecodeConstraints.describe
A small, display-only summary for observability (§3.3).
Recorded on events / persisted entry payloads so what constrained a generation is inspectable after the fact. Never replayed as context.
extra_body¶
tau_llm.constraints.DecodeConstraints.extra_body: dict[str, Any]
Per-call body params. Highest precedence: over Model.extra_body,
over τ defaults.
grammar¶
tau_llm.constraints.DecodeConstraints.grammar: SkipValidation[str | None]
Raw grammar text, in the model's declared dialect. The provider adds
the %llguidance header for llguidance models (never double-prefixing,
never prefixing a gbnf model's grammar).
has_constraint¶
has_constraint() -> bool
tau_llm.constraints.DecodeConstraints.has_constraint
Whether this carries an actual decode constraint (vs only tool_choice/extra_body).
json_schema¶
tau_llm.constraints.DecodeConstraints.json_schema: dict[str, Any] | None
A JSON Schema; sent as OpenAI-style response_format so the
server does its own schema→grammar conversion (llguidance consumes JSON
Schema natively — τ does not reimplement that compiler).
model_config¶
tau_llm.constraints.DecodeConstraints.model_config
No description. This object is marked but undocumented.
tool_choice¶
tau_llm.constraints.DecodeConstraints.tool_choice: str | dict[str, Any] | None
OpenAI-compat passthrough. Note "none" is what makes a
constraint legal alongside a declared tools array (see the provider).
verify¶
tau_llm.constraints.DecodeConstraints.verify: Callable[[str], bool] | None
No description. This object is marked but undocumented.
verify_output¶
verify_output(text: str) -> None
tau_llm.constraints.DecodeConstraints.verify_output
Assert text actually satisfies this constraint; raise if it does not.
τ never trusts a constraint blindly. The failure mode is real and observed (GRAMMAR_DECODING_RECON.md:36): on some tokenizers llguidance dies mid-generation, logs the error server-side, and lets generation continue unconstrained — completely invisible to the client, which receives a 200 and plausible text.
An unconstrained result returned as a constrained one is fabricated data, so a failed check raises rather than warns.
What can be checked per kind:
choices— exact membership. Total.json_schema— the output must parse as JSON. (Full schema validation is optional; a parse failure alone already catches constraint death.)grammar— no general check is possible against an arbitrary grammar. The caller may supplyverify; absent that, τ asserts non-empty output and the residual risk is documented.
Parameters
text: str— (no description)