Skip to content

[LLM Runner] Implement GrammarLogitProcessor using llguidance #19475

@kirklandsign

Description

@kirklandsign

Wrap LlgMatcher as LogitProcessor. compute_mask -> apply bitmask to logits -> sample -> consume_token. Support json_schema/regex/lark/gbnf. Depends on: LogitProcessor interface, tokenizer bridge, llguidance dep.

cc @larryliu0820 @mergennachin @cccclai @helunwencser @jackzhxng

Metadata

Metadata

Assignees

Labels

module: llmIssues related to LLM examples and apps, and to the extensions/llm/ codetriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type
No fields configured for issues without a type.

Projects

Status

To triage

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions