hf_text-generation-inference

Commit Graph

Author	SHA1	Message	Date
drbh	7dbaf9e901	fix: correctly index into mask when applying grammar (#1618 ) This PR fixes how the grammar mask is index when generating text and adds a new test to ensure the grammars work with non flash models	2024-03-01 18:22:01 +01:00
drbh	9b6db5f793	Support tools (#1587 ) This work in progress PR begins to add support for tools. Tools relies on grammar support and still has some unsolved challenges. Opening the PR for visibility and feedback	2024-02-28 11:10:27 +01:00

Author

SHA1

Message

Date

drbh

7dbaf9e901

fix: correctly index into mask when applying grammar (#1618 )

This PR fixes how the grammar mask is index when generating text and
adds a new test to ensure the grammars work with non flash models

2024-03-01 18:22:01 +01:00

drbh

9b6db5f793

Support tools (#1587 )

This work in progress PR begins to add support for tools. Tools relies
on grammar support and still has some unsolved challenges. Opening the
PR for visibility and feedback

2024-02-28 11:10:27 +01:00

2 Commits