6 Commits

Author	SHA1	Message	Date
OlivierDehaene	718096f695	feat: Support stop sequences (#7)	2022-12-12 18:25:22 +01:00
OlivierDehaene	042180d88f	fix(server): Only pad to multiple of 8 on GPUs	2022-12-08 19:37:37 +01:00
OlivierDehaene	a2985036aa	feat(server): Add model tests (#6)	2022-12-08 18:49:33 +01:00
Nick Hill	31d76e238d	fix(batching): Avoid theoretical hang in batcher loop (#5) ... - Avoid theoretical hang in batcher loop - Avoid a couple of clones in the router generate method - Keep attention mask tensors as integers - Remove num_heads attribute Co-authored-by: OlivierDehaene <Olivier.dehaene@gmail.com>	2022-12-05 10:10:59 +01:00
OlivierDehaene	4236e41b0d	feat(server): Improved doc	2022-11-07 12:53:56 +01:00
OlivierDehaene	427d7cc444	feat(server): Support AutoModelForSeq2SeqLM	2022-11-04 18:03:04 +01:00