Mirrors
/
hf_text-generation-inference
mirror of https://github.com/huggingface/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,136 Commits 177 Branches 50 Tags 606 MiB
Commit Graph

1136 Commits

Author SHA1 Message Date
Morgan Funtowicz 31747163e7 chore(trtllm): remove unused method 2024-10-21 14:10:23 +02:00
Morgan Funtowicz fb00f985ae chore(trtllm): post-rebase commit 2024-10-21 12:31:24 +02:00
Morgan Funtowicz 85c03e33a9 chore(trtllm): fmt 2024-10-21 10:00:27 +02:00
Morgan Funtowicz e3bce407be chore(trtllm): disable tokenizer parallelism by default 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 62f33d7ecd chore(trtllm): move dockerfile to right place 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 6687c06a21 feat(looper): minor optimizations to avoid growing too much the containers 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 027756c52d chore(cmake): download timestamp should be before URL 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 629153b44b feat(looper): check engine and executorWorker paths exist before creating the backend 2024-10-21 10:00:27 +02:00
Morgan Funtowicz f20ec28891 chore(cmake): use correct policy for download_timestamp 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 819c953771 misc(cuda): require 12.6 2024-10-21 10:00:27 +02:00
Morgan Funtowicz dd94ccc989 (fix): ore fixes for Dockerfile 2024-10-21 10:00:27 +02:00
Morgan Funtowicz f9f10a6636 (misc): improve trtllm download script robustness 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 0c3ba932cc (misc): disable logging in release mode 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 437c2aa142 (misc): update dependency in trtllm dockerfile 2024-10-21 10:00:27 +02:00
Morgan Funtowicz cb69c9a967 (misc): update dependency in trtllm dockerfile 2024-10-21 10:00:27 +02:00
Morgan Funtowicz c8a99af6c9 (fix): do not recreate the stateful hashmap at every it 2024-10-21 10:00:27 +02:00
Morgan Funtowicz eb13d8d1f3 (misc): increase verbosity of spdlog 2024-10-21 10:00:27 +02:00
Morgan Funtowicz ce0cd1fce8 (misc): build with trtllm 0.13.0 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 188e4dc64f (misc: build for sm_{75,80,86,89,90} by default 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 544c9d9dba (fix): HOPPER_SM_MAJOR is 9 not 8 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 213acc6e34 (misc) move to latest trtllm 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 507ff66692 (misc) rerun-if-changed all the cmake modules 2024-10-21 10:00:27 +02:00
Morgan Funtowicz b242f45c04 (misc) delete backend.rs 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 984ae9798f (post) impl postprocessing 2024-10-21 10:00:27 +02:00
Morgan Funtowicz fa63db0d07 (scheduler) rework submit/pull logic 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 42ccf4e77c (misc) no need to move for uint32_t items 2024-10-21 10:00:27 +02:00
Morgan Funtowicz b41875c139 (misc) simplify [make_]move_iterator by using c++20 type inference 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 0f50539b77 (Dockerfile.trtllm) delete for now 2024-10-21 10:00:27 +02:00
Morgan Funtowicz b1846fb4e6 (backend) refactor & cleanup 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 483f172938 (ffi) do not use reference capture in lambda as we are not capturing anything 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 3d0e90b631 (ffi) missing namespace for tle::Response 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 8e648ce425 (ffi) fix usage of wrong vector constructor making a capacity fill call 2024-10-21 10:00:27 +02:00
Morgan Funtowicz dddc9a44bd (build) fetchcontent use archives instead of git 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 089c5fe668 (server) forward auth_token to server::run 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 291eaa99fb use blocking_recv in looper to consume awaiting_requests at max before pulling in a single step 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 7bebc629af (misc) missing Result types for Rust 2024-10-21 10:00:27 +02:00
Morgan Funtowicz c2e21d8725 (backend) implement the post_processor background thread 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 0dca168bcb (misc) change scope identifiers 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 933ab67aa1 (ffi) encode the provided user prompt within each request thread 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 0b0c30fe8b (ffi) remove narrowing type warning 2024-10-21 10:00:27 +02:00
Morgan Funtowicz fb759bdd2a (looper) new looper initial implementation 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 5f7c0b67c3 (ffi) add template specialization to catch and convert to Rust Result<T, tensorrt_llm::common::TllmException> 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 33c962ef41 (ffi) add missing headers imports 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 2883c042ed (ffi) cleanup again 2024-10-21 10:00:27 +02:00
Morgan Funtowicz f4a74be384 (backend) expose PullNewTokens 2024-10-21 10:00:27 +02:00
Morgan Funtowicz b8a40a0af3 (backend) cleanup a bit 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 38b5263c61 (ffi) add max_new_tokens parameters 2024-10-21 10:00:27 +02:00
Morgan Funtowicz f6f689f509 (build) setup ccache if available 2024-10-21 10:00:27 +02:00
Morgan Funtowicz 2a339f99dd (trt) 2024-10-21 10:00:25 +02:00
Morgan Funtowicz 169e1f452f (server) expose new SchedulingError 2024-10-21 10:00:04 +02:00