Mirrors
/
hf_text-generation-inference
mirror of https://github.com/huggingface/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

QuantLinear is rocm compatible.

This commit is contained in:
Nicolas Patry 2024-10-23 18:02:50 +08:00
parent 82a6cb82e1
commit 849d8821ab
No known key found for this signature in database
GPG Key ID: 788A1EA699458B2F
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
server/text_generation_server/layers/gptq/__init__.py
View File

@ -10,7 +10,7 @@ from text_generation_server.utils.weights import Weight, Weights, WeightsLoader
if SYSTEM == "ipex":
from .ipex import QuantLinear
elif SYSTEM == "cuda":
elif SYSTEM in {"cuda", "rocm"}:
from .cuda import QuantLinear