Mirrors
/
hf_text-generation-inference
mirror of https://github.com/huggingface/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

fix: use path inside of speculator config (#1935) 

This PR access the path on the speculator similar to
`MLPSpeculatorHead.load` and `MedusaHeadV1.load`


these changes resolves this error locally when loading a `MedusaHeadV2`
```
TypeError: expected str, bytes or os.PathLike object, not dict
```
This commit is contained in:
drbh 2024-05-22 14:46:29 -04:00 committed by GitHub
parent 2f243a1a15
commit efb73fcb59
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
server/text_generation_server/layers/medusa.py
View File

@ -111,10 +111,10 @@ class MedusaHeadV2(nn.Module):
from safetensors import safe_open from safetensors import safe_open
import json import json
speculator = config.speculator speculator_path = config.speculator["path"]
medusa_config = str(Path(speculator) / "config.json") medusa_config = str(Path(speculator_path) / "config.json")
filename = str(Path(speculator) / "medusa_lm_head.safetensors") filename = str(Path(speculator_path) / "medusa_lm_head.safetensors")
with open(medusa_config, "r") as f: with open(medusa_config, "r") as f:
medusa_config = json.load(f) medusa_config = json.load(f)