Mirrors
/
hf_text-generation-inference
mirror of https://github.com/huggingface/text-generation-inference.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fixing Phi3.

This commit is contained in:
Nicolas Patry 2024-06-01 08:47:00 +00:00
parent 08b3eac2ce
commit 799a193b10
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
server/text_generation_server/models/custom_modeling/flash_llama_modeling.py
View File

@ -52,7 +52,8 @@ if SYSTEM == "rocm":
def load_attention(config, prefix, weights): def load_attention(config, prefix, weights):
bias = config.attention_bias # Only defined in granite.
bias = getattr(config, "attention_bias", False)
# if specific model type, load the correct attention # if specific model type, load the correct attention
if config.model_type == "phi3": if config.model_type == "phi3":