hf_text-generation-inference/server/text_generation_server/utils/speculate.py

12 lines
173 B
Python
Raw Permalink Normal View History

2023-12-11 04:46:30 -07:00
SPECULATE = None
2023-12-11 06:49:52 -07:00
2023-12-11 04:46:30 -07:00
def get_speculate() -> int:
global SPECULATE
return SPECULATE
2023-12-11 06:49:52 -07:00
2023-12-11 04:46:30 -07:00
def set_speculate(speculate: int):
global SPECULATE
SPECULATE = speculate