Truncate error text to fix service lockup / stall
What: * Update wrap_gradio_call to add a limit to the maximum amount of text output Why: * wrap_gradio_call currently prints out a list of the arguments provided to the failing function. * if that function is save_image, this causes the entire image to be printed to stderr * If the image is large, this can cause the service to lock up while attempting to print all the text * It is easy to generate large images using the x/y plot script * it is easy to encounter image save exceptions, including if the output directory does not exist / cannot be written to, or if the file is too big * The huge amount of log spam is confusing and not particularly helpful
This commit is contained in:
parent
6ac2ec2b78
commit
fec2221eea
|
@ -181,8 +181,15 @@ def wrap_gradio_call(func, extra_outputs=None):
|
||||||
try:
|
try:
|
||||||
res = list(func(*args, **kwargs))
|
res = list(func(*args, **kwargs))
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
# When printing out our debug argument list, do not print out more than a MB of text
|
||||||
|
max_debug_str_len = 131072 # (1024*1024)/8
|
||||||
|
|
||||||
print("Error completing request", file=sys.stderr)
|
print("Error completing request", file=sys.stderr)
|
||||||
print("Arguments:", args, kwargs, file=sys.stderr)
|
argStr = f"Arguments: {str(args)} {str(kwargs)}"
|
||||||
|
print(argStr[:max_debug_str_len], file=sys.stderr)
|
||||||
|
if len(argStr) > max_debug_str_len:
|
||||||
|
print(f"(Argument list truncated at {max_debug_str_len}/{len(argStr)} characters)", file=sys.stderr)
|
||||||
|
|
||||||
print(traceback.format_exc(), file=sys.stderr)
|
print(traceback.format_exc(), file=sys.stderr)
|
||||||
|
|
||||||
shared.state.job = ""
|
shared.state.job = ""
|
||||||
|
|
Loading…
Reference in New Issue