Truncate error text to fix service lockup / stall

What:
* Update wrap_gradio_call to add a limit to the maximum amount of text output

Why:
* wrap_gradio_call currently prints out a list of the arguments provided to the failing function.
   * if that function is save_image, this causes the entire image to be printed to stderr
    * If the image is large, this can cause the service to lock up while attempting to print all the text
 * It is easy to generate large images using the x/y plot script
 * it is easy to encounter image save exceptions, including if the output directory does not exist / cannot be written to, or if the file is too big
  * The huge amount of log spam is confusing and not particularly helpful
This commit is contained in:
Greg Fuller 2022-10-11 19:29:38 -07:00 committed by AUTOMATIC1111
parent 6ac2ec2b78
commit fec2221eea
1 changed files with 8 additions and 1 deletions

View File

@ -181,8 +181,15 @@ def wrap_gradio_call(func, extra_outputs=None):
try: try:
res = list(func(*args, **kwargs)) res = list(func(*args, **kwargs))
except Exception as e: except Exception as e:
# When printing out our debug argument list, do not print out more than a MB of text
max_debug_str_len = 131072 # (1024*1024)/8
print("Error completing request", file=sys.stderr) print("Error completing request", file=sys.stderr)
print("Arguments:", args, kwargs, file=sys.stderr) argStr = f"Arguments: {str(args)} {str(kwargs)}"
print(argStr[:max_debug_str_len], file=sys.stderr)
if len(argStr) > max_debug_str_len:
print(f"(Argument list truncated at {max_debug_str_len}/{len(argStr)} characters)", file=sys.stderr)
print(traceback.format_exc(), file=sys.stderr) print(traceback.format_exc(), file=sys.stderr)
shared.state.job = "" shared.state.job = ""