{{ analytics_tracking_code|safe }}

{{ llm_middleware_name }}

Current Model: {{ default_model }}

Estimated Wait Time: {{ default_estimated_wait }}
Processing: {{ default_active_gen_workers }}
Queued: {{ default_proompters_in_queue }}


Client API URL: {{ client_api }}

Streaming API URL: {{ ws_client_api if enable_streaming else 'Disabled' }}

OpenAI-Compatible API URL: {{ openai_client_api }}

{% if info_html|length > 1 %}
{{ info_html|safe }} {% endif %}

Instructions

  1. In Settings > Power User Options, enable Relaxed API URLS
  2. Set your API type to {{ mode_name }}
  3. Enter {{ client_api }} in the {{ api_input_textbox }} textbox.
    4. {% if enable_streaming %}
  4. Enter {{ ws_client_api }} in the {{ streaming_input_textbox }} textbox.
    5. {% endif %}
  5. If you have a token, check the Mancer AI checkbox and enter your token in the Mancer API key textbox.
  6. Click Connect to test the connection.
  7. Open your preset config and set Context Size to {{ default_context_size }}.
  8. Follow this guide to get set up: rentry.org/freellamas
{% if openai_client_api != 'disabled' and expose_openai_system_prompt %}
OpenAI-Compatible API

The OpenAI-compatible API adds a system prompt to set the AI's behavior to a "helpful assistant". You can view this prompt here.

{% endif %}
{{ extra_info|safe }}

Statistics

Proompters:

{% for key, value in model_choices.items() %}

{{ key }} - {{ value.backend_count }} workers

Estimated Wait Time: {{ value.estimated_wait }}
Processing: {{ value.processing }}
Queued: {{ value.queued }}

Client API URL: {{ value.client_api }}
Streaming API URL: {{ value.ws_client_api }}
OpenAI-Compatible API URL: {{ value.openai_client_api }}

Context Size: {{ value.context_size }}

Average Generation Time: {{ value.avg_generation_time | int }} seconds


{% endfor %}
git.evulid.cc/cyberes/local-llm-server