Mirrors
/
oai-reverse-proxy
mirror of https://gitgud.io/khanon/oai-reverse-proxy.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Reverse proxy server for various LLM APIs. Features translation between API formats, user management, anti-abuse, API key rotation, DALL-E support, and optional prompt/response logging.
191 Commits 11 Branches 0 Tags 1.8 MiB
TypeScript 96.5%
EJS 3.2%
Dockerfile 0.2%
Go to file
nai-degen 858a619ae2 fixes typecheck issue after rebasing 2023-07-19 11:21:53 -05:00
docker cleanup 2023-07-19 11:21:05 -05:00
docs updates docs 2023-07-19 11:20:17 -05:00
scripts adds python deps install script 2023-07-19 11:20:17 -05:00
src fixes typecheck issue after rebasing 2023-07-19 11:21:53 -05:00
.env.example Separate `MAX_OUTPUT_TOKENS` config for OpenAI/Claude (khanon/oai-reverse-proxy!21) 2023-06-08 00:58:57 +00:00
.gitattributes initial commit 2023-04-08 01:54:44 -05:00
.gitignore updates docs and README for Claude tokenizer 2023-07-19 11:17:56 -05:00
README.md updates docs 2023-07-19 11:20:17 -05:00
esbuild.js tries to fix huggingface docker build issues 2023-07-19 11:19:04 -05:00
package-lock.json cleanup 2023-07-19 11:21:05 -05:00
package.json tries newer version of zmq again 2023-07-19 11:21:05 -05:00
render.yaml Add docs and support for Render.com deployments (khanon/oai-reverse-proxy!9) 2023-05-15 21:47:30 +00:00
requirements.txt updates docs and README for Claude tokenizer 2023-07-19 11:17:56 -05:00
tsconfig.json downgrades zmq implementation for v5.x 2023-07-19 11:21:05 -05:00

README.md

OAI Reverse Proxy

Reverse proxy server for the OpenAI and Anthropic APIs. Forwards text generation requests while rejecting administrative/billing requests. Includes optional rate limiting and prompt filtering to prevent abuse.

Table of Contents

What is this?

If you would like to provide a friend access to an API via keys you own, you can use this to keep your keys safe while still allowing them to generate text with the API. You can also use this if you'd like to build a client-side application which uses the OpenAI or Anthropic APIs, but don't want to build your own backend. You should never embed your real API keys in a client-side application. Instead, you can have your frontend connect to this reverse proxy and forward requests to the downstream service.

This keeps your keys safe and allows you to use the rate limiting and prompt filtering features of the proxy to prevent abuse.

Why?

OpenAI keys have full account permissions. They can revoke themselves, generate new keys, modify spend quotas, etc. You absolutely should not share them, post them publicly, nor embed them in client-side applications as they can be easily stolen.

This proxy only forwards text generation requests to the downstream service and rejects requests which would otherwise modify your account.

Usage Instructions

If you'd like to run your own instance of this proxy, you'll need to deploy it somewhere and configure it with your API keys. A few easy options are provided below, though you can also deploy it to any other service you'd like.

Deploy to Huggingface (Recommended)

See here for instructions on how to deploy to a Huggingface Space.

Deploy to Render

See here for instructions on how to deploy to Render.com.

Local Development

To run the proxy locally for development or testing, install Node.js >= 18.0.0 and follow the steps below.

  1. Clone the repo
  2. Install dependencies with npm install
  3. Create a .env file in the root of the project and add your API keys. See the .env.example file for an example.
  4. Start the server in development mode with npm run start:dev.

You can also use npm run start:dev:tsc to enable project-wide type checking at the cost of slower startup times. npm run type-check can be used to run type checking without starting the server.

See the Optional Dependencies page for information on how to install the optional Claude tokenizer locally.