diemus/azure-openai-proxy

A proxy for Azure OpenAI API that can convert an OpenAI request into an Azure OpenAI request.

Azure OpenAI Proxy

Introduction

English | 中文

Azure OpenAI Proxy is a proxy for Azure OpenAI API that can convert an OpenAI request to an Azure OpenAI request. It is designed to use as a backend for various open source ChatGPT web project. It also supports being used as a simple OpenAI API proxy to solve the problem of OpenAI API being restricted in some regions.

Highlights:

• 🌐 Supports proxying all Azure OpenAI APIs
• 🧠 Supports proxying all Azure OpenAI models and custom fine-tuned models
• 🗺️ Supports custom mapping between Azure deployment names and OpenAI models
• 🔄 Supports both reverse proxy and forward proxy usage
• 👍 Support mocking of OpenAI APIs that are not supported by Azure.

Supported APIs

The latest version of the Azure OpenAI service currently supports the following 3 APIs:

Path	Status
/v1/chat/completions	✅
/v1/completions	✅
/v1/embeddings	✅

Other APIs not supported by Azure will be returned in a mock format (such as OPTIONS requests initiated by browsers). If you find your project need additional OpenAI-supported APIs, feel free to submit a PR.

Recently Updated

• 2023-04-06 supported /v1/models interface, fixed the issue of some web projects depending on models interface error.
• 2023-04-04 supported options interface, fixed the cross-domain check error issue of some web projects.

Usage

1. Used as reverse proxy (i.e. an OpenAI API gateway)

Environment Variables

Parameters	Description	Default Value
AZURE_OPENAI_PROXY_ADDRESS	Service listening address	0.0.0.0:8080
AZURE_OPENAI_PROXY_MODE	Proxy mode, can be either "azure" or "openai".	azure
AZURE_OPENAI_ENDPOINT	Azure OpenAI Endpoint, usually looks like https://{custom}.openai.azure.com. Required.
AZURE_OPENAI_APIVERSION	Azure OpenAI API version. Default is 2023-03-15-preview.	2023-03-15-preview
AZURE_OPENAI_MODEL_MAPPER	A comma-separated list of model=deployment pairs. Maps model names to deployment names. For example, gpt-3.5-turbo=gpt-35-turbo, gpt-3.5-turbo-0301=gpt-35-turbo-0301. If there is no match, the proxy will pass model as deployment name directly (in fact, most Azure model names are same with OpenAI).	gpt-3.5-turbo=gpt-35-turbo gpt-3.5-turbo-0301=gpt-35-turbo-0301
AZURE_OPENAI_TOKEN	Azure OpenAI API Token. If this environment variable is set, the token in the request header will be ignored.	""

Use in command line

curl https://{your-custom-domain}/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {your azure api key}" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

2. Used as forward proxy (i.e. an HTTP proxy)

When accessing Azure OpenAI API through HTTP, it can be used directly as a proxy, but this tool does not have built-in HTTPS support, so you need an HTTPS proxy such as Nginx to support accessing HTTPS version of OpenAI API.

Assuming that the proxy domain you configured is https://{your-domain}.com, you can execute the following commands in the terminal to use the https proxy:

export https_proxy=https://{your-domain}.com

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {your azure api key}" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Or configure it as an HTTP proxy in other open source Web ChatGPT projects:

export HTTPS_PROXY=https://{your-domain}.com

Deploy

Deploying through Docker

docker pull ishadows/azure-openai-proxy:latest
docker run -d -p 8080:8080 --name=azure-openai-proxy \
  --env AZURE_OPENAI_ENDPOINT={your azure endpoint} \
  --env AZURE_OPENAI_MODEL_MAPPER={your custom model mapper ,like: gpt-3.5-turbo=gpt-35-turbo,gpt-3.5-turbo-0301=gpt-35-turbo-0301} \
  ishadows/azure-openai-proxy:latest

Calling

curl https://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {your azure api key}" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Model Mapping Mechanism

There are a series of rules for model mapping pre-defined in AZURE_OPENAI_MODEL_MAPPER, and the default configuration basically satisfies the mapping of all Azure models. The rules include:

• gpt-3.5-turbo -> gpt-35-turbo
• gpt-3.5-turbo-0301 -> gpt-35-turbo-0301
• A mapping mechanism that pass model name directly as fallback.

For custom fine-tuned models, the model name can be passed directly. For models with deployment names different from the model names, custom mapping relationships can be defined, such as:

Model Name	Deployment Name
gpt-3.5-turbo	gpt-35-turbo-upgrade
gpt-3.5-turbo-0301	gpt-35-turbo-0301-fine-tuned

License

MIT

Star History

azure-openai-proxy's Issues

Stream 中文字符截斷亂碼問題。

問題重現與其他語言解決方案參考:
發生與此專案相同問題 ChatGPT-Next-Web

不知道 Golang 是否有辦法解決 Stream 中文字符截斷的問題。

在vercel里部署有什么特别的设置吗？

在vercel部署了一个，用默认的build配置，环境变量除了KEY没设置，其它都按文档里的配置了 https://azure-api-proxy.vercel.app/

测试语句(Key作了mask)，
curl https://azure-api-proxy.vercel.app/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer c4b7de425xxxxxxxxxxxxxxxxxxxxxxx" -d '{ "model": "gpt-35-turbo", "messages": [{"role": "user", "content": "Hello!"}]}'

返回，

The page could not be found

NOT_FOUND

是哪里没有做对吗？

#6 没有打印机效果，这个问题还没好。

我现在卡顿一会，一次返回一大段。 没有打印机效果。

support gpt-4

老师您好，请问一下咱们现在这套代码 能支持azure上的gpt4 的模型吗？

api key 使用问题

我想将作者的新项目中的apikey应用到另一个web项目上，直接使用该apikey会出现报错问题。

无代理无法调用azure openai api

在本地服务器中部署该项目的docker之后出现fetch error，采用postman按azure文档直接调用azure openai api也失败，是否现在不用代理就无法直接调用azure openai api呢？

/v1/models, unsupported request, body is empty, http: proxy error: unsupported protocol scheme ""

Hi,
I am trying use the proxy with chatBot-UI (https://github.com/mckaywrigley/chatbot-ui). The proxy is running and chatBot-UI appears to be sending it requests, but I am getting a proxy log error on opening chatBot-UI:
azure-openai-proxy error:

[GIN] 2023/04/04 - 07:30:12 | 502 |     123.423µs |      xxx.xxx.x.x | GET      "/v1/models"
2023/04/04 07:30:12 unsupported request, body is empty
2023/04/04 07:30:12 http: proxy error: unsupported protocol scheme ""

chatbot-UI error:

Error fetching models.
Make sure your OpenAI API key is set in the bottom left of the sidebar.
If you completed this step, OpenAI may be experiencing issues.

Any suggestions what I may be doing wrong?

无法开启流式，没有打字机效果

与ChatGPT-Next-Web连接，能访问，但是无法开启流式，没有打字机效果。有没有解决的办法？

不支持/v1/engines接口

测了下，不支持类似/v1/engines/text-embedding-ada-002/embeddings接口

Embeddings数组超过1个元素时微软接口会报错Too many inputs. The max number of inputs is 1.

用如下方式访问时

curl https://xxx.openai.azure.com/openai/deployments/yyy/embeddings?api-version=2023-03-15-preview  \
  -H "Content-Type: application/json" \
  -H "api-key: 1112222" \
  -d '{
      "input": ["1","2"]
    }'

openai的接口没这个问题，可以传递多个元素，但用一些库比如llama-index的时候会自动传递多个元素过来，当input的这个数组有超过2个元素时微软接口会报错，返回Too many inputs. The max number of inputs is 1错误
解决办法：
把数组的元素全部拼接起来成一个str，微软官方说法是input最大可以到8k
详细可以看微软的官方说明：
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference

能适配下poe的api吗

我重新适配了最新的api接口,能适配下poe吗

部分应用会在请求时发送 Options 预检导致无法成功代理

非常感谢作者的这个 Repo，不过遇到了 Options 请求的问题，请问这个有办法绕过吗？

请问是否支持指定deployment name？

试了下提示： "code": "DeploymentNotFound"

看日志：
/v1/chat/completions -> https://gpt-19354567-test.openai.azure.com/openai/deployments/chat/completions?api-version=2023-05-15

实际我的Azure接口是，url少个/test/

curl --location 'https://gpt-19354567-test.openai.azure.com/openai/deployments/test/chat/completions?api-version=2023-05-15' \
--header 'Content-Type: application/json' \
--header 'api-key: api-key' \
--data '{  "stream":true,
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
       
        {
            "role": "user",
            "content": "Hello"
        }
    ]
}'

请问这个deployments后面的部署名称可以自定义吗？/deployments/test/

請問預設是有 https 的嗎？

mac interl/arm64

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
8cd2e5ec29edadb5e425e2898b7d358261ab913dd6e887ad1fe095eefc8bc9cc

可以支援 v1/images 嗎

如果有支援 v1/images 就太好了

{ "statusCode": 500, "message": "Internal server error", "activityId": "xxxxxxxxxxxxxxxx" }

已经稳定使用了很多天。但是最近突然出现这个错误。
例如，

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxx" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

得到的回复：
{ "statusCode": 500, "message": "Internal server error", "activityId": "xxxxxxxxxxxxxxxx" }
使用的是香港的azure服务器。

奇怪的是，最开始的两到三次重启docker之后，可以发送一次信息得到正常的回应，之后还是500错误。
在两到三次之后，重启docker也不能正常使用。全部500.