Missing model name for 3.5 turbo Azure Deployment model

[dmitry-brazhenko]

Missing model name for 3.5 turbo Azure Deployment model

[bsstahl]

FYI - It looks like TikToken added it as an Azure deployment name:

    "gpt-3.5-turbo": "cl100k_base",
    "gpt-35-turbo": "cl100k_base",  # Azure deployment name

[winzig]

The official name of the model from OpenAI's API does include the period, e.g.:

• gpt-3.5-turbo
• gpt-3.5-turbo-0301
• gpt-3.5-turbo-16k-0613
• etc...

If you hit https://api.openai.com/v1/models (when authenticated) you can see all available models.

[bsstahl]

@winzig Is there a reason the Azure deployment name would be included in TikToken but not here? I thought this was intended as a port of that library.

[bsstahl]

I've edited the original comment so it is clear that the Azure Deployment name was added after the original creation of TikToken.

[dmitry-brazhenko]

We can add gpt-35-turbo as well. I do not see any problems with it