No access to claude api key, can't use

[smol-ai]

No access to claude api key, can't use

[swyxio]

1. you dont need it for the base experience, claude key is for the generated chrome extension
2. i'm working on a hosted thing that can hopefully share keys...

[BigwigsNFT]

Any ideas. This is great but without claude the token issue prevents from using it. Tried chunking and other options but no luck.

[jdingus]

Same here I'm assuming the context size from Claude is the defining factor.

openai.error.InvalidRequestError: This model's maximum context length is 8192 tokens. However, your messages resulted in 9037 tokens. Please reduce the length of the messages.

Seen some videos with people having some really detailed prompt.md files.

[swyxio]

yeah unfortunately I cant do much about claude access :/

[ishaan-jaff]

Hi @jdingus @d3287t328 @BigwigsNFT I believe I can help with this issue. I’m the maintainer of LiteLLM https://github.com/BerriAI/litellm - we allow you to use any LLM as a drop in replacement for gpt-3.5-turbo.

You can use LiteLLM in the following ways:

With your own API KEY:

This calls the provider API directly

from litellm import completion
import os
## set ENV variables 
os.environ["OPENAI_API_KEY"] = "your-key" # 
os.environ["COHERE_API_KEY"] = "your-key" # 

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages=messages)

Using the LiteLLM Proxy with a LiteLLM Key

this is great if you don’t have access to claude but want to use the open source LiteLLM proxy to access claude

from litellm import completion
import os

## set ENV variables 
os.environ["OPENAI_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your openai key
os.environ["COHERE_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your cohere key

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages=messages)