Trying to use local llamafile

[meshula]

Trying to use local llamafile

[meshula]

Ah, that's simple, thank you :) It didn't occur to me that not using https was an option because the instructions mention https. The fact that it should be https is a little bit obvious from the log. I should have mentioned the instructions I was following are here: https://docs.openinterpreter.com/guides/running-locally

interpreter --api_base http://localhost:8000/v1 -m llamafile -ak dummykey

is working for me.

[Notnaton]

You can add -m llamafile -ak dummykey, but it doesn't really matter. The api base makes sure it connects to the llamafile.
Interpreter adds openai to the model name for now to ensure it uses openai format.

[meshula]

I read the crash log more carefully (sorry I didn't mention it previously). I think I might have an incompatible urllib version or something like that. I notice the log also says offline=false. Maybe the llamafile itself isn't serving https. I'm just speculating...

pip show urllib3
Name: urllib3
Version: 2.2.0
Summary: HTTP library with thread-safe connection pooling, file post, and more.
Home-page:
Author:
Author-email: Andrey Petrov <[email protected]>
License:
Location: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages
Requires:
Required-by: requests

interpreter --api_base https://localhost:8080/v1 -m llamafile -ak dummykey

▌ Model set to openai/llamafile

Open Interpreter will require approval before running code.

Use interpreter -y to bypass this.

Press CTRL-C to exit.

> test

We were unable to determine the context window of this model. Defaulting to 3000.

If your model can handle more, run interpreter --context_window {token limit} --max_tokens {max tokens per response}.

Continuing...


        Python Version: 3.10.11
        Pip Version: 24.0
        Open-interpreter Version: cmd:Interpreter, pkg: 0.2.0
        OS Version and Architecture: macOS-14.3-arm64-arm-64bit
        CPU Info: arm
        RAM Info: 128.00 GB, used: 59.67, free: 5.55


        # Interpreter Info

        Vision: False
        Model: openai/llamafile
        Function calling: None
        Context window: None
        Max tokens: None

        Auto run: False
        API base: https://localhost:8080/v1
        Offline: False

        Curl output: Not local

        # Messages

        System Message: You are Open Interpreter, a world-class programmer that can complete any goal by executing code.
First, write a plan. **Always recap the plan between each code block** (you have extreme short-term memory loss, so you need to recap the plan between each message block to retain it).
When you execute code, it will be executed **on the user's machine**. The user has given you **full and complete permission** to execute any code necessary to complete the task. Execute the code.
If you want to send data between programming languages, save the data to a txt or json.
You can access the internet. Run **any code** to achieve the goal, and if at first you don't succeed, try again and again.
You can install new packages.
When a user refers to a filename, they're likely referring to an existing file in the directory you're currently executing code in.
Write messages to the user in Markdown.
In general, try to **make plans** with as few steps as possible. As for actually executing code to carry out that plan, for *stateful* languages (like python, javascript, shell, but NOT for html which starts from 0 every time) **it's critical not to try to do everything in one code block.** You should try something, print information about it, then continue from there in tiny, informed steps. You will never get it on the first try, and attempting it in one go will often lead to errors you cant see.
You are capable of **any** task.

        {'role': 'user', 'type': 'message', 'content': 'test'}


Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py", line 67, in map_httpcore_exceptions
    yield
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py", line 231, in handle_request
    resp = self._pool.handle_request(req)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_sync/connection.py", line 99, in handle_request
    raise exc
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_sync/connection.py", line 76, in handle_request
    stream = self._connect(request)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_sync/connection.py", line 154, in _connect
    stream = stream.start_tls(**kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_backends/sync.py", line 152, in start_tls
    with map_exceptions(exc_map):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1007)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_base_client.py", line 918, in _request
    response = self._client.send(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 915, in send
    response = self._send_handling_auth(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 943, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 980, in _send_handling_redirects
    response = self._send_single_request(request)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1016, in _send_single_request
    response = transport.handle_request(request)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py", line 230, in handle_request
    with map_httpcore_exceptions():
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py", line 84, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1007)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/litellm/llms/openai.py", line 361, in completion
    raise e
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/litellm/llms/openai.py", line 283, in completion
    return self.streaming(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/litellm/llms/openai.py", line 461, in streaming
    response = openai_client.chat.completions.create(**data, timeout=timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_utils/_utils.py", line 275, in wrapper
    return func(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 663, in create
    return self._post(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_base_client.py", line 1200, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_base_client.py", line 889, in request
    return self._request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_base_client.py", line 942, in _request
    return self._retry_request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_base_client.py", line 1013, in _retry_request
    return self._request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_base_client.py", line 942, in _request
    return self._retry_request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_base_client.py", line 1013, in _retry_request
    return self._request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/openai/_base_client.py", line 952, in _request
    raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.

.... repeats many times

[Notnaton]

Change --api_base http://localhost:8080/v1
You are currently using https not http, so you get an ssl error.