mkellerman / gpt4all-ui Goto Github PK
View Code? Open in Web Editor NEWSimple Docker Compose to load gpt4all (Llama.cpp) as an API and chatbot-ui for the web interface. This mimics OpenAI's ChatGPT but as a local instance (offline).
Simple Docker Compose to load gpt4all (Llama.cpp) as an API and chatbot-ui for the web interface. This mimics OpenAI's ChatGPT but as a local instance (offline).
Please fix
Followed the install, placed a model got this:
docker logs gpt4all-ui_api_1 --follow
INFO: Will watch for changes in these directories: ['/app']
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO: Started reloader process [1] using StatReload
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.11/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 59, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 66, in serve
config.load()
File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 471, in load
self.loaded_app = import_from_string(self.app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string
module = importlib.import_module(module_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/app/fastapi_server.py", line 50, in <module>
llama = llama_cpp.Llama(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/llama_cpp/llama.py", line 109, in __init__
raise ValueError(f"Model path does not exist: {model_path}")
ValueError: Model path does not exist: /models/7B/gpt4all-lora-quantized.ggml
Exception ignored in: <function Llama.__del__ at 0x7fb1cbd7b560>
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/llama_cpp/llama.py", line 804, in __del__
if self.ctx is not None:
^^^^^^^^
AttributeError: 'Llama' object has no attribute 'ctx'
Any ideas what to do?
IMO, this works way better than the official UI https://github.com/nomic-ai/gpt4all-ui
Followed everything and getting this error
I've set up a python script locally which contains a gui that allows modification to the compose yml's MODEL env variable, and then restart the service after model change.
Is there a way to switch model without reloading the docker instance?
I guess I need the dockerfile launch another local python script that intitiate the fastapi script as a subprocess.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.