rookasrudzianskas/boomx

An engine for running models in the cloud

BoomX: Containers for machine learning

BoomX is an open-source tool that lets you package machine learning models in a standard, production-ready container.

Process

boomx's Issues

Docker push after cog build routinely fails due to timeout

When attempting to push my boomx-built docker image to docker hub, I'm consistently getting the error use of closed network connection. This seems to be a timeout issue due to the upload being for such a large image (> 10 GB). I'm not sure if this is a problem others are running into when uploading images built with boomx and if so if there may be any way to address in the future.

Server reports "Already running a prediction" during setup

It should say it's doing setup and isn't ready, instead!

Use Docker Remote AP

We switched to entirely using the Docker CLI to interact with Docker, instead of using the Golang API. It's remarkably hard to do even basic things (docker run, build an image with buildkit, etc).

However, this is brittle because it is an unversioned API and we need to concatenate strings to pass input.

At some point, we should switch to using the Golang API. Buildx might make it easier to do builds.

End to end tests should clean up after themselves

Cleanup
The end to end tests should clean up, in order to clean the space, and save the active memory usage.

Unpickling errors

Our BOOMX fails on very peculiar issue related to unpickling which isn't very clear. Wondering if anybody has faced this and what the resolution was?

jsonPayload: {
exception: "Traceback (most recent call last):
  File "/root/.pyenv/versions/3.10.10/lib/python3.10/site-packages/cog/server/runner.py", line 113, in handle_error
    raise error
  File "/root/.pyenv/versions/3.10.10/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/root/.pyenv/versions/3.10.10/lib/python3.10/site-packages/cog/server/runner.py", line 334, in predict
    return _predict(
  File "/root/.pyenv/versions/3.10.10/lib/python3.10/site-packages/cog/server/runner.py", line 370, in _predict
    for event in worker.predict(input_dict, poll=0.1):
  File "/root/.pyenv/versions/3.10.10/lib/python3.10/site-packages/cog/server/worker.py", line 118, in _wait
    ev = self._events.recv()
  File "/root/.pyenv/versions/3.10.10/lib/python3.10/multiprocessing/connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
_pickle.UnpicklingError: invalid load key, '\x00'."
logger: "server.runner"
message: "caught exception while running prediction"
timestamp: "2023-04-05T08:19:51.798365Z"
}

For context, we're calling a command line utility from subprocess.run() inside the BOOMX. Wondering if it could cause issues. Furthermore, this issue is intermittent... so very sneaky to debug. Thanks!