paperspace/ml-in-a-box

Machine learning tool-set for Paperspace VMs

ML-in-a-box

This repository contains details for the Paperspace ML-in-a-box environments. Paperpsace supports both Ubuntu 22.04 and Ubuntu 20.04, each in separate directories.

Ubuntu 22.04 version is required for H100 machines and recommended for A100s. Ubuntu 20.04 version is recommended for any other machine type available on Paperspace.

These distinctions are mainly due to Nvidia driver versions needed for different generations of Nvidia hardware.

These templates can be used to spin up a Core Machine on Paperspace.

ml-in-a-box's Issues

Possible to bundle Ollama?

Hi,

I'm one of the maintainers of Ollama. We'll be releasing an official Linux build with Nvidia GPU support out of the box.

Would this be something of interest?

Any particular reason for the removal of Conda?

I'll admit that I'm not the biggest fan of Conda either, but in light of the fact that there are so many open-source LLMs out there that swear by conda as as a more efficient means of fine tuning models (vs running the explicit commands), this seems to be a godsend on that front.

Perhaps there's something that I'm missing. Because I was planning on downloading conda purely for the sake of convenience when running the appropriate code to fine tune the model once its been put up in the air.

Is this safe? Are there any latent issues (code-based, legal or otherwise), that you all came across that prompted you to remove conda? If so, please let me know! I'd appreciate that so much.

Thank you.

Is there a way to upgrade Cuda drivers installed?

I am getting the following warning:

UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 11070). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)

New MLiaB PyTorch is compiled for CUDA 11.6

The new MLiaB is CUDA 11.7 but the PyTorch 1.12.1 is compiled for CUDA 11.6.

This is an artifact of the time between when the MLiaB Docker was created and the current PyTorch 1.13.1 which defaults to 11.7.

Code that has strict checking for this may fail, e.g.,

Exception: Installed CUDA version 11.7 does not match the version torch was compiled with 11.6, unable to compile cuda/cpp extensions without a matching cuda version.

In this case it was solved by upgrading torch to 1.13.1, but we are noting the issue here as a generic case.

Currently not working

This is on a totally new install on a new instance

currently not working

totally new system, cannot use gpu in pytorch