dstack
is an open-source engine for running GPU workloads on any cloud.
It works with a wide range of cloud GPU providers (AWS, GCP, Azure, Lambda, TensorDock, Vast.ai, etc.)
as well as on-premises servers.
- [2024/03] dstack 0.16.1: Improvements to
dstack pool
and bug-fixes (Release) - [2024/02] dstack 0.16.0: Pools (Release)
- [2024/02] dstack 0.15.1: Kubernetes integration (Release)
- [2024/01] dstack 0.15.0: Resources, authentication, and more (Release)
- [2024/01] dstack 0.14.0: OpenAI-compatible endpoints (Release)
- [2023/12] dstack 0.13.0: Disk size, CUDA 12.1, Mixtral, and more (Release)
Before using dstack
through CLI or API, set up a dstack
server.
The easiest way to install the server, is via pip
:
pip install "dstack[all]" -U
If you have default AWS, GCP, or Azure credentials on your machine, the dstack
server will pick them up automatically.
Otherwise, you need to manually specify the cloud credentials in ~/.dstack/server/config.yml
.
For further details on setting up the server, refer to installation.
To start the server, use the dstack server
command:
$ dstack server
Applying ~/.dstack/server/config.yml...
The admin token is "bbae0f28-d3dd-4820-bf61-8f4bb40815da"
The server is running at http://127.0.0.1:3000/
Note It's also possible to run the server via Docker.
Once the server is up, you can use either dstack
's CLI or API to run workloads.
Below is a live demo of how it works with the CLI.
You specify the required environment and resources, then run it. dstack provisions the dev environment in the cloud and enables access via your desktop IDE.
Tasks allow for convenient scheduling of any kind of batch jobs, such as training, fine-tuning, or data processing, as well as running web applications.
Specify the environment and resources, then run it. dstack executes the task in the cloud, enabling port forwarding to your local machine for convenient access.
Services make it very easy to deploy any kind of model or web application as public endpoints.
Use any serving frameworks and specify required resources. dstack deploys it in the configured backend, handles authentication, and provides an OpenAI-compatible interface if needed.
Pools simplify managing the lifecycle of cloud instances and enable their efficient reuse across runs.
You can have instances provisioned in the cloud automatically, or add them manually, configuring the required resources, idle duration, etc.
Here are some featured examples:
Browse dstack.ai/examples for more examples.
For additional information and examples, see the following links: