DVCLive is a Python library for logging machine learning metrics and other metadata in simple file formats, which is fully compatible with DVC.
$ pip install dvclive
$ git init
$ dvc init
$ git commit -m "DVC init"
Copy the snippet below as a basic example of the API usage:
# train.py
import random
import sys
from dvclive import Live
with Live(save_dvc_exp=True) as live:
epochs = int(sys.argv[1])
live.log_param("epochs", epochs)
for epoch in range(epochs):
live.log_metric("train/accuracy", epoch + random.random())
live.log_metric("train/loss", epochs - epoch - random.random())
live.log_metric("val/accuracy",epoch + random.random() )
live.log_metric("val/loss", epochs - epoch - random.random())
live.next_step()
See Integrations for examples using DVCLive alongside different ML Frameworks.
Run couple of times passing different values:
$ python train.py 5
$ python train.py 5
$ python train.py 7
DVCLive outputs can be rendered in different ways:
You can use dvc exp show and dvc plots to compare and visualize metrics, parameters and plots across experiments:
$ dvc exp show
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Experiment Created train.accuracy train.loss val.accuracy val.loss step epochs
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
workspace - 6.0109 0.23311 6.062 0.24321 6 7
master 08:50 PM - - - - - -
โโโ 4475845 [aulic-chiv] 08:56 PM 6.0109 0.23311 6.062 0.24321 6 7
โโโ 7d4cef7 [yarer-tods] 08:56 PM 4.8551 0.82012 4.5555 0.033533 4 5
โโโ d503f8e [curst-chad] 08:56 PM 4.9768 0.070585 4.0773 0.46639 4 5
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
$ dvc plots diff $(dvc exp list --names-only) --open
Inside the DVC Extension for VS Code, you can compare and visualize results using the Experiments and Plots views:
While experiments are running, live updates will be displayed in both views.
If you push the results to DVC Studio, you can compare experiments against the entire repo history:
You can enable Studio Live Experiments to see live updates while experiments are running.
DVCLive is an ML Logger, similar to:
The main difference with those ML Loggers is that DVCLive does not require any additional services or servers to run.
Logged metrics, parameters, and plots are stored as plain text files that can be versioned by tools like Git or tracked as pointers to files in DVC storage.
You can then use different options to visualize the metrics, parameters, and plots across experiments.
Contributions are very welcome. To learn more, see the Contributor Guide.
Distributed under the terms of the Apache 2.0 license, dvclive is free and open source software.