Dask Fundamentals Tutorial for High Performance Computing

Dask is an open-source Python library for parallel/distributed computing.

This tutorial explains two fundamental problem paralleization concepts: domain decomposition and functional decomposition.
It exemplifies how these two concepts can be achieved using Dask.
It exemplifies how Dask enable parallel computing/distributed, out-of-core computations and scalability (from a local machine to cloud computing systems)

This tutorial start at 0.Introduction.

Run this tutorial using your own machine

Create a virtual enviroment using conda or pip.
Install the requirements.txt
Run the jupyterlab enviroment.

Run this tutorial using Vagrant

Vagrant automates the deployment of virtual machines. The Vagrantfile defines a ready to use jupyterlab virtual machine for this tutorial. Use the following commands to deploy the machine.

Install vagrant

sudo apt update
sudo apt install virtualbox
sudo apt install vagrant

Provision the virtual machine

vagrant up

Get into the virtual machine

vagrant ssh

Stop the virtual machine

vagrant ssh

Destroy the virtual machine

vagrant ssh

Use the following url to access the jupyterlab enviroment.

http://192.168.33.10:8000

This tutorial was presented in

HPCSS2023: Colombian HPC Summer School 2023, June 20 - 22, 2023.

Acknowledgements

This project is partially supported by CyberColombia
We thank Chameleon Cloud for its support in compute time.
We thank Coiled for their technical support and compute time provided on their ready-to-use Dask Cloud Computing Platform.
We thank Naty Clementi from Coiled for her support and suggestions to improve this tutorial.

Recommend Projects