hunoutl/starvz

R-based visualization techniques for the StarPU runtime

StarVZ

StarVZ consists in a performance analysis workflow that combines the power of the R language (and the tidyverse realm) and many auxiliary tools to provide a consistent, flexible, extensible, fast, and versatile framework for the performance analysis of task-based applications that run on top of the StarPU runtime (with its MPI layer for multi-node support). Its goal is to provide a fruitful prototypical environment to conduct performance analysis hypothesis-checking for task-based applications that run on heterogeneous (multi-GPU, multi-core) multi-node HPC platforms.

The source code of this framework is released under the GPLv3 license.

Instalation

# To install the R package
install.packages("devtools")
devtools::install_github("schnorr/starvz")

Non-R Dependencies for the whole workflow

• starpu_fxt_tool: Provided by StarPU
• pj_dump: Provided by PageNG
• rec2csv: Provided by recutils

Quick Usage

Full Workflow - From FxT to Visualization

With starpu_fxt_tool, pj_dump, and rec2csv in $PATH:

export EXP_FOLDER=/folder_to_fxt_files/

export STARVZ_TOOLS=$(Rscript -e 'cat(system.file("tools/", package = "starvz"), sep="\n")')

$STARVZ_TOOLS/starvz $EXP_FOLDER

This generates $EXP_FOLDER/starvz.png

This is equivalent to:

$STARVZ_TOOLS/phase1-workflow.sh $EXP_FOLDER
$STARVZ_TOOLS/phase2-workflow.R $EXP_FOLDER $STARVZ_TOOLS/../etc/default.yaml

The tools will be available after instalation inside the package folder subdirectory tools/.

This can be retrived by:

Rscript -e 'cat(system.file("tools/", package = "starvz"), sep="\n")'

Phase 2 (Visualizations) in R:

library(starvz)
data <- starvz_read("/folder_to_fxt_files/")
plot <- starvz_plot(data)
plot

plot will be a ggplot object that can be saved with ggsave

Extended Content

• Installation Instructions
• Phase 1 Workflow Instructions
• Phase 2 Workflow Instructions

Origin and Publications

A preliminary version of this framework has been released in the companion website (check the reproducible paper link below) of the VPA 2016 workshop (held during the SC16 conference). A second release of the framework is available in the companion website of an accepted article to Wiley’s Concurrent and Computation: Practice and Experience. A memory analysis extension was developed and discussed in the third publication at CCGRID 2019.

• Analyzing Dynamic Task-Based Applications on Hybrid Platforms: An Agile Scripting Approach, Vinicius Garcia Pinto, Luka Stanisic, Arnaud Legrand, Lucas Mello Schnorr, Samuel Thibault, Vincent Danjean, in 2016 Third Workshop on Visual Performance Analysis (VPA@SC), Salt Lake City, UT, 2016, pp. 17-24.
  • DOI and Reproducible Paper
• A Visual Performance Analysis Framework for Task-based Parallel Applications running on Hybrid Clusters, Vinicius Garcia Pinto, Lucas Mello Schnorr, Luka Stanisic, Arnaud Legrand, Samuel Thibault, Vincent Danjean, in Concurrency and Computation: Practice and Experience, Wiley, 2018, 30 (18), pp.1-31.
  • DOI, Draft, and Companion website
• Visual Performance Analysis of Memory Behavior in a Task-Based Runtime on Hybrid Platforms, Lucas Leandro Nesi, Samuel Thibault, Luka Stanisic and Lucas Mello Schnorr, in 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), Larnaca, Cyprus, 2019, pp. 142-151.
  • DOI and Companion website

Docker container

Please check this DockerFile to create a docker container with all the necessary requirements for a basic utilization of the starvz framework (in the form of an R package). Assuming that you have docker installed in your system, you may want to simply pull and run this container from Docker Hub, like this:

docker pull schnorr/starvz
docker run -it schnorr/starvz

After entering the container, run R and load the starvz package with:

library(starvz)

Team and Contact

• Lucas Mello Schnorr ([email protected])
• Vinicius Garcia Pinto ([email protected])
• Lucas Leando Nesi ([email protected])
• Marcelo Cogo Miletto ([email protected])

Getting help

Fell free to post an issue here in GitHub.