britishgeologicalsurvey/ash-model-plotting

Wrapper around Iris Python library for easy plotting of NAME, FALL3D and HYSPLIT model results.

ash-model-plotting's Issues

Setup logging in installed plot_ash_model_result

Summary

As a user I want logging when I call plot_ash_model_result so I can see what is happening, but I don't get it because of a bug.

Description

There is a bug because the installed version of plot_ash_model_results calls main() but the logging setup happens in if __name__ == '__main__'.

main() should be updated to call a setup_logging function before the plotting call.

Acceptance criteria

Install with pip install -e .

• Log messages appear when calling plot_ash_model_results for plotting

Parallel processing for plots

Summary

We would like parallel processing of the plots on order to speed up the plotting process.

Description

Plotting the data results in tens or hundreds of maps. These are independent of each other. The script could be made faster by using parallel processing. On a multi-core machine, this may give a speed up of around 3x.

There are a few strategies for parallel processing in Python: threading, multiprocessing and asyncio. These should be tried and compared. There is a good overview of these at: https://realpython.com/python-concurrency/

In the course of this work, it my turn out to be helpful to refactor plot_4d_cube to call plot_3d_cube to reduce repetition.

An overview of code profiling is found at: https://realpython.com/python-timer/

Acceptance criteria

• Parallel processing methods are investigated and described on Wiki page
• Benchmark comparisons are made using different methods
• Plotting stage of NAME script uses parallel processing

For parallel processing use the number of processes specified by SLURM

Summary

As a cluster user, I want to use the number of processes allocated by SLURM so that I don't slow the machine for others.

Description

By default, ash_model_plotting sets up the multiprocessing pool as Pool(), without specifying the number of processes. This makes it default to the number of processors on the machine.

We can specify an number e.g. Pool(2).

In plotting.py we can add a line near the top:

NUM_PROCESSES = os.getenv("SLURM ENVIRONMENT VARIABLE", None)

If the variable does not exist, it will default to None, which is the current behaviour.

Then the pool can use Pool(NUM_PROCESSES) to get the correct number.

I don't know the actual variable name. There are some options here: https://slurm.schedmd.com/sbatch.html#lbAK

To test

• When running with SLURM_ENVIRONMENT_VARIABLE set, the plotting uses the correct number of processes.

Upgrade to Iris 3

Summary

As a developer, I want to use Iris 3 to take advantage of new features.

Description

Iris 3 is out: https://scitools-iris.readthedocs.io/en/v3.0.0/whatsnew/3.0.html

Iris is installed via conda, so we should update installation instructions to specify a minimum version of 3.0. There are some breaking changes to the way that unspecified units are handled. These (or other changes) may break the tests and require updating of the code.

Acceptance criteria

• Installation instructions specify version >= 3.0
• Integration tests pass against Iris 3.0

Add option for Met Office - IMO - BGS MoU colour scheme

Summary

As a member of staff at BGS, I want to be able to make plots using the colour schemes agreed in the MoU.

Description

The branch new_color_map includes the MoU colour scheme. For it to work, it depends on the cubes being stored with the correct units. These should be recorded on this ticket.

A possible implementation of the colour map switching would be to replace the vaac_colours flag with a colour map option. It could choose from vaac, iceland_uk_mou or other builtin options.

Acceptance criteria

• Unit tests show that total_deposition is presented in standard units (g/m2)
• Unit tests show that total_column is presented in standard units
• Unit tests show that air_concentration is presented in standard units
• An option is added to the plotting functions allowing the MoU colours to be used