Comments (6)
For something that runs in ~5 minutes on 8 processors, setting the number of vertical levels to 8 (last number on line 13 of generate_data_sphere.py
, change from 48 to 8) should do the trick.
from flowsieve.
It hadn't even occurred to me that I didn't have any MPI tutorials. Thanks for point that out! I'll fix that this week :-)
from flowsieve.
Took a little longer than planned, sorry, but an MPI tutorial is now included. It follows almost exactly the 'Low Resolution' spherical case for consistency, but with multiple depth levels to allow multiple MPI ranks. When used on the recommended number of processors, it runs in a couple minutes.
from flowsieve.
40 processors is a bit much, so I have to get a supercomputer to test it. Would it be possible to reduce the requirements such that even with 8 processors you get something in a few minutes?
from flowsieve.
I've updated the ABOUT_TUTORIAL.md
to include a note / instructions on how to adjust the tutorial (changing a single number in the generate_data_sphere.py
scripts) to allow the tutorial to run on fewer processors in reasonable time.
The default setting is 24 processors running for ~10 minutes.
Does that seem reasonable?
Reducing the MPI-requirement
24 processors is a fairly heavy requirement if you are not running on a computing cluster.
You can simply run on fewer processors (highest efficiency if the number of processors divides evenly into 48 - the number of vertical levels), but at the cost of increasing the runtime.To reduce the processor cost without increasing runtime, you can decrease the number of vertical levels proportionately. E.g. you can reduce the vertical levels to 12 in order to run on 6 processors in a similar amount of time.
To adjust the number of vertical levels, you can adjust line 13 of
generate_data_sphere.py
, which readsNlon, Nlat, Ndepth = int(360//2), int(180//2), 48
. The last number,48
, specifies the number of vertical levels.When running the code, you can use any number of MPI ranks up to the number of verticals levels, but the most efficient use of processors occurs when the number of MPI ranks divides evenly into the number of vertical levels.
from flowsieve.
Managed to run with 8 vertical levels and 4 processors in a laptop for ~30 minutes.
from flowsieve.
Related Issues (20)
- Statement of need in documentation HOT 3
- What variables can be coarse grained by FlowSieve? HOT 5
- Running the tutorial on the sphere HOT 36
- When does filter commute with derivatives? HOT 5
- Scalar Tutorial HOT 4
- Makefile for Jasmin HOT 14
- question about irregular longitude and latitude, and many times HOT 4
- Cannot find -lhdf5 HOT 9
- Specify minimum requirements for installation in documentation HOT 7
- Installation errors with gcc 10 HOT 14
- Helmholtz decomposition in a high-resolution current HOT 2
- Add a copy of `constants.hpp` with the specific values in Tutorials HOT 3
- Filering on cartesian grid HOT 2
- How to make an appropriate system.mk file HOT 2
- The range to which coarse-grain is performed locally HOT 8
- ''from matpy import FiniteDiff'' failed HOT 2
- Compilation error
- Successfull compile but 'Assertion `input_nc_format == (3)' failed.' on JASMIN HPC
- Errors, compute_KE_spectra_and_slopes.cpp:(.text+0x139a): undefined reference to functions, such as `potential_vel_from_F HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flowsieve.