ompi-x / libomp Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
Implement the hooks for the creation of places so we can do more advanced partitioning of available resources when more than 1 MPI rank is running on the node.
We already identified the __kmp_partition_places() function (runtime/src/kmp_partition.cpp) that seems to initialize places based on the binding strategy; it might be a good place to start.
Also, the goal would ultimately be to control where and how Open MP places are created while sharing resources between MPI ranks.
I just ran a test against the latest gv_pmix
branch and noticed that the omp_get_thread_num()
returns one less than expected. For example, if NumCpus=12 and NumRanks=2, would expect to have 6 threads but instead we get 5 threads.
[PID:1037] DBG: LINE:1780 nthreads=5, _n_local_cpus=12, _n_local_ranks=2
[PID:1036] DBG: LINE:1780 nthreads=5, _n_local_cpus=12, _n_local_ranks=2
...<snip>...
[mpi_omp_hello.c:main:48] In parallel loop: calling omp_get_thread_num()...
[mpi_omp_hello.c:main:50] omp_get_thread_num() succeeded
Number of threads = 5
Looks like this is related to kmp_runtime.cpp#L1779.
Adding this as issue to track status while testing. Using gv_pmix
branch at da5ea2c
(after PR #4 was merged).
I enabled debug prints/instrumentation to __kmp_affinity_set_init_mask()
to show the binding of OMP threads to places. I see that some threads show all places
instead of a specific place X
.
Here's the command I used on machine with 12 cores (libomp/moc test show NCPUs=12).
NOTE: I lowered the KA_TRACE()
lines to debug level=5 in code, otherwise use KMP_A_DEBUG=101
.
beaker:$ env KMP_A_DEBUG=6 mpirun \
-x OMP_PLACES=threads \
-x OMP_CPU_AFFINITY=TRUE \
-x OMP_PROC_BIND=spread \
--map-by core -np 2 \
./mpi_omp_hello.dbg
Here's the relevant output sorted for easier reading and with the PID info:
__kmp_affinity_set_init_mask: binding T#0 (PID:13615) to place 0
__kmp_affinity_set_init_mask: binding T#1 (PID:13615) to all places
__kmp_affinity_set_init_mask: binding T#2 (PID:13615) to place 1
__kmp_affinity_set_init_mask: binding T#3 (PID:13615) to place 1
__kmp_affinity_set_init_mask: binding T#4 (PID:13615) to place 1
__kmp_affinity_set_init_mask: binding T#0 (PID:13614) to place 0
__kmp_affinity_set_init_mask: binding T#1 (PID:13614) to all places
__kmp_affinity_set_init_mask: binding T#2 (PID:13614) to all places
__kmp_affinity_set_init_mask: binding T#3 (PID:13614) to place 0
__kmp_affinity_set_init_mask: binding T#4 (PID:13614) to place 0
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.