jbornschein/mpi4py-examples

mpi4py examples

mpi4py-examples's Issues

How to test 10-task-pull-spawn.py?

10-task-pull-spawn.py should be run as Program should be started without argument. However,

pi@pi01:~/Downloads/mpi4py-examples $ python 10-task-pull-spawn.py 
--------------------------------------------------------------------------
There are not enough slots available in the system to satisfy the 10 slots
that were requested by the application:
  /usr/bin/python

Either request fewer slots for your application, or make more slots available
for use.
--------------------------------------------------------------------------
Traceback (most recent call last):
  File "10-task-pull-spawn.py", line 38, in <module>
    maxprocs=n_workers)
  File "mpi4py/MPI/Comm.pyx", line 1534, in mpi4py.MPI.Intracomm.Spawn
mpi4py.MPI.Exception: MPI_ERR_SPAWN: could not spawn processes

If I run with mpirun, program hangs:

pi@pi01:~/Downloads/mpi4py-examples $ mpirun --hostfile ~/hostfile python 10-task-pull-spawn.py 
--------------------------------------------------------------------------
All nodes which are allocated for this job are already filled.
--------------------------------------------------------------------------
Traceback (most recent call last):
  File "10-task-pull-spawn.py", line 38, in <module>
    maxprocs=n_workers)
  File "mpi4py/MPI/Comm.pyx", line 1534, in mpi4py.MPI.Intracomm.Spawn
mpi4py.MPI.Exception: MPI_ERR_SPAWN: could not spawn processes

How to use mpi4py on a cluster

Hello, thanks for sharing such good examples which have much learning value.
I have a question on when I want to run the .py files of mpi4py, is there any configuration or setting I should do on the cluster?
Looking forward to your reply. Thanks so much!

Example of mpi4py groups and intercommunication

Thanks for the very instructive examples of use of mpi4py.

I am actually trying to use mpi4py to gather processes of each node of a cluster [as explained on that StackExchange question]. My first idea was to directly use groups and intercommunications, but this post recommend to use the gather then split functions.

I am referring to you because you have a great experience with MPI using mpi4py and maybe one of your example could be simply adapted to achieve that. Also, It could be an interesting addition to your very good set of examples.

Could you please add a license?

Thank you so much for preparing these examples. They helped me a lot in familiarizing with mpi4py.

Would you be so kind to provide an open-source license for this project?

05-pseudo-whitening line 18: ModuleNotFoundError: No module named 'bernstein'

ModuleNotFoundError: No module named 'bernstein'

Where can I find bernstein package?

Cheers

WARNING: Open MPI failed to TCP connect to a peer MPI process. This should not happen.

......
--------------------------------------------------------------------------
WARNING: Open MPI failed to TCP connect to a peer MPI process.  This
should not happen.

Your Open MPI job may now fail.

  Local host: pi02
  PID:        24783
  Message:    connect() to 169.254.140.13:1026 failed
  Error:      Resource temporarily unavailable (11)
--------------------------------------------------------------------------
[pi01:28731] 3 more processes have sent help message help-mpi-btl-tcp.txt / dropped inbound connection
[pi01:28731] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
[pi01:28731] 3 more processes have sent help message help-mpi-btl-tcp.txt / client connect fail

My hostfile is:

$ cat ~/hostfile 
192.168.1.253   slots=4
192.168.1.251   slots=4
192.168.1.249   slots=4
192.168.1.247   slots=4

03-scatter-gather run bug....

pi@pi01:~/Downloads/mpi4py-examples $ mpirun --hostfile ~/hostfile 03-scatter-gather 
------------------------------------------------------------------------------
  Running on 16 cores
 ------------------------------------------------------------------------------
 After Scatter:
 [0] [0. 1. 2. 3.]
[1] [4. 5. 6. 7.]
[pi01][[24145,1],0][btl_tcp_endpoint.c:626:mca_btl_tcp_endpoint_recv_connect_ack] received unexpected process identifier [[24145,1],3]
[2] [ 8.  9. 10. 11.]
[pi03][[24145,1],8][btl_tcp_endpoint.c:626:mca_btl_tcp_endpoint_recv_connect_ack] received unexpected process identifier [[24145,1],10]

For 05-pseudo-whitening, IndexError: list index out of range

pi@pi01:~/Downloads/mpi4py-examples $ mpirun --np 16 --hostfile ~/hostfile  05-pseudo-whitening
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
Traceback (most recent call last):
  File "/home/pi/Downloads/mpi4py-examples/05-pseudo-whitening", line 26, in <module>
    in_fname = sys.argv[-2]
IndexError: list index out of range
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[1099,1],7]
  Exit code:    1
--------------------------------------------------------------------------