problem with MPI_Comm_Create_Group

[Bellahra]

problem with MPI_Comm_Create_Group

[ggouaillardet]

Please format your code and do copy/paste the outputs instead of posting pictures.

ierr should pretty much always be MPI_SUCCESS, but newcomm might be MPI_COMM_NULL.
MPI_Comm_size(MPI_COMM_NULL, ...) is illegal w.r.t. the MPI standard.

[wenduwan]

@Bellahra Thanks for reporting the issue. Could you please edit the post using this template?

https://github.com/open-mpi/ompi/issues/new?assignees=&labels=&projects=&template=bug_report.md&title=

[Bellahra]

Please format your code and do copy/paste the outputs instead of posting pictures.

ierr should pretty much always be MPI_SUCCESS, but newcomm might be MPI_COMM_NULL. MPI_Comm_size(MPI_COMM_NULL, ...) is illegal w.r.t. the MPI standard.

Ok, thank you. The question has been complemented and formatted.

[bosilca]

There are multiple issues with this code.

1. As @ggouaillardet indicated calling MPI_Comm_create_group with MPI_GROUP_EMPTY returns MPI_COMM_NULL, and it is illegal to call MPI_Comm_size on MPI_COMM_NULL.
2. On rank 0 (which is the rank that does not call MPI_Comm_create_group) the newcomm variable is not initialized, and that leads to badness in MPI_Comm_size.
3. Based on your question, I don't think this code does what you expect it to do. There is no group create by this code, ever. The reason is simple, you create the comm from MPI_GROUP_EMPTY so it will always result in MPI_COMM_NULL. If I understand what you are trying to do, I don't think using MPI_Comm_create_group is the right way to go, but instead you should take a look at MPI_Comm_split.

[Bellahra]

There are multiple issues with this code.

1. As @ggouaillardet indicated calling MPI_Comm_create_group with MPI_GROUP_EMPTY returns MPI_COMM_NULL, and it is illegal to call MPI_Comm_size on MPI_COMM_NULL.
2. On rank 0 (which is the rank that does not call MPI_Comm_create_group) the newcomm variable is not initialized, and that leads to badness in MPI_Comm_size.
3. Based on your question, I don't think this code does what you expect it to do. There is no group create by this code, ever. The reason is simple, you create the comm from MPI_GROUP_EMPTY so it will always result in MPI_COMM_NULL. If I understand what you are trying to do, I don't think using MPI_Comm_create_group is the right way to go, but instead you should take a look at MPI_Comm_split.

Thank you for your reply. I have tried your solution, and below is my code

program test7
    use mpi
    implicit none
    integer :: rank, n_ranks, ierr, n1_ranks, rank1
    integer :: group, newcomm
    call MPI_Init(ierr)
    call MPI_Comm_size(MPI_COMM_WORLD, n_ranks, ierr) 
    call MPI_Comm_rank(MPI_COMM_WORLD, rank, ierr)
    if (rank > 1) then
    print*,'1'
        call MPI_Comm_create(MPI_COMM_WORLD, MPI_GROUP_EMPTY, newcomm, ierr)
    else
        newcomm = MPI_COMM_NULL
    end if
    if (newcomm /= MPI_COMM_NULL) then
    print*,'begin'
        CALL MPI_Comm_group(newcomm, group, ierr)
        call MPI_Group_rank(group, rank1, ierr)
        call MPI_Group_size(group, n1_ranks, ierr)
        print*, "Rank ", rank, " belongs to the new group with size ", n1_ranks
    endif
    call MPI_Finalize(ierr)
end program test7

where I replaced the MPI_Comm_Split with its equivalent one, MPI_Comm_Create. I run the code with 5 CPU cores, but it takes tens of minutes to run, and up to now only one '1' has been printed. So I am not sure whether it can function correctly. Why does creating a new group consume so much time?

[bosilca]

I'm afraid you missed quite a bit of the underlying nature of the communicator creation. At the highest level the communicator creation functions are collective, aka. all processes involved must call them. Now, depending on the MPI functions you use, the definition of involved differs, but overall it falls into one of the following two categories:

1. All participants in the original communicator: This applies to MPI_Comm_create and to MPI_Comm_split;
2. All participants in the group involved: This applies to : MPI_Comm_create_group.
   For each of the communicator creation function this is well defined in the standard.

Now, let's go back to your new example, the one using MPI_Comm_create. This function is collective across all ranks in the original communicator, in your case MPI_COMM_WORLD. As you only make ranks > 1 go into this function, it is not collective and thus will eventually block.

Here is a working example:

program test7
    use mpi
    implicit none
    integer :: rank, n_ranks, ierr, n1_ranks, rank1
    integer :: group, newcomm, color
    call MPI_Init(ierr)
    call MPI_Comm_size(MPI_COMM_WORLD, n_ranks, ierr)
    call MPI_Comm_rank(MPI_COMM_WORLD, rank, ierr)
    color = 1 ! for now all ranks participate
    if (rank == 0) then ! let's exclude 1 from the resulting communicator
       color = MPI_UNDEFINED
    endif
    call MPI_Comm_split(MPI_COMM_WORLD, color, rank, newcomm, ierr)

    if (newcomm /= MPI_COMM_NULL) then
        print*,'begin'
        CALL MPI_Comm_group(newcomm, group, ierr)
        call MPI_Group_rank(group, rank1, ierr)
        call MPI_Group_size(group, n1_ranks, ierr)
        print*, "Rank ", rank, "in MPI_COMM_WORLD belongs to the new group with size ", n1_ranks, " as rank ", rank1
     else
        print*, "Rank ", rank, "in MPI_COMM_WORLD does not belongs to the new group"
    endif
    call MPI_Finalize(ierr)
end program test7

[Bellahra]

@bosilca Thank you most sincerely. Your explanation is very clear and helped me with exactly what I have misunderstood.

[wenduwan]

@Bellahra Please let us know if you have followup questions. Or this issue will be closed automatically.