Getting Java gateway process exited before sending its port number during init_spark()

[swapkh91]

Getting Java gateway process exited before sending its port number during init_spark()

[kira-lin]

Hi @swapkh91 ,
Sorry for the late reply. Ray 2.7.0 is released recently and might not be compatible with RayDP 1.6.0. Can you try Ray 2.6?

[swapkh91]

Hi @swapkh91 , Sorry for the late reply. Ray 2.7.0 is released recently and might not be compatible with RayDP 1.6.0. Can you try Ray 2.6?

@kira-lin thanks, I'll try and get back. Any limitation on java version?

[kira-lin]

Sorry for inconvenience. We only tested java 8. Java 11 should be fine though.

[swapkh91]

@kira-lin I tested it with ray 2.6.2, getting same error. I'll explain how I'm trying to connect, maybe some issue in the process

the ray cluster is on GKE
I have port forwarded it on my laptop through
kubectl port-forward --address 0.0.0.0 service/raycluster-autoscaler-head-svc 10001:10001

I then connect using

ray.init(address="ray://localhost:10001")
spark = raydp.init_spark(app_name='RayDP Example2',
                         num_executors=2,
                         executor_cores=2,
                         executor_memory='4GB')

Now this init_spark comamnd gives the above error

I checked the logs through dashboard

:job_id:03000000
:actor_name:RayDPSparkMaster
Error opening zip file or JAR manifest missing : /home/swapnesh/.local/lib/python3.10/site-packages/raydp/jars/raydp-agent-1.6.0.jar

Why is it showing the jar file path of my laptop? It is present there though, I checked

[kira-lin]

Why is it showing the jar file path of my laptop? It is present there though, I checked

Oops, this seems a bug. We'll try to fix this. For now, you can wrap this init_spark and things you want to do with spark in an remote actor, that should be fine. Thanks for identifying this bug.

[swapkh91]

Why is it showing the jar file path of my laptop? It is present there though, I checked

Oops, this seems a bug. We'll try to fix this. For now, you can wrap this init_spark and things you want to do with spark in an remote actor, that should be fine. Thanks for identifying this bug.

@kira-lin got it, I'll try that. Also, I noticed that raydp has dependency ray >= 2.1.0 as here. So this installs ray 2.7.1 when I do pip install raydp
I then have to manually do pip install --force-reinstall ray==2.6.2 to downgrade

[psr-ai]

He @swapkh91 I am also getting the same error. Did you find the solution for this?

[kira-lin]

hi @raiprabh ,

For now, you can wrap this init_spark and things you want to do with spark in an remote actor, that should be fine. Thanks for identifying this bug.

You can try this solution. We don't have enough bandwidth to work on this project now, so you are welcome to submit a PR to fix this if you have a solution @swapkh91 . We just need to use the path of the remote machines.

[Mayurji]

@kira-lin, Is there any update on this issue?