Method setFeaturesCols([class scala.collection.convert.Wrappers$JListWrapper]) does not exist

[nvidia]

Method setFeaturesCols([class scala.collection.convert.Wrappers$JListWrapper]) does not exist

[wbo4958]

@firestarman Could you have a look at this issue?

[firestarman]

Which notebook did you use, Python or scala ?

[sdev2030]

I am using the Python notebook. Error is when creating XGBoost classifier. (4th command cell)
https://github.com/NVIDIA/spark-xgboost-examples/blob/spark-3/examples/notebooks/python/mortgage-gpu.ipynb

[sdev2030]

Hi @firestarman,
I could get past the method not found error after redoing the initialization script and library install.
Now I am getting "IllegalArgumentException: features does not exist." error. Rapids-4-spark Libraries are installed successfully. Anything else I need to setup. Let me know. Thanks

[wbo4958]

Please refer to
#13 (comment)
#13 (comment)

[sdev2030]

Hi @wbo4958,
I followed the coding as suggested in issue #13 but it didn't solve the issue. Still getting "features does not exist" error.

Any thing else I need to change. Thanks for your support.

[wbo4958]

Hi @sdev2030, Could you check if rapids-4-spark jar has contained SQLPlugin,

jar tf rapids-4-spark.jar | grep SQLPlugin

[sdev2030]

Hi @wbo4958,
I checked it in the folder where the "rapids-4-spark*.jar" file is there and SQLPlugin is in the jar. If I just put the command as you mention above, it fails. Should I be having rapids-4-spark.jar without version numbers. Let me know.

[gnani4444]

Hi @wbo4958 @firestarman
This is not relevant to this issue,
Its from the previous issue

Can you share any notebook or code for hyper-parameter tuning using hyperopt, If you already have it.

I couldn't find featureImportances method for the model object. Can you guide me how to get feature importances.

Thanks in Advance

[wbo4958]

Hi @wbo4958,
I checked it in the folder where the "rapids-4-spark*.jar" file is there and SQLPlugin is in the jar. If I just put the command as you mention above, it fails. Should I be having rapids-4-spark.jar without version numbers. Let me know.

Hi @sdev2030, Is there any history WebUI and paste Spark Properties of Environment

[wbo4958]

Hi @wbo4958 @firestarman
This is not relevant to this issue,
Its from the previous issue

Can you share any notebook or code for hyper-parameter tuning using hyperopt, If you already have it.

I couldn't find featureImportances method for the model object. Can you guide me how to get feature importances.

Thanks in Advance

Hi @gnani4444 Could you file another issue?

[gnani4444]

@sdev2030
Can you add the conf directly to the cluster like below image in databricks cluster configuration.

[gnani4444]

Hi @wbo4958 @firestarman
This is not relevant to this issue,
Its from the previous issue
Can you share any notebook or code for hyper-parameter tuning using hyperopt, If you already have it.
I couldn't find featureImportances method for the model object. Can you guide me how to get feature importances.
Thanks in Advance

Hi @gnani4444 Could you file another issue?

Hi @wbo4958 , I filed another issue here

[sdev2030]

Hi @wbo4958,
when I set the spark conf at the cluster level..I am getting a new error now. Is 7.0 stable...doubting that now!

[gnani4444]

Hi @wbo4958,
when I set the spark conf at the cluster level..I am getting a new error now. Is 7.0 stable...doubting that now!

@sdev2030 Are you using Databricks runtime 7.0 or 7.0 ML or 7.0 ML GPU ?. Please make sure you are using 7.0 ML GPU.

[sdev2030]

Hi @wbo4958 @firestarman
Bit more progress. I was using p2 as worker node before, my apologies for missing to notice the m/c type before.
Once I switched the p3 for worker node, the training seems to fail at the 2nd stage. It appears like the processing goes thru fine and at the end it fails for some reason. I am attaching the errors. Please let me know if there is anything I need to change.
Thanks!

java error - gpu training.pdf

[sdev2030]

Run time Java error were related to Databricks 7.0 GPU issue. Once it got resolved, the training worked.
I still need to set the spark configs at the cluster level for the notebook to work. Setting inside the notebook didn't work.