-
Just open the folder with VS Code and reopen it in a container. ! IMPORTANT: it can take a while, it is installing some libraries and dependencies.
-
in the terminal just run:
sbt run
-
if you prefer, you can package the code and then run it with spark-submit.
-
Register udf_pandas provoke some error in teh DAG but the code works well.
macwinux / pyspark-in-scala Goto Github PK
View Code? Open in Web Editor NEWHow to comunicate pyspark with spark scala through sql