Data parallelism using the open-source cluster-computing framework : Apache Spark. Notebooks in PySpark and Scala languages.
-> Notebook : Pyspark_Databricks_Community_Edition.ipynb and Pyspark_Databricks_Community_Edition.py Basics of data parallelism using Spark (RDD), developed on the Databricks community edition plateform with PySpark
-> Notebook : Dataframe_Databricks_Community_Edition.ipynb and Dataframe_Databricks_Community_Edition.py Basics of data parallelism using Spark (Dataframe), developed on the Databricks community edition plateform with PySpark
-> Data used in these two notebooks : 2c443e0c8b99190f_c98fa37700000000_457998991_data_0__1__1_-c0514.parq
-> Script : Scala_basics.sc
Basics of Scala language