Apache Spark is an open-source cluster-computing framework. It is much better than the other options available at present.This is an attempt to test Apache Spark's key features with other setups. The features of Apache Spark that we are interested in are In-memory data abstraction, Partial DAG, Lineage base fault recovery, Data co-partition, Unification of Streaming, Batch and Interactive Processing and Hybrid Storage Architecture.
You will need to have Apache Spark installed on your system to run Scala file. You can download it from here. To get a better guide to install it on ubuntu, refer here You will need R Studio and R installed to run the R code. To install R studio refer here
checknb.scala is the Scala code for Naive Bayes Classification.
finalnb.r is the R code for Naive Bayes Classification.
checktrain.txt is the training dataset.
checktest.txt is the testing dataset.
Things you need to install to run the code on Spark:
JAVA
Scala
Things you need to install to run the R code:
- READR
install.packages("readr", INSTALL_opts = c('--no-lock'))
- STRINGR
install.packages("stringr", INSTALL_opts = c('--no-lock'))
- TOKENIZERS
install.packages("tokenizers", INSTALL_opts = c('--no-lock'))
Run the above three lines on R-Studio console.
-
SYED AFSAHUL HAQUE - afsahulsyed.
-
SUMIT KALRA. - Guidance - Lemon