iajitprasad Goto Github PK

bigdata_casestudyi_moviedataanalysis

This assignment is aimed at consolidating the concepts that was learnt during the MapReduce & Apache Pig.

bigdata_casestudyii_hive-hbase

This case study assignment is aimed at consolidating the concepts that was learnt during the various session of Hive & HBase of the course.

bigdata_casestudyiii_sensorusingspark

Working with Sensor using Apache Spark, Spark SQL

bigdata_casestudyiv_sparkstreaming

Demo of Spark application that reads the data from local file system & does some transformation & action on the fly. As & when files keep coming within the directory the application fetches the file automatically & performs successive transformation & action there by providing the desired output simultaneously. Application also moves the files onto HDFS perform some transformations & actions on HDFS as well & then compares both the results to produce a final message of success/failure in case the output file matches/does_not_match.

bigdata_casestudyv_ushospitalusingspark

Analysis of Hospital Data from United Stated using Apache Spark, Spark SQL

bigdata_session10assignment10.1

This assignment is aimed at consolidating the concepts that was learnt during the HBase Basics session of the course.

bigdata_session11assignment11.1

Advance HBase concepts. Importing large files from HDFS into HBase tables directly. Understanding HBase read/write & Architecture.

bigdata_session12assignment12.1

Analysis & import of Twitter Data onto HDFS. Tweet search based on certain keys that needs to added within configuration file & them Streaming Data in using Flume agent

bigdata_session14assignment14.1

Scala Basics. Exploring filter, apply, map, Lambda, count, length & various String Functions. Exploring list, tuples, arrays, sets, nested tuples & many more

bigdata_session15assignment15.1

Scala applications to find GCD (Greatest Common Divisor ) of two numbers, Fibonacci Series using Standard Loop & Recursion Method, and Square Root of a number using Babylonian Method of Divide & Average.

bigdata_session16assignment16.1

Creating a simple calculator using Scala

bigdata_session17assignment17.1

Inheritance & Multiple Inheritance, Partial Function and Match Case in Scala

bigdata_session18assignment18.1

Solving exercise problems using Spark RDD. Limitations of MapReduce and Features & Operations of Spark RDD.

bigdata_session19assignment19.1

Analysis of a college student dataset using Spark RDD. Demo of various operations on RDD such as countByValue, groupBy, groupByKey, reduceByKey,etc. Demo of map, flatmap, split, explicit, filter, type conversion, finding sum, count, distinct, aggregate, length of RDD. Demo of String, int comparisons, UNION & intersetion on RDD's.

bigdata_session1assignment1.1

This assignment helps to consolidate the concept learned in the opening session of BigData Engineering with Hadoop & Spark.

bigdata_session20assignment20.1

Travel Data Exercise using Scala SQL

bigdata_session21assignment21.1

Sports Data Exercise using Scala SQL and UDFs.

bigdata_session22assignment22.1

Understanding Kafka, starting a broker/leader that pushes messages to be consumed at consumer. Demonstrating ways to create multi-node replicas with multiple partitions. Different ways to create a topic(keyed/keyless) based on the key property. Formatting messages with custom values of key value separator & reading the same consumed messages at consumer using custom format using key separator property.

bigdata_session23assignment23.1

Apache Kafka. Working with Datasets using Java Classes.

bigdata_session24assignment24.1

Apache Spark Streaming examples using Netcat. Read streams of data to find sum, also find out the offensive words.

bigdata_session2assignment2.1

This assignment is aimed at consolidating the concepts that was learned during the HDFS session of the course.

bigdata_session3assignment3.1

This assignment is aimed at consolidating the concepts that was learned during the YARN session of the course.

bigdata_session4assignment4.1

This assignment is aimed at consolidating the concepts that was learned during the MapReduce Introduction session of the course.

bigdata_session5assignment5.1

This assignment is aimed at consolidating the concepts that was learned during the Advance MapReduce session of the course.

bigdata_session6assignment6.1

This assignment is aimed at consolidating the concepts that was learnt during the Data Indigestion Tool Sqoop session of the course.

bigdata_session7assignment7.1

This assignment is aimed at consolidating the concepts that was learnt during the Exploring Apache Pig session of the course.

bigdata_session8assignment8.1

This assignment is aimed at consolidating the concepts that was learnt during the Hive Basics session of the course.

bigdata_session9assignment9.1

This assignment is aimed at consolidating the concepts that was learnt during the Advance Hive session of the course.

data

effectivescala

Twitter's Effective Scala Guide