liupeiwenchina/introhadoop

Introduction to Hadoop

Introduction to Hadoop

A four session workshop to get started using tools in the Hadoop universe. We're going to use the Hortonworks Hadoop Sandbox.

##Objective: The workshop series offers a brief introduction to concepts of parallel distributed computing and the Hadoop universe. Participants will learn to navigate among the various tools, and to write programs for large scale data analysis. Examples will be provided in Python and R, knowledge of the Java programming language is not required.

Session 1: March 25

Concepts of parallel distributed programming, map-reduce, HDFS

Session 2: April 1

Map Reduce, MR Streaming API

Session 3: April 8

Hbase, Hive, Pig and other tools to query and access

Session 4: April 15

Spark

Preperation

1. Install either VMWare Virtual Machine or Virtual Box on your laptop. (VMWare offers a free Workstation Player for Windows and Linux https://my.vmware.com/en/web/vmware/free#desktop_end_user_computing/vmware_workstation_player/12_0, OS X users may use VMWare Fusion https://www.vmware.com/products/fusion/fusion-evaluation.html. Oracle's VirtualBox https://www.virtualbox.org/wiki/Downloads is free for all platforms.)
2. Download the Hortonworks Sandbox for the respective virtual machine from http://hortonworks.com/products/hortonworks-sandbox/#install (it's over 9 GB, it may take a while).
3. Download the SEC data set from https://www.dropbox.com/s/mjocpiqwcknvnto/SEC%20data.7z?dl=1 (about 2.2 GB)