mzhoume/cloudcomputinghw

Cloud Computing & Big Data, Homework

E6998 Sec 5 Cloud Computing Homework

Ming Zhou (UNI: MZ2591), Emily Hua (UNI: YH2901)

---

Homework 1

url: http://pre-env.jih6zvtkdb.us-east-1.elasticbeanstalk.com/

• Backend: Backend services, written in TypeScript with NodeJS and ExpressJS
• Frontend: Frontend views, written in TypeScript with AngularJS v2
• ElasticBeanstalk: Elastic Beanstalk configuration and app archive
• Lambda: (Unused), reserved for gluing backend services together

Tasks:

Backend:

• Fetch tweets in real time from Twitter Streaming API
• Use Elastic Search to store and index the tweets
• Expose RESTful APIs for searching with content, user and location
• Deploy on an auto-scalling environment

Frontend:

• Web UI for user to search for given keywords
• Use Google Map SDK to display the tweets in its original location
• Search tweets from the user chosen location and distance

---

Homework 2

url: http://ec2-52-70-84-118.compute-1.amazonaws.com/

kibana url: http://ec2-52-70-84-118.compute-1.amazonaws.com:5601/

kafka manager url: http://ec2-52-70-84-118.compute-1.amazonaws.com:9000/

zookeeper url: http://ec2-52-70-84-118.compute-1.amazonaws.com:2181/

• Backend: Backend services, written in TypeScript with NodeJS and ExpressJS, modular design
• Frontend: Frontend views, written in TypeScript with AngularJS v2 and BootStrap

Tasks:

Backend:

• Fetch tweets in real time from Twitter Streaming API
• Use kafka as the Event Queue replacement for AWS SQS
• Gather tweet sentiment data from watson from multi-threading worker poll
• Send processed tweets to AWS SNS to notify the persisting procedure
• Use Elastic Search to store and index the tweets

Frontend:

• Web UI for user to search for given keywords and username
• Use Google Map SDK to display the tweets in its original location
• Draw sentiment markers (with happy, neutral and sad face)
• Update the tweets list in real time, with sentiment displayed as background