jasonlnupolun/wisecrowdrec

A personalized Movie Recommender System, analyzing streaming tweets via Spark Streaming, Stanford CoreNLP, and Restricted Boltzmann Machines

Version 2: Node.js + Socket.io

demo video (< 1 minute)

Version 1: Spring MVC

Google Compute Engine (GCE)

$ cd WiseCrowdRec
$ sbin/gce-1-installTools.sh
$ sbin/gce-4-stopall.sh

$ screen
$ sbin/gce-3-run-sparkStreaming.sh
ctl+a, d

$ screen -X -S <SCREENID> kill

$ git push -u origin master

Check List:

• Streaming processing
  • Storm
  • Spark Streaming (Apache Spark 0.9.1)
• Deep learning
  • Restricted Boltzmann Machines (RBM) for Collaborative Filtering
• Machine learning lib
  • Mahout on Hadoop (Collaborative Filtering)
• nlp
  • Stanford CoreNLP
  • Calais
• Multithreaded, Parallel, and Distributed
  • Multithreading (in Java)
  • Parallel
  • Distributed
• WebSocket
• Rabbitmq
• ActiveMQ
• Apache Camel -> from(“file://xxxx").to("activemq://xxxx") (to ActiveMQ's JMS)
  • $ netstat -a -> to check if the activemq process has started
  • $ bin/activemq start
  • $ lsof -i:<port>
  • $ kill -9 {PID}
  • $ sudo lsof -i -n -P | grep TCP
• Storm JMS -> Java Message Service (JMS)
• Freebase
• Twitter4J
• Sign in with Twitter
• nosql - Cassandra
• data search - Elasticsearch
• Hadoop 2.2.0
• Tomcat 7
• Spring MVC
• Amazon EC2
• Google Compute Engine
• Google Web Toolkit (GWT)
• Node.js
• Socket.io
• D3.js Force-Directed Graph
• Jetty
• kafka
• Akka
• Zookeeper
• Scripts for running this project automatically
• Unit Test
• log4j
• Maven
• WebApp

---

• Multithreading Parallel RBM
  

• Project Snapshot

• WiseCrowdRec UI: RealTime Histogram