hadryan/moodzik

MoodZik is a web app that utilizes Machine Learning to analyze its user's face through their webcam, identify their current emotions, and compose music based on how they are feeling.

Moodzik

Overview:

MoodZik is a web app that utilizes Machine Learning to analyze a user's face through the webcam, identify their current emotions, and generate music from scratch based on how they are feeling.
The technologies / libraries used in this project are:

• Tensorflow & Keras
• Convolutional Neural Networks (Face / Emotion detection)
• Recurrent Neural Networks (Music generation)
• Pandas
• OpenCV
• NumPy
• Flask

Clone & Install dependencies:

Moodzik builds fine on Ubuntu 20.04.1 using Python 3.8.5

    git clone https://github.com/Belhoussine/Moodzik.git  
    ./Moodzik/setup

Run Moodzik locally:

    cd ./Moodzik/WebApp/
    python3 app.py
    firefox http://127.0.0.1:5000/

Face & Emotion Detection:

MindMap:

1. Face detection CNN:

• Get input from camera
• Use Haar_Cascade model
• Capture face:
  • Capture frame
  • Crop frame to identify face only
• Process captured face
• Feed to CNN_02

2. Mood detection CNN:

• Get face image from CNN_01

• Build CNN_02:

  • Design network architecture:
    • Convolution layers.
    • Fully connected Neural Network.
    • Define activation functions.
    • Define output labels.
    • Define optimizer / loss function.

• Train CNN_02:

  • Dataset:
    • Pre-processing:
      • Normalize data
    • Image processing:
      • Convert to grayscale
  • Train model
  • Save model as JSON

• Predict mood from image

• Display mood

3. Music Generation RNN:

• Get mood from CNN_02
• Train Music generator Neural Network:
  • Use Magenta
  • Use primers to generate music from
• Generate music based on mood:
  • Provide mood primers
  • Generate MIDI file based on mood

4. Play Music:

• Use Pygame.mixer on generated midi file

5. Publish web-app:

• Desing GUI
• Use flask to serve app
• Host in a remote server
• Purchase domain name
• Deploy