scott-park/lofi

ML-supported lo-fi music generator

Lofi

A ML-supported lo-fi music generator.
Explore »

About

We trained a VAE model in PyTorch to represent a lo-fi track as a vector of 100 features. A lo-fi track consists of chords, melodies, and other musical parameters. The web client uses Tone.js to make a dusty lo-fi track out of these parameters.

Architecture

• Client: The client is written in TypeScript and built with Webpack. It uses Tone.js to generate music.
• Model: The model is implemented in PyTorch. We synthesized various datasets, including Hooktheory and Spotify.
• Server: The server is a basic Flask instance that deploys the trained model checkpoint. The client communicates with the server using a REST API.

Setup

If you only want to tinker around with the client, you will only need the client folder. This will use the project's server as the backend.

If you want to deploy your own model, you can either train your own model (see the instructions in the model) or download the pre-trained checkpoint from here. Once you have deployed the server, change the server address inside client\src\api.ts.

Set up the client

1. Install node.js LTS.
2. Navigate to the client folder and run npm install to install the dependencies.
3. Run npm run serve to develop or npm run build to build a distributable.

By default, this uses the project's server as the backend. You can also train your own model and deploy your own server.

Train your own model

See the model folder for details. Once you have trained your model, put the checkpoint in the checkpoints folder.

Deploy your own server

See the server folder for details. You can use the provided Dockerfile. Don't forget to change the API url in the client.

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Acknowledgments

Big thanks to ZOOPRA UG for hosting the server!