gapdata/engine

The Centrifuge process, filter and saves the relevant documents as recommendations to the relevant users

Keep-Current

After studying a topic, keeping current with the news, published papers, advanced technologies and such proved to be a hard work. One must attend conventions, subscribe to different websites and newsletters, go over different emails, alerts and such while filtering the relevant data out of these sources.

In this project, we aspire to create a platform for students, researchers, professionals and enthusiasts to discover news on relevant topics. The users are encouraged to constantly give a feedback on the suggestions, in order to adapt and personalize future results.

The goal is to create an automated system that scans the web, through a list of trusted sources, classify and categorize the documents it finds, and match them to the different users, according to their interest. It then presents it as a timely summarized digest to the user, whether by email or within a site.

Who are we?

This project intends to be a shared work of Vienna Data Science Cafe Meet-Up members, with the purpose, beside the obvious result, to also be used as a learning platform, while advancing the Natural Language Processing / Machine Learning field by exploring, comparing and hacking different models.

Please feel free to contribute.

Project board is on Trello and we use Slack as our communication channel.

I want to help

We welcome anyone who would like to join and contribute. We meet regularly every month in Vienna through the Data Science Cafe meetup of the VDSG, show our progress and discuss the next steps.

Machine Learning & Natural Language Processing

If you're new to Machine Learning, we suggest reading the following sources:

• Machine learning collection
• Natural Language Processing in Python Book
• Keon's awesome-nlp!

Recommended tools

We lean heavily on existing tools as well as developing new methods. We are colaborating through Google Colab notebooks.

• TextBlob
• spaCy
• Vowpal Wabbit

The repository

This repository is the documents personalized recommendation engine. If you wish to assist in different aspects (Data Engineering / Web development / DevOps), we have divided the project to several additional repositories focusing on these topics:

• Web Development & UI/UX experiments can be found in our App repository
• Data Engineering tasks are more than welcomed in our Data Engineering repository
• Devops tasks are all across the project. We are trying to develop this project in a serverless architecture, and currently looking into Docker and Kubernetes as well as different hosting providers and plans. Feel free to join the discussion and provide your input!