richwellp/search-engine-data-collector

Summer '21 research project under Forward Data Lab group. Django website that allows easy data collection & annotation from integrated Google Search to find object 'source page' for data extraction. Auto ML model builder & inference with Random Forest.

Search-Engine-Data-Collector

Introduction

This repository contains code that launches a website allowing users to annotate Google Search results to find source pages A source page is defined to be a website that holds specific information about an object. In the case of an iPad, it could be the page that holds the technical specifications. In the case of a politician, it could be their official website. The goal is to find the page that holds information specific to that object to prepare for data extraction.

About

The webite uses the Django framework along with Bootstrap for frontend.

TODO

1. Add cars as template (Make, model, year)

2. Implement more ML models (ME)

3. When switching models/features, add the new model to the x-labels (also add x,y axis labels)

4. Implement engineered features

5. Add option to switch graph views

6. Fix sidebar (add pie-chart to side)

7. Add more data analytical options?

8. Collect lots of data

9. Make tab open in the same place (maybe as a pop-up?)