meghashyamc/scraper

Scrapes some product specific details from amazon.com's product pages and stores them/retrieves them from a database

scraper

Scrapes some product specific details from amazon.com's product pages and stores them/retrieves them from a database

Offers two endpoints:

:9000/scrape - This endpoint requires a request body JSON like so: {"url": "https://www.amazon.com/PlayStation-4-Pro-1TB-Console/dp/B01LOP8EZC"}

:9000/getscraped - This endpoint also requires a request body JSON exactly like /scrape.

/scrape extracts some product info from the amazon.com product page specified like name of the product, price, image URL etc.

Note: Only amazon.com's product pages (like the URL mentioned above) are supported.

Pre-requisites

To run scraper, the following must also be running:

• Redis on port 6379
• The binary from github.com/meghashyamc/dbconnect on port 9050

To run, migrate to the cmd directory and type go run main.go in a terminal.