New Pipeline: Instagram post scraper

[kuwala-io]

New Pipeline: Instagram post scraper

[arifluthfi16]

I am interested in taking this issue, i will be splitting the pipelines into 2 parts:

• Scrapper API
• And the actual Pipelines

just like the Google POI pipelines

Currently still working on the instagram scrapper and looking at what's possible.

[mattigrthr]

It should be possible to scrape public Instagram posts using hashtags, locations, and (public) users. This article provides some insights and ideas: https://blog.apify.com/scrape-instagram-posts-comments-and-more-21d05506aeb3/

Since @bmahmoudyan can't continue working on this issue, it's up for grabs again. :)

[mattigrthr]

The requirements for a new pipeline are the following:

• Saving the raw data (in this case the scraping results) in a file as is
• Transforming at least lat/lng to H3 and moving all nested properties to a column/table format using meaningful variable names (since we are switching to Postgres and dbt for transformations)
• Ideally saving the results in Parquet format since it’s much more storage efficient and optimized for parallel processing (just one command with PySpark)

[arifluthfi16]

PR for this issue #74