hegdechaitra/twitter_news_data

Scraping tweets and articles from news twitter handles. Can be used for teaser generation and news headline generation.

twitter_news_data

Tweets scraped from twitter for set of news twitter handles. The corresponding news article from respective website has also been scrapped.

The code successfully runs on Python 3.6.9

How to scrape twitter:

1. Install Twint using pip3 install --user --upgrade git+https://github.com/yunusemrecatalcam/twint.git@twitter_legacy2
2. run script as python get_tweets.py

How to scrape articles:

Script to scrape NBCNews articles is made available. Run python scrape_nbcnews.py

Output:

Tweets and corresponding article scraped for set of twitter handles has been made available under /data