ritvikmath/scrapingdata

Code for getting data from social media for your data science projects!

Scraping Data From Various Sources

scrapingdata's Issues

HTTP Error 400: Bad Request

Issue raised while accessing individual posts

i am getting a "keyerror : data" while accesing individual posts in the "post in posts['data']" line of the "scrape_facebook_page" function. Why is this happening and how can I fix this ?

scraping Twitter data via Tweepy using Premium version (for historical data)?

Appreciate your code so much, thank you. Worked like a charm after I updated some of the code for Python version 3 (deleted "b" in 'wb' and deleted "raw_" from "raw_input".

My issue is that I need historical data which requires premium API from Twitter. I purchased it but am totally lost in adding the relevant code, as there seems to be quite a bit more needed now -- could use your help!

Python 3

Could you provide the updated code in Python 3?

Error in scrape_facebook_page function

Hi,
I'm trying to execute the scraper on my machine (MacBook Pro - 10.15.1). I'm using Jupyter notebook (6.0.2) and Python 3.7.3.

When I try to execute the notebook I get following error:
TypeError: a bytes-like object is required, not 'str'

I also uploaded the respective output as a screenshot. Can anyone advise here?

name 'raw_input' is not defined

i am getting this error at last

Input error.

I changed raw_input to input, but then got up with this. Can you help out?
Hashtag Phrase corona

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-9964f143e2c9> in <module>()
     11 
     12 if __name__ == '__main__':
---> 13     search_for_hashtags(consumer_key, consumer_secret, access_key, access_secret, hashtag_phrase)

<ipython-input-2-81ccb421349a> in search_for_hashtags(consumer_key, consumer_secret, access_token, access_token_secret, hashtag_phrase)
     25 
     26         #write header row to spreadsheet
---> 27         w.writerow(['timestamp', 'tweet_text', 'username', 'all_hashtags', 'followers_count'])
     28 
     29         #for each tweet matching our hashtags, write relevant info to the spreadsheet

TypeError: a bytes-like object is required, not 'str' ```

HTTP Error 400: Bad Request

Kindly explain why this is happening.

TypeError: a bytes-like object is required, not 'str'

I did not change much on code except changing raw_input to input and now I am getting this error.

TypeError                                 Traceback (most recent call last)
<ipython-input-4-c6367b7ac85e> in <module>
     2 
     3 if __name__ == '__main__':
----> 4     search_for_hashtags(consumer_key, consumer_secret, access_token, access_token_secret, hashtag_phrase)
     5 

<ipython-input-2-568215187497> in search_for_hashtags(consumer_key, consumer_secret, access_token, access_token_secret, hashtag_phrase)
    25 
    26         #write header row to spreadsheet
---> 27         w.writerow(['timestamp', 'tweet_text', 'username', 'all_hashtags', 'followers_count'])
    28 
    29         #for each tweet matching our hashtags, write relevant info to the spreadsheet

TypeError: a bytes-like object is required, not 'str'

Writing to file

First of All thanks a lot for the code.

But I had one problem opening file with 'wb' attribute
I had to change it to 'w' only

Thanks