Comments (12)
John,
I believe I speak for all on this forum that we are all greatly appreciative of your time and effort, and of your graciousness in offering it for use.
Thank you.
Eric
from pytrends.
New version released 1.2.0 both on github and Pypi. Functionality remains the same. Next major release will use dictionaries for parameters passed in, I will keep both up on Pypi once that happens.
Known issues:
- Google will nag you that you've logged in from a new device each time.
from pytrends.
I am also seeing the same issue - checked login details and they are correct, plus checked latest version..
The google trends site still works as expected when using the url's and same account. Hope this can be resolved soon - such a useful tool!
from pytrends.
First off: I greatly appreciate the work done on pytrends. Thank you for providing this.
Pytrends worked perfectly for several weeks with zero glitches on Python 3.5.2. On Monday I received the same 400 error, and since then I have been stuck. I don't know what (if anything) changed. I too tried what @adstastic did in January (closed 400 error topic) and saw the exact same results (same csv file dump s/he saw). I also tried the pip install pytrends --"upgrade" you had recommended in March for Python 3.5.1 to ravimevcha from "Script stopped working.. 400 Bad Request error #30" but still the same 400 error.
I believe I've traced the issue to the self.opener.open(self.url_CookieCheck). My Google credentials are correct and work everywhere else (and I have no messages from Google relating to security or proving I am not a bot, and thus I don't believe Google locked my account. ) so I am at a loss as to how to proceed.
If I manually paste the google trends request url directly into the browser (I am using Opera; already logged into the same Google account for email/etc.) it gives me a good csv. I cleared the Opera cache, and don't show any cookies listed.
To be clear: I am only downloading between 20-50 trends TOTAL per day (each a list of up to five terms to be compared as a single trend/url/csv); this is not a mass-trend downloader for me.
I apologize if accidentally broke protocol but I had originally commented on this via a closed issue: HTTP Error 400: Bad Request #24. Actually there have been a number of similar comments on this closed issue in the past several days.
Any help is appreciated.
Thank you.
from pytrends.
I will be be taking a look at this today.
This package is something I support in my freetime so I appreciate
understanding in the time it takes to fix.
On Aug 27, 2016 07:59, "luc prieur" [email protected] wrote:
I am getting the same error. The problem is that Google has changed its
backend again. This means that this module, without some serious changes no
longer works. From my initial investigation, it goes beyond simple URL
change.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#47 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGxCB83lRT31J60moQCGpP4QFfAAZypdks5qkDSagaJpZM4JrzRB
.
from pytrends.
Got it. It's messy so I'll refactor after some coffee tomorrow but here is the gist in case I get hit by a meteor while I sleep.
class SessionGoogle:
def init(self, url_login, url_auth, login, pwd):
self.ses = requests.session()
login_html = self.ses.get(url_login)
soup_login = BeautifulSoup(login_html.content, "lxml").find('form').find_all('input')
dico = {}
for u in soup_login:
if u.has_attr('value'):
dico[u['name']] = u['value']
# override the inputs with out login and pwd:
dico['Email'] = login
dico['Passwd'] = pwd
self.ses.post(url_auth, data=dico)
def get(self, URL):
return self.ses.get(URL).text
url_login = "https://accounts.google.com/ServiceLogin"
url_auth = "https://accounts.google.com/ServiceLoginAuth"
session = SessionGoogle(url_login, url_auth, google_username, google_password)
req = session.get("http://www.google.com/trends/trendsReport?q=Pizza&hl=en-US&cmpt=q&content=1&export=1")
print(req)
from pytrends.
Here is the adapted code for an easy copy / paste job:
from bs4 import BeautifulSoup
class SessionGoogle:
def __init__(self, email, pwd):
self.url_login = "https://accounts.google.com/ServiceLogin"
self.url_auth = "https://accounts.google.com/ServiceLoginAuth"
self.email = email
self.pwd = pwd
self.ses = requests.session()
self.login_html = self.ses.get(self.url_login)
self.soup_login = BeautifulSoup(self.login_html.content, "lxml").find("form").find_all("input")
self.dico = {}
def log(self):
for u in self.soup_login:
if u.has_attr('value'):
# print(u.encode('utf-8'))
self.dico[u['name']] = u['value']
# override the inputs with out login and pwd:
self.dico['Email'] = self.email
self.dico['Passwd'] = self.pwd
self.ses.post(self.url_auth, data=self.dico)
def get(self, URL):
output = self.ses.get(URL).text
return output
session = SessionGoogle(username, password)
session.log()
req = session.get("http://www.google.com/trends/trendsReport?q=Pizza&hl=en-US&cmpt=q&content=1&export=1")
print(req)
from pytrends.
I am also seeing the same issue with Python 2. It worked on the 19th but not since then, with unchanged username/password and code.
from pytrends.
I get the same error with Python 3.4.3 i.e. the call to self.opener.open(self.url_CookieCheck)
fails. My authentication credentials haven't changed since last use. Perhaps Google has changed the behavior of that page.
from pytrends.
I am getting the same error. The problem is that Google has changed its backend again. This means that this module, without some serious changes no longer works. From my initial investigation, it goes beyond simple URL change.
from pytrends.
John, I find it extraordinary that guys like you maintain free packages
like pygtrend. I am very grateful. Thanks
On Aug 27, 2016 2:12 PM, "John Hogue" [email protected] wrote:
I will be be taking a look at this today.
This package is something I support in my freetime so I appreciate
understanding in the time it takes to fix.On Aug 27, 2016 07:59, "luc prieur" [email protected] wrote:
I am getting the same error. The problem is that Google has changed its
backend again. This means that this module, without some serious changes
no
longer works. From my initial investigation, it goes beyond simple URL
change.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<https://github.com/GeneralMills/pytrends/issues/
47#issuecomment-242915837>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/
AGxCB83lRT31J60moQCGpP4QFfAAZypdks5qkDSagaJpZM4JrzRB>
.—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#47 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAXoKIUAnS7bQchnKnnFaRr1O-OjghIOks5qkH39gaJpZM4JrzRB
.
from pytrends.
Working on it now. Looks like its finally time to move to Requests. I'm able to log into my Google Account now I just need to pass the cookie to the other report request.
@ericross2 thanks for your help. The reason it is working when you paste the URL is because you are most likely already logged into Google through your browser.
from pytrends.
Related Issues (20)
- The 'interest_by_region' does not match the UI.
- timeframe doesn't work - Google returned a response with code 429 HOT 13
- Pytrends work again with time frame 1 - H, 4 - H or 1 - D HOT 1
- Interested in being a Maintainer
- Proxy error HOT 1
- Impossible to get data for worldwide after any country has been set
- MaxRetryErr HOT 8
- It broke again HOT 57
- Code documentation :)
- Broken for short timeframe - question for experts HOT 4
- related_queries and related_topics not working (infinite loop) HOT 4
- Anyone getting "Holes" in the data again? HOT 2
- Many tests fail
- Error with all request: HTTPSConnectionPool(host='trends.google.com', port=443): Max retries exceeded HOT 17
- Update to Selenium script HOT 4
- is that pytrends still unavailiable everyone? HOT 4
- Selenium script giving exception HOT 2
- Error 429 for the past few days HOT 14
- Interested in Maintainers
- KeyError: 'content-type' when calling build_payload()
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytrends.