Comments (12)
I've eliminated fake-user agent in 1.2.0 which might have been setting your location to another geo-graphical area (US) than your home location when you manually use the browser.
Please let me know if this is still an issue.
from pytrends.
I do have the same issue. How is this possible? Does the data provided in the US differ from the data provided in Europe?
from pytrends.
It is indexed by the number searches in the geographic region you filter it to, therefore the numbers would be different. You can set it to worldwide.
from pytrends.
My problem is that the data differ, even if exactly the same parameters are entered in the module and the web interface respectively.
E.g I call:
trend({'q':'pizza','date': '1/2014 12m','geo':'US'}, 'dataframe')
The first value of the returned dataframe is:
2014-01-05 78.0
But when I go to Google Trends and in the menu I choose: United States, 2014 and then enter 'pizza' (search term), the value of the first node is:
05 Jan - 11 Jan: 79
The difference gets even higher for the following nodes and I dont see, why these can be different?
Interestingly I found following weird appearance: I checked the url that is requested by the module.
(I changed export to '5' so you can see tha graph, it has no effect on the data)
The graph shows the data, that is generated by the module. On the right bottom of the page there is a link that says 'View full report in Google Trends'. By clicking this link you reach the web interface of Google Trends with the parameteres already put in the menu. But the data differ to the data on the page before! Do you see the difference?
It would be very nice if you could help me with that.
Thank you.
from pytrends.
If you download the CSV from the link on the chart you shared you will see
that the date ranges do not line up. One starts at Jan 5th, the other
starts at Jan 4th.
On Tue, Sep 20, 2016 at 1:16 PM, Nzteb [email protected] wrote:
My problem is that the data differ, even if exactly the same parameters
are entered in the module and the web interface respectively.E.g I call:
trend({'q':'pizza','date': '1/2014 12m','geo':'US'}, 'dataframe')
The first value of the returned dataframe is:
2014-01-05 78.0
But when I go to Google Trends and in the menu I choose: United States,
2014 and then enter 'pizza' (search term), the value of the first node is:05 Jan - 11 Jan: 79
The difference gets even higher for the following nodes and I dont see,
why these can be different?Interestingly I found following weird appearance: I checked the url that
is requested by the module.
(I changed export to '5' so you can see tha graph, it has no effect on the
data)https://www.google.com/trends/fetchComponent?cid=TIMESERIES_
GRAPH_0&date=1%2F2014+12m&q=pizza&export=5&geo=USThe graph shows the data, that is generated by the module. On the right
bottom of the page there is a link that says 'View full report in Google
Trends'. By clicking this link you reach the web interface of Google Trends
with the parameteres already put in the menu. But the data differ to the
data on the page before! Do you see the difference?It would be very nice if you could help me with that.
Thank you.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
#45 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGxCB-bllN0iQTm0_buCHVspF_iFqJvTks5qsCLtgaJpZM4JlZ4m
.
from pytrends.
Thank you but I dont see this. When I download the CSV from Google Trends it starts at 5.Jan. I dont see that there could be a different CSV to download. The chart as well as the dataframe from the module says the range starts at 5. Jan, too. Could you show me an example where the data from the module exactly fit the data obtained by Google Trends?
from pytrends.
- I go to the link you sent. The first point is Jan 5th - 11th 2014, the data value is 78.
- I click on the link in the corner "View full report in Google Trends" That one also says Jan 5th - 11th 2014 but the data value is 79.
- I click on the drop down and download to CSV. The first row says 2014-01-04, value 79.
My guess is Google is doing one inclusive of the date on visualization, and exclusive of the date on the other. Since the data is indexed it will be sensitive to the inclusive or exclusion of that date.
If I request the actual K,V data from Google Trends or inspect the JSON in the page source from the link you shared I'll see {"v":new Date(2014,0,5),"f":"Jan 4 \u2013 11, 2014"},{"v":78.0,"f":"78"} for the first record.
from pytrends.
Thank you for going into this.
- I click on the drop down and download to CSV. The first row says 2014-01-04, value 79.
When I change my computer settings to an US timezone and re-log in into Google, the first row says 2014-01-04, value 79 , too. With Central Europe timezone settings (as in my last posts) it says 2014-01-05, value 79 .
My guess is Google is doing one inclusive of the date on visualization, and exclusive of the date on the other. Since the data is indexed it will be sensitive to the inclusive or exclusion of that date.
Yes, that sounds plausible. Yet we still dont know for sure which periods have been used for which type of data (Google Interface vs. Module/Json) because the Json has both dates in it as well. And I really need to fetch data with the module, that can exactly be replicated from the Google Interface, so Iam still curious about if that is possible?
- I go to the link you sent. The first point is Jan 5th - 11th 2014, the data value is 78.
Well, yes but... what is going on here:
When I click the link again (it is one day later now), the value is not 78 anymore but it changed to 77?! (And of course the same with requesting it with the pytrends module) I know that the interpretation of the data is not affected much in general because the relation between single datapoints is only weakly changed (indexing). Still, if the data fetched today for a specific period differs from the data fetched tomorrow for the same period, Iam wondering how exactly the data is created and if I can use it for research. What is it, that I am missing?
from pytrends.
I have been researching this as I have also been having the same problem.
This comment on a different project may be relevant: PMassicotte/gtrendsR#117 (comment)
from pytrends.
Thank you for the link. The comment clears up some questions. I observed the GTrends interface data on different days and it turns out that they are not consistent either. As the comment supposes, the difference seems to be higher for non-frequent search terms. If the reason for the inconsistency is caused by different samples, this probably can be fixed by fetching the same data at different days and averaging them and therefore getting lower variance of the 'estimate'.
The question now is if the difference between GTrends interface and the link I posted (again: which is used to provide the pytrends module data) has the same reason. If so, it would be fine. But I feel as there is still some uncertainty in it.
from pytrends.
When I've got time I may re-release a temporary branch using the previous method of getting the data which is much clunkier, but apparently works.... If some one wants to get a headstart on that they can look at the version before v3.0
from pytrends.
Google trends API changed. They are calculated differently. I will switch over to the new API soon but it will take time.
from pytrends.
Related Issues (20)
- It broke again HOT 57
- Code documentation :)
- Broken for short timeframe - question for experts HOT 4
- related_queries and related_topics not working (infinite loop) HOT 4
- Anyone getting "Holes" in the data again? HOT 2
- Many tests fail
- Error with all request: HTTPSConnectionPool(host='trends.google.com', port=443): Max retries exceeded HOT 17
- Update to Selenium script HOT 4
- is that pytrends still unavailiable everyone? HOT 4
- Selenium script giving exception HOT 2
- Error 429 for the past few days HOT 14
- Interested in Maintainers
- KeyError: 'content-type' when calling build_payload()
- Is there an upper limit on display results? HOT 1
- 429 Error HOT 7
- I am interested in reverse engineering Google trends HOT 1
- daily serach trends
- Too many 429 error responses. Max retries exceeded. #578 HOT 3
- Search volume HOT 1
- Does related queries work? I keep getting 429 HOT 16
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytrends.