Hey

[generalmills]

Hey

[dreyco676]

I'd love the help. I was thinking about how to parse the csv response into separate files for consumption while being flexible enough to handle changes.

Another issue that was brought to my attention is that there is a quota limit and we should try handling it and retrying if we reach it.

[mattreid9956]

So thanks for the commit rights, I will take a look, I have a few things to do over the next week or so but I will get involved soon.

About the quota limit.... That is an issue, I am really not sure how to avoid it, I think the randomised timer you have imposed is a good idea and may throw Google off the scent. That was the reason I added a sleep in the first place, but it sometimes hits the limit still. We should certainty add an exception handler for this. I have found that the quota limit is very random...

The problem with making the csv response code too involved is that Google could change the csv file format and then we're screwed. So I think you're correct in that for now it can easily be split into several smaller files: Interest Over Time, Regions, Cities, Searchers, Rising Searches. So that is 4 in total and they're usually separated by "\n\n" at the very least. Speak soon, should be good! I think as long as we get it into a useful output, then that is the main thing. I was interested in the Interest over time column for a given date range so that was what my code ended up doing and passing it to a python application called pandas.

Speak soon. Matt

[dreyco676]

We could just abstract the splitting of the csv response into just reading the first line of each section and then just write out the data until the next blank line and name the file searchterm_firstrow.csv. That way if they change City into Geographical Regions or switch them around we are still doing it correctly.