kencoken/imsearch-tools

Python utilities for automated download of images from various web sources

imsearch-tools's Issues

License?

Hi @kencoken : what would be the license for your code? Thanks!

Use pip to install

In the README, I think you mean pip install -r requirements.txt rather than python install -r requirements.txt?

flickr_api bugs

Tried query_demo.py with "num_results = 500" and "test_flickr_api=True". Failed with the following exceptions:
Executing Flickr API Search...
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
result = self._run(_self.args, *_self.kwargs)
File "/Users/peterwang/PyProjects/wearva/web_images/imsearchtools/engines/flickr_api.py", line 49, in _fetch_results_from_offset
num_results, self._results_per_req)
TypeError: not enough arguments for format string
<Greenlet at 0x106fd3410: <bound method FlickrAPISearch._fetch_results_from_offset of <imsearchtools.engines.flickr_api.FlickrAPISearch object at 0x105d8f590>>('hospital room', 0, aux_params={'sort': 'relevance', 'nojsoncallback': 1, 'format, num_results=500, headers={})> failed with TypeError

Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
result = self._run(_self.args, *_self.kwargs)
File "/Users/peterwang/PyProjects/wearva/web_images/imsearchtools/engines/flickr_api.py", line 49, in _fetch_results_from_offset
num_results, self._results_per_req)
TypeError: not enough arguments for format string
<Greenlet at 0x106fd34b0: <bound method FlickrAPISearch._fetch_results_from_offset of <imsearchtools.engines.flickr_api.FlickrAPISearch object at 0x105d8f590>>('hospital room', 100, aux_params={'sort': 'relevance', 'nojsoncallback': 1, 'format, num_results=500, headers={})> failed with TypeError

Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
result = self._run(_self.args, *_self.kwargs)
File "/Users/peterwang/PyProjects/wearva/web_images/imsearchtools/engines/flickr_api.py", line 49, in _fetch_results_from_offset
num_results, self._results_per_req)
TypeError: not enough arguments for format string
<Greenlet at 0x106fd3550: <bound method FlickrAPISearch._fetch_results_from_offset of <imsearchtools.engines.flickr_api.FlickrAPISearch object at 0x105d8f590>>('hospital room', 200, aux_params={'sort': 'relevance', 'nojsoncallback': 1, 'format, num_results=500, headers={})> failed with TypeError

Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
result = self._run(_self.args, *_self.kwargs)
File "/Users/peterwang/PyProjects/wearva/web_images/imsearchtools/engines/flickr_api.py", line 49, in _fetch_results_from_offset
num_results, self._results_per_req)
TypeError: not enough arguments for format string
<Greenlet at 0x106fd35f0: <bound method FlickrAPISearch._fetch_results_from_offset of <imsearchtools.engines.flickr_api.FlickrAPISearch object at 0x105d8f590>>('hospital room', 300, aux_params={'sort': 'relevance', 'nojsoncallback': 1, 'format, num_results=500, headers={})> failed with TypeError

Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/gevent/greenlet.py", line 327, in run
result = self._run(_self.args, *_self.kwargs)
File "/Users/peterwang/PyProjects/wearva/web_images/imsearchtools/engines/flickr_api.py", line 49, in _fetch_results_from_offset
num_results, self._results_per_req)
TypeError: not enough arguments for format string
<Greenlet at 0x106fd3730: <bound method FlickrAPISearch._fetch_results_from_offset of <imsearchtools.engines.flickr_api.FlickrAPISearch object at 0x105d8f590>>('hospital room', 400, aux_params={'sort': 'relevance', 'nojsoncallback': 1, 'format, num_results=500, headers={})> failed with TypeError

Traceback (most recent call last):
File "query_demo.py", line 104, in
num_results=num_results)
File "/Users/peterwang/PyProjects/wearva/web_images/imsearchtools/engines/flickr_api.py", line 99, in query
aux_params=aux_params)
File "/Users/peterwang/PyProjects/wearva/web_images/imsearchtools/engines/search_client.py", line 106, in _fetch_results
raise QueryException("No image URLs could be retrieved")
imsearchtools.engines.search_client.QueryException: No image URLs could be retrieved

Looks line 49 of flickr_api.py needs an extra set of (). Also, even with this fix, the current code doesn't allow more than self._results_per_req=100. I.e. instead of checking
if num_results > self._results_per_req:
and raising the exception, shouldn't there be a loop that cycles through all the pages instead?

license clarification

The project LICENSE.md and README.md files both specify that this is released under the MIT license. However, the text in LICENSE.md text is actually the one for the 2-clause BSD / FreeBSD.

So, under what license is this actually release? Is it the 2-clause BSD or is the MIT license.

can I download more than 100 images with GoogleWebsearch()??

I want to get images over 100.
Can I do this with imsearch-tools??

GoogleWebSearch return invalid image urls

For example, if my query is "ahi tuna", I will get urls like this:

http://s3.amazonaws.com/foodspotting-ec2/reviews/1881762/thumb_600.jpg%253F1339438780

However, it is not a valid url, instead the correct one is

http://s3.amazonaws.com/foodspotting-ec2/reviews/1881762/thumb_600.jpg

This is actually very common, for example, the query "abalone" will return examples like:

https://californiaoutdoors.files.wordpress.com/2014/10/measuring-abalone1.jpg%253Fw%253D500

http://www.gourmetsleuth.com/images/default-source/dictionary/abalone.jpg%253Fsfvrsn%253D4

Will you fix this?