cedws/finch

A proof-of-concept for enhancing and organising an image collection using Google's Vision API

finch's Issues

Finch attempts to process file types incompatible with the Vision API.

Resolved in 515dae9.

Provision Cloud Storage on-the-fly.

The Vision API has a request body limit of 10MB, which is easily reached when encoding high quality images to Base64.

It's possible to pass a Google Cloud Storage URI to the Vision API. If the images being processed are pushed to Cloud Storage, the process will be faster overall, because all web detection queries can be bundled into one request. It will also allow images of up to 20MB to be processed.

I would like Finch to provision a Cloud Storage container on-the-fly, upload the images to be processed, run the Vision API query, and then deallocate the container to avoid incurring unnecessary costs.

Investigate Yandex's reverse image search.

Yandex's reverse image search seems to be superior to Google's in every way. Google has also shown little interest in fixing the API quirks that I reported. I haven't been able to find an API for Yandex but perhaps there's another way (preferably without HTML scraping).

Filesize check is incorrect.

The Vision API limits the JSON request to 10MB, not the Base64 encoded image. This will cause issues in edge cases.

Use `serde` to serialize request JSON.

Check resolution of image without downloading.

ImageMagick's ping command allows the resolution of an image to be determined without fully downloading it. This could help speed things up and save bandwidth.

Push to package registry.

Implement Marr wavelet for more accurate perceptual hashes.

Implement tests.

Diagnose Marr wavelet output difference.

The output of the Marr wavelet implementation is different to pHash's.

Implement image reorganisation.

• Implement option for output image type
• Implement option for output image name

Finch fetches and overwrites images even if the webserver serves a dummy image.

After running Finch on a couple of thousand images, I noticed that some sites have expiry dates on the URLs given by the Vision API, and will serve dummy/error images whilst still returning 200 OK.

The only way I can think of fixing this is to introduce a perceptual hash algorithm to check that the fetched image is correct.

Use gRPC instead of REST API.

The gRPC offers increased performance, and no 10MB request body limit like the REST API has. This also means we can avoid provisioning a storage Bucket for the sake of circumventing these limits.

Finch treats directories as files.

The glob function is also returning directories. Finch attempts to upload directories as a regular file and results in a panic.

Finch should make requests asynchronously, not in parallel.

The parallel implementation is fairly inefficient. The goal is to open as many API requests as possible at once as each response can take several seconds. However, this is bottlenecked by the upload speed anyway. A better way to do this would be to asynchronously make API requests.

Score alternate versions based on the image aspect ratio.

Sometimes the first images are higher resolution, but don't have the same aspect ratio as the target image. Images should be scored based on the perceptual hash similarity and aspect ratio similarity.

Finch overwrites images regardless of whether the saved version is equal or greater resolution.

I seem to remember fixing this, but I guess it got lost along the way somewhere.