nasa-jpl-memex/image_space

Interactive Image similarity and Visual Search and Retrieval application

image_space's Issues

Find Similar Images slow or worse

I searched for 'girl' and opened the first image ("Sexy Schoolgirl Fetish", first metadata in the list is version: 1494086487025647600). When I click "Find similar images" it just processes - I left it for a couple of minutes to no avail.

Notification for "search completed" on click of "Find Similar Images" missing

Browser: Chrome
Version: Version 43.0.2357.124 (64-bit)

Steps to reproduce the issue:

1. Cick on Reset Search
2. Save any image
3. On the left panel, click on "i" of the saved image.
4. Once the modal appears, click on "Find Similar Images".

Result: The modal disappears but there is no notification that the search was completed successfully or with error.

Be smart about showing search buttons based on env variable presence

If background or dynamic searches are unavailable, do not display those search buttons.

Proxy Server Error on fetching image size distribution and Serial Number Disrtribution

Steps to reproduct issue -

1. Click on reset search
2. Click on 'Image Size Distribution' link or 'Serial Number Distribution' link on the left.

Result : The modal keeps trying to load data. Nothing happens.
On checking further, Console shows this error for both the options -

---

502 Proxy Error

<title>502 Proxy Error</title>

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /imagespace/api/v1/imagepivot.

Reason: Error reading from remote server

##

Missing features?

I don't see how the following works in the current implementation:

1. Where camera serial numbers are provided in the EXIF metadata, search the open web for pictures from the same camera.
2. Search by image size - I see where you can limit the search or initiate a new search based on image height or width once you've already done one search, but I don't see where you can easily specify image dimensions for a search or where you can launch a new search based on image size.

Pagination Widget

We should have a typical pagination widget to let people browse through, if they so wish, pages of results.

Instructions in separate dialog

Will need to make sure links to searches still work (i.e. they close the dialog).

Add image drag and drop functionality to tangelo app

Add Docker install to README

It is unclear that there is a docker container for image_space.
Would be great if we could make it explicit that there is one and then make it extremely easy for folks to find it and run it locally.
@jeffbaumes can you help me document this e.g. where is the docker container on DockerHub? Is it available for use?

How do you show the full size image?

@wmburke says: Previously clicking on the image would open it full size in a new browser window. Now, it doesn't seem possible to see the full size image at all.

I think this is important, especially if you're trying to see if it's the same person in two images - or get a better look at a tattoo, etc...

I would suggest another icon under the images (maybe the picture icon?) that when clicked opens it full size. This could be a browser window for now, but we should probably think about adding light box style functionality so that you could actually move through the images like you do through any web gallery.

List view for results

Add a toggle button for this. Log when a user switches modes.

Several .js files not located when deploying image_space locally

When I deploy locally (build instructions are pretty much bang on, took me around 30 minutes), unfortunately when I open the main image_space app on http://localhost:8080 I get a blank screen, I also get the following screen shot.

Only one of the three JS files is referenced within Gruntfile.js as below

 64         files[staticDir + '/main.min.js'] = [
 65             jsDir + '/main.js'
 66         ];

AFAICS the others are not listed in any other code within image_space.
This is an issue as they are referenced for import and are missing.

Suggestion: Missing notification on Logout and app state not changed on Logout

Browser: Chrome
Version: Version 43.0.2357.124 (64-bit)

Not sure how important the log in feature is for pilot testing, but we might want to add a notification once a user has logged out saying "You have signed out".

Also, once a user signs out, the app's state remains to the screen of last performed search.
For example,

Steps:

1. I have a user account created with username : sthapar
2. I performed a search on an image from the database.
3. I logged out.

Result: After logging out, the screen still shows the last search results page. We could bring it back to the Home Page of ImageSpace.

Highlight text that was searched for

Register/Login functionality

@wmburke says: If I understand correctly, this functionality is not currently functioning. Can we disable it or comment it out then?

Fix token becoming out of date for saved images

Possible issue with Docker install

Hi Folks,
Working on integration of Georgetown's REST services into image_space, I tried to recently build the docker image as

lmcgibbn@LMC-032857 /usr/local/image_space/imagespace(master) $ ./start-imagespace-docker.sh

this resulted in the below output

loads of steps...
...
Done, without errors.
 ---> bd9c6c0bfe75
Removing intermediate container 606df988bd6c
Step 24 : ENV KWDEMO_READY TRUE
 ---> Running in 3b5a070226ea
 ---> 8c73d76b8bf7
Removing intermediate container 3b5a070226ea
Step 25 : WORKDIR /home/kwuser/girder
 ---> Running in 953500bfd15e
 ---> 7de34aaaad43
Removing intermediate container 953500bfd15e
Step 26 : CMD sh -c python -m girder
 ---> Running in a18fcae6c882
 ---> fc2fbeb58515
Removing intermediate container a18fcae6c882
Successfully built fc2fbeb58515
/usr/bin/python: No module named cherrypy; 'girder' is a package and cannot be directly executed

The bottom message is concerning, I am not sure if this is OK or not.

Strip html from ad text

On login/logout re-render sidebar

Show highlighted query image in images similarity search results

Search by ad id

Check in javascript and/or girder for query length

Too long will kill REST API.

Make ad/source of image easily findable

We need to add this functionality to make the image searching useful in a broader context.

I've been thinking this, but we are also getting this feedback live from the User Surrogate Group right now.

Ability to save relevant images

I know this is not something previously discussed but as of now, I don't see a way to save images and find them again later. For the searches, you mention that you can bookmark the url in the browser, but I don't see where there is any permalink for an image.

Is this possible?

Image similarity uses

Suggestion from USG: show a single graph from an uploaded photo (or from a single other photo that was selected).

Also, the interactive query tool will be REALLY useful here for identifying bad connections and fake photos (there are a lot of fake photos, identifying real photos is also useful -- although probably needs to be done by a human).

Would be great if this could also be used as a connection type between ads - in conjunction with identifying as fake or real.

Sidebar height is off and not all text is visible in Firefox

The instructions in the sidebar on the left are not entirely visible. When I scroll to the bottom, I can't read the last line - seems the height of the div is off somehow.

I'm on Ubuntu 14.04 with Firefox 36.0.1.

It works fine in Chrome however...

Text search on original ad text

Update README to mention IMAGE_SPACE_LISTS

Required by FLANN, e.g.,

IMAGE_SPACE_LISTS=/home/kwuser/image_space/flann_index/lists.txt

Instructions for format of file upload

The instructions page can have a line stating the format of files that ImageSpace supports.

Eg: Allowed file format: .png, .jpg, .jpeg
That will also give users a hint to avoid formats such as pdfs, svgs etc.

Deploy

Are the deploy docs found here up to date? I just tried deploying this locally and the site is slightly different compared to the live site on the memex box.

cc @jeffbaumes

Demo images?

Should we provide demo images? It could be a set of links in the instructions where they could download them from.

(although I just uploaded a picture of squirrel and got some interesting hits :)

UI allows multiple saved versions of the same image

Only store one version.

A means by which to launch an image search from another app

For example, if the end user could right click on an image in TellFinder and choose to search this image in ImageSpace. Is that easy? It may be something we want to work out at the QPR.

Show image size while hovering over distribution graph

Link below image to search for similar images

Connection to open other tools - TellFinder - DIG - HG

Reverse "unblur" controls to "blur" controls

Need to show how images are similar

We currently have 3 or 4 ways of identifying images as similar when we search for similar images from a given image. We need to find a way to make it obvious how it is that these images are similar.

This is not an immediate problem that needs to be solved, but it should be solved before we enter a pilot.

When an image comes up in a list of similar images, it would be helpful to know how each one is similar. Do they share common EXIF data? Is it through the image histogram? Is there OCR'd text? Some of these can be considered soft connections and others hard. Analysts will want to know these connections.

Potentially, the expansion of the Image Size and Serial Number capabilities in the left side bar could help with this also - a sort of faceted search that would narrow the manner in which the shown images are connected. This is just a brainstormed thought - so after further consideration, it may not be a good answer, but I'm sharing it here anyway.

Searching for images similar to uploaded image having SVG format not working

Issue appears in : Broswer - Google Chrome, Version 43.0.2357.124 (64-bit)

Steps to reproduct issue -

1. Upload an SVG image.
2. Click on search icon at the bottom of uploaded image on the left

Result: The page keeps showing a loading icon and nothing appears.
Console shows the following error -

---

GET https://imagecat.memexproxy.com/imagespace/api/v1/imagesearch?url=https%3A%…3FSCiBFXzqzqHD0WSXQBYCRGQM06pi7vVCJwq6aLU3iQI31&histogram=%5B%5D&limit=100 500 (Internal Server Error)jQuery.ajaxTransport.send @ jquery.js:8623jQuery.extend.ajax @ jquery.js:8152Backbone.ajax @ backbone.js:1209_.extend.restRequest @ init.js:137imagespace.views.LayoutUserDataView.imagespace.View.extend.findSimilarImages @ UserDataView.js:130(anonymous function) @ UserDataView.js:101jQuery.Callbacks.fire @ jquery.js:3073jQuery.Callbacks.self.fireWith @ jquery.js:3185done @ jquery.js:8251jQuery.ajaxTransport.send.callback @ jquery.js:8598
init.js:120 500 Internal Server Error {"message": "ValueError: ValueError('No JSON object could be decoded',)", "trace": [["girder/api/rest.py", 279, "endpointDecorator", "val = fun(self, args, kwargs)"], ["girder/api/rest.py", 759, "GET", "return self.handleRoute('GET', path, params)"], ["girder/api/rest.py", 561, "handleRoute", "val = handler(*_kwargs)"], ["girder/api/access.py", 62, "accessDecorator", "return fun(_args, *_kwargs)"], ["/home/jbaumes/girder/plugins/imagespace/server/imagesearch_rest.py", 44, "getImageSearch", "'?url=' + params['url'] + '&num=' + str(limit), verify=False)"], ["/home/jbaumes/girder/env/lib/python2.7/site-packages/requests/models.py", 763, "json", "return json.loads(self.text, *_kwargs)"], ["/usr/lib64/python2.7/json/init.py", 338, "loads", "return _default_decoder.decode(s)"], ["/usr/lib64/python2.7/json/decoder.py", 365, "decode", "obj, end = self.raw_decode(s, idx=_w(s, 0).end())"], ["/usr/lib64/python2.7/json/decoder.py", 383, "raw_decode", "raise ValueError("No JSON object could be decoded")"]], "type": "internal"}

---

Dropping image opens the details window

Minor interface tweaks

@wmburke says: I think these should be simple and can be handled with text and/or html, so I'm grouping them together:

1. A button to show all images. It's not too bad to click in the search bar and then hit enter, unless you're coming from a page that already has the search field populated. Then you have to remember how to do it.
2. A note on the Image Size Distribution and Serial Number Distribution graphs that says that they are only showing results from the current search.
3. If there are no results from the current search for these two graphs, let's say that: "No results found for current search terms." As it is, it's just a bit confusing.

Make dark top bar to match more closely with Memex Explorer

Suggestion: Remove redundant attributes in search bar

Issue appears in : Broswer - Google Chrome, Version 43.0.2357.124 (64-bit)

Steps to reproduce this :

1. click on Reset Search
2. Click on 'i' of any image.
3. Click on '+' of an attribute to add to search.
4. Again Click on 'i' of the image and click on '+' of the same attribute.
   Screenshot of the same is attached.

Remove flask app

We aren't using the flask app anymore. We have it in the commit history if we ever wanted to go back. Let's remove it.

Clicking on image opens details window (maybe hover text for this)

Details button would no longer be needed

Search Result Widget

We should have a search result widget which displays the following

showing results of 1 - 20 of 100,396

Adding "Source Query" to search bar from a saved image's attributes throws error

Issue appears in : Broswer - Google Chrome, Version 43.0.2357.124 (64-bit)

Steps to reproduce this :

1. Click on Reset Search
2. Save any image by clicking on any image's save icon.
3. Scroll down to "Source Query" attribute which comes as the last attribute in the list.
4. Click on Search icon or plus icon on the right of Source Query attribute.

Result: Shows error on top right of the page.
Console shows -

---

500 Internal Server Error {"message": "KeyError: KeyError('response',)", "trace": [["girder/api/rest.py", 279, "endpointDecorator", "val = fun(self, args, kwargs)"], ["girder/api/rest.py", 759, "GET", "return self.handleRoute('GET', path, params)"], ["girder/api/rest.py", 561, "handleRoute", "val = handler(*_kwargs)"], ["girder/api/access.py", 62, "accessDecorator", "return fun(_args, **kwargs)"], ["/home/jbaumes/girder/plugins/imagespace/server/imagesearch_rest.py", 53, "getImageSearch", "return result['response']['docs']"]], "type": "internal"}

---

ImageSpace should call tika-python

We currently call Tika from the command line in Image Space. This should be updated to use tika-python.

Suggestion: Show notification when search is happening after clicking on '+' in Image Details

Steps to reproduce this :

1. click on Reset Search
2. Click on 'i' of any image.
3. Click on '+' of an attribute to add to search.

Result : Resulting images are shown, but no notification is shown that a search is being carried out when adding a new attribute to the search.

Remove tangelo initial app

We should also remove the tangelo initial app since we are using girder now. We have it in the commit history if we ever needed to go back.