googlecloudplatform/cloud-vision

Sample code for Google Cloud Vision

This repo has been deprecated. All new work will be found at a given language's repo:

C# / .NET | Go lang | Java | Node.js | PHP | Python | Ruby

Note: The Android and IOS samples haven't been moved to the main Android and IOS sample repos yet.

---

Google Cloud Vision API examples

This repo contains some Google Cloud Vision API examples.

The samples are organized by language and mobile platform.

Language Examples

Landmark Detection Using Google Cloud Storage

This sample identifies a landmark within an image stored on Google Cloud Storage.

• Documentation and Java Code
• Documentation and Python Code

Face Detection

See the face detection tutorial in the docs.

• Python Code
• Java Code

Label Detection

See the label detection tutorial in the docs.

• Python Code
• Java Code

Label Tagging Using Kubernetes

Awwvision is a Kubernetes and Cloud Vision API sample that uses the Vision API to classify (label) images from Reddit's /r/aww subreddit, and display the labelled results in a web application.

• Documentation and Python Code

Text Detection Using the Vision API

This sample uses TEXT_DETECTION Vision API requests to build an inverted index from the stemmed words found in the images, and stores that index in a Redis database. The resulting index can be queried to find images that match a given set of words, and to list text that was found in each matching image.

For finding stopwords and doing stemming, the Python example uses the nltk (Natural Language Toolkit) library. The Java example uses the OpenNLP library.

• Documentation and Python Code
• Documentation and Java Code

Mobile Platform Examples

Image Detection Using Android Device Photos

This simple single-activity sample that shows you how to make a call to the Cloud Vision API with an image picked from your device’s gallery.

• Documentation and Android Code

Image Detection Using iOS Device Photos

The Swift and Objective-C versions of this app use the Vision API to run label and face detection on an image from the device's photo library. The resulting labels and face metadata from the API response are displayed in the UI.

Check out the Swift or Objective-C READMEs for specific getting started instructions.

• Documentation (Objective-C)

• Documentation (Swift)

• iOS Sample Code

cloud-vision's Issues

Question about Text Detection Java Project Setup

Hi,

I'm trying to setup Text Detection Java Project. But, I don't know how to configure OpenNLP.

In README says:
"Set up OpenNLP
Download Tokenizer data and save it to this directory.
wget http://opennlp.sourceforge.net/models-1.5/en-token.bin"

When you say "save it to this directory", where should it be? Project's root directory?

Regards,

Android Example outdated. The used library will be deprecated soon

The documentation shows a message to not use com.google.apis:google-api-services-vision:v1 for newer projects because the library actively maintained will be com.google.cloud:google-cloud-vision that cannot be used in the same way you have done in the sample since some method are missing and It uses a different procedure to access the cloud service.
Since this is the only official sample by Google. It should be updated as soon as possible.

Example iOS app 403s

I followed the README and configured the Cloud Vision API key. Now I'm getting "Error code 403: Requests from this ios client application 'empty' are blocked". Any ideas?

Showing Progress

I was requesting if you could assist me in showing a progressbar by using the onProgressUpdate method in the asynctask

ImageSource supports "gcs" at moment, NO "imageUri" .

I saw the official RESTApi document, there should be an imageUri that supports normal public address. But I can not find it in Android SDK.

Will it be planned to add ?

BoundingPoly for LABEL_DETECTION

This may not be relevant to be an issue in this repository, but it might be in everyone's best interest.

I have been wondering when or if there are any plans of implementing boundingPoly's for the label detection feature.

I know it is hard but it just seems so great to imagine such a feature.

Many thanks.

p.s. if there are any workarounds for this matter, that would be great to know.

Add all samples to Travis config for unit & systems testing

This is a multi-language repo, so we will have to be a bit clever in how we set up the test matrix.

Platforms needed to build & test:

• Android
• Chrome
• Go
• Java
• Node.JS
• PHP
• Python
• iOS

Sample App for Node.js client

Hi, This isn't an issue, but I didn't know where to post.

I have a sample Facebook Messenger Bot using the Google Cloud Vision API.

I'm leaving the link for it here for people who might want to refer to it.

fb-vision-bot

Thank you.

Invalid JSON payload received. Unknown name "images" at \'requests[0]\': Cannot find field

     { error: 
         { code: 400,
           message: 'Invalid JSON payload received. Unknown name "images" at \'requests[0]\': Cannot find field.',
           status: 'INVALID_ARGUMENT',
           details: 
            [ { '@type': 'type.googleapis.com/google.rpc.BadRequest',
                fieldViolations: 
                 [ { field: 'requests[0]',
                     description: 'Invalid JSON payload received. Unknown name "images" at \'requests[0]\': Cannot find field.' } ] } ] } } } }

payload =  {
              requests: [
                {
                  images: {
                    "content": base64Data.replace("data:image/jpeg;base64,", "")
                  },
                  features: [
                    {
                      type: "TEXT_DETECTION"
                    }
                  ]
                }
              ]
            }

  axios.post('https://vision.googleapis.com/v1/images:annotate?key=AIzaSyAp9SJv4H5ibESc4aFG28SBdP51jNqOccc',
    JSON.stringify(payload),
    {
      headers: {
        "Content-Type": "application/json",
        "Content-Length": base64Data.replace("data:image/jpeg;base64,", "").length
      }
    }
  )
  .then(response => {
    console.log(response);
  })
  .catch(err => {
     console.log(util.inspect(err, {depth: null}));
  })

PHP example not working on AppEngine

The example works great locally, but fails in AppEngine. I get a curl error #7, cannot connect to host. From some reading online, it looks like https://vision.googleapis.com calls are blocked by AppEngine.

Can anyone confirm this? Any work arounds?

Do Vision API support Indian Language like Malayalam?

Hi Sir,
Sorry to post this on issues part. My Concern is , do google Vision API Support South Indian language like Malayalam. In web URL: https://cloud.google.com/vision/, I tried English image it works , but Malayalam image is not . If it allow , we plan a mobile app using Vision API for blind community for OCR Purpose in Malayalam. Waiting Your fast reply soon
N.B I attach a sample malayalam image to check you for OCR...

Anes

"Double Face" Problems

I've started to play with the Google cloud-vision api recently.

I have tested a very interesting problem: detect the 'double face' problem.

I have randomly selected three 'double face' images:
1: df1 ("https://cdn1.truelancer.com/upload-full/58198-Head-Turning-Illusion.jpg")
2: df2 ("http://pixdaus.com/files/items/pics/9/57/579957_1d372cf3eac3bdbdb39c9f0c89ce62d2_large.jpg")
3: df3 ("https://s-media-cache-ak0.pinimg.com/236x/9e/9f/bc/9e9fbc6f6d921a3a28d7849f67399bc8--profile-pics-face-profile.jpg"

I have used features "FACE_DETECTION" and "LABEL_DETECTION".

It is interesting, only df3 returned results for 'FACE_DETECTION", other all returned 'No features detected!'

But all of them will return correct labels when I used 'LABEL_DETECTION',

For example, df1 will return :

    mid     description     score

1 /m/0dzct face 0.9518243
2 /m/0f9swq chin 0.9191782
3 /m/0k0pj nose 0.9074579
4 /m/04hgtk head 0.8594943
5 /m/01g6gs black and white 0.8463373

.......

Any thoughts?

Use gimme to test samples on multiple versions of Go

Multilaguage support android

Which languages are supported by vision api for ocr.

Text detection: preserve document structure ?

Hi,

I'm currently trying the OCR provided by Cloud Vision in Python. The results are good. Most of the time it recognizes the text of the picture. But I want to know if there is some way to preserve document structure while getting the results.

For example I want to be able to read a spreadsheet line by line.

Thanks for your help!

Text detection working in debug apk but not working in release apk..

Text detection working in debug apk but not working in release apk with same api key..wrong json created in release apk..

Error code: 403

Hi there, I downloaded your project and followed the steps. I just replaced API_KEY with my key in ImagePickerViewController. But when I selected an image the following error appeared: Error code 403: Requests from this ios client application com.google.cloud.vision.sample.imagePicker are blocked. Can you guide me through this?

*When I used my browser key it worked but not working on my iOS key.

Many thanks.

URL

when I using URL for images it is show me nothing for features

Broken PHP Sample link

This link is broken

https://github.com/GoogleCloudPlatform/php-docs-samples/tree/master/vision/api

Javascript output

Hi, this is an ouput from Java using eclipse

and this is an output from javascript

Our goal is to identify the sentiment of the faces being detected. In java, it can only detect faces, but no sentiment analysis. In Javascript, it can detect and identify the sentiment. Our problem is, it does not return pictures for us to easily know how many faces are detected. Is there a way for this?

Table structure is not scanning properly.

I can't say this is an issue, need guidance on this,
I am having invoice like table structure and I need all details of it from vision API but its not returning accurate result.
Facing problems mentioned below, Can anyone help on this?

Problem 1 - Skipping some columns

Problem 2 - Skipping values of some columns

Cloud Vision API request fails if you try to restict API Key usage to a specific Android App

If you try to limit API key usage to a specific Android App, requests fails despite its package and digital signature are whitelisted, forcing you to use unrestricted access.

I want to print out all the text present in an image.

I am trying to use Vision APi for ANPR. Instead of passing queries, i just want to know the text in an image(license plate in the picture of the car). How do i do that?

Error with SwiftyJSON 3.1.1

When following the instructions and running pod init in the swift folder, the command prompt returns the following message.
Looks this is outdated

Installing SwiftyJSON (3.1.1)

[!] Error installing SwiftyJSON
[!] /Users/MeAccount/Applications/CocoaPods.app/Contents/Resources/bundle/bin/git clone https://github.com/SwiftyJSON/SwiftyJSON.git /var/folders/sp/gqx5nqn9095dh0qx64rww0dm0000gn/T/d20170731-32862-152q905 --template= --single-branch --depth 1 --branch v3.1.1

Cloning into '/var/folders/sp/gqx5nqn9095dh0qx64rww0dm0000gn/T/d20170731-32862-152q905'...
warning: Could not find remote branch v3.1.1 to clone.
fatal: Remote branch v3.1.1 not found in upstream origin

error Error: ENOENT, stat 'google-cloud/vision'

Please make it so that I can access what I need to access to complete https://cloud.google.com/vision/docs/reference/libraries#client-libraries-install-nodejs

See: npm/npm#18390

Go Cloud vision samples have moved

Can you link to these instead, please?

https://github.com/GoogleCloudPlatform/golang-samples/tree/master/vision

Errors : error Bad Request

How do I fix this?

Unable to execute command ./textindex.py ../../data/text/

The redis server is running. The credentials file for the API are set.

But when I try to execute- ./textindex.py /Users/.../.../cloud-vision-master/data/text

I get-
Traceback (most recent call last):
File "./textindex.py", line 56, in
from googleapiclient import discovery
ImportError: No module named googleapiclient

Vision-Label Detection returns blank response for a specific image

Hi,
When using the Label Detection in Vision API, during my testing one specific image doesnt get labelled at all. Infact it breaks the label.py code since the response is empty.

Steps to reproduce :
Use label.py to score image attached with this issue.

My Testing (I Changed label.py code to print complete 'response' object):

c:\BigData\google\api\python-docs-samples\vision\api\label>python label.py ./res
ources/nailpaint.jpg
{'responses': [{'labelAnnotations': [{'score': 0.96202141, 'mid': '/m/0p833', 'd
escription': 'perfume'}]}]}

c:\BigData\google\api\python-docs-samples\vision\api\label>python label.py ./res
ources/nailclipper.jpg
{'responses': [{}]}

Missing parent POM?

Did I miss a sub-file somehow? I downloaded the ZIP file, and got this

pwd
/Users/ME/Downloads/cloud-vision-master/java/face_detection

mvn clean compile assembly:single
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for com.google.cloud:vision-samples:1.0.0: Failure to find com.google.cloud:shared-configuration:pom:1.0.0 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ com.google.cloud:vision-samples:1.0.0, /Users/ME/Downloads/cloud-vision-master/java/pom.xml, line 26, column 11
 @ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project com.google.cloud.vision.samples:face-detection:1.0-SNAPSHOT (/Users/ME/Downloads/cloud-vision-master/java/face_detection/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for com.google.cloud:vision-samples:1.0.0: Failure to find com.google.cloud:shared-configuration:pom:1.0.0 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ com.google.cloud:vision-samples:1.0.0, /Users/ME/Downloads/cloud-vision-master/java/pom.xml, line 26, column 11 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

annotateImageRequest

annotateImageRequest doesnt allow to pass an URL, while using NodeJS its possible to make it.

https://developers.google.com/resources/api-libraries/documentation/vision/v1/java/latest/com/google/api/services/vision/v1/model/AnnotateImageRequest.html

Example NodeJS have introduced::
https://github.com/tejitak/node-cloud-vision-api/blob/master/test_annotate_remote.js

Debugging: Request create an Image Object with URL param while our SetImage object just accept Image

Cleaning Up Cloud Vision OCR Text Detection responses?

Is there a feature that I can pass into my request that will parse the OCR text I'm getting back into, for example, english words and names? I'm having difficulty doing such on the client side while maintaining performance.

Make sure Travis passes for non-branch pull requests

Ran into this in #49.

Maven fails on non-branch PRs. https://travis-ci.org/GoogleCloudPlatform/cloud-vision/jobs/116719540

Running com.google.cloud.vision.samples.facedetect.FaceDetectAppIT
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.252 sec <<< FAILURE! - in com.google.cloud.vision.samples.facedetect.FaceDetectAppIT
detectFaces_withFace_returnsAtLeastOneFace(com.google.cloud.vision.samples.facedetect.FaceDetectAppIT)  Time elapsed: 0.173 sec  <<< ERROR!
java.lang.IllegalArgumentException: no JSON input found
    at com.google.cloud.vision.samples.facedetect.FaceDetectAppIT.setUp(FaceDetectAppIT.java:44)
detectFaces_badImage_throwsException(com.google.cloud.vision.samples.facedetect.FaceDetectAppIT)  Time elapsed: 0.001 sec  <<< ERROR!
java.lang.IllegalArgumentException: no JSON input found
    at com.google.cloud.vision.samples.facedetect.FaceDetectAppIT.setUp(FaceDetectAppIT.java:44)

Go fails on non-branch PRs https://travis-ci.org/GoogleCloudPlatform/cloud-vision/jobs/116719539

++go vet ./go/...
++'[' -f /home/travis/build/GoogleCloudPlatform/cloud-vision/client-secret.json ']'
++go test -v ./go/...
=== RUN   TestLabel
--- FAIL: TestLabel (0.13s)
    Error Trace:    label_test.go:40
    Error:      Received unexpected error "google: error getting credentials using GOOGLE_APPLICATION_CREDENTIALS environment variable: unexpected end of JSON input"

FileUriExposedException in Android Sample

android.os.FileUriExposedException: file:///storage/emulated/0/Pictures/temp.jpg exposed beyond app through ClipData.Item.getUri()

Seems the file access for picture taken is not in a proper way.

CloudVision, returns an error "Requests from this Android client application <empty> are blocked"

sample.google.com.cloudvision D/MainActivity﹕ failed to make API request because {
"code": 403,
"errors": [
{
"domain": "global",
"message": "Requests from this Android client application are blocked.",
"reason": "forbidden"
}
],
"message": "Requests from this Android client application are blocked.",
"status": "PERMISSION_DENIED"
}

OCR missing some numbers

Hi,

I'm currently running Google vision through Python on an image file attached below.

Most of the numbers are read, but some numbers are missing. For example, row 16 column 4 value -177 missing. Most of row 15 is missing. There are other errors, but the main point is how to fix this issue? I don't really care about the accuracy of reading the Chinese words. I've tried making the image into black and white, removing background, making it 8 bit, making it binary, and various combinations of them. None achieved perfect recognition of the numbers.

Bad request - 400

I'm using an untouched Android sample, and the API returns me a bad request 400 looking like this:

{ "code": 400, "message": "Invalid JSON payload received. Unknown name \"c\": Cannot find field.", "errors": [ { "message": "Invalid JSON payload received. Unknown name \"c\": Cannot find field.", "domain": "global", "reason": "badRequest" } ], "status": "INVALID_ARGUMENT" }

The JSON looks like this:

{c=[{c=[{c=10, d=LABEL_DETECTION}], d={c=_9j_4AAQSkZJRgABAQAAAQABAAD_2wBDAKABQoUKAP_9k}}]}

I shortened the Base64 image string obviously.

camera option not working

TEXT_DETECTION is never getting result for Android device

I was getting result for LABEL_DETECTION perfectly but when I change it toTEXT_DETECTION, I am always getting zero result. I tried for many OCRs but no success. Experiment was done only for Android.

I did following changes in MainActivity.java file.

- Feature labelDetection = new Feature();
-                            labelDetection.setType("LABEL_DETECTION");
-                            labelDetection.setMaxResults(10);
-                            add(labelDetection);

+ Feature textDetection = new Feature();
+                           textDetection.setType("TEXT_DETECTION");
+                           textDetection.setMaxResults(10);
+                           add(textDetection);

I tried with various text, such as one from wikipedia.

Chrome label detection not functional

I installed a build of the Chrome extension from the webstore and although "Face detection" displays an image and "Text detection" copies to my clipboard, "label detection" doesn't appear to have any effect.

Deprecate PHP sample in `cloud-vision/php/`

There is a better sample at:
https://github.com/GoogleCloudPlatform/php-docs-samples/tree/master/vision

Invalid JSON payload received. Unknown name \"language_hints\" at 'requests[0]': Cannot find field.

Trying out vision api, using this request payload (minified the base 64) always returns error 400 on unkown language_hints, can't find where should language_hints should be placed

{
   "requests":[{
      "image": {
         "content":"iVBORw0KGgoAAAANSUhEUgAABuAAAAE....="
      },
      "features": [{
         "type": "DOCUMENT_TEXT_DETECTION",
         "maxResults": 200,
         "languageHints" : ["pt"],
      }], 
   }]
}

and sending this returns

{
    "error": {
        "code": 400,
        "message": "Invalid JSON payload received. Unknown name \"language_hints\" at 'requests[0]': Cannot find field.",
        "status": "INVALID_ARGUMENT",
        "details": [
            {
                "@type": "type.googleapis.com/google.rpc.BadRequest",
                "fieldViolations": [
                    {
                        "field": "requests[0]",
                        "description": "Invalid JSON payload received. Unknown name \"language_hints\" at 'requests[0]': Cannot find field."
                    }
                ]
            }
        ]
    }
}

TypeError(repr(o) + " is not JSON serializable")

Running the example faces.py using google-api-python-client (1.5.0) in python (3.5.1)
I get the following error TypeError(repr(o) + " is not JSON serializable")

Traceback (most recent call last):
  File "C:/git/code/python/scripts/GoogleVisionAPI/faces.py", line 83, in <module>
    main('face-input.jpg', 'out.jpg')
  File "C:/git/code/python/scripts/GoogleVisionAPI/faces.py", line 68, in main
    faces = detect_face(image)
  File "C:/git/code/python/scripts/GoogleVisionAPI/faces.py", line 41, in detect_face
    request = service.images().annotate(body={'requests': batch_request,})
  File "C:\Anaconda3\lib\site-packages\googleapiclient\discovery.py", line 743, in method
    actual_path_params, actual_query_params, body_value)
  File "C:\Anaconda3\lib\site-packages\googleapiclient\model.py", line 149, in request
    body_value = self.serialize(body_value)
  File "C:\Anaconda3\lib\site-packages\googleapiclient\model.py", line 258, in serialize
    return json.dumps(body_value)
  File "C:\Anaconda3\lib\json\__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "C:\Anaconda3\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "C:\Anaconda3\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "C:\Anaconda3\lib\json\encoder.py", line 180, in default
    raise TypeError(repr(o) + " is not JSON serializable")

Client Not Synched Up to Server Update _KEY_MAP values Missing

Hi,

There seems to be some changes in JSON response returned from the server, but the client library is not up-to-date.

For instance, I see that there is now a key called fullTextAnnotation returned by the server, but the corresponding _KEY_MAP is missing this key in the client library, causing KeyError.

Would there be a possibility of cross checking the versions in the future?

Unicode fractions not detected for TEXT_DETECTION

When using the cloud vision api for OCR, it doesn't like unicode fractions. I'm using it for a recipe project I'm working on, so fractions are kind of a big deal ;).

Loving the Vision API!

Following the text detection guide I'm always getting no hits in the index lookup

The redis server is running. The credentials file for the API are set.
Now when I execute:
./textindex.py ../../data/text/

I get:
../../data/text/sunbeamkitties.jpg already added to index.
../../data/text/mountain.jpg already added to index.
../../data/text/sabertooth.gif already added to index.
../../data/text/succulents.jpg already added to index.
../../data/text/bonito.gif already added to index.
../../data/text/wakeupcat.jpg already added to index.

Then I use python to try to find text:

import textindex
index = textindex.Index()
index.print_lookup('cats')
No hits found.
index.print_lookup('up')
No hits found.

Something is not working... I've tried with the 'up' string that is clearly in wakeupcat.jpg file, any idea?

Thank you.

We can not access the URL currently. Please download the content and pass it in.

When working on the remote label detection i get this message on many (like really many) accessible and existing images.
e.g.
http://heizung-dinges.de/assets/images/Koebig2.jpg

Also some other times i will get the message Bad image data. On different existing and accessible images.

Re-requesting the API is not working, same exact errors will occur.

KeyError: 'faceAnnotations' on images not containing any faces

When I pass an image containing no faces, I should get 0 as a result, instead I get this:

faces = response['responses'][0]['faceAnnotations'] KeyError: 'faceAnnotations'

I'm using your python code for face detection. And I'm not sure every one of my images will contain a face or not.

Broken Links in README.md [404]

Broken Links
Landmark Detection Using Google Cloud Storage:
-Documentation and Java Code

Face Detection
-Python Code
-Java Code

Label Detection
-Python Code
-Java Code

Text Detection Using the Vision API
-Documentation and Java Code

HttpError 403 when requesting /images:annotate from GCE"

Hi,

I get the following exception when running the example application on GCE. The cloud Vision API has been activated for the project.

Traceback (most recent call last):
File "vision-test.py", line 35, in
response = service_request.execute(num_retries=num_retries)
File "/usr/lib/anaconda/lib/python2.7/site-packages/oauth2client/util.py", line 140, in positional_wrapper
return wrapped(_args, *_kwargs)
File "/usr/lib/anaconda/lib/python2.7/site-packages/googleapiclient/http.py", line 729, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://vision.googleapis.com/v1/images:annotate?alt=json returned "Project has not activated the vision.googleapis.com API. Please enable the API for project google.com:cloudsdktool (#32555940559).">

Thanks!