fgrehm/serenata-ocr

A Serverless API for OCRing Serenata de Amor's documents (currently limited to Chamber of Deputies receipts)

Serenata OCR

A Serverless API for OCRing Serenata de Amor's documents (currently limited to Chamber of Deputies receipts). Powered by Claudia.JS and Google Cloud Vision.

From zero to an OCR API in minutes

https://asciinema.org/a/149404

Initial setup

In terms of tools / development stuff, while a Docker environment is in the works, this is what you'll need:

• git clone [email protected]:fgrehm/serenata-ocr.git
• cp config.json{.example,}
• NodeJS 6.10 (:warning: This is important, it is the version executed in AWS Lambda).
• yarn install or npm install
• Claudia.JS CLI (npm install -g claudia)
• AWS credentials configured for claudia as outlined in this tutorial

For OCRing with Google Cloud Vision you'll need:

• Cloud Vision needs to be enabled for your project. Can be done from https://console.cloud.google.com/apis/library/vision.googleapis.com/?q=vision.
• An API Key with permissions to perform Cloud Vision requests. Can be created from https://console.cloud.google.com/apis/credentials.
• open config.json and add the key there.

Deployment

As mentioned above, make sure your AWS credentials are configured as outlined in this tutorial. Once you have that done, proceed to your first deployment of the API:

claudia create --region us-east-1 \
               --api-module app \
               --timeout 60 \
               --memory 512 \
               --set-env-from-json config.json

At the end of claudia create you'll get an url, to test it run:

API="https://YOUR_API_ID.execute-api.us-east-1.amazonaws.com/latest/chamber-of-deputies/receipt"
# One liner if you have `jq` installed
API="https://$(jq -r '.api.id' claudia.json).execute-api.us-east-1.amazonaws.com/latest/chamber-of-deputies/receipt"

# OCR a receipt and get the full text of the PDF
curl "${API}/1789/2015/5631380" > 5631380.json

# Play with the data
jq '.config + .extra' 5631380.json
jq '.ocrResponse.fullTextAnnotation.text' 5631380.json

Documentation

🚧 Proper documentation is in the works 🚧

• From a high level, this is what gets done under the hood:
  • The receipt PDF associated with the reimbursement is downloaded from the Chamber of Deputies website.
  • ImageMagick is used to convert the PDF to a PNG image with: convert -density <density> receipt.pdf -quality 100 -deskew 40% -append receipt.png
  • The PNG is uploaded to Google Cloud Vision and the results are sent back to the client.
• For custom parameters supported by the API, see app.js for now.
• For local execution, see local.js for now (run with node local.js).
• Example responses at examples/
• Some useful utilities at Deskfile
• More info? Please read the code for now, it is super small:
  • Start off from app.js
  • Then see the code at src/

Wanna help?

See the issue tracker for inspiration.

Troubleshooting

Feel free to create an issue.

Function times out

Maybe the document is too big for your function to handle so give it more :muscle:

claudia update --timeout 90 --memory 1024

serenata-ocr's Issues

New provider: Tesseract.js

http://tesseract.projectnaptha.com

Tesseract.js is a pure Javascript port of the popular Tesseract OCR engine.

Escape parameters passed `exec`s

Right now that's a huge security issue here, another idea would be to validate user input before pushing it into execs.

Testing infrastructure

Now that the prototype works ™️ , it is time to take a step back and implement some much needed automated tests along with setting up a CI.

Experiment with cropping images

In theory we shouldn't need to deskew images (at least for google cloud vision) but I noticed that it yields better results. I wonder if cropping is going to help as well.

If it doesn't, at least we can save some $ in terms of data transferred to providers.

New provider: OCR.space

If cloud vision becomes too expensive, we can provide a way to leverage https://ocr.space/ocrapi, their free plan includes 25k operations, better than the 1k provided by Google.

Authenticate with HTTP Basic Auth or a simple token

This is the minimum needed before we set up a production environment.

New provider: NewOCR

http://www.newocr.com

Seems a bit expensive ($0.005 per image) but it'd be good to try for comparison with the other services available

Documentation

• API parameters
• Response structure
• Development with Docker
• Costs
• ...

New provider: Microsoft Azure Computer Vision API

Haven't tried it yet but I think it is worth a shot, specially because they have support for OCRing handwritten documents:

• https://azure.microsoft.com/en-us/pricing/details/cognitive-services/computer-vision/
• https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

Give pdf.js another shot

The purpose is check if the image quality is better than what's generated by ImageMagick. Unfortunately it is currently blocked by mozilla/pdf.js#8489.

Return full text by default, allow for fetching raw provider's response

With the upcoming support for multiple providers, it is going to be hard for clients to handle the specifics of each one of them, not to say that AWS charges for data transfered as well and Google Cloud Vision API responses are huge.

By default, the response should be the full text of the document, passing in raw=true would give back the complete provider's response.

Cache PDF files in S3

Just so that we don't need to hit the chamber of deputies website all the time