Giter Site home page Giter Site logo

danschultzer / receipt-scanner Goto Github PK

View Code? Open in Web Editor NEW
290.0 17.0 56.0 3.63 MB

Receipt scanner extracts information from your PDF or image receipts - built in NodeJS

License: Other

JavaScript 97.32% HTML 2.68%
ocr receipt-scanner receipts optical-character-recognition extract-data extract-information

receipt-scanner's Introduction

receipt-scanner

Receipt scanner extracts information from your PDF or image receipts.

Travis NPM version Codecov David

Example

import scanner from 'receipt-scanner'

scanner(streamOrFilePath)
  .parse(function (err, results) {
    if (err) return console.error(err)
    else console.log(results)
  })

CLI

receipt-scanner path/to/image.png

Results

{
    "path/to/image.png": {
        "amount": "1,390.00",
        "date": "2016-06-19"
    }
}

Getting started

$ brew install opencv3 poppler tesseract --with-all-languages

$ brew link --force opencv3

$ npm install git+https://[email protected]/danschultzer/receipt-scanner.git -g

Now run:

receipt-scanner path/to/image.png

Command Line Interface

$ receipt-scanner --help

Usage: receipt-scanner [options] <path...>

Options:

  -h, --help             output usage information
  -f, --format <format>  format to return, json (default) or text
  -p, --progress         add a progress bar
  -s, --summary          show summary details
  -v, --verbose          show verbose information

Optional dependencies

These dependencies are only necessary if you're going to use imagemagick or graphicsmagick image preprocessor.

Preprocessor Install command
Graphicsmagick $ brew install graphicsmagick
Imagemagick $ brew install imagemagick

API

Custom image preprocessor

You can use, and chain, specific image preprocessors by using the imagePreprocessor method like so:

var gm = require('gm')

function customPreprocessor (fileOrStream, outfile, cb) {
  gm(fileOrStream)
   .resize(400, 200)
   .in('-level', '25%,75%')
   .write(outfile, function (error) {
     cb(error, outfile)
   })
}

import scanner from 'receipt-scanner'

scanner(streamOrFilePath)
  .imagePreprocessor(customPreprocessor)
  .parse(function (err, results) {
    if (err) return console.error(err)
    else console.log(results)
  })

The default preprocessor used is opencv. It's also possible to add configuration settings by pushing an array to imagePreprocessor like so:

scanner(stream_or_file_path)
  .imagePreprocessor(['opencv', { verbose: true }])

Custom text parser

You can add a custom text parser by using the textParser method like so:

function customTextParser (text) {
  var regexp = new RegExp('Description: (.*)', 'ig')
  var output = []
  while (var matches = regexp.exec(text)) {
    output.push(matches[1])
  }
  return { matches: output, match: output[0] }
}

import scanner from 'receipt-scanner'

scanner(streamOrFilePath)
  .textParser(customTextParser)
  .parse(function (err, results) {
    if (err) return console.error(err)
    else console.log(results)
  })

The value will be added to the response object for the customTextParser key.

Customizing amount or date parser

You can customize either parser by setting config like so:

import scanner from 'receipt-scanner'

scanner(streamOrFilePath)
  .textParser(['date', { parser: 'first' }])
  .parse(function (err, results) {
    if (err) return console.error(err)
    else console.log(results)
  })

Date parser will by default find the earliest date, but as shown in the example you can also find the first. The amount parser will find the total first, and if nothing is found, then find the biggest amount.

Date parser config options

parser: What parser to run, earliest or first

Amount parser config options

parsers: What parsers to run in order. Default is ['total', 'largest'].

Ticker

A ticker callback can be added with the ticker method.

import scanner from 'receipt-scanner'

scanner(streamOrFilePath)
  .ticker(function (percent) {
    // Update ticker with current percent amount
  })
  .parse(function (err, results) {
    if (err) return console.error(err)
    else console.log(results)
  })

Parse text

If you've already extracted the text, and just want to parse it for the relevant information you can use parseText.

import scanner from 'receipt-scanner'

var results = scanner().parseText(text)

It'll return the same results as when you use parse(callback).

How is text parsed?

Receipt scanner takes an ambiguous approach to date and amounts. Amounts formatting is guessed from the number of amounts found with comma or with dots for decimal separation.

What's the binaries for?

poppler: For pdftotext module and pdfimages binary (PDF processing)

imagemagick: For gm module (image preprocessing)

graphicsmagick: For gm module (image preprocessing)

opencv3: For node-opencv (image preprocessing)

vips: For sharp module (image preprocessing)

tesseract --all-languages: For node-tesseract module (OCR)

Tests

$ npm test

You can use npm test watch to keep tests running, and npm run cover for coverage.

Benchmark

Run npm run benchmark to get success rate using the receipt-scanner-testdata repository.

To generate random data set to benchmark run npm run generate-benchmark-sample.

LICENSE

(The MIT License)

Copyright (c) 2016 Dan Schultzer, Benjamin Schultzer & the Contributors Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

receipt-scanner's People

Contributors

danschultzer avatar schultzer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

receipt-scanner's Issues

Increase code coverage

Work is under way to get code coverage of 90%+. In particular the following needs to be fully under code coverage:

  • opencv-photo2scan.js (several paths depending on paper mask, rotation, grayscale/color)
  • imagemagick-prepare-ocr.js
  • graphicsmagick.js

While this will increase test duration greatly, it'll also keep future development easier. opencv-photo2scan definitely needs some specs to ensure that changes won't break existing successful image manipulation.

It might be an idea to have a secondary set of tests for this (that's enabled in CircleCI), instead of being part of npm test. I think having a repository of images to run through and check success rate would be very useful.

node-tesseract throws error

I get an error whenever I try to use receipt-scanner whether if I was using it from CLI or from node.

the error is

fs.js:165
    throw new ERR_INVALID_CALLBACK();
    ^

TypeError [ERR_INVALID_CALLBACK]: Callback must be a function
    at makeCallback (fs.js:165:11)
    at Object.fs.unlink (fs.js:1034:14)
    at /Users/****/.npm-global/lib/node_modules/receipt-scanner/node_modules/node-tesseract/lib/tesseract.js:99:14
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:440:3)

MacOS Sierra (10.12) opencv build error

/tmp/opencv3-20160921-26223-1hcco0d/opencv-3.1.0/modules/videoio/src/cap_qtkit.mm:46:9: fatal error: 'QTKit/QTKit.h' file not found
#import <QTKit/QTKit.h>
        ^
1 error generated.

Not related to receipt-scanner, but if you install opencv3 on MacOS 10.12, the current brew file won't work. You'll have to run brew upgrade opencv3 --HEAD. This will be fixed soon.

0.5.0 release

I would like to see if we can increase success rate with OpenCV before releasing the next version. This will be a full 0.5.0 release.

  • Increase success rate on OpenCV preprocessor
    • Gradient lightning
    • Paper hitting edges of canvas/frame (contours has a difficult time there)
    • Bending of whole page (a sin wave alon vertical and horizontal)
    • Crumpled paper (random distortion across paper with difference in implosion/lightning)

ERROR: failed to run: pkg-config when running npm install git+https://[email protected]/danschultzer/receipt-scanner.git

When I run npm install git+https://[email protected]/danschultzer/receipt-scanner.git I get the following error:

> node-gyp rebuild

/Users/brechtmissotten/Projects/personal/receipt-parser/node_modules/opencv/utils/find-opencv.js:28
              throw new Error("ERROR: failed to run: pkg-config" + opencv + " " + flag + " - Is OpenCV installed?");
              ^

Error: ERROR: failed to run: pkg-config "opencv >= 2.3.1" --cflags - Is OpenCV installed?
    at /Users/brechtmissotten/Projects/personal/receipt-parser/node_modules/opencv/utils/find-opencv.js:28:21
    at ChildProcess.exithandler (child_process.js:304:5)
    at ChildProcess.emit (events.js:193:13)
    at maybeClose (internal/child_process.js:1001:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:266:5)
gyp: Call to 'node utils/find-opencv.js --cflags' returned exit status 1 while in binding.gyp. while trying to load binding.gyp

I tried re-installing and also https://stackoverflow.com/a/46743865/3163075 to no avail.

Might be related to peterbraden/node-opencv#325

Update installation instructions

I've spent whole day trying to this software. I've stepped on #62 and after several hours juggling arround differend documentations and guides I did have any luck to successfully install the software

Bug in config.onlyFindTextInsidePaperContour

There is a bug in config.onlyFindTextInsidePaperContour

when tested agianst test file readableColor.png we'll get
Uncaught AssertionError: expected [ReferenceError: contours is not defined] to equal null.

when tested with config.onlyFindTextInsidePaperContour and config.removeNoise set to true
Uncaught AssertionError: expected [ReferenceError: contours is not defined] to equal null

Further investigation

Shows there might be a bug upstream in opencv3 when tested with config.onlyFindTextInsidePaperContour and config.verbose set to true

OpenCV Error: Assertion failed (count >= 0 && (depth == CV_32F || depth == CV_32S)) in arcLength, file /tmp/opencv3-20160929-23220-ldcoax/modules/imgproc/src/shapedescr.cpp, line 285
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /tmp/opencv3-20160929-23220-ldcoax/modules/imgproc/src/shapedescr.cpp:285: error: (-215) count >= 0 && (depth == CV_32F || depth == CV_32S) in function arcLength

Abort trap: 6

config.onlyFindTextInsidePaperContour, config.verbose and config.removeNoise set to true

OpenCV Error: Assertion failed (count >= 0 && (depth == CV_32F || depth == CV_32S)) in arcLength, file /tmp/opencv3-20160929-23220-ldcoax/modules/imgproc/src/shapedescr.cpp, line 285
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /tmp/opencv3-20160929-23220-ldcoax/modules/imgproc/src/shapedescr.cpp:285: error: (-215) count >= 0 && (depth == CV_32F || depth == CV_32S) in function arcLength

Abort trap: 6

tests work as expected with readable.jpg and readableGray.png

Possible break point isolatePaper()

Changing cv.Constants.CV_8UC1...cv.Constants.CV_8UC3

  • cv.Constants.CV_16UC1...cv.Constants.CV_16UC3
  • cv.Constants.CV_CV_32SC1...cv.Constants.CV_32SC3
  • cv.Constants.CV_CV_32FC1...cv.Constants.CV_32FC3

readableColor.png
OpenCV Error: Assertion failed (mask.depth() == CV_8U && (mcn == 1 || mcn == cn)) in copyTo, file /tmp/opencv3-20160929-23220-ldcoax/modules/core/src/copy.cpp, line 342
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /tmp/opencv3-20160929-23220-ldcoax/modules/core/src/copy.cpp:342: error: (-215) mask.depth() == CV_8U && (mcn == 1 || mcn == cn) in function copyTo

Abort trap: 6
readableGray.png
OpenCV Error: Assertion failed (mask.depth() == CV_8U && (mcn == 1 || mcn == cn)) in copyTo, file /tmp/opencv3-20160929-23220-ldcoax/modules/core/src/copy.cpp, line 342
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /tmp/opencv3-20160929-23220-ldcoax/modules/core/src/copy.cpp:342: error: (-215) mask.depth() == CV_8U && (mcn == 1 || mcn == cn) in function copyTo

Abort trap: 6

readable.jpg work as expected

Add unit test for OpenCV preprocessor configuration options

We'll need unit tests for OpenCV preprocessor configuration options.

It'll need to be in test/lib/image_preprocessor/preprocessor/opencv-photo2scan_spec.js, and should cover the following two config options:

config.removeNoise
config.onlyFindTextInsidePaperContour

We'll use #14 to get full code coverage on the default setup, but this will not test special config options in preprocessors, so we'll need to hit those with unit testing.

npm install or yarn install fails

OS information

ProductName:	Mac OS X
ProductVersion:	10.12.6
BuildVersion:	16G1314

Here are the npm logs https://pastebin.com/97xLe4Rn

It looks like node-tesseract is getting timed out.

In package.json I tried changing

"node-tesseract": "git://github.com/desmondmorris/node-tesseract.git",
"opencv": "git://github.com/peterbraden/node-opencv.git",

to

 "node-tesseract": "0.2.7",
 "opencv": "6.0.0",

but had no luck

Here is the npm log https://pastebin.com/3cvNUSv4

yarn install also fails. I can provide logs if required.

dependancies update?

just installed it and got errors.

the first was the ln command is version dependant to opencv3 version 3.1.0_3. the current version is 3.1.0_4.

the other was a missing webp install. perhaps include in the first brew install command?

Android app?

This looks like a great project! Are there plans for an Android app?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.