pcolar/ocr-parser

Programming project for INLS560

ocr-parser's Issues

Create option for collecting output sequence number

Modify the code to accept a range of input files:
-- files with the same date will count up the page sequence number
-- open and close each input file cleanly

Flexibility in exporting parsed data

In the current version under development, the code has a fix data output to the CSV file. Ideally, the code would read the header of the CSV file to determine which data elements to parse.

Practical application would require standardized element names as part of an extensible controlled vocabulary.

File selection API

For Windows and Mac OSX
-- call the parse program using a range of input files based on the selection in Finder or Windows file manager

Define levels of metadata produced in the workflow

Possible levels:

Collection
Digitization batch
Reel
Item
Derived metadata
Parsed metadata

Incorporate use of the CSV library in parsing program

Using the CSV library will simplify the parsing program and make it easier to maintain.

Incomplete metadata field

Complete partial metadata from the OCR file.

Data may be incomplete: 'January 12, 199"
no field separation: 'Page1', 'PageOne'

Refactor parsing program

A high priority is moving the parsing regex code to a library, where it can be extensible and not clog the main code.

Methods to combine collected metadata

Several spreadsheets and data files exist from separate levels of the workflow.
Define matching elements and inheritance

Convert date strings into normalized dates

a function to convert date strings into standardized date formats

'JUNE 15, 1946' -> 19460615

Workflow for missing metadata

One or more metadata elements are not found in the OCR file

Submission Guildelines

Document how to become involved and submit code or other work to the project.

Parse the OCR text file

Code to parse the text files output by the digitization workflow.

Currently identified fields include:
Volume
Issue
Date
Page Number

Convert text number to digits

a function to convert numbers written as text to digits:

one -> 1
twenty seven -> 27

Create Python script to query user and build run environment

Some of our command line parameters can be loaded from shell environment variables using the 'os' library: os.environ["shell_variable"]
A small python script can query the user and create the shell variables - which will persist for the life of the terminal session

Convert Roman numerals to digits

Function to convert roman numerals to digits

i.e. IX -> 9