mekery/pdfdig

PDFDig is a useful tool to dig content from pdf document for text mining.

PDFDig Introduction

PDFDig is a useful tool to dig content from pdf document for text mining, which is written in Python and based on pdftotext and PDFMiner.

• Home: https://github.com/mekery/pdfdig
• Documentation: http://mekery.github.com/pdfdig/

Features

• Convert pdf to txt.
• Search in pdf document, working like grep.
• Build table of content(TOC) of pdf document.
• Get pdf metadata.

PDFDig Utility

pdftotext.py

pdftotext.py converts pdf to text. There are tens of similar utilities can do this job, while few of them, including pdftotext, can process line-break, hyphen and extra white spaces appropriately, and some of them render the pdf in physical order, which are unsuitable for multi-column pdf documents.

pdftotext.py uses pdftotext to get the text content of pdf document, then normalizes the text content.

pdfgrep.py

pdfgrep.py enables you to search and count in pdf files. pdfgrep.py searches in grep-style, which means you can use regular expression in search and get matching lines.

Reference

• pdftotext: http://en.wikipedia.org/wiki/Pdftotext
• PDFMiner: http://www.unixuser.org/~euske/python/pdfminer/

pdfdig's Issues

Mac compatibility and UTF-8 support

--- pdftext.py
+++ (clipboard)
@@ -37,11 +37,10 @@
         """
         Check if pdftotext available on running platform.
         """
-        # Todo: Mac
         pathstring = os.environ.get('PATH')
         platform_prefix = platform.system()[0]
-        if  platform_prefix in 'L,C,M':
-            # L:Linux, C:Cygwin, M:Mac OS X
+        if  platform_prefix in 'L,C,D':
+            # L:Linux, C:Cygwin, D:Mac OS X (Darwin)
             paths = pathstring.split(':')
             for path in paths:
                 pdftotext = os.path.join(path, 'pdftotext')
@@ -65,7 +64,7 @@
         pages = []

         while True:
-            command = 'pdftotext -{0} -f {1} -l {1} {2} -'.format(layout, self.pagecount+1, pdf)
+            command = 'pdftotext -enc UTF-8 -{0} -f {1} -l {1} {2} -'.format(layout, self.pagecount+1, pdf)
             #page_content = os.popen(command).readlines()
             page_content = os.popen(command).read()

Can't find package when use Virtualenv

pdftotext.py can't find package when use Virtualenv

'Text' object has no attribute 'content'

When I run 'python pdftotext pdffile.pdf' I get this:

$ python pdftotext.py -o boop.txt ~/Documents/Archive_PDF_Content_Stripper/pdfminer/samplePDFs/VDD_S466M1534E01_D_AfterQG2.pdf

pdftotext command not found
Traceback (most recent call last):
File "pdftotext.py", line 74, in
if name == 'main': sys.exit(main(sys.argv))
File "pdftotext.py", line 63, in main
pages = pc.content
AttributeError: 'Text' object has no attribute 'content'