joewiz/s3

S3 helper

S3 Helper

Adapted from https://github.com/dknochen/xaws for accessing S3 buckets with eXist-db

Not intended for general use yet

Supply your AWS credentials in modules/aws_config.xqm (duplicate the template file, rename to aws_config.xqm)

Build

1. Single xar file: The collection.xconf will only contain the index, not any triggers!

   ant

s3's Issues

cache-all.xq needs to run with write privileges

cache-all.xq must run on the production site. it caches listings of static resources on s3, so we can display the file size, last modified date, etc., in links to these resources (and so we know not to link to resources that do not exist).

in order to avoid hardcoding passwords into source, the best way is for pre-install.xql should set the correct uid/gid privileges for this script to be run. (i think?)

Need Help with PDFtk Server and XFDF Files

Joe,

I apologize for contacting you here, but I have tried every other means I could find and got no response.
Could you please take a moment to help me with using XFDF files with PDFtk?

I'm running pdftk 2.02 on macOS 10.12.6.

You wrote a great blog back in 2014 that was very helpful to me:
Filling PDF forms with PDFtk, XFDF, and XQuery

Unfortunately, I get a fatal error when trying to fill a PDF from a XFDF. The same PDF works fine with a FDF file, but I now need to handle high-end Unicode characters that appear in non-English languages. So, I have to use XFDF.

Here is the error I get:

$ pdftk ExamplePDF.pdf fill_form ExampleData.xml output filltest1.pdf

Unhandled Java Exception in create_output():
java.lang.RuntimeException: Root element is not Bookmark.
at pdftk.com.lowagie.text.pdf.XfdfReader.startElement(pdftk)
at pdftk.com.lowagie.text.pdf.SimpleXMLParser.parse(pdftk)
at pdftk.com.lowagie.text.pdf.SimpleXMLParser.parse(pdftk)
at pdftk.com.lowagie.text.pdf.XfdfReader.(pdftk)

This is using the PDF form that Sid provided in his download, and your example XFDF.
Both are attached. Importing to the same PDF and using the same XFDF file with AdobeAcrobat Pro works fine.

I get the same exact error with my form and data, yet when I use Adobe Acrobat Pro X to import to the form using the same XFDF file, it works fine.
I've searched and searched, and cannot get any info on this error.

I'm running pdftk 2.02 on macOS 10.12.6.

Can you offer any help?

$ pdftk ExamplePDF.pdf fill_form ExampleData.xml output filltest1.pdf
 
Unhandled Java Exception in create_output():
java.lang.RuntimeException: Root element is not Bookmark.
   at pdftk.com.lowagie.text.pdf.XfdfReader.startElement(pdftk)
   at pdftk.com.lowagie.text.pdf.SimpleXMLParser.parse(pdftk)
   at pdftk.com.lowagie.text.pdf.SimpleXMLParser.parse(pdftk)
   at pdftk.com.lowagie.text.pdf.XfdfReader.<init>(pdftk)

I have no idea what the errors mean, or how to fix.

This is using the PDF form that Sid provided in his download, and your example XFDF.
Both are attached. Importing to the same PDF and using the same XFDF file with AdobeAcrobat Pro works fine.

I get the same exact error with my form and data, yet when I use Adobe Acrobat Pro X to import to the form using the same XFDF file, it works fine.
I've searched and searched, and cannot get any info on this error.

Example PDF and XFDF Files.zip

Can you offer any help?

app should prompt for login

this app should only be accessible to users with write privileges. the app is currently dumb and doesn't prompt for a login. please modify to prompt for login upon first access.

i will need to apply this to other apps that we currently restrict to staff, so please be aware that this method will be applied to other apps.

Add handling for object:upload with eXist

• Switch to AWS Signature v4
• Add tests for existing functions
• Enable object:upload and other functions as needed

bucket:list not working

hsg retrieves the list of available ebooks on S3, storing the following information about each ebook: path, file size, last modified date. For example:

<resources prefix="frus/frus1969-76v18/ebook/">
    <resource>
        <filename>frus1969-76v18.epub</filename>
        <s3-key>frus/frus1969-76v18/ebook/frus1969-76v18.epub</s3-key>
        <size>1249981</size>
        <last-modified>2013-05-22T13:30:38.000Z</last-modified>
    </resource>
    <resource>
        <filename>frus1969-76v18.mobi</filename>
        <s3-key>frus/frus1969-76v18/ebook/frus1969-76v18.mobi</s3-key>
        <size>1756344</size>
        <last-modified>2013-05-22T13:30:55.000Z</last-modified>
    </resource>
</resources>

From this, we can construct a link to http://s3.amazonaws.com/static.history.state.gov/frus/frus1969-76v18/ebook/frus1969-76v18.epub. My ported code no longer works, even though I haven't made any significant changes to my knowledge. Here is a reproducible example, assuming you've installed this app locally and already have pasted S3 credentials into your local /db/apps/s3/modules/aws_config.xqm file:

xquery version "1.0";

(: list contents of static.history.state.gov S3 bucket :)

import module namespace aws_config = "http://history.state.gov/ns/xquery/aws_config" at '/db/apps/s3/modules/aws_config.xqm';
import module namespace bucket = 'http://www.xquery.co.uk/modules/connectors/aws/s3/bucket' at '/db/apps/s3/modules/xaws/modules/uk/co/xquery/www/modules/connectors/aws-exist/s3/bucket.xq';

let $bucket := 'static.history.state.gov'
let $marker := ''
let $prefix := 'frus/'
return
    bucket:list($aws_config:AWS-ACCESS-KEY, $aws_config:AWS-SECRET-KEY, $bucket, '/', $marker, '', $prefix)

This script works on hsg2 code, but not on hsg3. On hsg3, this script returns:

<httpclient:response xmlns:httpclient="http://exist-db.org/xquery/httpclient" statusCode="403">
    <httpclient:headers>
        <httpclient:header name="x-amz-bucket-region" value="us-east-1"/>
        <httpclient:header name="x-amz-request-id" value="6E7978080E79B7A4"/>
        <httpclient:header name="x-amz-id-2"
            value="hEeR2mRnam9dV96CCBhjjm7fXQjTfzkR6ihov+jEdLkAaBdkWyxgsKXG49IeojvAiSYJbcjlKDg="/>
        <httpclient:header name="Content-Type" value="application/xml"/>
        <httpclient:header name="Transfer-Encoding" value="chunked"/>
        <httpclient:header name="Date" value="Sun, 08 Nov 2015 17:32:29 GMT"/>
        <httpclient:header name="Server" value="AmazonS3"/>
    </httpclient:headers>
    <httpclient:body mimetype="application/xml" type="xml">
        <Error>
            <Code>SignatureDoesNotMatch</Code>
            <Message>The request signature we calculated does not match the signature you
                provided. Check your key and signing method.</Message>
            <AWSAccessKeyId>AKIAILAK6WKQT7I7ZREA</AWSAccessKeyId>
            <StringToSign>GET x-amz-date:Sun, 08 November 2015 12:32:30 -0500
                /static.history.state.gov/</StringToSign>
            <SignatureProvided>63e5cba2487918c6be1ef2365decb307205e1662</SignatureProvided>
            <StringToSignBytes>47 45 54 0a 0a 0a 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 53 75 6e 2c
                20 30 38 20 4e 6f 76 65 6d 62 65 72 20 32 30 31 35 20 31 32 3a 33 32 3a 33 30 20
                2d 30 35 30 30 0a 2f 73 74 61 74 69 63 2e 68 69 73 74 6f 72 79 2e 73 74 61 74 65
                2e 67 6f 76 2f</StringToSignBytes>
            <RequestId>6E7978080E79B7A4</RequestId>
            <HostId>hEeR2mRnam9dV96CCBhjjm7fXQjTfzkR6ihov+jEdLkAaBdkWyxgsKXG49IeojvAiSYJbcjlKDg=</HostId>
        </Error>
    </httpclient:body>
</httpclient:response>

On hsg2, this returns the expected results:

<httpclient:response xmlns:httpclient="http://exist-db.org/xquery/httpclient" statusCode="200">
    <httpclient:headers>
        <httpclient:header name="x-amz-id-2"
            value="a3bIB6gUjXWEvM22yUkzNgwbajD5eC1lhxYgV1PKYkCoeK4F9pOHiZYwInYiUXJQkc6LFNddBew="/>
        <httpclient:header name="x-amz-request-id" value="8B2E9FE4D2857F05"/>
        <httpclient:header name="Date" value="Sun, 08 Nov 2015 17:34:20 GMT"/>
        <httpclient:header name="x-amz-bucket-region" value="us-east-1"/>
        <httpclient:header name="Content-Type" value="application/xml"/>
        <httpclient:header name="Transfer-Encoding" value="chunked"/>
        <httpclient:header name="Server" value="AmazonS3"/>
    </httpclient:headers>
    <httpclient:body mimetype="application/xml" type="xml">
        <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
            <Name>static.history.state.gov</Name>
            <Prefix>frus/</Prefix>
            <Marker/>
            <MaxKeys>1000</MaxKeys>
            <Delimiter>/</Delimiter>
            <IsTruncated>false</IsTruncated>
            <Contents>
                <Key>frus/</Key>
                <LastModified>2011-04-22T15:43:42.000Z</LastModified>
                <ETag>"d41d8cd98f00b204e9800998ecf8427e"</ETag>
                <Size>0</Size>
                <Owner>
                    <ID>6b1e007de41207a06cd1dde9cd0b729e5cfc97ab9b7e68e64d72c0717c48d6b0</ID>
                    <DisplayName>Fedcloudit70_00007</DisplayName>
                </Owner>
                <StorageClass>STANDARD</StorageClass>
            </Contents>
            <CommonPrefixes>
                <Prefix>frus/frus1865p4/</Prefix>
            </CommonPrefixes>
            <CommonPrefixes>
                <Prefix>frus/frus1914-20v01/</Prefix>
            </CommonPrefixes>
            <CommonPrefixes>
                <Prefix>frus/frus1914-20v02/</Prefix>
            </CommonPrefixes>
            <CommonPrefixes>
                <Prefix>frus/frus1914Supp/</Prefix>
            </CommonPrefixes>
            <CommonPrefixes>
                <Prefix>frus/frus1915Supp/</Prefix>
            </CommonPrefixes>
            <CommonPrefixes>
                <Prefix>frus/frus1916Supp/</Prefix>
            </CommonPrefixes>
            [... similar results truncated...]
        </ListBucketResult>
    </httpclient:body>
</httpclient:response>