report to a database

[dinahhandel]

report to a database

[dericed]

Updating the title of this. The lto2filemaker script uses the Filemaker Server XML api which is awkward and horribly slow, so changing this thread to consider other database options. @retokromer suggests MariaDB, @privatezero is doing something similar with MySQL.

I suggest these functions for a database:

• store entirety of the LTO schema in database tables
• allow some sort of command line ltosearch tool to query what file is where, when written, etc.

Other considerations?

[dinahhandel]

This is still true! And, more importantly, we need to ensure that lto2filemaker is reporting/reading the file path from the rsync log correctly.

[dericed]

Possibly with some @jfarbowitz style, we could switch out from filemaker to some more open form of database (sqlite/mysql).

[retokromer]

I second @dericed 's suggestion for a more robust database. MariaDB is the free replacement for MySQL: https://github.com/MariaDB

[dinahhandel]

Another issue to resolve with lto2filemaker, which is specific to CUNY TV and our data migration, is to make sure that the database reflects the tape that the material has migrated to. This can be done by reporting additional information from the schema file to the database, such as the tape number. Presently, after writing a tape, we receive the following output about updating the database:
Uploading tape metadata for tape name=E00002. Please wait…
Uploading file metadata for tape name=E00002. Please wait…
The database already knows about a file in E00002.schema called name=AWW00001.mov
The database already knows about a file in E00002.schema called name=AWW00002.mov
The database already knows about a file in E00002.schema called name=AWW00003.mov
The database already knows about a file in E00002.schema called name=AWW00004.mov
The database already knows about a file in E00002.schema called name=AWW00005.mov
The database already knows about a file in E00002.schema called name=AWW00006.mov
The database already knows about a file in E00002.schema called name=AWW00007.mov
The database already knows about a file in E00002.schema called name=AWW00008.mov
The database already knows about a file in E00002.schema called name=AWW00009.mov
The database already knows about a file in E00002.schema called name=BDC00235.mov
The database already knows about a file in E00002.schema called name=BDC00246.mov
The database already knows about a file in E00002.schema called name=BDC00247.mov
The database already knows about a file in E00002.schema called name=BDC00248.mov
The database already knows about a file in E00002.schema called name=BDC00249.mov
The database already knows about a file in E00002.schema called name=BDC00250.mov
The database already knows about a file in E00002.schema called name=BDC00251.mov
The database already knows about a file in E00002.schema called name=BDC00252.mov
The database already knows about a file in E00002.schema called name=BDC00253.mov
etc.

[dinahhandel]

@dericed don't forget about this!!! :)

[dericed]

B)

[retokromer]

I agree with @dericed

The SQL commands for MySQL (controlled now by Oracle) and MariaDB (a fork when Oracle tock over MySQL) should be identical. We replaced some years ago, in 2013 if I remember carefully.

[retokromer]

Yes, in 2013 (e.g. https://retokromer.ch/publications/FilmXML.html#ch7).

[dericed]

partly (maybe fully) resolved by #25. Comments welcome.

[privatezero]

Yes! Comments/critiques please! Still a bunch to be worked out!

[retokromer]

Is this still an open issue, @dinahhandel, or is it resolved by #25?

[dericed]

#25 is partly resolving, but there's plans to report on data from the schema file to a database. I believe @privatezero has this in the works.

[retokromer]

Thank you for the update! (Then I don’t close ;-)

[dericed]

IIUC, this part was resolved by #46. Thanks to @privatezero