librecat/catmandu-filestore

File support for Catmandu projects

LibreCat - an institutional repository

LibreCat is a new institutional repository system developed by LibreCat Group which has as its key features:

• institutional repository
• publication list manager for reseachers
• institutional research data archive.

The development started in 2013 in Bielefeld and was made available on GitHub from the start. Since 2015 the code is in production at Bielefeld. In 2016 Ghent University started using the cataloging backend in production.

Features

• Cataloging of many record types: Book, Book (Editor), Book Chapter, Book Review, Conference Abstract, Conference (Editor), Conference Paper, Dissertation, Encyclopedia Article, Journal Article, Special Issue, Newspaper Article, Preprint, Report, Translation, Translation (Section), Working Paper, Thesis, Research Data, Project, Award, Research Group
• Drag and drop upload of full-text publications
• Copycat from DOI, PubMED, ArXiv and Web of Science
• Google Scholar indexation support
• Citation styles configurable from Zotero Style Repository
• Full MathJAX Latex support to add mathematical formulas in abstracts and titles
• Pluggable authentication modules
• Delegate input and management to user others
• Multilingual support
• ElasticSearch indexing
• Pluggable file store backend
• Command line support using 'Catmandu'
• OAI-PMH and SRU
• REST / content negotiation
• Signposting
• The LibreCat is open source and shipped with the same license as the Perl language: http://dev.perl.org/licenses/

Install

See our Wiki at: https://github.com/LibreCat/LibreCat/wiki

catmandu-filestore's Issues

Test suite fails (with Catmandu 1.2009 and newer?)

The test suite fails on some of my smokers:

...
Trace begun at /home/cpansand/.cpan/build/2019121213/Catmandu-FileStore-1.151-Wbjo1I/blib/lib/Catmandu/DirectoryIndex/Number.pm line 29
Catmandu::DirectoryIndex::Number::format_id('Catmandu::DirectoryIndex::Number=HASH(0x55dba9de2648)', 000000002) called at /home/cpansand/.cpan/build/2019121213/Catmandu-FileStore-1.151-Wbjo1I/blib/lib/Catmandu/DirectoryIndex/Number.pm line 73
Catmandu::DirectoryIndex::Number::add('Catmandu::DirectoryIndex::Number=HASH(0x55dba9de2648)', 000000002) called at t/Catmandu-DirectoryIndex-Number.t line 95
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 2 just after 10.
t/Catmandu-DirectoryIndex-Number.t .... 
Dubious, test returned 2 (wstat 512, 0x200)
All 10 subtests passed 
... (more failing test files following) ...

Statistical analysis suggests that this happens with newer Catmandu versions:

****************************************************************
Regression 'mod:Catmandu'
****************************************************************
Name           	       Theta	      StdErr	 T-stat
[0='const']    	      1.0000	      0.0000	20977463844481968.00
[1='eq_1.2002']	      0.0000	      0.0000	   5.70
[2='eq_1.2006']	      0.0000	      0.0000	   8.12
[3='eq_1.2007']	      0.0000	      0.0000	  11.26
[4='eq_1.2009']	     -1.0000	      0.0000	-12111344397527302.00
[5='eq_1.2010']	     -1.0000	      0.0000	-20214389962006772.00

R^2= 1.000, N= 65, K= 6
****************************************************************

Current DirectoryIndex Map died hard when the database does't map filesystem layout

I have use case where I use the Map DirectoryIndex for the temporary storage of files which get cleaned up with a crontab scripts. When the file layout doesn't match directory mapping stored in the index, the DirectoryIndex will die.

Can this behavior made configuratble?

- default : die when filesystem in inconsistent
- optional: skip directories not found with a warning message

unable to upload file using IO::String (or something similar)

Example:

my $file_store = Catmandu::Store::File::Simple->new(root => "data");
$file_store->index->add({ _id => 1 });
my $dir = $file_store->index->files(1);

my $str = "hello";

#error: "stat() on unopened filehandle GEN0 at [..]File/Copy.pm line 102."
$directory->upload(
  Catmandu::Util::io( \$str ),
  "test.txt"
);

#same error here
$directory->upload(
  IO::String->new($str),
  "test.txt"
);

Gazillions of log messages "Use of uninitialized value ..."

Use of uninitialized value in addition (+) at /srv/www/pub/LibreCat/local/lib/perl5/Catmandu/Store/File/Simple/Bag.pm line 97.

Logs, produced by LibreCat, explode.

Use of uninitialized value $n

When no output is written (e.g. the IO is connected to /dev/null) the Bag complains about unitialized values.

Use of uninitialized value $n in addition (+) at /opt/librecat/local/lib/perl5/Catmandu/Store/File/Simple/Bag.pm line 111.
Use of uninitialized value $n in numeric ne (!=) at /opt/librecat/local/lib/perl5/Catmandu/Store/File/Simple/Bag.pm line 104.
Use of uninitialized value $n in addition (+) at /opt/librecat/local/lib/perl5/Catmandu/Store/File/Simple/Bag.pm line 111.
Use of uninitialized value $n in numeric ne (!=) at /opt/librecat/local/lib/perl5/Catmandu/Store/File/Simple/Bag.pm line 104.
Use of uninitialized value $n in addition (+) at /opt/librecat/local/lib/perl5/Catmandu/Store/File/Simple/Bag.pm line 111.