caltechlibrary/pubarchiver

Package up microPublication.org and other journals for archiving into Portico and PMC

pubarchiver's Issues

Add consistency checks

A simple implementation would be to periodically ask the journal API for the list of what's available and compare that to the recorded uploads, then email someone if there's a discrepancy.

Volume number not set correctly in PMC output

The volume number portion of the PMC file names is set based on the current year, while it's supposed to be set based on the year of the article's publication.

"No content" returns from datacite not handled gracefully

If datacite.org returns no content for a given URL, you get an exception:

microarchiver messages.py error(): No content found for https://api.datacite.org/dois/10.17912/micropub.biology.000167
Traceback (most recent call last):
  File "/home/mhucka/system/lib/python3.6/site-packages/microarchiver/__main__.py", line 186, in main
    MainBody(source, after, output_dir, do_zip, report, get_xml, preview, say).run()
  File "/home/mhucka/system/lib/python3.6/site-packages/microarchiver/__main__.py", line 283, in run
    self.write_articles(dest_dir, articles)
  File "/home/mhucka/system/lib/python3.6/site-packages/microarchiver/__main__.py", line 394, in write_articles
    xml = self._metadata_xml(article)
  File "/home/mhucka/system/lib/python3.6/site-packages/microarchiver/__main__.py", line 421, in _metadata_xml
    raise error
microarchiver.exceptions.NoContent: No content found for https://api.datacite.org/dois/10.17912/micropub.biology.000167

This should be handled more gracefully.

Add alternative report formats

Request via email:

Would it be possible to have a different output or output format so that I can find information in the attachment without having to launch another app? Currently, when I click on the report, the columns are too narrow to see the specific article DOI value, so I have to open it in sheets or scroll over to see the link. While it seems minor, I am logging these data in another sheet so I can track the article through to pubmed upload, which means I'm entering things on another sheet --the less mousing I have to do the better.

Suspicious line in PMC upload script.

The use of tail at this line in the code,

https://github.com/caltechlibrary/microarchiver/blob/0aab75f72b712fa69e56c7979440a7e58bd26a3b/scripts/upload-to-pmc#L85

is very suspicious. I strongly suspect this should be cat but it needs more careful analysis.

Broken image link in README.md

What happened here?

PMC upload script sends email twice if there's a failure

The mail command at the end of the script,

https://github.com/caltechlibrary/microarchiver/blob/0aab75f72b712fa69e56c7979440a7e58bd26a3b/scripts/upload-to-pmc#L114-L116

unconditionally sends mail, even if the mail command at

https://github.com/caltechlibrary/microarchiver/blob/0aab75f72b712fa69e56c7979440a7e58bd26a3b/scripts/upload-to-pmc#L94

has sent mail due to a failure. It needs to be rewritten so that only one message is sent.

_save_articles doesn't test if saving article succeeded, leading to poor error detection

_save_articles doesn't test if saving article succeed, and thus can't deal cases where article is in the article.xml but something goes wrong when actual download is attempt. need to catch the error, then write exceptions so they can be tried next time

Add ability to write to Google spreadsheet

The microPublication team requested the ability to write to a Google sheet the info about what has been uploaded.