codeisscience/software-usage-reporting

Tiered software usage reporting recommendations from #BOSC2019

software-usage-reporting's Issues

Review journal software usage reporting policy

• Find meta-research paper if exist
• If it doesn't, compile major compbio journal policies

The goal is to identify gap.

Recommended best practices for notebooks

There is currently no known guidelines

• paper on simple rules to write and analyse notebooks: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007007
• the Turing way credit section is under development - we can back contribute once we're done?
• there is a project to make Notebooks FAIR at eScience (@mkuzak to follow-up)

review nf-core report formats

The following is courtesy @heuermh

In nf-core, it is a best-practice (and generated by the nf-core getting-started template) to capture software versions

https://github.com/nf-core/rnaseq/blob/09e8214704d81332ad1c391dc8d72f801c8def23/main.nf#L370

echo $workflow.manifest.version &> v_ngi_rnaseq.txt
echo $workflow.nextflow.version &> v_nextflow.txt
fastqc --version &> v_fastqc.txt

scrape_software_versions.py &> software_versions_mqc.yaml

I'm not confidently sure how this file is used … looks like the multiqc tool may use it to generate an HTML report?

https://github.com/nf-core/rnaseq/blob/09e8214704d81332ad1c391dc8d72f801c8def23/main.nf#L1318

file ('software_versions/*') from software_versions_yaml.collect()

Look at RO-Crate / CWLProv

common-workflow-language/common-workflow-language#770

Recommended command line best practice to expose version

We should recommend best practice for version information (and check what already exist for that), given a command line tool.

TODO: Find/link to tool author guides

For instance:

Level 0

• MUST have a programmatic way to get version info
• Parameter SHOULD be --version but MAY be arbitrarily named

$ tool -custom-version-parameter`
> 1.2.3

Level 1

• MUST respond to parameter --version
• MUST include tool name
• MUST include version number
• Version number SHOULD be semantic version (TODO: How can you tell?)

$ python --version
Python 3.7.1

Level 2

• MUST support/include detailed version info
• SHOULD support --version --verbose
• SHOULD list important dependencies and their versions
• MAY recursively ask dependencies for detailed version info (TODO: How to show nesting?)

$ python --version -V
Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0]

$ latex --version --verbose
pdfTeX 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian)
kpathsea version 6.2.3
Copyright 2017 Han The Thanh (pdfTeX) et al.
There is NO warranty.  Redistribution of this software is
covered by the terms of both the pdfTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the pdfTeX source.
Primary author of pdfTeX: Han The Thanh (pdfTeX) et al.
Compiled with libpng 1.6.34; using libpng 1.6.34
Compiled with zlib 1.2.11; using zlib 1.2.11
Compiled with poppler version 0.62.0

Level 3

• MUST support structured version info
• MUST support --version=cff
• MUST return only valid CITATION.cff YAML/JSON content on stdout
• SHOULD include a CFF references section
• Dependencies SHOULD be listed as software under references
• MAY recurse call --version=cffon tools and embed result in their references item

TODO: Check if anything already supports something like this

$ tool --version=cff
cff-version: 1.0.3
message: If you use this software, please cite it as below.
authors:
  - family-names: Druskat
    given-names: Stephan
    orcid: https://orcid.org/0000-0003-4925-7248
title: My Research Tool
version: 1.0.4
doi: 10.5281/zenodo.1234
date-released: 2017-12-18

What to include in the lightweight reporting format?

Reminder: linked-data and/or full W3C Prov is too much.

This is meant to be human and machine generated

From our in person discussion at a #BOSC2019 birds of a feather session and the follow on at the #BOSC2019 CoFest:

1. Name of the software
2. Version used
3. Exact parameters used

Items one and two are also part of any decent software citation (see the Force 11 Software Citation Guidelines) and it is item number 3 (exact parameters used) that is unique to this lightweight reporting format