ebi-ait/covid-excel-utils

Tools to transform and validate covid data portal metadata files.

covid-excel-utils's Issues

Allow Configurable Excel Loading

Change ExcelLoader to take a configuration object that specifies:

• The bounds of Metadata (Maybe: MinColumn, MaxColumn, MinRow MaxRow)
• The bounds of Data (Maybe: Min Column, Min Row)
• Where to put/inject an Error Summary
• Error Summary as notes or as cell data?
• Errors as notes or as cell data?

Also Possibly:

• Specified Data Sheets / MetaData Sheets
• Some way of specifying the object / attribute creation pattern:

covid example:
object name = Row 1
attribute name = Row 2
units = Row 5

Example A:
object name = sheet name
attribute name = Row 1
units = Row 2

Example B
object_name.attribute_name = Row 1

Remove Parentheses from field names

BioSamples authentication error handling is missing

I checked the code relating to BioSamples submission and I can not see anywhere if we handle errors relating to AAP fault or when BioSamples would return other then success. It looks like we only handle the happy path in the code.

Allow Validation Against Configurable Checklist

Allow any specific checklist to be specified.

• Perhaps specified in the in cli config
• Perhaps specified in the excel metadata

Probably create a ChecklistValidator class from the base of SchemaValidator or DockerValidator

• Detect if a local json copy of the checklist needs to be made
• download and cache if necessary
• use the json schema representation of the checklist when validating using the SchemaValidator

Change destination column for error results

Current Issue

When validating and saving results to the excel file, 2 columns are added between the existing column A and B.
This is causing two issues:

1. Data Validation Input Messages (pictured below) are not shifted with the insertion and so will refer to the incorrect columns:

2. The background colour of cells no longer matches the original. The in-place workaround is to clear styling from loaded cells but this introducers it's own problems so is not ideal.

Proposed Solution

The first column of the covid excel file is used for instructions to the user that the loader can remove so that column A is used to report the number or errors.

Report errors without shifting the notes / styling

1. Stop inserting the error summary into B and C
2. Change the error summary to only need one column "Number of Errors", saved into column A
3. Clear the contents of column A before loading
4. Tweak the styling used on the summary column because blocks of red are hard to read and the user can easily add a filter to hide rows with 0 errors.

Ensure cells can be reverted to the correct colour when an error is fixed.

5. Stop reverting the styling of captured cells
6. Capture the styling of the first non-error data cell for each object
7. Apply the right colour before validation to revert any old validation results (currently doing this but with white)

Out of Scope

Keep in mind that we want to head in the direction of #12, but adding that functionality is out of scope to this issue.

blank_v3_genome_assemblies example spreadsheet has values in wrong row

I noticed that in the spreadsheet blank_v3_genome_assemblies.xlsx there are values for 'accepted values' in the 5th row rather than the 4th row.