bihealth/digestiflow-server

:cocktail: Web-based database system for flow cell management (incl. REST API)

Digestiflow Server

This project contains the code for the Digestiflow Server component of the Digestiflow Suite. The sibling projects are:

• Digestiflow CLI: command line client
• Digestiflow Demux: (semi)-automated demultiplexing of Illumina flow cells.

Digestiflow Suite

Digestiflow Server is the central component. It provides a a database system for managing Illumina flow cells and libraries with a web-based UI and a REST API. Its features include managing sample sheets, quality reports, a minimal "lab notebook" for note-taking and messaging, and sending out notifications on status changes.

Digestiflow CLI allows you to extract meta data and adapter index information from sequencer output directories. Usually, this is run in a regular (e.g., cron) job to monitor the directories where your sequencers store their output to automatically register new flow cells.

Digestiflow Demux allows you to demultiplex your flow cell data (semi)-automatically. Using the sample sheets entered in to Digestiflow Server, performs demultiplexing. This can also be run in a regular job to demultiplex flow cells as soon as sample sheets are provided in Digestiflow Server and marked as "ready for demultiplexing" there.

Installation

Please refer to the Installation section of the Documentation on how to install Digestiflow Server.

The recommended way is to Install with Docker Compose.

digestiflow-server's Issues

Allow commas in custom cycles in sample sheet

Problem with BCL adapter check

Warning shown for R1 if error only there for R2.

display demux_reads on properties page

Permissions not considered in search results?

Add field for flowcell-specific demux parametes

Some flow cells need special treatment, e.g. for 10x data, one needs to add

    --create-fastq-for-index-reads \ 
    --minimum-trimmed-read-length=8 \ 
    --mask-short-adapter-reads=8 \ 

Allow filtering flow cells by state

Add information which authentication method was used

Empty library barcode does not validate

Use new sentry API

Update cache when registering new histogram stats

Manual creation of flow cell does not work

Leads to 5xx as project is not set.

Warn if sample sheet barcodes and demux reads mismatch

index histogram only checks first index even if index pool like 10x

E.g. TTGGCATA show up red in the overview, even at 4 % because it is the 3rd sequence in SI-GA-C1 (CCACTTAT,AACTGGCG,TTGGCATA,GGTAACGC). CCACTTAT is shown in black as one would expect.

Release v1.0.0

Is reverse-complement in context menu broken?

• [ ]

Lookup indices in barcodes if not found in sample sheet

Problem with flow cell organism

I think it's improper mapping.

Also, need to be very explicit that flow cell data may not be saved.

Barcode sequence check does not work properly for Chromium index sequences

Adapter sequence not found, must check for being contained in the list instead of being equal to the list value.

Search for similar BCL adapters if not found

Fix manual entering of flow cells with NextSeq

Should be fairly simple to update the regular expression as flow cell vendor IDs are fixed length, then slot length can be fixed to one and the separating undescore can become optional

• https://github.com/bihealth/digestiflow-server/blob/master/flowcells/forms.py#L41

FLOW_CELL_NAME_RE = (
    r"^(?P<date>\d{6,6})"
    r"_(?P<machine_name>[^_]+)"
    r"_(?P<run_no>\d+)"
    r"_(?P<slot>\w)"
    r"_?(?P<vendor_id>[^_]{9})"
    r"(_(?P<label>.+))?$"
)

Store original library order in sample sheets

This is necessary to keep line order for edit-save-edit stable (in case people don't have lexicographically sorted sample sheets).

Do not allow manual updates of planned/current reads for auto-added flow cells

Implement flowcell-specific timeline

Add note about citation to manual and UI

Paste broken in context menu?

Fix posting of histogram information through API

Add more fields to sample sheet

Add web hooks for more service integrations

Organism human not displayed in sample sheet view

Instead "unknown" is shown. Displays nicely in table.

Get index histogram cutoff from API

Add statistics for index combinations

Also search for manual label

i7 library kit displayed instead of i5

Add "ansible" folder

Fix compatibility with MiSeq

MiSeq folder names don't specify the slot but the system expects it everywhere.

Add aliases to barcode names

People might have more than one name for the barcode.

Duplicate barcodes lead to HTTP 500

Barcode lengths must be checked vs. demultiplexing mask

Allow searching for reverse-complement

Add sentry integration

Missing lane information should be marked as invalid

Download of file attachments shows binary content

Allow searching through messages

Implement @mentions

Histogram indication for flow cells without barcode

In this case, the UI should indicate that no histogram can be expected.

Add option for disabling by lane splitting

Validator must also interpret library demux reads

Display per-sample overrides in sample sheet view

Currently only visible in editor.

Cache the errors

Will require backgrounds jobs for future migrations.

Add "paste" into the barcode set/sample sheet editors