nbisweden/igdiscover-legacy

Analyze antibody repertoires and discover new V genes from high-throughput sequencing reads

⚠️ ❗ ⚠️ ❗ ⚠️

This repository is for an old IgDiscover version. Please see the new repository at https://gitlab.com/gkhlab/igdiscover22/ or the homepage at https://www.igdiscover.se/ for the most recent and maintained IgDiscover version.

⚠️ ❗ ⚠️ ❗ ⚠️

igdiscover-legacy's Issues

Possibly use 'exact' column to compute cross-mapping ratio

Update IgBLAST

We are using 1.4.0, but 1.6.0 has been released recently.

Make runs reproducable

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

Random subsampling leads to runs on the same data being different. We need to be able to set the seed for the random number generator to a specified value so that two runs produce the same results.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/9

Print IgBLAST speed and/or progress

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

The 'igblast' subcommand should print out how many assignments were made per second.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/39

Ensure that the 'black' cluster in the clusterplots is detected

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

The bottom right cluster of each clusterplot is currently unmarked. A consensus is computed for all unmarked sequences, so we do not miss it, but since all unmarked sequences are included (also those that are not in the bottom right), the consensus may be bad.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/22

Remove intermediate files by default

It is not necessary to keep large files in intermediate iterations: Candidates for deletion are assigned.tab.gz, igblast.txt.gz and filtered.tab.gz.

Do not rename sequences identical to a starting database sequence

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

The compose script should not append the _Sxxxx suffix to genes that have a whitelist_diff of zero.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/27

Check whether input database is ok

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

When running igdiscover init, we should make sure that the FASTA files

• exist
• are parseable
• do not contain duplicate sequences
• do not contain duplicate sequence ids
• are named V.fasta, J.fasta and (optional) D.fasta
• if no D.fasta exists, a dummy file should be created

If we do not, makeblastdb will fail at a much later stage.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/19

Remove barcodes before running IgBLAST

Both barcodes and the RACE-specific G-prefix should be removed before running IgBLAST. This may be a problem when an unstranded protocol is used.

_pickle.PicklingError

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

We suffer from snakemake issue 257. The current workaround is to avoid Python 3.5 and use 3.4 instead.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/28

Reported cluster_size for artificially added "db" sequences may be wrong

Improve CDR3 detection by using a more recent IgBLAST

The CDR3 detection regular expression by D'Addario et al. that we use is roughly like this [FY] [FHVWY] C ... W [AGV] (the ... is the CDR3, the things to the left and right are allowed amino acids). However, we do not detect some CDR3s with it. For example, in one of the datasets, the C is mutated and becomes a W.

Assign clonotype to a small number of sequences

Given a list of clonotypes and a FASTA file with previously unseen sequences, assign each of the sequences in the FASTA file to its matching clonotype.

For this, we need a way to run IgBLAST separately on a single sequence.

Depends on #41

Make IgDiscover publicly available

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

• Switch Bitbucket repository to 'public'
• Upload a package to PyPI
• Upload a package to bioconda
• Connect repository on readthedocs

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/1

Make a new release soon

Ensure also that the changelog is up to date.

Be more helpful when validating the input database

igdiscover init validates the input database and checks whether there are duplicate sequences.

The error we get complains only about the first duplicate sequence that it finds, so it is necessary to re-run init in order to fix and find all problems, which is annoying, especially if the GUI is used.

make differences = 0 the default

Make rename: false the default?

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/35

Have an option for 'unknown' species

This would:

• not require a J database file
• use a dummy J database instead
• set ignore_j to true

Some detected lambda CDR3s are too long

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

Sometimes the regex detects one amino acid too many at the 5' end.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/20

Require fewer sequences for errorplots

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

Currently, an errorplot is made only if at least 200 sequences are assigned to a gene, which is too strict especially on small library sizes.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/26

Allow to relax pre-processing filter criteria

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

In particular, requiring that the V coverage is at last 90% is not appropriate when the 5' primers are within the V segment.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/25

For cross-mapping correction, ignore last 5 bases

When computing the distance between two V sequences, ignore differences in those last bases that are part of the CDR3

• Among multiple V sequences, the one with the highest number of CDR3s is usually the full-length one - that one should be kept.

Make germline filters configurable

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

The thresholds for the germline and pre-germline filters are currently hardcoded in the Snakefile. These should become configuration options.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/10

Re-write CDR3 regular expressions in terms of amino acids

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/18

TclError when running igdiscover init

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

When running igdiscover init remotely without providing --db and --reads1, a backtrace is printed:

_tkinter.TclError: no display name and no $DISPLAY environment variable

There should be a nicer error message.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/6

Require sequence structure at 3' end: primer + barcode

This should help to remove strange sequences that should not be analyzed.

Use sqt.dna.nt_to_aa

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

I have implemented a faster nt_to_aa function in the sqt library. We need to use it because the parse subcommand spends much of its time in that routine.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/23

Ensure that default configuration file actually corresponds to defaults

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

That is, test whether reading in an empty configuration file gives the same configuration as reading in the igdiscover.yaml file that we put in each analysis directory.

This ensures that we document the correct defaults in the file.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/13

Discover J genes

The correct J gene is typically the longest one in a cluster

Support barcodes in the 3' end

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/8

When computing has_stop, tolerate stops close to the 3' end

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/24

Remove warning: "wildcard constraints in inputs are ignored"

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/38

Support interleaved input

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

Please vote for this issue if this applies to you.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/7

Set up testing with Travis

See bitbucket-pipelines.yml for how it was done previously.

Create a report as final result

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

Possibly in HTML format. With statistics and plots. Could possibly use pweave.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/16

Make a demo result available for download

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

We should run IgDiscover on a publicly available dataset and make the result folder available for download. This would be great for people that want to get an impression of what the program can do, but do not want to spend 8 hours just to process a realistic dataset.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/37

Spawn only as many igblastn processes as cores

Currently, in the igblast subcommand, a new igblastn process is spawned for each chunk of reads that is processed. This is quite some overhead.

Tests for more subcommands

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

Unit testing should include more subcommands. It should also cover a full run of the pipeline on a test dataset.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/43

Stop iterating when no new sequences are discovered

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/2

Omit species from database file names

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

That is, the database files should no longer be named speciesname_V.fasta, speciesname_D.fasta and speciesname_J.fasta, but only V.fasta, D.fasta and J.fasta.

This is a requirement for solving issue #15.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/33

Add J region sequence to assigned.tab

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/5

Ensure the meaning of "db" in candidates.tab is documented

Whitelist sequences in the input database by default

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/11

Update test data set to reflect new file names

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

The database FASTA files in the "testdata-0.2" dataset still include the species name.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/36

Count and report clonotypes

For now, two sequences belong to the same clonotype if

• they have the same V and J assignment (ignore D)
• their CDR3s have the same length
• their CDR3s are identical to some adjustable threshold (70-100%)

Input is the filtered.tab.gz file. Output should be a table in which sequences coming from the same clonotype are listed next to each other.

Draw a graph of similar novel sequences

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/34

Better subsampling

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

If there are many sequences assigned to a single gene, we may miss some low-expressed ones when we cluster only 1000 randomly picked ones. We can instead subsample from those that have some minimum edit distance to the database gene in order to make sure that we do get more interesting sequences.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/14

Add ability to analyze light chains

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

The light chain doesn’t contain a D gene, but the pipeline currently complains if there is no D gene database.

Other fixes may also be necessary. Perhaps the CDR3 regex needs to be modified.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/3

Display mailing list address in documentation

Originally reported by: Marcel Martin (Bitbucket: marcelm, GitHub: marcelm)

---

and also when running IgDiscover.

---

• Bitbucket: https://bitbucket.org/igdiscover/igdiscover/issue/29