tillmacher/taxontabletools

TaxonTableTools - A comprehensive, platform-independent graphical user interface software to explore and visualise DNA metabarcoding data

TaxonTableTools

Background

TaxonTableTools (TTT) aims to provide easy to use tools for biologists and non-bioinformaticians to analyse and visualize their DNA metabarcoding data quickly and reproducible via a graphical user interface.

TaxonTableTools is an evolving software and there will be bugs and issues at few points. If so, please leave the report in the git repository or drop me an email. Furthermore, new content and functions will be gradually added. Suggestions and recommendations for new features are always welcome!

Requirements

• Python version 3.6 or 3.7, 3.8, and 3.9
• TTT will most likely not work with python 3.10 and 3.11
• Pip python package installer
• OS independent (tested on Windows 10, Ubuntu 16 & 18 and MacOS Catalina)
  • Some issues were reported during the installation on Windows
  • Some issues were reported during the installation on MacOS

Installation

TaxonTableTools requires Python version 3.6 or 3.7 and can be easily installed using pip.

First, make sure you run the correct pip version via:

python3.9 -m pip --version

Which should return "pip 21.3.1 from /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pip (python 3.9)" or similar.

Then install numpy and TaxonTableTools via pip:

python3.9 -m pip install numpy

python3.9 -m pip install taxontabletools

TaxonTableTools can then be started via:

python3.9 -m taxontabletools

Updates can be installed via:

python3.9 -m pip install --upgrade taxontabletools

The installation of Krona tools is optional and needs to be done separately. Note that Krona tools is currently not supported on Windows.

Tutorial

The easiest way to learn to how to use TaxonTableTools is to check out the tutorial!

Quick start

• When first launched TTT will ask to define an output directory. This is where all your projects and respective output files will be stored. A new folder "Projects" will be created in this directory.

• The next window will always appear when launching TTT. Here you can create new projects or load an already existing one. The output directory can also be changed here.

• Create a new project by typing the desired name or leave blank to create a "Default_project". Click on "Create new" to proceed.

• A blank template TaXon table is available here.

Graphical user interface

Examples

Ecological traits

OTU-specific traits can now be added to the TaXon table. Traits can e.g. be:

• habitat
• diet type
• distribution
• ecological traits

Requirements for trait source tables (.xlsx):

• Column A: One of the TTT-specific taxonomic level (Phylum, Class, Order, Family, Genus, Species)
• Column B: Name of the trait

Example for aquatic invertebrates

Requirements for manual import:

• The traits columns must be located between the 'Status' and 'seq' columns!
• Do not alter the location of the other TaXon table columns.

There is no limit of OTU traits columns that can be added. A TaXon table containing OTU traits is compatible with all modules (OTU traits will simply be ignored if not required).

Example for aquatic invertebrates

Change log (only for major changes)

v 1.4.8

• New feature:

• Added support for easier read and taxonomy table conversion from APSCALE.

• Important changes:

  -> Taxonomy table sheet name for the APSCALE import changed to 'Taxonomy table’

  -> Read table column name for OTU sequences changed to 'Seq'

  -> Those are the default names generated in APSCALE and makes importing data easier.

• Bug fixes:

• Y-axes for alpha diversity and rarefaction plots start at 0.

• Fixed crash of the venn diagram module.

• Fixed potentially remaining zero read OTUs after read-based rarefaction.

v 1.4.5

TTT change log v 1.4.5

• Added trait import to the data conversion modules.

v 1.4.4

TTT change log v 1.4.4

Bug fixes

• Fixed incorrect text boxes.

NMDS

• Changed back NMDS to raw stress until sclearn updates to normalized stress.
• TTT will be updated as soon as normalized stress is available!

v 1.3.1

• Metadata entries for specific samples can now be left blank to exclude the sample from the analyses. This avoids the need to create new metadata tables for each subset of a TaXon table
• Taxon lists have been overhauled and now include presence/absence indications for samples or metadata groups
• Fixed incorrect y-axis title in the per taxon plot
• Enhanced y-axis title of shared OTUs per bin plot
• Fixed Qiime2 conversion
• Fixed OTU and read correlation layout bugs

v 1.3.0

• Enhanced TaXon table check. Missing taxonomy is now marked with a flag
• User preferences can now be saved
• User will now be informed about available updates
• User can choose what the clustering units (e.g. OTUs, ASVs) shall be called
• Genus and species names are displayed in italic
• Added additional replicate analysis plot
• TaXon table filter tools print the number of removed OTUs
• Metadata tables can be opened in TTT
• Added new tool to subtract reads present in negative controls from all samples
• Added option to distinguish different taxa in the rarefaction plot
• Added site occupancy heatmap option
• Added parallel category chart
• Added option to display single samples in one Krona chart
• Added non.metric multidimensional scaling (NMDS) analysis
• Added bray-curtis option to diversity and ordination analyses
• Added vernacular names download in the taxon list
• Log files can be opened in TTT
• Several layout changes
• Various minor bug fixes

v. 1.2.4

• Fixed "TaXon table consuming" bug on some Windows 10 machines

v. 1.2.3

• Fixed alpha diversity bug
• Missing file in installation path fixed.
• Corrected minor spelling and layout mistakes.

v. 1.2.0

Major update:

• The identifier "IDs" was changed to "ID" in the TaXon table header!
• Please adjust old TaXon tables accordingly!
• Otherwise TTT will crash!

New features:

• Metadatables can now directly be adjusted from within TTT.
• Browsing TaXon tables will directly show the current project folder.
• The OTU prompt is more relaxed. Hash OTUs are accepted now.
• Qiime2 feature tables can now be converted to the TTT format.
• Additional options for customizing colors have been added.
• Font sizes can now be adjusted.
• Diversity and ordiniation analyses can now be performed on different taxonomic levels.
• Diversity and ordiniation analyses automatically transform data to boolean values. Thus, also non presence/absence converted tables can be used.
• Moved various options to drop down menus.

Bug fixes:

• Fixed "shared OTUs" tool. More than 3 replicates are now supported.
• Changed x*axis value to read proportions in the per taxon analysis.
• Added missing package 'statsmodels' to installation.

How to cite

If you use TTT:

• Macher, T.H., Beermann, A. J., & Leese, F. (2021). TaxonTableTools—A comprehensive, platformindependent graphical user interface software to explore and visualise DNA metabarcoding data. Molecular Ecology Resources. doi: https://doi.org/10.1111/1755*0998.13358

If you create Krona charts, please also cite:

• Ondov, B. D., Bergman, N. H., & Phillippy, A. M. (2011). Interactive metagenomic visualization in a Web browser. BMC Bioinformatics, 12(1), 385. doi: 10.1186/1471210512*385

taxontabletools's Issues

Issues when installing TTT on MacOS

Issue: Installation fails when trying to install the scikit-bio dependency MacOS

Solution:

• Install brew on your Mac (https://brew.sh/)
• Install gcc using brew
  • $ brew install gcc
• Install openssl using brew
  • $ brew install openssl
• Install wget using brew
  • $ brew install wget
• Install geos using brew
  • $ brew install geos
• Then follow this: https://stackoverflow.com/questions/68839809/issues-with-scikit-bio-on-mac-m1
  • Hint: If you do not use the default version of Python in your PATH you have to specify the installation:
  • $ python3.9 setup.py install (instead of just 'install .')

Unable to used TTT

Hey Macher

I am not able to use the TTT. I have python version 3.9 in my Mac. I have prepare the Table Taxon file as suggested but this is showing error in the TTT.
Could you please help me in the same.

Regards
Rishikesh

Using TTT with ASVs

Hello,
I'm wondering if you see any reason why I couldn't use TTT for output from other pipelines that produce ASV tables rather than OTUs (ANACAPA for example)? Of course the tables will be formatted differently and will need to be modified to fit your template.
Thanks for your thoughts on this.
Sarah

AttributeError: 'DataFrame' object has no attribute 'append'

Hi Till,

I am honoured to inform you that I have earned gold medal in the program crashing category again. I was competing in the 'Read Proportions > Calculate bar chart' section and encountered an [x_x] unexpected error [x_x], despite the accurate plotting of the heatmap and pie chart. This was a first-time occurrence, and the function had previously been functioning properly.

best, Onur.

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/bin/taxontabletools", line 33, in <module> sys.exit(load_entry_point('taxontabletools==1.4.8', 'console_scripts', 'taxontabletools')()) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/taxontabletools/__main__.py", line 1696, in main read_proportions_bar(taxon_table_7_path, read_proportions_bar_taxonomic_level, path_to_outdirs, width_read_props_bar, heigth_read_props_bar, template, font_size, color_discrete_sequence) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/taxontabletools/read_proportions.py", line 292, in read_proportions_bar plot_df = plot_df.append(df) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/core/generic.py", line 6204, in __getattr__ return object.__getattribute__(self, name) AttributeError: 'DataFrame' object has no attribute 'append'

Issues when installing TTT on Windows

Installation of scikit-bio fails

• Issue: Installation fails when trying to install the scikit-bio dependency hdmedians on Windows
• Solution: Install C++ Build Tools (more information: https://stackoverflow.com/questions/40504552/how-to-install-visual-c-build-tools)

TTT freezes when writing a plot (Windows)

Solution: Downgrade kaleido to version 0.1.0post1

pip3 install kaleido==0.1.0post1

TaXon tables get corrupt when plotting Taxonomic resolution / richness

Hello! My TaXon tables get consumed after running "Taxonomic resolution" or "Taxonomic richness".
Their file size becomes 0 KB and Excel can't open them, "because the file format or file extension is not vaild." The plots are saved as expected, but I need to backup the TaXon table before running either of the two commands.
Other functions work fine. First I thought that it's a consequence of the warning about non-"binomial taxonomy" in my input table, but I fixed the taxonomy and the problem persists.
I set up TTT on Windows 10 last week.

Unexpected error: <class 'AttributeError'>

Issue: Attempting to work through the tutorial using the tutorial files. However, whenever I reach try to merge replicates using the 'replicate consistency' function or use table filtering to extract Arthropods. I am met with the following message: "Unexpected error: <class 'AttributeError'>.