qiime2/dev-docs

Developer documentation for QIIME 2

qiime2 (the QIIME 2 framework)

Source code repository for the QIIME 2 framework.

QIIME 2™ is a powerful, extensible, and decentralized microbiome bioinformatics platform that is free, open source, and community developed. With a focus on data and analysis transparency, QIIME 2 enables researchers to start an analysis with raw DNA sequence data and finish with publication-quality figures and statistical results.

Visit https://qiime2.org to learn more about the QIIME 2 project.

Installation

Detailed instructions are available in the documentation.

Users

Head to the user docs for help getting started, core concepts, tutorials, and other resources.

Just have a question? Please ask it in our forum.

Developers

Please visit the contributing page for more information on contributions, documentation links, and more.

Citing QIIME 2

If you use QIIME 2 for any published research, please include the following citation:

Bolyen E, Rideout JR, Dillon MR, Bokulich NA, Abnet CC, Al-Ghalith GA, Alexander H, Alm EJ, Arumugam M, Asnicar F, Bai Y, Bisanz JE, Bittinger K, Brejnrod A, Brislawn CJ, Brown CT, Callahan BJ, Caraballo-Rodríguez AM, Chase J, Cope EK, Da Silva R, Diener C, Dorrestein PC, Douglas GM, Durall DM, Duvallet C, Edwardson CF, Ernst M, Estaki M, Fouquier J, Gauglitz JM, Gibbons SM, Gibson DL, Gonzalez A, Gorlick K, Guo J, Hillmann B, Holmes S, Holste H, Huttenhower C, Huttley GA, Janssen S, Jarmusch AK, Jiang L, Kaehler BD, Kang KB, Keefe CR, Keim P, Kelley ST, Knights D, Koester I, Kosciolek T, Kreps J, Langille MGI, Lee J, Ley R, Liu YX, Loftfield E, Lozupone C, Maher M, Marotz C, Martin BD, McDonald D, McIver LJ, Melnik AV, Metcalf JL, Morgan SC, Morton JT, Naimey AT, Navas-Molina JA, Nothias LF, Orchanian SB, Pearson T, Peoples SL, Petras D, Preuss ML, Pruesse E, Rasmussen LB, Rivers A, Robeson MS, Rosenthal P, Segata N, Shaffer M, Shiffer A, Sinha R, Song SJ, Spear JR, Swafford AD, Thompson LR, Torres PJ, Trinh P, Tripathi A, Turnbaugh PJ, Ul-Hasan S, van der Hooft JJJ, Vargas F, Vázquez-Baeza Y, Vogtmann E, von Hippel M, Walters W, Wan Y, Wang M, Warren J, Weber KC, Williamson CHD, Willis AD, Xu ZZ, Zaneveld JR, Zhang Y, Zhu Q, Knight R, and Caporaso JG. 2019. Reproducible, interactive, scalable and extensible microbiome data science using QIIME 2. Nature Biotechnology 37:852–857. https://doi.org/10.1038/s41587-019-0209-9

dev-docs's Issues

wrong function call in Associating Formats with a Type?

Bug Description
@HSapers turned up what looks like an oversight in Associating Formats with a Type, where we show plugin.register_semantic_types instead of plugin.register_semantic_type_to_format. The latter function is never used on this page of the docs.

Are we correct that this is an error? If so, @HSapers would be happy to cut a PR fixing the issue.

API docs typeface is hard to read

imp: update dev docs with python 3.8 examples

Improvement Description
Our dev docs currently reference python 3.6 in our conda tutorial. We should update this to python 3.8, and update the channel that contains the qiime2 label to r2022.8.

Steps required:

1. Replace all instances of python 3.6 to python 3.8
2. Replace all instances of https://conda.anaconda.org/qiime2/label/r2019.1 to https://conda.anaconda.org/qiime2/label/r2022.8

References
https://dev.qiime2.org/latest/tutorials/conda-tutorial/

Set up deploy target to build `dirhtml` target first

Comments
This is probably super simple, but since I don't know how Makefile inheritance works, saving this for a rainy day.

add document covering pros and cons of creating a plugin versus contributing to an existing one

Addition Description
It would be helpful to have a document that describes pros and cons of building a new plugin versus contributing to an existing (possibly "core distribution") plugin.

References
@thermokarst and I started discussing this in the context of this PR.

Section 11: Getting Additional Help

No particular outline for this section.

Section 7: Testing Plugins

May be useful to mention busywork here.

Potentially Unused Variables Containing Older Build Versions

Description
There are two variables in the conf.py file which contain old build version values of QIIME 2. If these variables are not used, they should be removed from the file.

Steps to reproduce the behavior

• Go to source/conf.py file and see lines 65 and 67.

Screenshot

Computation Environment

• OS: macOS Mojave
• QIIME 2 Release 2019.4

Comments
The priority for this issue is low. Good first issue.

Section 2: The Plugin Object and Entrypoints

We do not have a particular outline for this section. But it should describe the plugin object and how it is discovered by the framework.

Dev quickstart

Section 3: Actions

Should include the following sections:

1. Methods
2. Visualizers
3. Pipelines
4. Transformers

Section 4: Storing Data in QIIME 2

Proposed Behavior
Should include the following sections:

• Semantic Types
• Formats and Directory Formats
• Archive Structure
• Provenance

Add "under construction" banner

While we are bootstrapping things, if someone stumbles across the rendered docs it would be nice for them to have some kind of warning about the in-progress / incomplete nature of these docs.

Set up scrolling code blocks

Section 5: Using Metadata

We do not have a particular outline for this section.

transition all relevant content to new developer documentation

• move relevant content to https://github.com/caporaso-lab/developing-with-qiime2
• archive this repository

Section 10: API Reference

Should include:

• Plugin Object
• Directory Format Classes
• Format Classes
• Action
• Context
• Utils
• Metadata
• Results Object
• Artifact
• Visualization
• Types
• Type Expression
• Predicates
• Transformers

Document semantic validation

Some initial work was done in #51, but it's pretty incomplete.

Create a README

Glossary

Update all dev docs to reflect python 3.6 change

Improvement Description
There are still references to Python 3.5 in the dev docs, which should be updated to reflect the fact that QIIME 2 is now using Python 3.6.

Comments
I'm happy to do this, though I thought this might be a good beginner-friendly task that someone else could take up as well.

References
I found a few in Publishing a plugin, but there may be other places too.

Provision new SSL cert

Section 9: Internal Details

Proposed Behavior
Should include these sections:

1. Garbage Collection
2. Metaprogramming
3. Pickling
4. Transformation API

search is broken

either we fix it or trash it

Many quick search links yield 404 errors.

Bug Description
Many but not all links on quick search results pages fail to generate descriptions, and produce 404 Not Found errors.This could be an issue with nginx's handling of the links, too, I guess. 🤷‍♂️

Based on a little quick poking, pages nested immediately below dev.qiime2.org/latest are handled properly. These are all basically table of contents pages, and index.html files. Child pages (content pages) nested farther down fail.

E.g.

• Success: dev.qiime2.org/latest/api-reference/index.html?highlight=api
• Fail: dev.qiime2.org/latest/api-reference/sdk.html?highlight=api fails

Steps to reproduce the behavior

1. Go to 'dev.qiime2.org'
2. Enter a common search term in the Quick Search bar (e.g. 'API'). Click 'Go'
3. Look at results, click some links

Expected behavior
Search results let users click through to actual content.

Screenshots
In this screenshot, all links with ... descriptions fail:

Computation Environment

• OS: linuxmint 19.3
• Browser: firefox 74, chromium 81.0.4044.138

Questions

1. Is this an issue with the generated links, or with nginx's handling of them?

Section 6: Plugin Coordination (Namespace Considerations)

No particular outline for this section.

Can the sphinx templating be factored out from the content?

Questions
For example, could we make a git submodule for the sphinx config, allowing us to re-render older docs without needing to backport a bunch of configuration?

Add conda requirements file for sphinx & friends

Section 8: Publishing Plugins

This will be all about conda, and should probably mention busywork.