Decide on documentation method for CIME

[esmci]

Decide on documentation method for CIME

[rljacob]

@gold2718 suggested:
"Sphinx (http://www.sphinx-doc.org/en/stable/) uses reStructuredText
(rst) and github understands (can render) this format. It also produces
HTML, PDF (rst => LaTeX => PDF), ePub and other formats.

It's free, stable, and installable from python (it started as a python
documentation project)."

[gold2718]

More on Sphinx and ReST:
ReST is a markup language. Some sites I use to learn/use the language are:
https://wiki.typo3.org/ReST_Syntax
https://github.com/kiith-sa/RestructuredText-tutorial
http://thomas-cokelaer.info/tutorials/sphinx/rest_syntax.html

Sphinx is a tool for rendering many formats from ReST documents. On my mac, I simply did:
sudo port install py27-sphinx

This gives you (among other things) a command called sphinx-quickstart. This handy tool sets up a directory structure, template document, and a makefile.
You can then create different formats with commands like:
make latexpdf
make html
make singlehtml

and many others. I don't think github pages does all that. In addition, ReST is a commonly used standard apart from github (e.g., all the Python documentation).

I'm working on a tool to do most of a translation of our current docbook documentation into ReST ('most of' means some hand editing will still be desired).

[rljacob]

Does this workflow mean you could not edit the wiki through the web interface and have the source ReST documents updated? We would have to stick to the one-way: edit source -> push -> render in wiki.
edit in wiki -> pull to get changes would not work?

[gold2718]

No, a github-development workflow works fine.
The point of ReST and Sphinx and the makefile (over, e.g., markdown), is that while we can easily edit and view documents on github, we can, with very little extra effort, also publish single and multi-page HTML as well as PDF versions of our documents.
To have both, set up a documentation source directory with sphinx-quickstart. Then, the source documents can be edited with any tool, including github. Sphinx only needs to be used when producing other formats (e.g., PDF). I have tested this combined workflow, it is really easy.

[rljacob]

I see, and according to this we can mix markdown languages, you just have to get the file extension correct: https://help.github.com/articles/adding-and-editing-wiki-pages-locally/

[rljacob]

One more thing: how do we include figures? Adding binary files to git is usually a bad idea.

[gold2718]

Small binary files are not a problem, especially if they don't change often. Including a few jpegs (.JPG) should not affect things too much (and github renders those as well as PNG, GIF, PSD, and SVG formats). Github doesn't seem to mind images, they even provide tools for comparing image diffs.
https://help.github.com/articles/rendering-and-diffing-images/
I think the rule of thumb for documentation figures should be that they should be uploaded when they are finished or close-to-finished. Larger image files (MB size) should be handled more carefully in terms of uploading more than one version.

[mvertens]

I propose we go with Sphinx and mark this as done. Thoughts?

[rljacob]

Fine with me.

[rljacob]

You can check out the cime wiki with:
git clone [email protected]:ESMCI/cime.wiki.git

I propose we then add our Sphinx-markdown-written documentation to that repo. There are examples of how github wikis are organized:
https://github.com/showcases/projects-with-great-wikis

And you can clone those to see how they did it.
One limitation of the github wiki is all the filenames have to be unique even if they're separated by subdirectories.

[rljacob]

This wiki shows how to use multiple directories.
git clone https://github.com/snowplow/snowplow.wiki.git

[rljacob]

Forget the wiki. Sphinx and github pages might be the way to go:
https://daler.github.io/sphinxdoc-test/includeme.html

http://datadesk.latimes.com/posts/2012/01/sphinx-on-github/

http://raxcloud.blogspot.com/2013/02/documenting-python-code-using-sphinx.html

[gold2718]

I'm following directions from github (https://help.github.com/articles/creating-project-pages-manually/)
After creating the orphan branch per the link above, I Created the Sphinx setup with:

sphinx-quickstart CIME-Users-Guide --sep y --dot _ --project "CIME User's Guide" --author 'Rob Jacob' -v 1.0.0 --release 1.0.0 --language en --suffix .rst --master CIME-Users-Guide --epub n --ext-autodoc y --ext-doctest n --ext-intersphinx n --ext-todo n --ext-coverage n --ext-pngmath n --ext-mathjax n --ext-ifconfig n --ext-viewcode n --makefile y --no-batchfile y

I committed the files and pushed the gh-pages branch back to ESMCI but I can't get things working smoothly. The pages site is: http://esmci.github.io/cime/

[gold2718]

I'm not a fan of the github pages approach. Since github pages only displays HTML, someone has to do a publish step (clone, run sphinx, push) for that site to get updated. At least with the wiki, the documentation is real time. The extra step is to develop a publish script that will turn the wiki repo into other formats (e.g., HTML, PDF). We could still maintain a github pages repo (separate from CIME) if we want to use that site for our published docs.

[gold2718]

I think there is a relatively stable -- if not trivial -- workflow for documentation. Wiki pages can be edited either directly on github or via a cloned repo. I have added some tips for working with the wiki.
I have a proof-of-concept github pages document at http://esmci.github.io/cime/ The process of updating this page from a clone of the wiki repo can easily be scripted but I would like for us to get some real documentation produced before I put much time into it. One nice feature of this approach is that the github.io page can be tagged as the cime gh-pages branch (to match CIME releases) even if we decide to include documentation for multiple versions on that site.