jcarrano/antidox

Sphinx autodoc & autoindex using Doxygen

Antidox: Use Doxygen sanely from within Sphinx

An antidote to doxy-madness

Summary

antidox is a Sphinx extension that can read Doxygen XML "databases" and insert documentation for entities in Sphinx documents, similar to Breathe.

It is intended to be fast and simple, though easily customizable.

Document generation (i.e. conversion between doxy-xml and reStructuredText) is driven by XML stylesheets (powered by lxml,) while indexing and selection of documentable entities is done by a SQL database (sqlite3.)

Here is an example project showing showing this extension in action.

Example usage

Generate the documentation for an entire header, and include all entities defined in that header:

.. doxy:c:: lua_run.h::*
    :children:

The syntax <path>::<identifier> can be used to disambiguate between entities with the same name in different files.

To document a Doxygen group:

.. doxy:c:: [CborPretty]
    :children:

You can manually specify which children should be documented:

.. doxy:c:: be_uint16_t
    :children: u16

Cross references are provided by a custom role, e.g.:

:doxy:r:`be_uint16_t::u16`

The complete syntax is decribed in the docs.

Stub generation

The gen_stubs.py script shows how stub files can be automatically generated. You can adapt this script to your own project.

Generating API docs this way is fast and convenient, but may be suboptimal, since the spirit of this extension (and of Sphinx) is to generate narrative documentation and not merely an API reference.

Note: Beta

Though usable, this extension is still under development. Backwards compatibility will be kept for all releases with the same major/minor version.

Be aware, however, that after updating this extension you may need to do a clean build of your docs to see the results.

Objectives

• Reuse API docs made with Doxygen in a Sphinx project.
• Provide a smooth transition between 100% automatic API docs (what Doxygen generates) and semi-manual documentation (autodoc-style).
• Have sensible defaults for automatic documentation generation while allowing customization.
• Deal with big projects efficiently: the main tool in use now (Breathe) has resource usage issues when dealing with large XML files.

antidox's Issues

Enable external XML file

And make the build depend on that file.

It should be possible to import the default stylesheet. See https://lxml.de/resolvers.html#i-o-access-control-in-xslt for more info.

Document how to run doxygen before the sphinx build.

We can steal that from breathe.

Add Templates

Add at least basic support for the following entity types. Aesthetics can be solved later:

• class
• struct
• union
• define
• variable
• typedef
• enum
  • enumvalue
• function
• Aggregate Compounds
  • group
  • file

Low priority/extras:

• dir
• exception
• namespace
• page
• example

Add doxyfile as dependency.

Tell sphinx that the output depends on doxygen's documentation.

Target resolution fails for paths that are a prefix of another.

If there is a file a/b.h and another one b.h, resolve_target() will raise AmbiguousTarget.

Add example.

Make an example doxygen project with an example sphinx project.

Maybe we can steal it from somewhere.

Add proper packaging

setup.py and all that stuff.

Implement correct document purges.

• Verify that it is a problem.
• use env-purge-doc

Publish the example project.

We need something to show off.

Implement autoindex/group listing

Provide a means for smooth transition from doxygen html to sphinx.

Of course we can't expect people to write hundreds of rst documents at once, so there must be a way to have entities documented in sphinx without doing it manually.

Fix how the database is initialized.

Plan A:

• Before the (possibly parallel) build starts: load Doxy DB in memory, dump it to string (iterdump), set the string as an attribute of the app.
• (Maybe this cannot be done) Mark the xml documents (maybe only index.xml) with note_dependency
• In the reading phase, read the load containing the DB into each worker. This is much faster than rereading all the xml documents, esp if parallel build is used.

A SQL command dump is smaller than a sqlite saved database (12M vs 7.5M for the RIOT docs).

Document stub generation.

Use the gen_stubs.py script as an example.

Add documentation

This is a documentation tool. It should have documentation!

Turn the README into a marketing/showoff pamphlet and move the real technical data into the manual.

Abandon the philosophy of "offline capable" XSL.

The current documentation states that:

The templating and XML handling logic is designed so that in the future it is possible to run the XSL transformation online, using generic tools. For this reason, there should not be any custom functions defined and no stylesheet parameters that depend on the plugin to set them.

Recent profiling has proven that XSLT processing comprises only a minor percentage of total time (in fact, after aa4525a most time is taken by the writing step.)

By abandoning the "offline" philosophy it will be possible to simplify code and enhance customizability:

• Add custom XSL function that can query the DoxyDB.
• Add template parameters that are supplied from the directive's parameters.

An obvious candidate is the :noindex: parameter, that is still unimplemented and would be trivial if XSL template parameters were allowed.

Going back to the above quote, the idea is already being betrayed in the XSL implementation. From the docs:

To apply the XSL transformation, the XML element corresponding to the entity being documented (e.g. a or a ) is extracted from its containing document and the transform is run as if that element was the root element. This means XPath expressions may give different results when the the same stylesheet is applied to a whole doxygen XML.

future of sqlite3 in doxygen

Hi @jcarrano,

I noticed this repo on a search I occasionally run for new code interacting with Doxygen's sqlite3 database. It didn't look like you're actually using it--just that you mention it?

In any case, I wanted to give you a heads up that I'm working on a pretty big update to Doxygen's sqlite3 generator. If I can get the contribution accepted, the schema (and quality of the data it holds) may change in a future Doxygen version.

Also, are you aware of Breathe (https://github.com/michaeljones/breathe)? At a glance, it sounds like you're both working to leverage Doxygen's output, via XML, to generate Sphinx documentation.

Not working with Sphinx-3

Description

The setup should request sphinx<3 until this is fixed.

Error

Running Sphinx v3.1.0
making output directory... done
(Re-)Reading Doxygen DB
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 3 source files that are out of date
updating environment: [new config] 3 added, 0 changed, 0 removed
reading sources... [100%] xxxxxxx
Exception occurred:
  File "/home/jcarrano/.local/lib/python3.8/site-packages/sphinx/domains/c.py", line 3442, in clear_doc
    for fullname, (fn, _id, _l) in list(self.objects.items()):
ValueError: not enough values to unpack (expected 3, got 2)
The full traceback has been saved in /tmp/sphinx-err-n6iurq1w.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

Write a better README

Rebuild database after files are changed.

Detect changed XML files and rebuild the database instead of reloading it from Sphinx's state.

Add c/h files as dependencies.

Tell sphinx that the output depends on the C/h files where the entities are defined.

This needs #5 to be useful.

Alternatively, depend on the XML files (but that would need an external method to check if the content changed, because Doxygen will always touch the files).