spyder-ide / docrepr Goto Github PK
View Code? Open in Web Editor NEWGenerate rich representations for docstrings
License: BSD 3-Clause "New" or "Revised" License
Generate rich representations for docstrings
License: BSD 3-Clause "New" or "Revised" License
I was following the Sphinx tutorial's section on Automatic documentation generation from code, which has the following docstring:
def get_random_ingredients(kind=None):
"""
Return a list of random ingredients as strings.
:param kind: Optional "kind" of ingredients. <----------- !!
:type kind: list[str] or None <------------------------- !!
:raise lumache.InvalidKindError: If the kind is invalid.
:return: The ingredients list.
:rtype: list[str]
"""
return ["eggs", "bacon", "spam"]
Notice how it uses :param:
and :type
to document both the name and the type of a parameter. When Sphinx renders this docstring (via the autodoc
extension), it looks great:
The above is all about Sphinx.
Now to Docrepr.
When I attempt to render the same docstring with Docrepr, those two directives don't seem to be parsed properly. For example, if I
pip install /path/to/master/branch/docrepr
git clone https://github.com/fasiha/lumache.git && cd lumache
python test_docrepr.py
where test_docrepr.py
contains the following:
from docrepr import sphinxify # html generator
from IPython.core.oinspect import Inspector # oinfo generator
import lumache
oinfo = Inspector().info(lumache.get_random_ingredients)
oinfo['name'] = 'get_random_ingredients'
url = sphinxify.rich_repr(oinfo)
print('file://' + url)
the output file looks like this:
It looks like the two directives, param
and type
both got their own bullet, as if Sphinx didn't recognize that they describe the same parameter?
Is there something special that needs to go into conf.py
for Sphinx to understand this directive? I tried adding a line to conf.py,
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.autosummary',
]
but this didn't change the output of Docrepr.
Am I right in my guess that Docrepr's conf.py needs to be tweaked for this to work? Are there other Sphinx/reST directives that Docrepr doesn't understand?
Thanks for taking a look!
@martinRenou Other than dropping Py3.6 support and adding it to the config file, is there any way to avoid requiring the --asyncio-mode=auto
CLI arg (and the dependency) every time we run tests, even if we don't run the visual regression tests?
On top of that, there seems to be a bug where running pytest --help
, even with --asyncio-mode=auto
, displays the error every time unless the config file is manually edited to include it. This probably should be reported somewhere upstream, but in the meantime it isn't so nice for potential new contributors to not even be able to run pytest --help
without knowing the magic flag to add.
If this is not possible until we drop Py3.6 (planned for 0.3.0), I'll add an explicit prominent mention of this to the contributing guide and perhaps the readme too.
Currently, nothing is rendered at all for plots in docrepr on Windows, even though it appears to work on Linux in an identical environment. This likely has to do with temp file/directory handling or other OS-specific differences. See the discussion on #30 and further discussion on #32.
It would be really nice to have this for v0.2.0, since it is unclear if it was working on Windows before (but seems likely so), but it can be pushed if necessary. Thanks!
docrepr defines some configuration options in its __init__.py
file:
https://github.com/spyder-ide/docrepr/blob/master/docrepr/__init__.py#L20
However, for these options to take effect, you need to do two imports, like this:
import docrepr
from docrepr.sphinxify import rich_repr
We need to find a better way to do this (but at the moment I can't think of a good one :-).
The problem is that without loading those options, things like MathJax don't work correctly.
Pinging @SylvainCorlay about it.
It fails for multiple reasons:
add_directive
does not accept the alt
keyword argumentsphinxify instantiates Sphinx with the same input and output dir, but Sphinx 1.8.1 fails if this is the case:
import docrepr
docrepr.sphinxify.sphinxify("", ".")
---------------------------------------------------------------------------
ApplicationError Traceback (most recent call last)
<ipython-input-10-6651f661623b> in <module>
1 import docrepr
----> 2 docrepr.sphinxify.sphinxify("", ".")
~/conda/envs/torch/lib/python3.6/site-packages/docrepr/sphinxify.py in sphinxify(docstring, srcdir, output_format, temp_confdir)
298 sphinx_app = Sphinx(srcdir, confdir, srcdir, doctreedir, output_format,
299 confoverrides, status=None, warning=None,
--> 300 freshenv=True, warningiserror=False, tags=None)
301
302 # Run the app
~/conda/envs/torch/lib/python3.6/site-packages/sphinx/application.py in __init__(self, srcdir, confdir, outdir, doctreedir, buildername, confoverrides, status, warning, freshenv, warningiserror, tags, verbosity, parallel, keep_going)
158
159 if self.srcdir == self.outdir:
--> 160 raise ApplicationError(__('Source directory and destination '
161 'directory cannot be identical'))
162
ApplicationError: Source directory and destination directory cannot be identical
I suppose a separate temporary directory needs to be created.
I've noticed this a while ago, but didn't want to hold up the v0.2.0 release with it.
There's a significant visual regression with docstring display using Sphinx 4 (repro'd in 4.0.3 and 4.4.0; renders correctly in 3.5.3). The headings don't display as they did before, and collapse mode no longer works.
I tested all the way back to the earliest working commits using the fabfile.py
collapse test and the difference between Sphinx 3 and Sphinx 4 was still present, in otherwise identical environments. From skimming the DOM, it looks like either the script that does it isn't firing at all, or it isn't finding the correct elements to modify.
@martinRenou , any change you or one of your colleagues could help fix this?
Running the numpy.sin test works as before.
python -I -bb -X dev -m pytest --open-browser -k numpy_module --asyncio-mode=auto -W default
gives:
Running the numpy.sin test works as before
python -I -bb -X dev -W error -m pytest --open-browser -k sin --asyncio-mode=auto
gives:
Running the collapse test, however, shows very different results.
python -I -bb -X dev -W error -m pytest --open-browser -k collapse --asyncio-mode=auto
gives:
It is not clear if it is a command line tool, web server or a library that requires some IDE host
When using the plot_directive, the HTML file seems to look for Matplotlib screenshots under the _images
directory, while they are at the same level as the HTML file:
From a first small investigation, this _images
directory suffix seems to come from Matplotlib itself, we don't set it here in docrepr.
I plan to add a suite of standard pre-commit hooks that run locally and in the CIs to check for (and in some cases, fix) issues, lint for and code quality issues, auto-format the code and ancillary files and more, based on what we have in our other repos. While I'd like to have this for v0.2.1, it won't block the release and I'm fine with bumping it as needed.
As originally discussed in PR #21
See our modern standard pytest.ini on e.g. the QtPy repo; feel free to pull that in here (minus the coverage options, unless you want that). @SylvainCorlay assuming you want to go ahead with this, it'll of course be up to you how much you want to make the tests idiomatic Pytest (with fixtures, parameterization, etc) and/or how much you want to add, beyond just getting the basics up and running.
We can also add the CI workflow in #24 as part of the same PR once the basic tests are set up (perhaps as a tag-team effort), or I can take care of that in a followup PR. Might make sense to do it separately, since I've had trouble kicking off the first GH Actions run from branches on forks and its best to limit that as much as practicable.
More or less blocker of #24
I'd love to be able to make a function/magic to display a rich HTML repr of the docstring in the Notebook. Unfortunately it seems a naive implementation doesn't display properly. I guess I've got two questions:
If it is possible I'm happy to give it a go if I'm pointed in the right direction - I'm not too sharp with web technologies so am at a bit of a loss where to begin.
As discussed in PR #21 . See spyder-ide/qtpy#270 for more details and spyder-ide/qtpy#272 for the basic template this will follow.
As originally mentioned in PR #21
Here, we can use a simplified version of our basic workflow template, as used on many of our other repos. QtPy (e.g. as of spyder-ide/qtpy#272 ) is the most recent example, though in that case (like most of the other extant ones, for various specific reasons), we used Miniconda to set up the envs and a bash script to run through them, so we could test different Qt binding versions on the same job. Here, that could be greatly simplified to something most closely resembling what I have in Sub Manager (which was itself based on the previous Spyder repo GitHub actions I developed), though with the further improvements made to those in QtPy and elsewhere.
More or less blocked on #23
Also reported by @goanpeca: docrepr should use sass or less in its css, so that it's easy to customize its appearance.
Hi there,
I noticed that signature are correctly displayed for class
objects, but not for function
objects using docrepr v.0.1.0
(pip install, python 3.4.3, windows 7 x64).
Actually the whole "Definition:" line is missing in the case of functions.
By the way, docrepr is really great when you want to generate a simple doc using Numpy style docstrings. At least it is far simpler than digging into Sphinx to get Numpy's style correctly rendered (can't get neither numpydoc nor napoleon working, but in my case PEBKAC as I am a complete noob at Sphinx at this time.)
Hope you'll keep going with docrepr!
@martinRenou Looks like our visual regression tests just started failing on the Class Sphinx test on all platforms, presumably due to a dep update in the CIs in the past 24 hours, "Web Help" is being mistakenly shown when I presume it shouldn't, given custom toy classes certainly don't have it. I've seen this elsewhere on some other cases; I presume the code that detects whether web help is available (AFAIK, just for stdlib code?) is rather buggy? I can look into it, but I appreciate your help too, and this would be a good chance to test out the bot in a real-world setting.
I just read https://blog.jupyter.org/inspector-jupyterlab-404cce3e1df6 and I am excited to see that this repo was revived last fall, thank you for working on it again! Currently it seems like I need to have docrepr installed in every virtual environment where I want to use it. I usually have a separate environment for my jupyterlab install and then use nb_conda_kernels to work with other environments. For this workflow it would be convenient if docrepr could just be installed once (in the jupyterlab env) and then work for jupyterlab launched from that env, regardless of which kernel a specific notebook is using. Would it be possible to implement this feature or is it relatively hard/time consuming?
I just noticed that IPython builds are failing, with an error like this:
File "/home/docs/checkouts/readthedocs.org/user_builds/ipython/envs/latest/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 90, in <module>
import docrepr.sphinxify as sphx
File "/home/docs/checkouts/readthedocs.org/user_builds/ipython/envs/latest/lib/python3.5/site-packages/docrepr/sphinxify.py", line 53, in <module>
CACHEDIR = tempfile.gettempdir() + osp.sep + 'docrepr-' + username
TypeError: Can't convert 'NoneType' object to str implicitly
I guess that maybe the USER
environment variable isn't set on RTD?
Since 0.2.0 was the first Docrepr release in quite a while, and there are some moderately to perhaps more critical bugs remaining for 0.2.1, we'll probably want to release 0.2.1 as soon as a couple weeks after 0.2.0 is out on PyPI and conda, particularly if any important/critical bugs are found. I opened this issue as a general venue to discuss and track progress toward the next patch release. The plan after this is to release 0.3.0 dropping Python 3.6 support and possibly larger changes, as well as any further bugfix releases needed on the 0.2.x branch
Hard-blockers:
Soft blockers:
Targeted for v0.2.0, but can be deferred to a later patch or minor release:
Other misc items:
As mentioned in #28 (comment), there is a copy of Matplotlib's plot_directive
since this commit.
We should probably remove this copy and use Matplotlib's directive instead. This would reduce some maintenance burden, e.g. we wouldn't have to update our copy for latest Sphinx version support, as Matplotlib devs did it already.
Also incorporates #20
Python 3.6 just went EOL, and it imposes additional difficulties in our tests, CIs and elsewhere to maintain support for it. We'll want to drop it for 0.3.0. This may also let us do other cool things, like adding static type hints and Mypy checking.
Following the various fixes and improvements that were merged recently, would there be any objection with me tagging a release of docrepr as of the current state of the master branch?
The version available on PyPI is completely non-functional, and having the current version available is currently our main blocker to move forward with the downstream use case in Jupyter. We (@SylvainCorlay, @martinRenou, @fasiha) definitely plan on continuing improving docrepr.
cc @CAM-Gerlach
I am trying to us docrepr to render docstrings in JupyterLab, but it seems like only top level headings in the docstring are rendered, not other headings, code, etc. This is with the following versions:
# Name Version Build Channel
docrepr 0.2.0 pyh050c7b8_0 conda-forge
jupyterlab 3.3.4 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge
Right now, the vendored CSS includes 7 year old versions of Bootstrap (3.3.5) and pygments from 2015; the latter at least should be a mostly drop-in upgrade and we can at least bump the former to 3.4.1, if not v4 or v5.
More concerning, the vendored JS dates from the same vintage, including the corresponding Bootstrap 3.3.5, JQuery 1.11.3 and an old copy of Python.org's "copy button" script (which I'm not sure even works now, as I don't see the button and it explicitly notes it "only works with JQuery 1.8.3 or less".
Also, mathjax is apparently attempting to be fetched from cdn.mathjax.org, which is shut down and generates a console warning (I guess its not vendored locally?). Not sure what's going on there.
Finally, the HTML template still has a XHTML 4 Transitional doctype; looks pretty straightforward to bump that to HTML 5 (I didn't notice any other particular problems there).
@SylvainCorlay , do you think any of your developers could take a look at helping pay off this techdebt? Its not ultra-urgent but Its going to be a problem sooner or later, and may particularly concern security-conscious corporate deployments.
Hi,
I found this issue via the test suite, and traced it to the fact that the source files copied by sphinx itself when instantiating the Sphinx object (sphinx_app
in sphinxify.py
) were read-only. The error looks like:
___________________________ test_sphinxify[outline] ____________________________
build_oinfo = <function fixture_build_oinfo.<locals>._build_oinfo at 0x7fffd551a1f0>
set_docrepr_options = <function fixture_set_docrepr_options.<locals>._set_docrepr_options at 0x7fffd551ae50>
open_browser = <function open_browser.<locals>._open_browser at 0x7fffd54de310>
compare_screenshots = <function compare_screenshots.<locals>._compare_screenshots at 0x7fffd54de4c0>
test_id = 'outline', obj = <ufunc 'sin'>, oinfo_data = {'name': 'sin'}
docrepr_options = {'outline': True}
@pytest.mark.asyncio
@pytest.mark.parametrize(
('test_id', 'obj', 'oinfo_data', 'docrepr_options'),
_test_cases_to_params(TEST_CASES),
ids=list(TEST_CASES.keys()),
)
async def test_sphinxify(
build_oinfo, set_docrepr_options, open_browser, compare_screenshots,
test_id, obj, oinfo_data, docrepr_options,
):
"""Test the operation of the Sphinxify module on various docstrings."""
if (oinfo_data.get('docstring', None) == PLOT_DOCSTRING
and sys.version_info.major == 3
and sys.version_info.minor == 6
and sys.platform.startswith('win')):
pytest.skip(
'Plot fails on Py3.6 on Windows; older version of Matplotlib?')
oinfo = build_oinfo(obj, **oinfo_data)
set_docrepr_options(**docrepr_options)
> url = docrepr.sphinxify.rich_repr(oinfo)
docrepr/tests/test_output.py:240:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/gnu/store/dmcslfpdfyv74mfwwbqliksr255xk8iw-python-docrepr-0.2.0/lib/python3.9/site-packages/docrepr/sphinxify.py:450: in rich_repr
class_doc = sphinxify(wrapped_class_docstring, srcdir)
/gnu/store/dmcslfpdfyv74mfwwbqliksr255xk8iw-python-docrepr-0.2.0/lib/python3.9/site-packages/docrepr/sphinxify.py:405: in sphinxify
merge_directories(destdir, srcdir)
/gnu/store/dmcslfpdfyv74mfwwbqliksr255xk8iw-python-docrepr-0.2.0/lib/python3.9/site-packages/docrepr/utils.py:52: in merge_directories
shutil.copytree(source, destination, dirs_exist_ok=True)
/gnu/store/b6j1qw1a5rkbfvcy7lc9fm95abbzpa4x-python-3.9.9/lib/python3.9/shutil.py:566: in copytree
return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
entries = [<DirEntry 'docstring.html'>, <DirEntry 'search.html'>, <DirEntry '_static'>, <DirEntry '.buildinfo'>, <DirEntry 'searchindex.js'>, <DirEntry 'objects.inv'>]
src = '/tmp/guix-build-python-docrepr-0.2.0.drv-0/tmpev83mper'
dst = '/tmp/guix-build-python-docrepr-0.2.0.drv-0/docrepr-None/tmpbm5c0r1l'
symlinks = False, ignore = None
copy_function = <function copy2 at 0x7ffff73c5d30>
ignore_dangling_symlinks = False, dirs_exist_ok = True
def _copytree(entries, src, dst, symlinks, ignore, copy_function,
ignore_dangling_symlinks, dirs_exist_ok=False):
if ignore is not None:
ignored_names = ignore(os.fspath(src), [x.name for x in entries])
else:
ignored_names = set()
os.makedirs(dst, exist_ok=dirs_exist_ok)
errors = []
use_srcentry = copy_function is copy2 or copy_function is copy
for srcentry in entries:
if srcentry.name in ignored_names:
continue
srcname = os.path.join(src, srcentry.name)
dstname = os.path.join(dst, srcentry.name)
srcobj = srcentry if use_srcentry else srcname
try:
is_symlink = srcentry.is_symlink()
if is_symlink and os.name == 'nt':
# Special check for directory junctions, which appear as
# symlinks but we want to recurse.
lstat = srcentry.stat(follow_symlinks=False)
if lstat.st_reparse_tag == stat.IO_REPARSE_TAG_MOUNT_POINT:
is_symlink = False
if is_symlink:
linkto = os.readlink(srcname)
if symlinks:
# We can't just leave it to `copy_function` because legacy
# code with a custom `copy_function` may rely on copytree
# doing the right thing.
os.symlink(linkto, dstname)
copystat(srcobj, dstname, follow_symlinks=not symlinks)
else:
# ignore dangling symlink if the flag is on
if not os.path.exists(linkto) and ignore_dangling_symlinks:
continue
# otherwise let the copy occur. copy2 will raise an error
if srcentry.is_dir():
copytree(srcobj, dstname, symlinks, ignore,
copy_function, dirs_exist_ok=dirs_exist_ok)
else:
copy_function(srcobj, dstname)
elif srcentry.is_dir():
copytree(srcobj, dstname, symlinks, ignore, copy_function,
dirs_exist_ok=dirs_exist_ok)
else:
# Will raise a SpecialFileError for unsupported file types
copy_function(srcobj, dstname)
# catch the Error from the recursive copytree so that we can
# continue with other files
except Error as err:
errors.extend(err.args[0])
except OSError as why:
errors.append((srcname, dstname, str(why)))
try:
copystat(src, dst)
except OSError as why:
# Copying file access times may fail on Windows
if getattr(why, 'winerror', None) is None:
errors.append((src, dst, str(why)))
if errors:
> raise Error(errors)
E shutil.Error: [('/tmp/guix-build-python-docrepr-0.2.0.drv-0/tmpev83mper/_static/plot_directive.css', '/tmp/guix-build-python-docrepr-0.2.0.drv-0/docrepr-None/tmpbm5c0r1l/_static/plot_directive.css', "[Errno 13] Permission denied: '/tmp/guix-build-python-docrepr-0.2.0.drv-0/docrepr-None/tmpbm5c0r1l/_static/plot_directive.css'")]
/gnu/store/b6j1qw1a5rkbfvcy7lc9fm95abbzpa4x-python-3.9.9/lib/python3.9/shutil.py:522: Error
I've fixed it by changing except TypeError
to except (TypeError, shutil.Error)
(which means the copying falls back to the manual routine below the shutil.copytree
failed call). A better approach could be to ensure all files are writable after creating the Sphinx object.
Any chance this package will be updated any time soon? We want to use it as a dependency in our package, but it seems to need a bit of updating, couldn't get it to work.
I was tracking down the history of the sphinxify licensing to clarify how this sphinxify module is BSD while the core Sage sphinxify is currently GPLv2+. Here is what I found:
I don't think there is anything to do and this ticket can be closed immediately, but I just wanted to document the history to prevent confusion.
Personal side note: If I recall correctly, I remember having a conversation about the BSD license on the original sphinxify module at SageDays 19 with Tim and a few others, where we decided it would be fine to have a BSD-licensed file in the Sage Notebook codebase, but that we'd just make sure that file explicitly stated it had the BSD license
@martinRenou mentioned he was working on this, so opening this to track progress and milestone it appropriately. We could push it to v0.2.1, but I'd be way more comfortable maintaining this having tests that actually check whether the rendered content matches what we expect it to be. In particular, checking file content and setting up a downloadable archive from the GH Actions job (if not a full rendered preview on Netlify or another service) would be very nice to have. I can also hack something together, but since @martinRenou mentioned he'd be working on it and will likely get to it sooner, I'll leave this to him unless otherwise requested.
Reported by @goanpeca: We shouldn't show a tab that has empty content, i.e. the No doc available message as the first one.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.