ooreilly/mydocstring

A tool for extracting and converting Google-style docstrings to plain-text, Markdown, and JSON

Hi there 👋

mydocstring's Issues

supporting classes

hi there,

I'm really enjoying mydocstring.

I was wondering if there is any plan to support documenting Classes?

If not, you demonstrate how to get the classes data in one of the Issues here: classes example

i was wondering if there is a similar method that also applies the markdown formatting?

many thanks

examples in the repo not working after initial install

Hi there,

i installed mydocstring as described in the readme:

pip install mydocstring

when i run the example.sh script from within the appropriate conda env i get the following error:

File "C:\ProgramData\Miniconda3\envs\mf_main\Scripts\mydocstring.exe\__main__.py", line 7, in <module> File "c:\programdata\miniconda3\envs\mf_main\lib\site-packages\mydocstring\docstring.py", line 39, in main cmd = command.Command(options) File "c:\programdata\miniconda3\envs\mf_main\lib\site-packages\mydocstring\command.py", line 53, in __init__ returns=self.docstring['return_type']) KeyError: 'return_type' Traceback (most recent call last): File "c:\programdata\miniconda3\envs\mf_main\lib\runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "c:\programdata\miniconda3\envs\mf_main\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\ProgramData\Miniconda3\envs\mf_main\Scripts\mydocstring.exe\__main__.py", line 7, in <module> File "c:\programdata\miniconda3\envs\mf_main\lib\site-packages\mydocstring\docstring.py", line 39, in main cmd = command.Command(options) File "c:\programdata\miniconda3\envs\mf_main\lib\site-packages\mydocstring\command.py", line 53, in __init__ returns=self.docstring['return_type']) KeyError: 'return_type' bash: pause: command not found Microsoft Windows [Version 10.0.18362.476] (c) 2019 Microsoft Corporation. All rights reserved.

any suggestions would be most welcome?

class method not parsed if `self` is on a newline after opening bracket

Consider the class

bash-3.2$ cat tst.py
class TestClass:

    def __init__(
            self,
            arg1
    ):
        """
        Args:
            arg1 : an arg
        """
        pass

Parsing this fails because self is on a newline.

Proposed fix to PyExtract::extract_method pattern to change (self[\w\W] to (\s*self[\w\W]

Not working

Calling

mydocstring Hook.py . --markdown

Where Hook.py

class Hook:
    def before_build_model(self, estimator):
        """
        This function is called before calling the `model_fn` function

        Args:
            estimator: A reference to the estimator

        Returns:

        """
        pass

    def after_build_model(self, estimator):
        """
        This function is called after calling the `model_fn` function
        Args:
            estimator: A reference to the estimator

        Returns:

        """
        pass

    def before_run_epoch(self, estimator, epoch, data, batch_n, tot_res):
        """
        This function is called before run a new epoch.
        Args:
            estimator: A reference to the estimator
            epoch: The index of the current epoch
            data: The current data passed to the epoch
            batch_n: The number of batches
            tot_res: An object with keys the current mode operations and values and array of results

        Returns:

        """
        pass

    def after_run_epoch(self, estimator, epoch, data, batch_n, tot_res):
        """
        This function is called after run a new epoch. It can be implemented in order to create
        custom hooks. For example, `tot_res` is a dictionary that contains all the results for each operation for each batch.
        For example we may want to log the epoch history during train.
        ```
        def after_run_epoch(self, estimator, epoch, data, batch_n, tot_res):
            if estimator.mode == Mode.TRAIN:
                # you can check the estimator mode
                print(tot_res) # { 'loss' : [0.8, 0.7, 0. 6]...}
        ```

        Args:
            estimator: A reference to the estimator
            epoch: The index of the current epoch
            data: The current data passed to the epoch
            batch_n: The number of batches
            tot_res: An object with keys the current mode operations and values and array of results

        Returns:

        """
        pass

    def before_run_batch(self, estimator, res, i):
        """
        This function is called before run a new batch.

        Args:
            estimator: A reference to the estimator
            res: An object with keys the current mode operations and the single result for each one of them
            i:  The batch index

        Returns:

        """
        pass

    def after_run_batch(self, estimator, res, i, tot_res):
        """
        This function is called after run a new batch.

        Args:
            estimator: A reference to the estimator
            res: An object with keys the current mode operations and the single result for each one of them
            i:  The batch index

        Returns:

        """
        pass

Generate this error:

  File "/usr/local/bin/mydocstring", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/mydocstring/docstring.py", line 39, in main
    cmd = command.Command(options)
  File "/usr/local/lib/python3.6/site-packages/mydocstring/command.py", line 27, in __init__
    self.docstring = extract.extract(self.filename, self.name)
  File "/usr/local/lib/python3.6/site-packages/mydocstring/extract.py", line 202, in extract
    return extractor.extract(query)
  File "/usr/local/lib/python3.6/site-packages/mydocstring/extract.py", line 63, in extract
    return types[self.dtype]()
  File "/usr/local/lib/python3.6/site-packages/mydocstring/extract.py", line 179, in extract_module
    return self.find(pattern)
  File "/usr/local/lib/python3.6/site-packages/mydocstring/extract.py", line 130, in find
    raise NameError(r'Unable to extract docstring for `%s`' % self.query)
NameError: Unable to extract docstring for ``

Cheers

Code in docstring does not parse properly

Code

In the following demonstration I have tried parsing a simple docstring.

from mydocstring.parse import GoogleDocString

s = """
Hello

Examples:
    ```python
    a = {
        'a': 1,
        'b': 1,
        'c': 1,
    }
    ```
"""

parser = GoogleDocString(s)
p = parser.parse()

for section in p:
    for k,v in section.items():
        print(f'{k}=>{v}')

Output

header=>
text=>
Hello

args=>[]
header=>Examples
text=>\```python
a = {
}
\```

args=>[{'field': '', 'signature': '', 'description': "1,\n    'b': 1,\n    'c': 1,"}]

Here as you can see the dictionary in the code is being parsed as arguments. This will break the rendering as well. Do we have a workaround for this? 😄

Copyright statements missing in headers

@ooreilly , another "small" thing I noticed: it seems that you don't have copyright/license statements in the headers of your files!

I would like to use your excellent parser.py to build the documentation of a project, and would like to copy it in our repo. Would that be ok for you/ would you add your copyright statement before?

Thanks!

Issue when parsing some test google docstrings

@ooreilly thanks for sharing this excellent library!

I just wanted to mention that I found some issues when parsing some test Google docstrings.
For example, for this function

def module_level_function(param1, param2=None, *args, **kwargs):
    """This is an example of a module level function.

    Function parameters should be documented in the ``Args`` section. The name
    of each parameter is required. The type and description of each parameter
    is optional, but should be included if not obvious.

    If \*args or \*\*kwargs are accepted,
    they should be listed as ``*args`` and ``**kwargs``.

    The format for a parameter is::

        name (type): description
            The description may span multiple lines. Following
            lines should be indented. The "(type)" is optional.

            Multiple paragraphs are supported in parameter
            descriptions.

    Args:
        param1 (int): The first parameter.
        param2 (:obj:`str`, optional): The second parameter. Defaults to None.
            Second line of description should be indented.
        *args: Variable length argument list.
        **kwargs: Arbitrary keyword arguments.

    Returns:
        bool: True if successful, False otherwise.

        The return type is optional and may be specified at the beginning of
        the ``Returns`` section followed by a colon.

        The ``Returns`` section may span multiple lines and paragraphs.
        Following lines should be indented to match the first line.

        The ``Returns`` section supports any reStructuredText formatting,
        including literal blocks::

            {
                'param1': param1,
                'param2': param2
            }

    Raises:
        AttributeError: The ``Raises`` section is a list of all exceptions
            that are relevant to the interface.
        ValueError: If `param2` is equal to `param1`.

    """

the parser seems to fail when dealing with

    The format for a parameter is::

        name (type): description
            The description may span multiple lines. Following
            lines should be indented. The "(type)" is optional.

            Multiple paragraphs are supported in parameter
            descriptions.

and

       The ``Returns`` section supports any reStructuredText formatting,
        including literal blocks::

            {
                'param1': param1,
                'param2': param2
            }

this is certainly a corner case for users, but might be worth looking into?

Deprecation warnings due to regular expressions

For Python version 3.6 and up, the test emits a couple of deprecation warnings. See the Actions tab.

=============================== warnings summary ===============================
extract.py:414
  /home/runner/work/mydocstring/mydocstring/mydocstring/extract.py:414: DeprecationWarning: invalid escape sequence \s
    'token_split': '^\s*\d+\.'

parse.py:562
  /home/runner/work/mydocstring/mydocstring/mydocstring/parse.py:562: DeprecationWarning: invalid escape sequence \(
    match = re.findall('\(([\w\W]*?)\)\s*(?:->\s*(\w+))?', args)

parse.py:698
  /home/runner/work/mydocstring/mydocstring/mydocstring/parse.py:698: DeprecationWarning: invalid escape sequence \s
    match = re.findall('(\s*)(%s[\w\W]+)' % keyword,

-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================== 30 passed, 3 warnings in 0.14s ========================