Docstrings

[kaitai-io]

Docstrings

[GreyCat]

Results checklist:

• C++
• C#
• Java
• JavaScript
• Perl
• PHP
• Python
• Ruby

[GreyCat]

Current status:

Perl

I'm totally unable to grasp what particular combination of =item, =cut, =over, =head1, etc, is required to generate documentation for a class. Actually, I don't even get if it's a good idea to generate POD interleaved with code (as documentation suggests), or just POD section at the end of file (as majority of modules do). @sergeyzelenyuk, could you lend a hand here?

Python

I've stumbled upon generation of docstrings for plain attributes (i.e. not declared anywhere). This SO question suggests that stuff's still bad with that sort of documentation in Python: one can either place it into class-related docstring (which is unparseable and can't be used in any IDE), or one has to generate weird constructs with @property for that. @dgladkov, any ideas on what shall be used here?

[GreyCat]

I've added doc attribute to type specification as well. Now classes can be annotated too, including top-level class, which probably should be most important piece of documentation.

[GreyCat]

Python docstrings are more or less there (probably "less", as there is no attribute docstrings generated, as Python seem to lack attribute documentation).

[arekbulski]

Python properties do have docstrings, read this:
https://stackoverflow.com/questions/16025462/what-is-the-right-way-to-put-a-docstring-on-python-property

I unchecked Python on the list for you.

[GreyCat]

The problem is that we don't generate these trivial getters and we don't really hide actual instance members by using something like that:

@property
def x(self):
    """Get x"""
    return getattr(self, '_x', 42)

So, shall we start generating them and rename actual members to something like _m_foo? I'd be worried about the performance at the very least.

[arekbulski]

Can you show me what is actually being generated by current implementation?

[GreyCat]

For example, https://github.com/kaitai-io/ci_targets/blob/master/compiled/python/docstrings.py

[arekbulski]

Hmm arent we talking about (already) solved problem?

[GreyCat]

I'm not sure what you're implying. The fact that Python now lacks docstrings for properties that are not implemented using @property getter method still stands. These are usually encountered as regular seq attributes.

For example, compare these attributes in Ruby, and note that Python counterpart lacks such lines at all. This line in _read() is sufficient to create property and that's all.

[arekbulski]

I am too sleepy to see straight, thanks for putting up with me. So in the example "self.one" is a field (not a property), and fields do not have docstrings? I think you mixed the terminology? Fields (class attributes) and properties are 2 different terms. In python example you linked, properies are documented, fields are not. There is no such thing as "properties without @Property" as far as terminology goes, but I got what you mean. So you suggest to use read-only properties instead of attributes? I'd say yes, there is no cost/drawback in using properties instead. Except slight more generated code.

[GreyCat]

Are you sure? It seems to me that performance drop significant, i.e.

import timeit

class FooWithField:
    def __init__(self):
        self.pew = 42

class FooWithPropertyRaw:
    def __init__(self):
        self._m_pew = 42

    @property
    def pew(self):
        """Get pew"""
        return self._m_pew

class FooWithPropertyGetAttr:
    def __init__(self):
        self._m_pew = 42

    @property
    def pew(self):
        """Get pew"""
        return getattr(self, '_m_pew')

foo1 = FooWithField()
foo2 = FooWithPropertyRaw()
foo3 = FooWithPropertyGetAttr()

print(timeit.timeit(lambda: foo1.pew)) # => 0.092588186264
print(timeit.timeit(lambda: foo2.pew)) # => 0.212033987045
print(timeit.timeit(lambda: foo3.pew)) # => 0.277426958084

So, just adding @property + method with simple return is ~2.3x more expensive than just a field. Doing "proper" attribute getting using getattr is ~3x more expensive.

[arekbulski]

Ah right, I remember using fields and properties years back when I was a C# fan. Then it had same performance, and I do remember reading about it. I didnt quite check for Python. Good thinking there.

You are misinterpreting the numbers. Its not 2.3x times slower, its +0.12 microsecond slower. This amount adds to parsing time, and user handling of that data, both are much heavier than that. A drop in the bucket, so to speak. I think docs might be worth an extra microsecond, dont you think?

There is nothing proper in third example. I would not recommend it over second out of readability alone. Discount it.

[KOLANICH]

One more drawback/advance (depends on application) is laziness. For debugging it is certainly a drawback since when debugging we need to find all the issues like incorrect enum values or invalid characters/wrong encodings. So we run our format against a set of files and look for exceprions.

[arekbulski]

Correct "exceprions". I dont understand what properties have to do with it. Properties are not lazy, nor make parsing lazy by any meaning. Can you explain by example?

[KOLANICH]

I have confused them with instances which are currently generated via properties.