support reading docstrings from conftest.py

[satelliteqe]

support reading docstrings from conftest.py

[elyezer]

Is the idea here of having project globals?

We already have context for module, classes and test methods. Adding support for conftest will open for requiring adding package level contexts. We can have that, but the focus right now is to deliver xml-test-case command. That said, if you want to contribute with a patch that would be awesome.

All the test collection and parsing is provided by Testimony and this function (https://github.com/SatelliteQE/testimony/blob/master/testimony/__init__.py#L58-L60) is the one that adds rules to select a test module. In addition to that will be required to update how the contexts are handled so we can add more levels.

If we have conftest and package context which one should be considered as more generic? I think the answer would be conftest, which will follow this hierarchy: conftest, then package, then module, then class and finally test method.

[spoore1]

Eleyezer,

The idea is kinda of like project globals I guess. I have started adding certain labels to the docstrings for my modules if they would be identical for all methods in the class(es) in the file. Just a way to simplify what goes into each function/method docstring. Effectively treating them like module level globals. Thus, I guess this would be a package (or directory) level global.

Sure, I'll take a look at writing a patch for this request. Should I open a separate issue in testimony?

As for order of priority for the context levels, I would have thought package then conftest. TBH my initial thought was super simple just add conftest.py support like any other module. Then just require defining in only one place or you could get strange results. Can't you have multiple directories with different conftests in a single package?

package/__init.py
package/dir1/{conftest.py, test_feature1.py}
package/dir2/{conftest.py, test_feature2.py}

Also, you're thinking the package level contexts should be included in init.py right?

Thanks,
Scott

[elyezer]

I am not sure if multiple conftest.py can exist inside the same package. If we decide on supporting multiple conftest we need to improve even more the contexts so conftest higher in the directory tree will be used if in a subdirectory does not have one, else the subdirectory will override the more generic one. Thinking a little bit more about that, I think we should only support package level context and don't support conftest at all. Package context will help on have globals and that is more suited for every test runner.

Yes, package level should analyze the __init__.py docstring.

[spoore1]

I guess this depends on test coding style. You can have multiple conftest.py files in a directory tree that isn't necessarily a python package. Each conftest.py applies to the tests in its own directory. For example, my team simply installs our tests from git and executes py.test command. We have test "suites" as separate directories with separate conftest.py files.

As for order of precedence, the conftest in the subdirectory "should" override the one in a parent directory I think. The data closest to the test should be the most "authoritative", right? At least in how we're talking about processing docstrings?

That is still a good point about having to maintain parent directory conftest data to fully support that. Maybe just __init__.py is right for the start. I think that's definitely the easier path. There is no sub-directory/child-node type inheritance expected there right? Even if groups aren't using packages, they could add __init__.py files to directories to supply directory level defaults for their test modules.

My first draft was for conftest but, I'll modify for __init__ and go from there. If we decide to include conftest later, it can be added separately, I think.

[elyezer]

Either way is fine to me, subpackages inherit tokens from parent packages or only the modules within a package inherit from its __init__.py file.

About conftest, yes I agree that having package context will open even more the possibilities, then we can check if supporting conftest will continue being required.

[spoore1]

I didn't realize this one was still open. I think it can be closed now right? We just support the init.py docstrings instead of the conftest ones I originally suggested right?

[elyezer]

@spoore1 yeah we can close this. Thank you!