pytest-dev/pluggy

A minimalist production ready plugin system

pluggy's Issues

regression for firstresult hook without implementation

devpi/devpi#458

is triggered by https://github.com/pytest-dev/pluggy/pull/58/files#diff-85f4be4e30ec3b4f3d53843b24d72141R109

@tgoodlet i beleive a test is needed for those edge behaviours

GitHub release of 0.5.2

Hi, would it be possible to make a release of 0.5.2 here on GitHub?

Project name based collection triggers regression

I'll put up a test which demonstrates this shortly.
We also don't seem to have any tests which don't also include adding a hookspec the lack of which is the source of the problem.

Turn _MultiCall into a function

Now that #23 has landed it seems fit that we evaluate the _MultiCall type.

Afaict this class contains a single method execute() other then __init__ (and __repr__ but who uses it?) and that method was only necessary for the original recursive implementation.

I'm wondering can we simplify the whole thing into a simple function?

Reasons for:

• _MultiCall keeps no state since it's execute method is always called directly after instantiation
• _MultiCall instances are never reused
• having a function is less code, faster and arguably easier to cythonize

Resolve E741 issues

The latest release of flake8 adds support for E741 ambiguous variable name type errors, resulting in a broken check tox target.

PendingDeprecationWarning: generator 'pytest_pycollect_makeitem' raised StopIteration

Hi,

I'm getting lots of these warnings when running some of the pywikibot tests using pytest and python 3.5.

WARNING: C:\Users\a\AppData\Local\Programs\Python\Python35\lib\site-packages\_pytest\vendored_packages\pluggy.py:250: PendingDeprecationWarning: generator 'pytest_pycollect_makeitem' raised StopIteration
  wrap_controller.send(call_outcome)

See: https://travis-ci.org/wikimedia/pywikibot-core/jobs/167265098#L493

Document the `pluggy._Result` API

Currently pluggy._CallOutcome is implied to be part of the internal api by the leading _.

In practice it is used by hookwrappers.

An example:

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    rep = outcome.get_result() 
    ...

The outcome is accessed directly by the user.
Therefore I propose we expose it correct in plugg.__all__ and expose it in the api reference.

trigger a deprecationwarning for Result.excinfo

pluggy/pluggy/callers.py

Lines 32 to 34 in 1037dc9

	@property
	def excinfo(self):
	return self._excinfo

we kept the code for pytest, but we should at least deprecate it and remove the use in future

tags missing

pluggy breaks on bad attributes in plugins

see pytest-dev/pytest#1073

firstresult hook wrapper return list

As per pytest-dev/pytest#2730 0.5.0 broke firstresult hook wrappers such that a single result is not returned from _Result.get_result().

We need a test and fix for this asap!

adapting towncrier and release automation

we should take a look for handling this here as well

the KeyError on forgotten hook arguments is non-descriptive

instead there should be a clear error on what arguments are missing

Error: warnings has no attribute DeprecationWarning

While testing pluggy off the branch from #72, I came across this error:

        if '__multicall__' in hookimpl.argnames:
            warnings.warn(
                "Support for __multicall__ is now deprecated and will be"
                "removed in an upcoming release.",
>               warnings.DeprecationWarning
            )
E           AttributeError: module 'warnings' has no attribute 'DeprecationWarning'

pluggy/pluggy/__init__.py

Lines 656 to 661 in 557cd3f

	if '__multicall__' in hookimpl.argnames:
	warnings.warn(
	"Support for __multicall__ is now deprecated and will be"
	"removed in an upcoming release.",
	warnings.DeprecationWarning
	)

And indeed warnings module doesn't have a DeprecationWarning.

We should add a test to catch this warnings (and any others we might have).

Iterate plugin results

Currently we have the firstresult option. An interesting addition would be a way to iterate over the plugins. The big difference to the default of returning all results, is that the consumer can stop iterating at any point. This is useful for things where plugins might cause user interaction. One example is a password hook that tries to get a password from a keychain and if that doesn't work the next plugin is tried, which asks the user for the password in a prompt. With firstresult we don't get the fallback and with the current default of returning all results, the user would always be prompted, even if the password from the keychain would work.

Port to modern inspect

For func signature inspection in py3 we're supposed to use the new inspect.signature().
An example of how to provide dual support for py2/3 can be found here.

Use relative imports for test suite

As per discussion in pytest-dev/pytest#2716 I want to see if we can avoid self-hosted testing of pluggy by using relative imports in the test suite. I think the only hurdle will be making tox work alongside. TravisCI shouldn't have any problems afaik.

Use setuptools_scm for versioning

So we don't have to manually track versions.

Make sure to use method use by pytest of generating a _version.py file and importing it to obtain the version; the alternate method of using pkg_resources does not work from within frozen executables.

Marking hookspec as requiring at least one implementation

For some hooks it would be great if I could make an implementation for it mandatory, so that it doesn't just do nothing but raises an Error when called.
What are your thoughts on that?

Strict mode(s) - change validation rules dynamically?

As per pytest-dev/pytest#1821 @RonnyPfannschmidt has mentioned it's worth discussing if pluggy.PluginManager should have a strict mode where every single plugin which is loaded is verified against a spec instead of only performing this validation each time PluginManager.check_pending() is called.

In pytest check_pending() is only done once just prior to the pytest_cmdline_main(config=config) call meaning unverified hooks can be loaded any time afterwards.

This issue is to open discussion and brainstorming.

Add some benchmark tests

There's been some discussion (#15, #23) surrounding performance mostly as it pertains to the _Multicall.execute() loop.

I propose we add some tests which use pytest-benchmark or something similar to audit changes we make to critical code paths. This encourages being objective about future changes.

Can't load instances which define hook specs

You currently can't instantiate a class which defines hook specs prior to loading

import pluggy

hookspec = pluggy.HookspecMarker('example')

class SpecSpace(object):
    @hookspec
    def myhook(self, stuff):
        """My custom hook spec.
        """

pluggy.PluginManager('example').add_hookspecs(SpecSpace())

because pluggy.varnames() doesn't successfully detect and remove the self name.

I realize this is an odd usage (and implied to be illegal based on the argname to add_hookspecs()) but it is conceivable. A contrived example might be where a user wishes to define the spec and the default hook implementation using the same function:

import pluggy

hookspec = pluggy.HookspecMarker('example')
hookimpl = pluggy.HookimplMarker('example')

class SpecSpace(object):
    def __init__(self, arg):
        self.arg = arg 

    @hookimpl
    @hookspec
    def hook1(self, stuff):
        """My custom hook spec.
        """
        if self.arg = 'doggy' and stuff == 'blah':
             # do this one thing
        else:
            # do this other thing

ss = SpecSpace('blah')
pm = pluggy.PluginManager('example')
pm.add_hookspecs(ss)
pm.register(ss)

I guess it's kind of silly...

Either way the implementation of pluggy.varnames is quite outdated and should be changed to more properly levarage the inspect module. If we prefer to not ever allow instances then a more explicit error should be thrown in PluginManager.add_hookspecs()

RTD build keeps failing

See the latest here: https://readthedocs.org/projects/pluggy/builds/

Looks like the culprit is:

 File "conf.py", line 23, in <module>
    dist = pkg_resources.get_distribution('pluggy')

So I guess we aren't installing pluggy prior to docs building? That's a RTD config thing right?

I'd be happy to help with this but I don't have access :(

0.5.0 release!

Once pytest-dev/pytest#2719 get's in we should do a new release and get the fast(er) version of pluggy's core out in the wild as well as the fixes since 0.4.0. I wasn't sure if it should be a patch or a full minor.

@hpk42 @nicoddemus @RonnyPfannschmidt I'd be glad to take on this reponsibility from now on if you all are ok with it. I will need to have access to the pypi account however.

Let me know what you guys think!

Drop py2.6 support

As the drop from pytest via pytest-dev/pytest#2812 we can now remove lingering code that is only in place to support old python.

See pytest-dev/pytest#2822 for comparable CI related changes that are likely appropriate.

Some stuff I can think of off hand includes:

• set logic for checking missing hookcall args can be modernized to use new {,} declarations and dict.viewkeys().
• str.format() method now supports automatic numbering in py2.7+
• leveraging collections.OrderedDict to simplify some of the internal data structures for managing hooks

support defaults to enable deprecation

this is to help pytest-dev/pytest#1372

the idea is, that if a hook is invoked with a missing, but specced argument, then the default is used instead

so if a hook caller would call hook.pytest_deselected(items=...)' and the hook was defined withdef pytest_deselected(items, reason='No Reason Given)`
then the receivers would receive the defined default reason

in order to support deprecation of such usage we could later devise a mark for deprecated defaults

0.5.2 release

I just got an email that 0.5.1 is being shipped on NetBSD.
We need to get 0.5.2 out soon!

Things blocking:

• #79
• pytest-dev/pytest#2737

I think it should be pretty quick to get these complete :)

Document how hook call order is affected by plugin registration

As discussed in #62, we should explicitly document how hook call order is done regarding how plugins are registered.

From the top of my head, hooks are called in this order:

1. setuptools plugins (in arbitrary order, except those marked with try_first and try_last);
2. core plugins;

INTERNALERROR: KeyError: 'startdir'

Hi, I'm using py.test and django. Py.test seems to use pluggy behind the scenes.

Can anyone shed some light on this error?

$ py.test
Test session starts (platform: darwin, Python 3.5.2, pytest 3.0.2, pytest-sugar 0.7.1)
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/main.py", line 94, in wrap_session
INTERNALERROR>     config.hook.pytest_sessionstart(session=session)
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/pytest_sugar.py", line 217, in pytest_sessionstart
INTERNALERROR>     lines = self.config.hook.pytest_report_header(config=self.config)
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 593, in execute
INTERNALERROR>     args = [all_kwargs[argname] for argname in hook_impl.argnames]
INTERNALERROR>   File "/Users/jon/.virtualenvs/webapp/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 593, in <listcomp>
INTERNALERROR>     args = [all_kwargs[argname] for argname in hook_impl.argnames]
INTERNALERROR> KeyError: 'startdir'

Duplicate firstresult tests?

I noticed that the tests here and here appear to be essentially duplicates under different names?
Is there any reason I should not remove one of them?

Move to new style classes

As per pytest-dev/pytest#2147 we should also move to new-style classes throughout!

support the creation fo an actual plugin dependency tree and use its root for consideration/order

in pytest when we use conftests with pytest_plugins,
those specifications are not disabled when using other paths

as such, as soon as trees of conftests that use pytest_plugins, we get a inconsistent dependency state of plugins

additionally some plugins dod epend on other plugins either optionally or directly

experiments and a larger discussion are needed to give this issue the required detail for concrete action

Remove __multicall__ support

Now that #23 has been resolved via #58 we can move towards removing the old _LegacyMultiCall and remaining restrictions surrounding it's support in the core (mostly the _HookCaller afaict).

I was hoping to remove support in a pre-1.0 version and coordinating with pytest appropriately.

Reorganize test classes into modules

The testing/test_pluggy.py module is not only a bit long but also overwhelming.

I propose we break up the current organization of as follows:
TestPluginManager -> test_plugin_manager.py
TestAddMethodOrdering -> test_add_method_ordering.py
TestMultiCall -> test_multicall.py
TestHookRelay -> test_hook_relay.py
TestTracer -> test_tracer.py

I personally think it just lends to a better immediate understanding of what tests are available/need to be written around each internal component of the pluggy architecture.

Temporary registration via context manager

This is a proposal for a very simple new feature. I actually implemented it in a side project pysipp and was thinking it was something we could add to the PluginManger.

The jist of it is adding a way to temporarily register plugins using a context manager api:

with pluginmanager.register_all([pluginmod1, pluginmod2]) as pm:
    #... do stuff that requires above plugins to be active...

# continue with stuff that doesn't require above plugins to be registered.

So it would basically just be adding PluginManager.register_all() (or whatever name you guys think is best) method that is a simple wrapper around register/unregister.

Add a logo

I think I found a good one using a plug symbol and we should be able to insert it using the built-in sidebarlogo in the layout.html sphinx-template.

Stick docs link in the Github project title

@nicoddemus @hpk42 even as a collaborator I don't seem to be able to modify the title to stick in the docs link :(

I was just thinking it might be a good way to get newcomers more quickly immersed by tempting them with a docs link at the inset ;)

Anyway let me know what you guys think!

raise a more comprehensible errro when hooks are invoked wrong

i was confronted with code that invoked a hook with positional arguments instead of kwargs,
the default python error in that case left me completely puzzled for a while

a more comprehensible error would be really nice to have

better error when a attribute named like a hook is an module

pytest-dev/pytest#1686

trouble happened when i tried to use a package as a plugin while it had a module named pytest_store

Have a unified way of showing info about installed plugins

When implementing tox-dev/tox#544 a discussion arose about what it should look like when you output version information about the tool.

tox-dev/tox#628 (comment)

So I leave that suggestion here, that pluggy could provide a higher level API to output version information and maybe with different levels of exactness/verbosity (e.g. full paths, vs. just names and versions).

stop using the stylechecking pytest plugins

linting is not testing, and we should stop pretending and also add linting/metrics to the GitHub integrations

Importing pytest_* modules should not trigger PluginValidationError

In pytest-dev/pytest-bdd#224 @soulrebel had an issue when importing pytest_bdd in his conftest.py file:

_pytest.vendored_packages.pluggy.PluginValidationError: unknown hook 'pytest_bdd' in plugin <module 'project_qa.tests.functional.conftest' (<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7f035b2d4f28>)>

This seems odd - importing a pytest_* module should work normally, no? Maybe pluggy should at least check that unknown hooks aren't module objects?

high level overview

The description "as used by py.test" is not very helpful. To be able to compare with other known plugin systems like Trac, SCons , Roundup or Spyder I need to answer the questions:

1. how plugin discovery is made
2. how plugin loading happens
3. when plugin initialization occurs

Execute some build environments using latest pytest and/or master

As discussed in pytest-dev/pytest#2730

Fix up the readme

As per @nicoddemus's suggestion It'd be good to have an introductory example and a little more excitement there ;)

Better handling of VersionConflict

I am seeing the following error (end of traceback), when running py.test:

  File "…/pytest/_pytest/config.py", line 917, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "…/pluggy/pluggy.py", line 501, in load_setuptools_entrypoints
    plugin = ep.load()
  File "…/pyenv/velodrome/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2354, in load
    self.require(*args, **kwargs)
  File "…/pyenv/velodrome/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2371, in require
    items = working_set.resolve(reqs, env, installer)
  File "…/pyenv/velodrome/lib/python3.4/site-packages/pkg_resources/__init__.py", line 844, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (coverage 4.0b3 (…/pyenv/velodrome/lib/python3.4/site-packages), Requirement.parse('coverage<4.0,>=3.7.1'))

The exception does not provide a hint about the problematic plugin.

Deprecate __multicall__ support

As per the comment in pluggy._MultiCall we should be able to drop the line here. Grepping the pytest sources reveals that only a few spots still reference it.

This change also allows us to remove the recursion induced by _wrapped_call() getting passed _MultiCall.execute. This means we can remove _wrapped_call and simply loop through hookwrapper pre and post yield calls at the beginning and end of execute(). I'd actually be interested to see if contextlib.contextmanager wrapping could be used directly for this.

Turn warnings into errors in the test suite

Currently the test suite is issuing this warnings:

=============================== warnings summary ===============================
testing/test_method_ordering.py::test_add_tracefuncs[spec-is-class]
  /home/travis/build/nicoddemus/pluggy/.tox/py34/lib/python3.4/site-packages/pluggy/__init__.py:678: UserWarning: Argument(s) ('arg',) which are declared in the hookspec can not be found in this hook call
    .format(tuple(notincall))
testing/test_method_ordering.py::test_add_tracefuncs[spec-is-instance]
  /home/travis/build/nicoddemus/pluggy/.tox/py34/lib/python3.4/site-packages/pluggy/__init__.py:678: UserWarning: Argument(s) ('arg',) which are declared in the hookspec can not be found in this hook call
    .format(tuple(notincall))
testing/test_pluginmanager.py::test_call_with_too_few_args
  /home/travis/build/nicoddemus/pluggy/.tox/py34/lib/python3.4/site-packages/pluggy/__init__.py:678: UserWarning: Argument(s) ('arg',) which are declared in the hookspec can not be found in this hook call
    .format(tuple(notincall))
-- Docs: http://doc.pytest.org/en/latest/warnings.html

Turning them into errors or at least mark them as "known" is a good way to catch incompatibilities early.

Can't pypi-install pluggy - PYPI returned unsafe url: ''

Trying to pypi-install (wrapper that generates a debian package from a pip install) I get the following error:

# pypi-install --verbose 5 pluggy
downloading to /tmp/tmpoqv1mk
querying PyPI (https://pypi.python.org/pypi) for package name "pluggy"
found default release: 0.4.0
Traceback (most recent call last):
  File "/usr/bin/pypi-install", line 67, in <module>
    main()
  File "/usr/bin/pypi-install", line 48, in main
    allow_unsafe_download=options.allow_unsafe_download)
  File "/usr/lib/python2.7/dist-packages/stdeb/downloader.py", line 95, in get_source_tarball
    raise ValueError('PYPI returned unsafe url: %r' % download_url)
ValueError: PYPI returned unsafe url: ''

Is there something missing in setup.py ? in the description on pypi ?

warn/fail on plugin instances whose methods have a hook argument instead of self