pyinvoke/invocations

Reusable Invoke tasks

What is this?

Invocations is a collection of reusable Invoke tasks, task collections and helper functions. Originally sourced from the Invoke project's own project-management tasks file, they are now highly configurable and used across a number of projects, with the intent to become a clearinghouse for implementing common best practices.

Currently implemented topics include (but are not limited to):

• management of Sphinx documentation trees
• Python project release lifecycles
• dependency vendoring
• running test suites (unit, integration, coverage-oriented, etc)
• console utilities such as confirmation prompts

and more.

Roadmap

While Invocations has been released with a major version number to signal adherence to semantic versioning, it's somewhat early in development and has not fully achieved its design vision yet.

We expect it to gain maturity in tandem with the adoption and development of Invoke post-1.x. It's also highly likely that Invocations will see a few major releases as its API (and those of its sister library, patchwork) matures.

For a high level roadmap re: when Invocations will get significant updates, see the maintainer's roadmap page.

Development

This project uses the same dev methodology as Invoke proper - please see its development page here.

invocations's Issues

blessings is dead

The blessings project seems to be dead:

• erikrose/blessings#148
• erikrose/blessings#160

Time to switch to something else? For example blessed?

use unittest.mock instead of mock

Could you please consider to use built-in unittest.mock when mock is not available?

Background:
https://trello.com/c/S6eADbii/64-remove-python-mock
https://fedoraproject.org/wiki/Changes/DeprecatePythonMock

Docs || Examples

Where can i find some examples or docs?

Loosen pin on tabulate package

The tabulate package is pinned to ==0.7.5 which is a version released back in 2015.

Is it possible to change the restraint to >=0.7.5? I'm using the databricks-cli package which has a pin that requires >=0.7.7.

As far as I can tell, there is no breaking changes since 0.7.5.

Thanks!

usage example

Is invocations meant to be installed with pip or it is a collection of scripts one should copy and paste in own invoke.py?

What is the the simplest usage of invocations? Could be very helpful in the docs (addressed before in #9)

Many thanks for clarification.

tasks.py: watch command uses "sys" module without import

By inspection of the main "tasks.py".

OOPS, wrong repo
=> SHOULD-BE: invoke (not: invocations).

autodoc of invoke fails with newer versions of sphinx

My builds have started to fail:

Running Sphinx v2.4.4

Exception occurred:
  File "/Users/sambriggs/.local/share/virtualenvs/cookiecutter-pypackage-oT8-zJK2/lib/python3.7/site-packages/sphinx/deprecation.py", line 49, in __getattr__
    return getattr(self._module, name)
AttributeError: module 'sphinx.ext.autodoc' has no attribute 'add_documenter'
The full traceback has been saved in /var/folders/vz/gcfll7gd04z1sbzr9kv2phn80000gn/T/sphinx-err-qijxis7s.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

The traceback looks like:

# Sphinx version: 2.4.4
# Python version: 3.7.6 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 2.11.1
# Last messages:

# Loaded extensions:
Traceback (most recent call last):
  File "/Users/sambriggs/.local/share/virtualenvs/cookiecutter-pypackage-oT8-zJK2/lib/python3.7/site-packages/sphinx/cmd/build.py", line 275, in build_main
    args.tags, args.verbosity, args.jobs, args.keep_going)
  File "/Users/sambriggs/.local/share/virtualenvs/cookiecutter-pypackage-oT8-zJK2/lib/python3.7/site-packages/sphinx/application.py", line 245, in __init__
    self.setup_extension(extension)
  File "/Users/sambriggs/.local/share/virtualenvs/cookiecutter-pypackage-oT8-zJK2/lib/python3.7/site-packages/sphinx/application.py", line 397, in setup_extension
    self.registry.load_extension(self, extname)
  File "/Users/sambriggs/.local/share/virtualenvs/cookiecutter-pypackage-oT8-zJK2/lib/python3.7/site-packages/sphinx/registry.py", line 465, in load_extension
    metadata = setup(app)
  File "/Users/sambriggs/.local/share/virtualenvs/cookiecutter-pypackage-oT8-zJK2/lib/python3.7/site-packages/invocations/autodoc.py", line 90, in setup
    autodoc.add_documenter(TaskDocumenter)
  File "/Users/sambriggs/.local/share/virtualenvs/cookiecutter-pypackage-oT8-zJK2/lib/python3.7/site-packages/sphinx/deprecation.py", line 49, in __getattr__
    return getattr(self._module, name)
AttributeError: module 'sphinx.ext.autodoc' has no attribute 'add_documenter'

The offending line carries an explanation of why the extension is calling an internal API:

NOTE: sooo for now, since a bunch of our other shit breaks on Sphinx 1.7, we are just explicitly calling autodoc's add_documenter. Sadface.

But it looks like now is the time that it needs fixing!

invocations/invocations/autodoc.py

Line 90 in b3fdef7

autodoc.add_documenter(TaskDocumenter)

Restrict enum34 dependency to python < 3.4

Hi folks,

I recently migrated a python project from 3.4 to 3.6 and bumped into an issue caused by a dependency on enum34.

Having a dig around, that dependency is drawn from invocations itself, but is caused because the backport doesn't include features that the 3.6 enum stdlib does include.

Can we consider pinning the enum34 dependency to python <3.4?

The sphinx plugin doesn't seem to work with sphinx-autodoc-typehints and sphinx-autodoc-defaultargs

I use these extensions:

• https://github.com/tox-dev/sphinx-autodoc-typehints
• https://github.com/zwang123/sphinx-autodoc-defaultargs

They work for regular public functions, but anything annotated with @task doesn't pull in the type hint and default value from the function definition.

Is it possible to configure the invoke ext to work with this somehow?

I should add I'm using this guys code snippet to patch in support for type hints in the first place: pyinvoke/invoke#357 (comment)

Implicit dependency on pip

invocations.packaging.release imports pip:

invocations/invocations/packaging/release.py

Line 30 in b819db8

from pip import __version__ as pip_version

while it is not listed as project's runtime dependency.

2.6.0: pytest if failing

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-invocations-2.6.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-invocations-2.6.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/invocations-2.6.0, configfile: pytest.ini, testpaths: tests
plugins: Faker-13.11.1, relaxed-1.1.5
collected 0 items / 1 error

================================================================================== ERRORS ==================================================================================
______________________________________________________________________ ERROR collecting test session _______________________________________________________________________
Direct construction of pytest_relaxed.classes.SpecModule has been deprecated, please use pytest_relaxed.classes.SpecModule.from_parent.
See https://docs.pytest.org/en/stable/deprecations.html#node-construction-changed-to-node-from-parent for more details.
============================================================================= warnings summary =============================================================================
../../../../../usr/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:168
  /usr/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:168: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    exec(co, module.__dict__)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
ERROR
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================================================= 1 warning, 1 error in 0.29s ========================================================================

2.3.0: sphinx fails on rendering documentation

Looks like it is come issue with using sphinx.ext.autodoc

+ /usr/bin/python3 setup.py build_sphinx -b man --build-dir build/sphinx
running build_sphinx
Running Sphinx v4.3.2

Exception occurred:
  File "/home/tkloczko/rpmbuild/BUILD/invocations-2.3.0/invocations/autodoc.py", line 90, in setup
    autodoc.add_documenter(TaskDocumenter)
AttributeError: module 'sphinx.ext.autodoc' has no attribute 'add_documenter'
The full traceback has been saved in /tmp/sphinx-err-9qv9ykir.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

[tkloczko@ss-desktop SPECS]$ cat /tmp/sphinx-err-9qv9ykir.log
# Sphinx version: 4.3.2
# Python version: 3.8.12 (CPython)
# Docutils version: 0.17.1 release
# Jinja2 version: 3.0.3
# Last messages:

# Loaded extensions:
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/setup_command.py", line 169, in run
    app = Sphinx(self.source_dir, self.config_dir,
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 237, in __init__
    self.setup_extension(extension)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 394, in setup_extension
    self.registry.load_extension(self, extname)
  File "/usr/lib/python3.8/site-packages/sphinx/registry.py", line 442, in load_extension
    metadata = setup(app)
  File "/home/tkloczko/rpmbuild/BUILD/invocations-2.3.0/invocations/autodoc.py", line 90, in setup
    autodoc.add_documenter(TaskDocumenter)
AttributeError: module 'sphinx.ext.autodoc' has no attribute 'add_documenter'

Make the package work without bundled packages in invoke

This package expect invoke to contain vendored content which is quite often removed by distributions.

Attached patch will make the imports optional falling back to system packages:

invocations-no-bundled.txt

Fix testrun under python3

With python3 if you open anything it is interpreted as an unicode and you would have to decode it.

Alternatively one can use codecs.open as I attach here:

invocations-py3.txt

Implicit dependency on readme-renderer

invocations.packaging.release imports readme_renderer:

invocations/invocations/packaging/release.py

Line 31 in b819db8

import readme_renderer.rst # transitively required via twine in setup.py

while it's not listed as runtime dependency (even if readme-renderer is dependency of dependency (twine)).

Test suite failure: "Direct construction of SpecModule has been deprecated, please use SpecModule.from_parent."

When running the test suite with inv test I'm getting the following error:

============================= test session starts ==============================
platform darwin -- Python 3.9.10, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /nix/store/mqdppmn4n37dkqwx71g88a8pa5h676gj-python3-3.9.10/bin/python3.9
cachedir: .pytest_cache
rootdir: /private/tmp/nix-build-python3.9-invocations-2.6.0.drv-0/source, configfile: pytest.ini, testpaths: tests
plugins: relaxed-1.1.5
collected 0 items / 1 error                                                    

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
Direct construction of SpecModule has been deprecated, please use SpecModule.from_parent.
See https://docs.pytest.org/en/stable/deprecations.html#node-construction-changed-to-node-from-parent for more details.
=============================== warnings summary ===============================
../../../../nix/store/mq15vnx91rm5vzdkf1qnixs5p9k5py2g-python3.9-pytest-6.2.5/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:170
  /nix/store/mq15vnx91rm5vzdkf1qnixs5p9k5py2g-python3.9-pytest-6.2.5/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:170: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    exec(co, module.__dict__)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
ERROR 
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
========================= 1 warning, 1 error in 0.17s ==========================

What am I doing wrong? What versions of pytest are supported?

Failing tests in 2.0.0 release?

I may be misunderstanding how the project is setup, but when I download & extract the 2.0.0 release (from PyPI) and try to run tests, things fail:

> curl -fsSL 'https://files.pythonhosted.org/packages/source/i/invocations/invocations-2.0.0.tar.gz' | tar -xz'
> cd invocations-2.0.0
> python3 setup.py test
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
running egg_info
writing invocations.egg-info/PKG-INFO
writing dependency_links to invocations.egg-info/dependency_links.txt
writing requirements to invocations.egg-info/requires.txt
writing top-level names to invocations.egg-info/top_level.txt
reading manifest file 'invocations.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*' found under directory 'docs/_build'
warning: no files found matching 'tasks-requirements.txt'
warning: no previously-included files matching '*.pyc' found under directory '*'
warning: no previously-included files matching '*.pyo' found under directory '*'
warning: no previously-included files matching '*' found under directory '**/__pycache__'
adding license file 'LICENSE'
writing manifest file 'invocations.egg-info/SOURCES.txt'
running build_ext
/home/kshi/invocations-2.0.0/.eggs/semantic_version-2.6.0-py3.9.egg/semantic_version/base.py:94: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if self.prerelease and self.minor is 0 and self.patch is 0:
/home/kshi/invocations-2.0.0/.eggs/semantic_version-2.6.0-py3.9.egg/semantic_version/base.py:94: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if self.prerelease and self.minor is 0 and self.patch is 0:
/home/kshi/invocations-2.0.0/.eggs/semantic_version-2.6.0-py3.9.egg/semantic_version/base.py:100: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if self.prerelease and self.patch is 0:
/home/kshi/invocations-2.0.0/invocations/pytest.py:141: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if report is "html":
Traceback (most recent call last):
  File "/home/kshi/invocations-2.0.0/setup.py", line 28, in <module>
    setup(
  File "/home/kshi/.venv/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.9/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.9/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/kshi/.venv/lib/python3.9/site-packages/setuptools/command/test.py", line 232, in run
    self.run_tests()
  File "/home/kshi/.venv/lib/python3.9/site-packages/setuptools/command/test.py", line 250, in run_tests
    test = unittest.main(
  File "/usr/lib/python3.9/unittest/main.py", line 100, in __init__
    self.parseArgs(argv)
  File "/usr/lib/python3.9/unittest/main.py", line 124, in parseArgs
    self._do_discovery(argv[2:])
  File "/usr/lib/python3.9/unittest/main.py", line 244, in _do_discovery
    self.createTests(from_discovery=True, Loader=Loader)
  File "/usr/lib/python3.9/unittest/main.py", line 154, in createTests
    self.test = loader.discover(self.start, self.pattern, self.top)
  File "/usr/lib/python3.9/unittest/loader.py", line 349, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "/usr/lib/python3.9/unittest/loader.py", line 405, in _find_tests
    tests, should_recurse = self._find_test_path(
  File "/usr/lib/python3.9/unittest/loader.py", line 483, in _find_test_path
    tests = self.loadTestsFromModule(package, pattern=pattern)
  File "/home/kshi/.venv/lib/python3.9/site-packages/setuptools/command/test.py", line 50, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/usr/lib/python3.9/unittest/loader.py", line 191, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/home/kshi/.venv/lib/python3.9/site-packages/setuptools/command/test.py", line 50, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/usr/lib/python3.9/unittest/loader.py", line 205, in loadTestsFromName
    test = obj()
  File "/home/kshi/.venv/lib/python3.9/site-packages/invoke/tasks.py", line 123, in __call__
    if not isinstance(args[0], Context):
IndexError: tuple index out of range

• How are the tests supposed to be run on the PyPI tarball?
• Can this be documented somewhere?

I ask because I'm trying to get invocations to be successfully packaged for my OS, and being able to run tests for packages is an important criteria for verifying quality/functionality of packages.

Newer `semantic_version`

invocations currently pins its requirements to semantic_version>=2.4,<2.7. The latest semantic_version that matches is 2.6.0 released seven years ago. This causes problems for operating systems packagers because they usually pack recent versions of software and it is very hard to provide multiple versions to fulfill similar ancient requirements.

For example, in OpenIndiana the semantic_version package is at version 2.10.0.

Please update invocations to work with newer semantic_version. Thank you.

Consider using extra_requires in setup.py + optional imports

• Early in invocations' life we tried using optional imports, e.g. the watch-docs task will yell at you if you don't have watchdog installed.
• This was found to not scale super great, and since all the other dependencies we grew were pure-Python, we decided not to continue that practice.
• At this point we do have a nontrivial number of deps, and we're also asking users of Invoke to install Invocations for even the smallest helper like confirm. While users should arguably never be all THAT upset to install something requiring no extra C level deps, it's still not ideal to need a dozen or more.
• We're experimenting with setuptools' extras_require elsewhere and it may work well here, e.g. invocations vanilla has no deps besides Invoke, invocations[watching] adds watchdog, invocations[complete] adds everything.
• Unfortunately this still requires try/except wrapping around imports, which as above is not the best thing, but it may be a reasonable sacrifice, especially now that we can offer one 'flavor' of invocations per module or subpackage (as opposed to having, say, 2-4 different warnings per module because of its 2-4 dependencies).
  • I.e. before, we'd need say 4 try/except clauses, 4 different error messages, etc
  • Now we can probably get away with a larger single try/except and a single message saying please pip install invocations[flavor]?

RFE: is it possible to start making github releases?🤔

Is it possible next time on release new version make the github release to have entry on https://github.com/pyinvoke/invocations/releases? 🤔

I'm asking because only on make gh release is spread notification about new release to those who have set watch->releases.
My automation process those notification trying make preliminary automated upgrade of building packages which allow save some time on maintaining packaging procedures.

More about gh releases is possible to find on
https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository
https://github.com/marketplace/actions/github-release
https://pgjones.dev/blog/trusted-plublishing-2023/

Pypi package does not include tests

Distribution vendors use the tests to validate sanity of the releases they ship as such it is good idea if the tests are standalone to provide them.

Just matter of adding the folders to MANIFEST.in