marrow/package

Plugin discovery and management, dependency graphing, and object import path canonicalization.

package's Issues

Allow resolution of package-relative file paths.

Currently the load utility is effectively a functional import interface for objects from Python modules, similar to calling __import__ directly, with subsequent attribute lookup to retrieve the target object. (Since __import__('foo.bar.baz').bar.baz is silly.) There are use cases, especially involving template engines, where resolving a dot-notation path to the on-disk file it matches is most important; the target might not even be Python code. Incorporate this behaviour in a secondary helper function, path.

Alter extension exclusion behaviour to not self-exclude.

It'd be handy to allow extensions to declare themselves, effectively, as singletons. E.g. with a provides of {'feature'} and simultaneously excludes of {'feature'}. This is intended to declare a given feature which may only be satisfied by a single extension at a time.

Load utility fails to utilize default in the event of an import error.

Currently the marrow.package.loader:load utility assumes the __import__ call will work, and merely passes the default through to the eventual call to traverse. This is a problem if the import itself is invalid; ImportError is raised when, instead, if a default is defined it should be used. The exception should be re-raised if no default is provided.

Generalize WebCore additions and improvements.

WebCore utilizes marrow.package extensively for plugin/extension services and has developed a useful abstraction for the collection of named extension callbacks; essentially protocol methods of the extension objects which itself has a protocol to advertise new callbacks to collect. (Allowing an extension to define callbacks for its own use, not just as defined by the host application.)

Protocol components to port:

ExtensionManager

• Declaration of SIGNALS; a set of attribute names to collect. E.g.

   SIGNALS = {'start', 'stop'}

  With the above, start and stop attributes of loaded extensions will be captured and executed in order. E.g. plugin A and B would have their callbacks executed: A.start, B.start, A.stop, B.stop.

• Allowance for declaration of callback ordering by prefixing the attribute name with a minus. E.g.

   SIGNALS = {'start', '-stop'}

  With the above, start attributes of loaded extensions will be captured and executed in order, and stop attributes would be captured and executed in reverse order. E.g. plugin A and B would have their callbacks executed: A.start, B.start, B.stop, A.stop. This allows for easier simulation of "middleware" behaviour.

• Allowance for re-naming / aliasing callbacks to be gathered. A SIGNAL entry may either be a string literal naming the attribute to collect and naming the collection of those callbacks, or a tuple naming the group and attribute origin independently. E.g.

   SIGNALS = {('middleware', '__call__')}

  The above would collect all __call__ methods into a signal group named middleware.

• Feature flag tracking. Allow extensions to declare provides as well as needs (hard dependency) or uses (optional dependency) sets. These are generally free-form string identifiers defined by the application hosting plugins.

• Utilize the topological sort of extensions based on provides/needs/uses tag-based associations. Due to the inherently ordered nature of the callbacks being collected, collection (and execution) is dependent on explicit ordering, or implied ordering through dependency resolution.

For details on the WebCore implementation, see: https://github.com/marrow/WebCore/blob/develop/web/core/extension.py

For an example outlining WebCore's complete extension API, see: https://github.com/marrow/WebCore/blob/develop/example/extension.py

Resolve plugin names.

When resolving the importable name of a given object:

>>> from marrow.package.canonical import name
>>> from rita.wrap.integrator.cvm import CVManagerJobSync

>>> name(CVManagerJobSync)
'rita.wrap.integrator.cvm:CVManagerJobSync'

If the object happens to be a registered plugin in a target namespace, the name of the plugin should be returned:

>>> name(CVManagerJobSync, 'rita.sync')
'cvm'

If a namespace is provided and the referenced object is not registered, a LookupError (or derivative) should be raised. Optionally, direct imports should be allowable:

>>> name(SomethingElse, 'rita.sync', imports=True)
'__main__:SomethingElse'