schollii/pypubsub

A Python publish-subcribe library (moved here from SourceForge.net where I had it for many years)

pypubsub's Issues

Using wx.lib.pubsub with PyInstaller

wxPython 4 (Phoenix) has now been officially released and supports Python 3. Py2exe has not been fully converted for Python 3 and is now no longer maintained. Neither is Esky the software updater package that is (was) callable from wxPython. The way forward seems to be via PyInstaller and PyUpdater. Unfortunately when attempting to use PyInstaller on a wxPython application that uses pubsub it fails to import the items that are in wx\lib\pubsub\core\kwargs.

Looking further into PyInstaller I see it has a powerful mechanism for locating problematic imports via Python scripted "Hooks" and that many such have been published at https://github.com/pyinstaller/pyinstaller for various Python packages. There is one there for wx.lib.pubsub but looking at the open Issues there have been several, so far unsuccessful attempts to complete it going back several years, in particular, issues #1367, #1530, #2215 and #2233.

Is there any recommendation as to how pubsub can be used by PyInstaller?

David Hughes
Forestfield Software

Running tests on 3.3.0

I am looking to package your software for Gentoo Linux. Iorder to maintain Python2/3 compatibility I have opted to start with the older verison:

The tests fail with this log.

The issue seems to be the following:

ImportError: cannot import name policies

It is also a relative import, so I assume it should be somewhere in the directory tree of the package, however, I cannot locate this file. Can you help me out?

The tests are being run with nosetests -v || die.

Include tests in archive

This is a request to include the unit tests in the .zip release archives. It is useful to be able to run the tests after building a package to make sure everything is working.

Deprecation warning

It looks like there is a small issue with a Deprecated function. See below

c:\anaconda2\envs\py36\lib\site-packages\pubsub\core\callables.py:147: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
(allParams, varParamName, varOptParamName, defaultVals) = getargspec(func)

pub.subsribe fails when using python 3 typehints in callable

Here is the error log:
File "C:\Users\J84375\AppData\Local\Programs\Python\Python36\lib\site-packages\pubsub\core\publisher.py", line 160, in subscribe subscribedListener, success = topicObj.subscribe(listener, **curriedArgs)

File "C:\Users\J84375\AppData\Local\Programs\Python\Python36\lib\site-packages\pubsub\core\topicobj.py", line 353, in subscribe args, reqd = topicArgsFromCallable(listener, ignoreArgs=curriedArgs)

File "C:\Users\J84375\AppData\Local\Programs\Python\Python36\lib\site-packages\pubsub\core\topicargspec.py", line 51, in topicArgsFromCallable argsInfo = getListenerArgs(_callable, ignoreArgs=ignoreArgs)

File "C:\Users\J84375\AppData\Local\Programs\Python\Python36\lib\site-packages\pubsub\core\callables.py", line 221, in getArgs return CallArgsInfo(func, firstArgIdx, ignoreArgs=ignoreArgs)

File "C:\Users\J84375\AppData\Local\Programs\Python\Python36\lib\site-packages\pubsub\core\callables.py", line 147, in __init__ (allParams, varParamName, varOptParamName, defaultVals) = getargspec(func)

File "C:\Users\J84375\AppData\Local\Programs\Python\Python36\lib\inspect.py", line 1072, in getargspec

raise ValueError("Function has keyword-only parameters or annotations" ValueError: Function has keyword-only parameters or annotations, use getfullargspec() API which can support them

Removing the type hints from the callable signature removes the error. However, it would be very nice to use typehints!

Packages which depend on pypubsub

I have finally gotten the pypubsub package to a status where it could be included in the main Gentoo Linux software repository (making it accessible to all Gentoo users).

However, the one package which I needed it for (wxpython) apparently no longer depends on it. We're currently debating whether we still want to include it or not. Do you know if there are any high-profile packages which need it or whether it has any standalone (rather than library) usability?

Add CI tests

Given that this is a fairly widely used library (e.g. according to GitHub 744 repos depend on it), it seems like adding CI testing to this repo would be worth while? If you use GitHub actions it should be relatively fast and completely free.

Tests are confusing and non-intuitive to run, requiring magic incantations.

• it depends on the current working directory that pytest is run from, because things like open('test4_prov_module_actual.py', 'r') and self._parse_tree(_get_elem(open(xml, mode="r") (where xml is just "xmlprovider_topics.xml"), instead of using os.path.join(os.path.dirname(__file__), 'xmlprovider_topics.xml')
• src/pypubsub is not autodetected and added to the PYTHONPATH, whereas the common custom of using "modulename" as a directory in the root directory of the repository, would work fine.
• using tests/suite/ as the test directory, instead of just tests/, means pytest needs to be explicitly passed the test directory, instead of just detecting it automatically.

The ideal case would be that one could git clone https://github.com/schollii/pypubsub && cd pypubsub && python -m pytest and everything would just work without a fuss.
Currently it instead requires an additional cd tests/suite && PYTHONPATH=$PWD../../src/ python -m pytest for no discernable reason.

DeprecationWarning

def foo():
    pass

import wx.lib.plot
from pubsub import pub
pub.subscribe(foo, 'topic')

Result of my reduced code in Python 3.6:

D:\Python36\lib\site-packages\pubsub\core\callables.py:147: DeprecationWarning:
inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
(allParams, varParamName, varOptParamName, defaultVals) = getargspec(func)

(not without wx.lib.plot, and only in Python 3)

INotificationHandler path is wrong

In the documentation, it seems to be pubsub.pub.INotificationHandler, but I think the correct name is pubsub.core.INotificationHandler.

deleting all topics

Need a simple way to remove all topics/subscribers/reset pubsub without stopping the interpreter. Working in Pythonista which does not reset the interpreter when a script finishes.

Merge Alignment branch to master

Wait a bit to see if more issues raised, then merge to master as 4.0.4.

Might be good to rename the 2 tags vx.y.z to rel_x.y.z to clearly distinguish branches from releases. Or rename the v3.4 branch to py2.7 and rename the rel_3.4.2 tag to v3.4.2, maybe that's better.

Replace obsolete references to sf.net

Several places on the internet still refer to pubsub.sf.net. When possible, these link should be updated.

PyPubSub 3.4.2 install on python 2.70

Hi, is this release only for Python 2.70, 64-bit, it does not seems to work on 32-bit

Version 3.30 install perfect - it also seems 3.4.2 has nothing new in it?

Release .zip

Hello,

I am running into issues with installing the "pypubsub" package on Arch Linux.
This package depends on the file "https://downloads.sf.net/project/$_project/$_project/$pkgver/$_pkgname-$pkgver.zip" which no longer exists because of your switch to Github.

Will you provide new releases as .zip files in the future and if so: where do I find those releases?

An alternative is a package that automatically builds your GIT repository: how stable will the master branch of this repository be?

Greetings,
Renze

Python 2 support

Any chance that you would be willing to consider supporting Python 2 again? The embedded copy of PyPubSub was removed from wxPython Phoenix and a external dependency was added on PyPubSub. However, Phoenix still supports Python 2, so this has resulted in Phoenix being uninstallable in certain cases.

Ref wxWidgets/Phoenix#887

Get the topic name, which "called" a listener

Hello,

I have this (not working) listener:

def uhu_listener(payload):
print ("uhu_listener:")
print ("\t topic: ", ??? GET TOPIC WHICH CALLED ME ???)
print("\t` payload: ", payload)

and this main entry:

if name == "main":
payload = 3
pub.subscribe(uhu_listener, topicName="uhu1")
pub.subscribe(uhu_listener, topicName="uhu2")
pub.subscribe(uhu_listener, topicName="uhu3")
pub.sendMessage("uhu2", payload=payload)

So, uhu_listener listens to 3 Topics (uhu1, uhu2 and uhu3).

Is there a way to determine the topic, which "called" the uhu_listener method?

BR,
Reinhard

New Publishers

As noted at pubsub.readthedocs, the expression

from pubsub import pub

... creates a “default” instance of pubsub.core.Publisher and binds several local functions to
some of its methods and those of the pubsub.core.TopicManager instance that it contains.
However, an application may create as many independent instances of Publisher as
required (for instance, one in each thread; with a custom queue to mediate message transfer between > threads)."

As noted in this quote, the pub module does quite at bit of work to expose methods and attributes, and it is not immediately obvious how to build a new Publisher() object with the correct attributes exposed. I'd be really interested in knowing if there is a standard way to do this. Alternatively, perhaps we could add a new module that provides this.

Generic topic in the tree

I have been looking at your module for an application that I am refactoring and your module looks very good. One thing that would make my application much easier to write is if I can have a generic subscipriton in the middle of the topic tree. For example if I had a topic tree:

a1:
  b1:
    c1
  b2:
    c1

And if I had the ability to subscribe generically at the b level so that I would get a1-b1-c1 and a1-b2-c1. The obvious answers in the current model are:

1. Move c up a level and b down a level but that will only flip the issue and not really solve it (i.e. I want a generic c selection as well.
2. Stop at the b level and handle c in code. But this means more application logic rather than subscription logic.
3. Subscribe to a1-b1-c1 and a1-b2-c1 which means having more permutations to generate and manage.

My thought is if the topic manager could handle a iterable of hashable objects and each level would then just be a lookup of a hash plus a look up of a generic (like the Any object in pydispatcher.)
Thoughts?

Compatibility with Python3.7

Hi,

I am having difficulty running pypubsub w/ python 3.
Steps:
1- Brand new SD card: Used NOOB to install Raspian Buster
2- Updated the OS
3- Then installed pupubsub (pip install pypubsub)
I can run the test program using Python2, but when I try to use Python3 I get a module import error (ModuleNotFoundError: No module named 'pubsub').

I need to run Python3 because other modules I will need to use for the main app need Python3. Does pypubsub support Python3?

Thanks

Depends on wxpython?

As far back as 3.3.0 I notice imports like import wx? This doesn't sound right. I am in fact packaging your software in order to satisfy the dependencies of the newer wxpython versions. Are you aware of this circular dependency?

Can't use closure function for subscribe callback function

from pubsub import pub

def localCall(func):
def real_func(data):
print("In real function : {}".format(data))
func(data)
return real_func

def localUpdate(data):
print("In local update")

pub.subscribe(localCall(localUpdate), "update")
pub.sendMessage("update", data="updateUI")

In the example above, the function localUpdate can't be called. I have no idea why the closure function can't be called?

Support keyword-only arguments of Python 3

With issue #27 fixed, we can now revisit #6.

By design Pypubsub only cares whether args are optional or required (don't have a default value), so the update should be much simpler than done in PR #13, but further investigation needed. Other problem with PR#13 is that travis-CI indicates some tests broken by the PR.

Verify with tests reported in #6 and PR #13

Feature request: subscribe non-global callables

Seems I am only able to pub.subscribe(some_callable_in_global_namespace, 'topic'). If that is so can we change that to allow for non-globals to subscribe.

My current work-around is something like this:

def init(kwargs={}):
    global start, inited
    inited = True
    def start(kwargs={}):
        global pause, started
        started = True
        def pause(kwargs={}):
            global paused
            paused = True
            # do stuff
        pub.subscribe(pause,'pause.myaddonA')
    pub.subscribe(start, 'start.myaddonA')
pub.subscribe(init, 'init.myaddonA')

Release 4.0.1

License file and release notes are confusing and non-intuitive to find.

Storing the license file as an oddly named file in src/pypubsub is very much not discoverable, and seems to be done for no reason other than to store it as package_data even though no one who was completely unable to discover either that or the release notes in the standard locations in the github web interface, is likely to go digging around in the egg directory.

On which note it looks like zip_safe=False may be being set just to make these files available outside of a zip, even though they're not relevant to the functionality of python at all. In fact, most people would just look at the setuptools metata via e.g. pip show pypubsub, and if they need the license for legal reasons they don't need it in the egg distribution so it would make more sense to have it in the root of the repository.

I've literally never seen it done this way before. :(

Also, moving it to the root of the repository and giving it a standard name means github can automatically detect and display the status in the header bar for the repo. :)

AUTO_TOPIC_OBJ implementation missing

AUTO_TOPIC_OBJ is documented yet missing from the actual implementation?

KeyError with topics named "event"

I'm currently failing to send any messages with the topic name of event.<something> (replacing <something> with some text).
For example I'm trying to send event.track_playback_started but it fails and returns a KeyError. I can however send other messages like state.track_playback_started. Even a topic something like "myapplicationname_event" fails to send with the key error. Only if I remove "event" from it it succeeds.

Is "event" a reserved topic? I can't find anything about it in the documentation. The only reserved topic I can find is pub.ALL_TOPICS but obviously I'm not using that one.

How to view Handled messages

Hi,

I am in the process of debugging and application that uses PyPubSub for its messages. I am using the snoop methods (https://pypubsub.readthedocs.io/en/v4.0.3/usage/usage_advanced_debug.html#id4) to see which messages are published. However, I'd really like to also log whenever a handler receives a message. Is there a means of doing so easily?

Paddy

Subscribe from another Script

Hi,

I am trying to use pypubsub in order to subscribe to a topic from another python script. Is this feasible?

My setup is: One python script continuously looping with pub.sendMessage('rootTopic', arg1=id, arg2=tvec, arg3=rvec) and another script as follows.

from pubsub import pub

def listener0(arg1=None, arg2=None, arg3=None): 
    print("listener0: ", arg1, arg2, arg3)

if __name__ == '__main__':
    
    while True:
        pubListener, first = pub.subscribe(listener0, 'rootTopic')
        print (pubListener.name())
        assert first == True
        assert pubListener.isDead() == False   

I cannot see any message in the console apart from:

listener0_3800
listener0_3800
Traceback (most recent call last):
  File "main.py", line 12, in <module>
    assert first == True
AssertionError

So seems that first is true for the first is true for the first two instances and then is false.

PS: Before splitting the code in two parts I have tried the sub/pub on the same script and it works

Let me know,

Matteo

policies and py2and3 imports (3.3.0)

I am having issues while trying to run the tests with some imports, in particular this bit (in pypubsub/src/pubsub/core/topicdefnprovider.py ) :

from .. import (
        policies,
        py2and3
)

any idea where these modules should be coming from?

Content Filtering

Does pypubsub support content or header filtering for clients? Topic filtering looks great, but filtering based on a specific kwarg or arg would be a useful addition.

A listener has died

I am trying to implement the MVC pattern using Python3.7, wxpython (4.0.4) and Pypubsub 4.0.3.

See below a code example. When I run this the following gets printed to the console:

PUBSUB: New topic "pressed" created
PUBSUB: Subscribed listener "TestController.pressed" to topic "pressed"
PUBSUB: Listener "TestController.pressed" of Topic "pressed" has died

Last line shows Listener of Topic "pressed" has died. I don't understand why.
Also when I run this through the debugger (pycharm) and put a breakpoint where the listener is created, wait a bit and continue the Listener doesn't die and all works as it should.
Any suggestion whether this might be a bug, or am I doing something wrong ?

import sys
import wx
from pubsub import pub
from pubsub.utils import useNotifyByWriteFile

useNotifyByWriteFile(sys.stdout)


class TestModel:
    def do_something(self):
        print("doing something")


class TestView(wx.Window):
    def __init__(self, parent):
        wx.Window.__init__(self, parent, -1)

        sizer = wx.BoxSizer(wx.VERTICAL)
        self.btn = wx.Button(self, -1, "Press")
        sizer.Add(self.btn, 0, wx.EXPAND | wx.ALL)
        self.SetSizer(sizer)

        self.btn.Bind(wx.EVT_BUTTON, self.on_press)

    def on_press(self, evt):
        print("button pressed")
        pub.sendMessage("pressed", val=1)


class TestController:
    def __init__(self, parent):

        self.view = TestView(parent)
        self.model = TestModel()

        pub.subscribe(self.pressed, "pressed")

    def pressed(self, val):
        print("pressed event received")
        self.model.do_something()


class MainFrame(wx.Frame):
    def __init__(self):
        wx.Frame.__init__(self, None, title="a test", size=(800, 600))
        sizer = wx.BoxSizer(wx.VERTICAL)

        controller = TestController(self)
        sizer.Add(controller.view, 0, wx.EXPAND)

        self.SetSizer(sizer)

        self.Show()


if __name__ == "__main__":
    app = wx.App()

    start = MainFrame()

    sys.stdout = sys.__stdout__

    app.MainLoop()

Could you please add a sdist release on Pypi?

Hello Oliver,

I would like to use PyPubSub-4.0.3 in my Gentoo Overlay for Home Assistant, because the pyinsteon component seems to need it. First, I hope you don't mind. Therefor I need a release file in tar.gz format. It exists at Github, but Pypi only has it in wheel format, currently I use the release file from Github.

Next, I am not sure which name I should choose: The Github repo is called: pypubsub, on Pypi it is called: PyPubSub, but the Pypi Wheel file is: Pypubsub, the library itself installs as pypubsub, the egg calls itself Pypubsub. OK, this works for now, but it could happen that somebody adds it to a repo with one of the other names, this would lead to file collisions in future, perhaps you could clean this up and use one single name on all locations.

Currently I used 'pypubsub' for the ebuild, this avoided all conversions in the build process.

Generally, a tar.gz in sdist format on Pypi is preferred, because this could take use of Gentoo's mirror system. Could you please add one?

Most of the other components/integrations for Home Assistant do both.

Thanks
\B.

Update wxPython wiki

Update for Python 3.7

Verify no changes needed
Update docs

Support Type hints in listener signatures

This is follow-up on #6, which has a PR that cannot be accepted due to test breakage and also seems to be more change than necessary (need investigation). The issue was likely caused by pypubsub use of deprecated inspect.getargspec instead of getargsfullspec. Since this was fixed in another PR, typehints are likely fine and the only portion of #6 that needs addressing is signatures with keyword-only args, to be addressed in #26 due to test failures with PR #13.

Conda-forge package

It would be very helpful to have this package on conda-forge https://conda-forge.org/
in order to have a reliable conda source for pypubsub.
So far, the package is only available on some third-party channel:
https://anaconda.org/ivoflipse/pypubsub

If someone with more knowledge of the package would support me, I could set up the conda-forge pull request.

Async support

I haven't looked too deeply in what would be involved to implement this properly, but before I do I was wondering whether you have any thoughts on this.

The main thing I'd like is to be able to provide a listener callback that is a co-routine instead of a 'normal' function. My current workaround is to convert any 'await'-able calls inside my listener to sync, but I thought it might be nicer if pypubsub could handle async callbacks natively.

As a first step, it might be good enough to check if the listener is a co-routine, and convert/await it inside the relevant pypubsub-component. Might be opening up a big can of multi-thread-worms and not worth it, not sure.

Ideally of course, pypubsub would handle async stuff natively, but that is most likely too much to ask, and probably better to look at something like https://github.com/abadger/pubmarine , but overall I'm very happy with pypubsub and it fits most of my requirments perfectly.

Pause messages of type

Hi @schollii. I am interested to know whether it is possible to put a stop on messages being sent for a while. The idea is a follows:

• A sends a message x
• It is consumed by B, C, D, ..., H
• Each of B, C, D, ... , H process their message, and send a message, each of which is consumed by A.
• Upon receipt of each message, A publishes another message y(B), y(C), ..., which is consumed by other entities. Let's say each message y is of topic "Z" and the kwargs are a dictionary with say key1 = "AA" and key2 = "BB".

What I would like is that I prevent A from sending the messages y(B), Y(C), ... etc. until the "last" message is received from H.

The kind of thing I am imagining is:

with pub.withhold("Z", kwargs["key1"] == "AA", kwargs["key2"] == "BB"):
    pub.sendMessage(x)

pub.sendMessage("Z", **kwargs)

The reason for considering this idea is that each of the messages y generates redundant activity so that it needs only be sent once.

I understand that this process may go against the spirit of asynchronous messaging, but I feel it is an interesting pattern.

Complete the new 3.4 branch release

The 3.4 version is needed because the 3.3 source was damaged during the transition from sourceforge to github a few years ago: the zip installer works and pubsub works, but installations based on the setup.py, including tests etc were not working properly because the zip was not designed to contain everything.

Version 3.4 is being released to support python 2.7 only, and as such is a deprecated version! but being released so we have an official complete release that works in python 2.7.

After the issues raised by TheChymera regarding version 3.3 of pypubsub, the only thing left to fix is the documentation such as READMEs, docs etc.

MyPy can't actually use the type hints because the package doesn't claim to be typed

This package has type information, but ti doesn't ship a magic py.typed marker file. So MyPy refuses to actually use the type information in the package and instead throws an error saying that I need to somehow induce the library to ship a py.typed file, or else write all my own stubs with duplicate type information.

Listener inadequate errors randomly appearing

Hello,

I am using pypubsub in the context of a program that uses cocotb to test some HDL code. I have a LARGE number of publisher and subscribers, and in the past few days I have started to receive this kind of errors:

INFO     cocotb:simulator.py:302 #   2500.00ns ERROR    root                               Listener "MyFun._myhandler" (from module "MyFun") inadequate: required args (msg) not allowed (could curry them), topic has no required args (params (msg) are req'd in listener, optional in topic )
INFO     cocotb:simulator.py:302 #                                                         Traceback (most recent call last):
INFO     cocotb:simulator.py:302 #                                                           File "MY_PATH/test.py", line 138, in run_test
INFO     cocotb:simulator.py:302 #                                                             myfun = MyFun(config)
INFO     cocotb:simulator.py:302 #                                                           File "MY_PATH/MyFun.py", line 67, in __init__
INFO     cocotb:simulator.py:302 #                                                             pub.subscribe(self._myhandler, mytopic)
INFO     cocotb:simulator.py:302 #                                                           File "MY_PATH/.tox/py3/lib/python3.7/site-packages/pubsub/core/publisher.py", line 160, in subscribe
INFO     cocotb:simulator.py:302 #                                                             subscribedListener, success = topicObj.subscribe(listener, **curriedArgs)
INFO     cocotb:simulator.py:302 #                                                           File "MY_PATH/.tox/py3/lib/python3.7/site-packages/pubsub/core/topicobj.py", line 356, in subscribe
INFO     cocotb:simulator.py:302 #                                                             argsInfo = self.__validator.validate(listener, curriedArgNames=curriedArgs)
INFO     cocotb:simulator.py:302 #                                                           File "MY_PATH/.tox/py3/lib/python3.7/site-packages/pubsub/core/listener.py", line 274, in validate
INFO     cocotb:simulator.py:302 #                                                             self.__validateArgs(listener, paramsInfo, curriedArgNames)
INFO     cocotb:simulator.py:302 #                                                           File "MY_PATH/.tox/py3/lib/python3.7/site-packages/pubsub/core/listener.py", line 333, in __validateArgs
INFO     cocotb:simulator.py:302 #                                                             raise ListenerMismatchError(msg, listener, extraArgs)
INFO     cocotb:simulator.py:302 #                                                         pubsub.core.callables.ListenerMismatchError: Listener "MyFun._myhandler" (from module "MyFun") inadequate: required args (msg) not allowed (could curry them), topic has no required args (params (msg) are req'd in listener, optional in topic )

Please do note that I have not touched the file in question (I was working on a totally unrelated file, with totally unrelated topics...).
Has anyone got a clue about why this is happening?
I can easily circumvent the error by putting the argument as optional in the handler and then putting an assert to ensure that the argument value is not None, but this is quite annoying...

Thanks a lot!