Comments (10)
@teharris1 IIUC what you are saying, given two topics a1.b1.c1 and a1.b2.c1, you would like to have a listener subscribe to a1.b1, such that it would get messages for both "subtopics"? Pypubsub already does that:
pub.subscribe(your_listener, 'a1.b1')
pub.sendMessage('a1.b1.c1')
pub.sendMessage('a1.b2.c1')
You can do various things like make pypubsub automatically include the actual topic sent, and you can use **kwargs in listener so it will also get the lower level args.
Subscribing to 'a1', same idea. You can also subscribe to all topics via ALL_TOPICS.
from pypubsub.
Thank you for your quick reply. Yes, your response is my option 3 above. What I am hoping for is something like this:
pub.subscribe(my_listener, ['a1', pub.ALL_TOPICS, 'c1'])
pub.sendMessage('a1.b1.c1') # or pub.sendMessage(['a1', 'b1', 'c1'])
pub.sendMessage('a1.b2.c1') # or pub.sendMessage(['a1', 'b2', 'c1'])
And receive callbacks on both (i.e. only 'a1', any 'b' and only 'c1' would match). Of course all topic values would have to be hashable so they can act as a key.
from pypubsub.
Subscription to topic 'a' achieves this. If you need to filter just use AUTO_TOPIC to receive topic in listener and check topic name.
from pypubsub.
Right but then I need to handle topics b
and c
, which is what I am trying to avoid. I believe this change does 2 things for your uses:
- It allows for any hashable object to be a topic (similar to how pydispatcher works)
- It allows for a more flexible use of subtopic subscriptions meaning the sender does not need to send differently while the subscriber can more easily select which subtopics it cares about.
from pypubsub.
Sorry not sure I follow why current functionality is insufficient. There is a pypubsub channel on gitter, can we chat there?
from pypubsub.
@teharris1 So based on the extra info you provided on gitter, I concluded that you are trying to use topics as data, rather than as types of messages that carry data. I proposed 2 solutions:
class Data:
def __init__(self, a=None, b=None, ... m=None):
self.a = a
...
self.m = m
def listener(data: Data, b1: int):
...use data.a, ... data.m based on value of b1
pub.subscribe(listener, 'A')
...
pub.sendMessage('A', b1=b1, data=Data(b=..., f=...))
pub.sendMessage('A', b1=other_b1, data=Data(c=..., k=..., n=...))
And here is another, which I find simpler and more expressive:
def listener1(b, f, **kwargs): # listener for b1=1
...
def listener2(c, n, k, **kwargs): # listener for b1=2
...
pub.subscribe(listener1, 'A')
pub.subscribe(listener2, 'A')
# when b1=1:
pub.sendMessage('A', b=..., f=...)
# when b1=2:
pub.sendMessage('A', c=..., k=..., n=...)
If you could let us know if either of these approaches works for you.
from pypubsub.
First, thank you for reaching back out.
Yes, either of those options are very viable solutions. But they defeat the purpose of the original question.
What I am trying to figure out is a way to allow a sender to always send messages the exact same way and route to a command object that will take the appropriate action. The sender has as little implementation logic as possible.
What you are describing above is esetially to have the command object (or handler object depending on the implementation) manage the logic below the highest level. This certainly will work, no doubt.
What i am hoping to acomplish is to simplify the command object logic (and have fewer command objects) by allowing the publisher to route messages to specific command objects rather than have a top level command object that routes messages to a specific command obect.
I can accomplish this, as you described on gitter, by having mutiple subscriptions for all of the permutations. This is also a viable option but requires a lot of subscriptions. I was hoping to take advantage of the subtopic concept in your implementation (which most pub/sub implementations don't have, i.e. pydispatcher). So I like your module and feel the idea of allowing a generic subscription in the middle of the topic tree is not breaking your module's design but rather enhancing it to be more flexible.
I also like that your implementation does not meaningfully degrade the speed of pub/sub with the subtopic concept and see that my request would potentially have a negative impact on speed (i.e. when looking for 'a1-b*-c1' you will have to look for 'c1' in 'b1', 'b2' and 'b3' which may only produce 1 result.) However, this impact would only be realized if the use of the module values flexiblity over speed as I do.
If this is not in scope of your module I understand and have no hard feelings. Thanks for your help.
from pypubsub.
It would add considerable complexity to the sendMessage
implementation, which currently does not track "topics down below". Plus there is yet another approach: create a wrapper for your star listeners:
class StarListener:
def __init__(self, listener, topicFilter):
self.__listener = listener
self.__topicFilter = topicFilter.split('.')
pub.subscribe(listener, pub.ALL_TOPICS)
self.__topicCache = []
def __call__(self, topic=pub.AUTO_TOPIC, **kwargs):
if self.__checkMatch(topic):
self.__listener(**kwargs)
def __checkMatch(self, topic):
if topic.getName() in self.__topicCache:
return True
actualTopicName = topic.getNameTuple()
# actual: [a, b, c, d, e]
# filter: [a, *, c, *, e]
for actualName, filterName in zip(actualTopicName, self.__topicFilter):
if filterName == actualName:
continue
if filterName == '*':
continue
if filterName.endswith('*') and actualName.startswith(filter[:-1]):
continue
return False
self.__topicCache.add(topic.getName())
return True
# test:
def yourListener(arg1, arg2):
print(arg1, arg2)
listener = StarListener(yourListener, 'a.*.c.de*.f')
pub.subscribe(listener)
pub.sendMessage('a.b1.c.dea.f', ...) # yes
pub.sendMessage('a.b2.c.dea.f', ...) # yes
pub.sendMessage('a.b1.c.dda.f', ...) # no
In the above simple implementation,
- the
listener
wrapper subscribes to the root (ALL) topics, because you don't want a topic a,b,c to match "d,f,*, a, b,c". This could be improved somewhat since the portion before the first star is (I think) adequate, thus saving some cpu. - listeners are weak-referenced in pypubsub so you must keep a strong ref to
listener
. But this could (testing needed) prevent in some caseslistener
from being garbage collected (I'm thinking in the case whereyourListener
is a method, and solistener
is a member of the instance of that method, thus leading to a circular reference which could prevent garbage collection). For that, you could store weak ref toyourListener
inStarListener
. - The above design lends itself to more sophisticated filtering (just add
if ... continue
blocks), perf improvements. - You should make the call have as specific kwargs as possible. For example if you know that all listeners want abc and dfg, and then more args based on topic, then use
__call__(self, abc, dfg, **kwargs)
as pubsub does some checks for you, helps with debugging events. - Subtopics will be included, but this could be changed, e.g. if you want exact match then add a rule
if len(self.topicFilter) != len(actualTopicName): return False
- You could easily cache the topics matched so you don't have to go throught the set of rules more than once for a given topic (this is shown, but not essential)
- You loose the automated message data specification checking feature of pypubsub.
from pypubsub.
That is a really good idea. I was also able to breakdown two of the fields into unique combinations and found that the permutations are smaller than I thought (i.e. hundreds, not thousands). This will allow me to create a dict
of these combinations and turn them into a single topic, which meaningfully simplified my problem. I am comfortable now using your base library for the issue without the original request.
Thanks so much for all your help.
from pypubsub.
That's great to hear!
Some day I'd love to understand better what you are using this for. Could be nice to put something in https://pypubsub.readthedocs.io/en/v4.0.3/about.html#pypubsub-users, I haven't updated it in a while.
I'm closing this now.
from pypubsub.
Related Issues (20)
- policies and py2and3 imports (3.3.0) HOT 23
- Complete the new 3.4 branch release HOT 3
- Subscribe from another Script HOT 3
- Replace obsolete references to sf.net HOT 1
- Update wxPython wiki
- Merge Alignment branch to master
- PyPubSub 3.4.2 install on python 2.70
- Packages which depend on pypubsub HOT 1
- How to view Handled messages HOT 3
- Async support HOT 3
- Could you please add a sdist release on Pypi? HOT 5
- Can't use closure function for subscribe callback function
- Pause messages of type HOT 3
- Compatibility with Python3.7 HOT 1
- KeyError with topics named "event" HOT 2
- Add CI tests
- MyPy can't actually use the type hints because the package doesn't claim to be typed
- Listener inadequate errors randomly appearing
- INotificationHandler path is wrong
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pypubsub.