Comments (8)
Glad you are enjoying Thespian! We are enthusiastic about the potentials for Actor-based applications, so both suggestions and support for the effort are appreciated.
The mailing list might be a better place for these types of questions: more people are likely to see and respond, and the discussion is a little more searchable for future queries than a closed issue.
You might want to use the coordinator pattern. This pattern needs to be described in the documentation somewhere, but basically you have a top-level coordinator actor whose job is to start the Calc actor and provide that address to all clients. The coordinator's only responsibility is to create and provide the address of the current valid Calc actor on demand (and single-responsibility like this is another good Actor pattern).
def startup():
asys = ActorSystem()
asys.createActor(Coordinator, globalName='coordinator')
def client():
asys = ActorSystem()
calc = asys.ask(asys.createActor('coordinator', globalName='coordinator'), "get calc addr")
result = asys.ask(calc, work())
class Coordinator(Actor):
def __init__(self):
self.calc_addr = None
def receiveMessage(self, message, sender):
if message == "get calc addr":
if not self.calc_addr:
self.calc_addr = self.createActor(Calc)
return self.calc_addr
elif isinstance(message, ChildActorExited):
if message.childAddress == self.calc_addr:
self.calc_addr = None
In the above (skeleton) code, the Coordinator simply returns the address of a Calc actor on demand, creating it if necessary. If it is told that the Calc actor has exited, it will re-create one on the next request.
The reason for using the Coordinator as the registered globalName actor instead of just referring to the Calc directly via a globalName is that there is no other entity to tell (i.e. send a ChildActorExited to) when a "top-level" actor (in this case, the Coordinator) has exited and therefore cause it to be restarted. By using a very simple top-level coordinator actor that is responsible for deploying the actual worker actors and keeping them around, the overall application gains a great deal of fault tolerance.
This clearly assumes that there is a "primary" system where the Coordinator lives and where the clients run to submit their requests. You can use requirements to ensure that Calc actors are created on (less long-lived) worker systems that use the primary system as the convention leader. If you need more tolerance for failures of the "primary" you could have a Coordinator running on each system and use an election-based system to determine which was the current managing Coordinator. (This is another reason for using the Coordinator: adding complexity to the management does not affect the "business logic" in the Calc actor---separation of concerns and single responsibility patterns at work).
One thing I would suggest would be to consider whether or not your Calc actor must be a singleton. A singleton is definitely a valid operational mode, but Actors lend themselves to easy parallelism... would your model support multiple Calc actors, with the coordination elsewhere (like your persistence actor, which is another good separation of concerns)? This would allow you to scale your application by using the Troupe or having multiple operational Calc actors on different nodes.
And a final note regarding Thespian maturity:
Right now, globalName only applies to the current ActorSystem, and the customary configuration of ActorSystems is a star pattern around that central or "primary" system. Future plans for Thespian include having the ability for multiple or self-electing convention leaders that would change the star topology into more of a mesh, and such a change would probably also involve making the globalName applicable inter-system instead of the current intra-system application.
Regards,
Kevin Quick
from thespian.
Thanks for the advice. I'm kicking the Coordinator idea around in my head.
My concern is that when the "primary" convention leader system with the Coordination actor needs to be taken offline, the whole system will fail because the other actor system can't contact the Coordinator. Is this always the case when using a convention, the convention leader must alway be online?
I can't make the Calc run in parallel because the next calculation is dependent on the previous calculated state.
I don't suppose there is a way to broadcast the latest calculated state to a CurrentState actor in all systems?
from thespian.
At the underlying thespian level, yes, the convention leader must always be present for the other convention members to function. That's the reference to the star configuration I made in the last post, and there is a plan to implement a leader election system in the future to remove this central dependency.
From the Coordinator perspective, you can enhance the coordinators communicating on the other systems to share information and handle the scenario when one leaves; you would need to implement your own leader election process for that. There is no broadcast, so you would need to pass Coordinator addresses to each other and perform explicit individual notifications.
To get around the thespian single-leader constraint above, you can use the preRegister/deRegister calls from the Coordinator to re-establish communications. This is going to get complicated though. The centralized convention leader has been the typical configuration that has been used in the past, and removing this constraint is definitely something that is needed; you are pushing the boundary of current implementation and if it's possible to live with the single leader configuration for now and wait for the Thespian enhancement that might be your better option.
If you want to get involved in enhancing Thespian itself in this area, that would be welcomed, but in this situation this is already on the roadmap so it will still happen even if you don't have the bandwidth to get personally involved (and probably sooner now that there is demand).
-Kevin Quick
From: Kevin Colyar [email protected]
Sent: Friday, September 23, 2016 9:36:38 AM
To: godaddy/Thespian
Cc: Kevin W Quick; Comment
Subject: Re: [godaddy/Thespian] Advice: Actor State in Convention (#20)
Thanks for the advice. I'm kicking the Coordinator idea around in my head.
My concern is that when the "primary" convention leader system with the Coordination actor needs to be taken offline, the whole system will fail because the other actor system can't contact the Coordinator. Is this always the case when using a convention, the convention leader must alway be online?
I can't make the Calc run in parallel because the next calculation is dependent on the previous calculated state.
I don't suppose there is a way to broadcast the latest calculated state to a CurrentState actor in all systems?
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/20#issuecomment-249240939, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFX0bhI3rO2P1KfWU9p_rqzl0TC_rjjmks5qtAAWgaJpZM4KElQy.
from thespian.
I have created #21 to specifically address the limitation of a single convention leader; feel free to upvote via a thumbs-up reaction to indicate level of interest or need for this.
from thespian.
I'd love to contribute to Thespian any way I can. I was looking into Raft,
and I was wondering, at least for the time being, if using something like
PySyncObj https://github.com/bakwc/PySyncObj would solve my state issue
until a mesh convention is implemented.
I could hold a synchronized state outside of my ActorSystem in the clients
and use Thespian for parallel calculations, persistence, etc. Thoughts?
On Fri, Sep 23, 2016 at 10:59 AM Kevin Quick [email protected]
wrote:
At the underlying thespian level, yes, the convention leader must always
be present for the other convention members to function. That's the
reference to the star configuration I made in the last post, and there is a
plan to implement a leader election system in the future to remove this
central dependency.From the Coordinator perspective, you can enhance the coordinators
communicating on the other systems to share information and handle the
scenario when one leaves; you would need to implement your own leader
election process for that. There is no broadcast, so you would need to pass
Coordinator addresses to each other and perform explicit individual
notifications.To get around the thespian single-leader constraint above, you can use the
preRegister/deRegister calls from the Coordinator to re-establish
communications. This is going to get complicated though. The centralized
convention leader has been the typical configuration that has been used in
the past, and removing this constraint is definitely something that is
needed; you are pushing the boundary of current implementation and if it's
possible to live with the single leader configuration for now and wait for
the Thespian enhancement that might be your better option.If you want to get involved in enhancing Thespian itself in this area,
that would be welcomed, but in this situation this is already on the
roadmap so it will still happen even if you don't have the bandwidth to get
personally involved (and probably sooner now that there is demand).-Kevin Quick
From: Kevin Colyar [email protected]
Sent: Friday, September 23, 2016 9:36:38 AM
To: godaddy/Thespian
Cc: Kevin W Quick; Comment
Subject: Re: [godaddy/Thespian] Advice: Actor State in Convention (#20)Thanks for the advice. I'm kicking the Coordinator idea around in my head.
My concern is that when the "primary" convention leader system with the
Coordination actor needs to be taken offline, the whole system will fail
because the other actor system can't contact the Coordinator. Is this
always the case when using a convention, the convention leader must alway
be online?I can't make the Calc run in parallel because the next calculation is
dependent on the previous calculated state.I don't suppose there is a way to broadcast the latest calculated state to
a CurrentState actor in all systems?You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/godaddy/Thespian/issues/20#issuecomment-249240939>, or
mute the thread<
https://github.com/notifications/unsubscribe-auth/AFX0bhI3rO2P1KfWU9p_rqzl0TC_rjjmks5qtAAWgaJpZM4KElQy.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABDhe13StcOROjlxPUGml0YE-mHJD9sks5qtBNlgaJpZM4KElQy
.
from thespian.
That sounds like a reasonable approach. Obviously you'd have duplicated addressing/configuration between Thespian and PySyncObj (thanks for the interesting link there...) but that shouldn't be too hard to manage until Thespian supports mesh.
Feel free to reach out to me directly to coordinate any Thespian work you want to get involved in (and thanks!).
-Kevin Quick
From: Kevin Colyar [email protected]
Sent: Friday, September 23, 2016 1:32:04 PM
To: godaddy/Thespian
Cc: Kevin W Quick; Comment
Subject: Re: [godaddy/Thespian] Advice: Actor State in Convention (#20)
I'd love to contribute to Thespian any way I can. I was looking into Raft,
and I was wondering, at least for the time being, if using something like
PySyncObj https://github.com/bakwc/PySyncObj would solve my state issue
until a mesh convention is implemented.
I could hold a synchronized state outside of my ActorSystem in the clients
and use Thespian for parallel calculations, persistence, etc. Thoughts?
On Fri, Sep 23, 2016 at 10:59 AM Kevin Quick [email protected]
wrote:
At the underlying thespian level, yes, the convention leader must always
be present for the other convention members to function. That's the
reference to the star configuration I made in the last post, and there is a
plan to implement a leader election system in the future to remove this
central dependency.From the Coordinator perspective, you can enhance the coordinators
communicating on the other systems to share information and handle the
scenario when one leaves; you would need to implement your own leader
election process for that. There is no broadcast, so you would need to pass
Coordinator addresses to each other and perform explicit individual
notifications.To get around the thespian single-leader constraint above, you can use the
preRegister/deRegister calls from the Coordinator to re-establish
communications. This is going to get complicated though. The centralized
convention leader has been the typical configuration that has been used in
the past, and removing this constraint is definitely something that is
needed; you are pushing the boundary of current implementation and if it's
possible to live with the single leader configuration for now and wait for
the Thespian enhancement that might be your better option.If you want to get involved in enhancing Thespian itself in this area,
that would be welcomed, but in this situation this is already on the
roadmap so it will still happen even if you don't have the bandwidth to get
personally involved (and probably sooner now that there is demand).-Kevin Quick
From: Kevin Colyar [email protected]
Sent: Friday, September 23, 2016 9:36:38 AM
To: godaddy/Thespian
Cc: Kevin W Quick; Comment
Subject: Re: [godaddy/Thespian] Advice: Actor State in Convention (#20)Thanks for the advice. I'm kicking the Coordinator idea around in my head.
My concern is that when the "primary" convention leader system with the
Coordination actor needs to be taken offline, the whole system will fail
because the other actor system can't contact the Coordinator. Is this
always the case when using a convention, the convention leader must alway
be online?I can't make the Calc run in parallel because the next calculation is
dependent on the previous calculated state.I don't suppose there is a way to broadcast the latest calculated state to
a CurrentState actor in all systems?You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/godaddy/Thespian/issues/20#issuecomment-249240939>, or
mute the thread<
https://github.com/notifications/unsubscribe-auth/AFX0bhI3rO2P1KfWU9p_rqzl0TC_rjjmks5qtAAWgaJpZM4KElQy.
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABDhe13StcOROjlxPUGml0YE-mHJD9sks5qtBNlgaJpZM4KElQy
.
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/20#issuecomment-249295154, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFX0bgIt--EuLyI-_L_suEEsrxqul8yQks5qtDdEgaJpZM4KElQy.
from thespian.
Hi Kevin,
I've got Issue #21 open for addressing other convention topologies; with that issue still open, are you OK with closing this issue thread?
from thespian.
Sure, I'll follow the conversation over there.
from thespian.
Related Issues (20)
- Low performance sending low latency messages between actors
- Is this project still active? HOT 2
- MultiprocTCPBase cannot determine socket address when computer is offline
- Dynamically creating actors HOT 1
- Getting exceptions While Using "multiprocTCPBase" and logdefs in virtualenvironment HOT 7
- Why do I not see a reply message logging for these "Person" actors? HOT 2
- how to run thespian with an event loop HOT 1
- Error using multiprocQueueBase and multiprocTCPBase HOT 3
- Example http_server2.py doesn't work like described HOT 1
- questions: how can I avoid main thread exits when run actors with multiprocQueueBase HOT 9
- Problem with reaching an actor (from the outside) after a while HOT 5
- question: Is there any way to use/integrate external message broker ? (for example rabbitMQ) HOT 2
- self.createActor() with globalName from inisde an ActorTypeDispatcher increases size of self HOT 1
- Assorted Windows Hiccups HOT 1
- TCPTransport: multiple "dictionary changed size during iteration" issues
- multiprocTCPBase: Example keeps breaking on `InvalidActorSpecification` on my machine HOT 4
- Setting TCP_NODELAY on macOS sometimes fails HOT 9
- Dead letter handling with multiprocQueueBase implementation HOT 3
- MultiProcessQueue crashes on KeyboardInterrupt HOT 5
- `logdirector`'s "Dirty Trick" just bit me. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from thespian.