Advice: Actor State in Convention

[thespianpy]

Advice: Actor State in Convention

[kwquick]

Glad you are enjoying Thespian! We are enthusiastic about the potentials for Actor-based applications, so both suggestions and support for the effort are appreciated.

The mailing list might be a better place for these types of questions: more people are likely to see and respond, and the discussion is a little more searchable for future queries than a closed issue.

You might want to use the coordinator pattern. This pattern needs to be described in the documentation somewhere, but basically you have a top-level coordinator actor whose job is to start the Calc actor and provide that address to all clients. The coordinator's only responsibility is to create and provide the address of the current valid Calc actor on demand (and single-responsibility like this is another good Actor pattern).

def startup():
    asys = ActorSystem()
    asys.createActor(Coordinator, globalName='coordinator')

def client():
    asys = ActorSystem()
    calc = asys.ask(asys.createActor('coordinator', globalName='coordinator'), "get calc addr")
    result = asys.ask(calc, work())

class Coordinator(Actor):
    def __init__(self):
        self.calc_addr = None
    def receiveMessage(self, message, sender):
        if message == "get calc addr":
            if not self.calc_addr:
                self.calc_addr = self.createActor(Calc)
            return self.calc_addr
        elif isinstance(message, ChildActorExited):
            if message.childAddress == self.calc_addr:
                self.calc_addr = None

In the above (skeleton) code, the Coordinator simply returns the address of a Calc actor on demand, creating it if necessary. If it is told that the Calc actor has exited, it will re-create one on the next request.

The reason for using the Coordinator as the registered globalName actor instead of just referring to the Calc directly via a globalName is that there is no other entity to tell (i.e. send a ChildActorExited to) when a "top-level" actor (in this case, the Coordinator) has exited and therefore cause it to be restarted. By using a very simple top-level coordinator actor that is responsible for deploying the actual worker actors and keeping them around, the overall application gains a great deal of fault tolerance.

This clearly assumes that there is a "primary" system where the Coordinator lives and where the clients run to submit their requests. You can use requirements to ensure that Calc actors are created on (less long-lived) worker systems that use the primary system as the convention leader. If you need more tolerance for failures of the "primary" you could have a Coordinator running on each system and use an election-based system to determine which was the current managing Coordinator. (This is another reason for using the Coordinator: adding complexity to the management does not affect the "business logic" in the Calc actor---separation of concerns and single responsibility patterns at work).

One thing I would suggest would be to consider whether or not your Calc actor must be a singleton. A singleton is definitely a valid operational mode, but Actors lend themselves to easy parallelism... would your model support multiple Calc actors, with the coordination elsewhere (like your persistence actor, which is another good separation of concerns)? This would allow you to scale your application by using the Troupe or having multiple operational Calc actors on different nodes.

And a final note regarding Thespian maturity:

Right now, globalName only applies to the current ActorSystem, and the customary configuration of ActorSystems is a star pattern around that central or "primary" system. Future plans for Thespian include having the ability for multiple or self-electing convention leaders that would change the star topology into more of a mesh, and such a change would probably also involve making the globalName applicable inter-system instead of the current intra-system application.

Regards,
Kevin Quick

[kevincolyar]

Thanks for the advice. I'm kicking the Coordinator idea around in my head.

My concern is that when the "primary" convention leader system with the Coordination actor needs to be taken offline, the whole system will fail because the other actor system can't contact the Coordinator. Is this always the case when using a convention, the convention leader must alway be online?

I can't make the Calc run in parallel because the next calculation is dependent on the previous calculated state.

I don't suppose there is a way to broadcast the latest calculated state to a CurrentState actor in all systems?

[kwquick]

At the underlying thespian level, yes, the convention leader must always be present for the other convention members to function. That's the reference to the star configuration I made in the last post, and there is a plan to implement a leader election system in the future to remove this central dependency.

From the Coordinator perspective, you can enhance the coordinators communicating on the other systems to share information and handle the scenario when one leaves; you would need to implement your own leader election process for that. There is no broadcast, so you would need to pass Coordinator addresses to each other and perform explicit individual notifications.

To get around the thespian single-leader constraint above, you can use the preRegister/deRegister calls from the Coordinator to re-establish communications. This is going to get complicated though. The centralized convention leader has been the typical configuration that has been used in the past, and removing this constraint is definitely something that is needed; you are pushing the boundary of current implementation and if it's possible to live with the single leader configuration for now and wait for the Thespian enhancement that might be your better option.

If you want to get involved in enhancing Thespian itself in this area, that would be welcomed, but in this situation this is already on the roadmap so it will still happen even if you don't have the bandwidth to get personally involved (and probably sooner now that there is demand).

-Kevin Quick

---

From: Kevin Colyar [email protected]
Sent: Friday, September 23, 2016 9:36:38 AM
To: godaddy/Thespian
Cc: Kevin W Quick; Comment
Subject: Re: [godaddy/Thespian] Advice: Actor State in Convention (#20)

Thanks for the advice. I'm kicking the Coordinator idea around in my head.

My concern is that when the "primary" convention leader system with the Coordination actor needs to be taken offline, the whole system will fail because the other actor system can't contact the Coordinator. Is this always the case when using a convention, the convention leader must alway be online?

I can't make the Calc run in parallel because the next calculation is dependent on the previous calculated state.

I don't suppose there is a way to broadcast the latest calculated state to a CurrentState actor in all systems?

You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/20#issuecomment-249240939, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFX0bhI3rO2P1KfWU9p_rqzl0TC_rjjmks5qtAAWgaJpZM4KElQy.

[kwquick]

I have created #21 to specifically address the limitation of a single convention leader; feel free to upvote via a thumbs-up reaction to indicate level of interest or need for this.

[kevincolyar]

I'd love to contribute to Thespian any way I can. I was looking into Raft,
and I was wondering, at least for the time being, if using something like
PySyncObj https://github.com/bakwc/PySyncObj would solve my state issue
until a mesh convention is implemented.

I could hold a synchronized state outside of my ActorSystem in the clients
and use Thespian for parallel calculations, persistence, etc. Thoughts?

On Fri, Sep 23, 2016 at 10:59 AM Kevin Quick [email protected]
wrote:

At the underlying thespian level, yes, the convention leader must always
be present for the other convention members to function. That's the
reference to the star configuration I made in the last post, and there is a
plan to implement a leader election system in the future to remove this
central dependency.

From the Coordinator perspective, you can enhance the coordinators
communicating on the other systems to share information and handle the
scenario when one leaves; you would need to implement your own leader
election process for that. There is no broadcast, so you would need to pass
Coordinator addresses to each other and perform explicit individual
notifications.

To get around the thespian single-leader constraint above, you can use the
preRegister/deRegister calls from the Coordinator to re-establish
communications. This is going to get complicated though. The centralized
convention leader has been the typical configuration that has been used in
the past, and removing this constraint is definitely something that is
needed; you are pushing the boundary of current implementation and if it's
possible to live with the single leader configuration for now and wait for
the Thespian enhancement that might be your better option.

If you want to get involved in enhancing Thespian itself in this area,
that would be welcomed, but in this situation this is already on the
roadmap so it will still happen even if you don't have the bandwidth to get
personally involved (and probably sooner now that there is demand).

-Kevin Quick

---

From: Kevin Colyar [email protected]
Sent: Friday, September 23, 2016 9:36:38 AM
To: godaddy/Thespian
Cc: Kevin W Quick; Comment
Subject: Re: [godaddy/Thespian] Advice: Actor State in Convention (#20)

Thanks for the advice. I'm kicking the Coordinator idea around in my head.

My concern is that when the "primary" convention leader system with the
Coordination actor needs to be taken offline, the whole system will fail
because the other actor system can't contact the Coordinator. Is this
always the case when using a convention, the convention leader must alway
be online?

I can't make the Calc run in parallel because the next calculation is
dependent on the previous calculated state.

I don't suppose there is a way to broadcast the latest calculated state to
a CurrentState actor in all systems?

You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/godaddy/Thespian/issues/20#issuecomment-249240939>, or
mute the thread<
https://github.com/notifications/unsubscribe-auth/AFX0bhI3rO2P1KfWU9p_rqzl0TC_rjjmks5qtAAWgaJpZM4KElQy

.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABDhe13StcOROjlxPUGml0YE-mHJD9sks5qtBNlgaJpZM4KElQy
.

[kwquick]

That sounds like a reasonable approach. Obviously you'd have duplicated addressing/configuration between Thespian and PySyncObj (thanks for the interesting link there...) but that shouldn't be too hard to manage until Thespian supports mesh.

Feel free to reach out to me directly to coordinate any Thespian work you want to get involved in (and thanks!).

-Kevin Quick

---

From: Kevin Colyar [email protected]
Sent: Friday, September 23, 2016 1:32:04 PM
To: godaddy/Thespian
Cc: Kevin W Quick; Comment
Subject: Re: [godaddy/Thespian] Advice: Actor State in Convention (#20)

I'd love to contribute to Thespian any way I can. I was looking into Raft,
and I was wondering, at least for the time being, if using something like
PySyncObj https://github.com/bakwc/PySyncObj would solve my state issue
until a mesh convention is implemented.

I could hold a synchronized state outside of my ActorSystem in the clients
and use Thespian for parallel calculations, persistence, etc. Thoughts?

On Fri, Sep 23, 2016 at 10:59 AM Kevin Quick [email protected]
wrote:

At the underlying thespian level, yes, the convention leader must always
be present for the other convention members to function. That's the
reference to the star configuration I made in the last post, and there is a
plan to implement a leader election system in the future to remove this
central dependency.

From the Coordinator perspective, you can enhance the coordinators
communicating on the other systems to share information and handle the
scenario when one leaves; you would need to implement your own leader
election process for that. There is no broadcast, so you would need to pass
Coordinator addresses to each other and perform explicit individual
notifications.

To get around the thespian single-leader constraint above, you can use the
preRegister/deRegister calls from the Coordinator to re-establish
communications. This is going to get complicated though. The centralized
convention leader has been the typical configuration that has been used in
the past, and removing this constraint is definitely something that is
needed; you are pushing the boundary of current implementation and if it's
possible to live with the single leader configuration for now and wait for
the Thespian enhancement that might be your better option.

If you want to get involved in enhancing Thespian itself in this area,
that would be welcomed, but in this situation this is already on the
roadmap so it will still happen even if you don't have the bandwidth to get
personally involved (and probably sooner now that there is demand).

-Kevin Quick

---

From: Kevin Colyar [email protected]
Sent: Friday, September 23, 2016 9:36:38 AM
To: godaddy/Thespian
Cc: Kevin W Quick; Comment
Subject: Re: [godaddy/Thespian] Advice: Actor State in Convention (#20)

Thanks for the advice. I'm kicking the Coordinator idea around in my head.

My concern is that when the "primary" convention leader system with the
Coordination actor needs to be taken offline, the whole system will fail
because the other actor system can't contact the Coordinator. Is this
always the case when using a convention, the convention leader must alway
be online?

I can't make the Calc run in parallel because the next calculation is
dependent on the previous calculated state.

I don't suppose there is a way to broadcast the latest calculated state to
a CurrentState actor in all systems?

You are receiving this because you commented.
Reply to this email directly, view it on GitHub<
https://github.com/godaddy/Thespian/issues/20#issuecomment-249240939>, or
mute the thread<
https://github.com/notifications/unsubscribe-auth/AFX0bhI3rO2P1KfWU9p_rqzl0TC_rjjmks5qtAAWgaJpZM4KElQy

.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABDhe13StcOROjlxPUGml0YE-mHJD9sks5qtBNlgaJpZM4KElQy
.

You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/20#issuecomment-249295154, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFX0bgIt--EuLyI-_L_suEEsrxqul8yQks5qtDdEgaJpZM4KElQy.

[kwquick]

Hi Kevin,

I've got Issue #21 open for addressing other convention topologies; with that issue still open, are you OK with closing this issue thread?

[kevincolyar]

Sure, I'll follow the conversation over there.