Comments (1)
Hi @gerowam , thanks for the report and the corresponding analysis. I can confirm that I see just over 800 bytes of consumption for each createActor()
call with a globalName
specified.
The increased memory consumption is a necessity of the distributed nature of the Actor environment. In concordance with your concern about storing the global address locally, the Thespian implementation must check with the Admin each time a "new" address is requested, and there must be local storage associated with this new address; any attempt to locally cache that information would suffer the same out-of-date issue you noted.
Probably a better way to address this situation is to have an Actor that provides management capabilities for other Actors (let's call this the "Maker" actor in the following discussion, and the term "Global" refers to the corresponding global actor, while the term "Client" refers to an actor wishing to communicate with the global actor). In this configuration, the Thespian globalName
is not necessarily needed, and the system will operate as described below:
- Client needs to communicate to Global, so Client issues a (user-defined) "GetGlobalAddress" message to Maker. In this situation, Maker may have a
globalName
that identifies it, but since the functionality provided by Maker is very limited, it is fairly safe and not expected to crash or otherwise fail, thus the client can issue a singlecreateActor(Maker, globalName = "maker")
call and save the resulting address onself
. - When Maker receives the "GetGlobalAddress", it checks an internal dictionary (on
self
) to see if it has an address for Global. Initially, it does not, so it will then issue acreateActor(Global)
call (here noglobalName
is needed). It then adds the address obtained from thatcreateActor
to the internal dictionary and then returns the address to the Client in the response to the "GetGlobalAddress" message. - The Maker also records in another
self
dictionary the address of each Client that it gave the global address to. - Subsequent Client "GetGlobalAddress" calls can be handled directly by the Maker from the main
self
dictionary. - If the Global Actor crashes or otherwise exits, the Maker will handle the
ChildActorExited
message by removing theself
dictionary entry containing the address of Global. It may optionally send a (user-defined) "GlobalAddressNoLongerValid" message to the Client actors that were registered in the secondaryself
dictionary; this is optional but can make recovery faster, since this tells the Client actors that they should also flush any local copies of the Global actor address they are maintaining. - The next time a Client issues a "GetGlobalAddress" to the Maker, the
self
dictionary will not contain an address for Global, so it will re-perform thecreateActor
process for Global to get a new instance. - If the Client Actors have cached the Global actor address, and the "GlobalAddressNoLongerValid" is not used, the clients will eventually receive a
PoisonMessage
on their attempts to send to the (no longer running) Global. At that point they should discard their local address for Global and re-issue a "GetGlobalAddress" to Maker.
In the above scenario, Maker is a very simple implementation, but since Maker also acts as the parent actor for Global (or any number of Global actors), it will actively know when Global exits and another version needs to be created. The Client actors require a slightly longer protocol to get the address of Global from Maker, but they don't need any additional complexity. In this scenario, the growth of memory needed for actor address management should be limited to only those cases where the Global actor exits and needs to be re-created.
from thespian.
Related Issues (20)
- Low performance sending low latency messages between actors
- Is this project still active? HOT 2
- MultiprocTCPBase cannot determine socket address when computer is offline
- Dynamically creating actors HOT 1
- Getting exceptions While Using "multiprocTCPBase" and logdefs in virtualenvironment HOT 7
- Why do I not see a reply message logging for these "Person" actors? HOT 2
- how to run thespian with an event loop HOT 1
- Error using multiprocQueueBase and multiprocTCPBase HOT 3
- Example http_server2.py doesn't work like described HOT 1
- questions: how can I avoid main thread exits when run actors with multiprocQueueBase HOT 9
- Problem with reaching an actor (from the outside) after a while HOT 5
- question: Is there any way to use/integrate external message broker ? (for example rabbitMQ) HOT 2
- Assorted Windows Hiccups HOT 1
- TCPTransport: multiple "dictionary changed size during iteration" issues
- multiprocTCPBase: Example keeps breaking on `InvalidActorSpecification` on my machine HOT 4
- Setting TCP_NODELAY on macOS sometimes fails HOT 9
- Dead letter handling with multiprocQueueBase implementation HOT 3
- MultiProcessQueue crashes on KeyboardInterrupt HOT 5
- `logdirector`'s "Dirty Trick" just bit me. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from thespian.