Dynamically adding a pool

[epgsql]

Dynamically adding a pool

[seth]

Hi Murali,

There is not currently a way to dynamically add a pool, but I agree it is a useful feature and there is no reason not to add it.

I'm working on a somewhat extensive set of improvements currently staged in the multi-pool and async-start branches. Adding dynamic pool creation would fit well there.

I'd love to have some details, even if only vague, on how you are using pooler. If public comments aren't a good forum for you, feel free to drop me an email seth at userprimary dot net.

[kbmurali]

Hi Seth,

Thanks for prompt response.

I have used your pooler for RIAK connections and it worked well when I knew the pool configuration in advance. I have a context where I would need to create a pool of ports that spawn C program at run time. The requirement is to create the pool on some user event that defines the C program to be spawned and number of port instances. The issue is that I have pooler already running for RIAK connections in that same application and hence I cannot dynamically start it any more with C port pool config. It would have been nice if I could have spawned yet another pooler gen_server instance with a different id.

One another observation is that 1 pooler managing multiple pools is okay however under volumes, there may be 'take_member' messages queuing up in the pooler mail box adding minor latencies - perhaps ability to spawn separate pooler instances may alleviate such problem as well. Just a thought.

I really appreciate you have made your implementation available via github and thank you for pretty-good implementation.

Thanks,

Murali

[seth]

Thanks for the extra info. When using pooler with Riak, do you use the random load balancing in pooler?

Good observations about multiple pools. In the branch I mentioned, I have things refactored such that each pool gets its own gen_server to reduce contention under load. Makes much more sense once you start managing unrelated pools of things.

[kbmurali]

I am using the default load balancing strategy. Did not specify the load balancing scheme explicitly.

Do you recommend any specific scheme?

[seth]

Hi there,

Wanted to let you know that I've just added the ability to dynamically add pools to the feature branch pooler-1-0. This branch has a number of other enhancements and a few changes that you will need to take into account to make use of it (see the NEWS file on that branch). I'm planning to merge this branch to master in a week or two. Any feedback is of course appreciated.

[kbmurali]

Thanks Seth for promptly incorporating this feature. I will play with the new build shortly and will get back to you with any further questions. Appreciate your timely action on this.

[seth]

The new code is now on master, including pooler:new_pool. There are examples of its use in the tests and readme. If you run into troubles, please open a new issue. I'm going to close this one for now. Thanks again for sending suggestions.