Spawning worker(s) at app start

[cocaine]

Spawning worker(s) at app start

[oktocat]

Hi.

Spawning new worker may be an issue only on cold-start of app, because if you already have some workload it means that you already have enough workers to handle that workload. Cold-start is rare situation so we don't really think that's an issue. And on the other hand, having some amount of idle workers is unnecessary spending of resources (i.e memory and connections). Also, we don't know and can't know how to calculate the right number of pre-spawned app workers without real workload, and it can not be configured via profile, because that number should depend on workload. Maybe sometimes in the future it will be done with some heuristic, but not with simple worker pre-spawning.

About second issue. What's the point of having app workers being idle?

[mwf]

Spawning new worker may be an issue only on cold-start of app, because if you already have some workload it means that you already have enough workers to handle that workload. Cold-start is rare situation so we don't really think that's an issue.

Cold-start is rare only if you HAVE some "constant" workload =) And if it's rather instable, first-worker starts will occur from time to time... For someone they may be a little annoying.

This

And on the other hand, having some amount of idle workers is unnecessary spending of resources (i.e memory and connections).

and this

Also, we don't know and can't know how to calculate the right number of pre-spawned app workers without real workload, and it can not be configured via profile, because that number should depend on workload.

is admin's business, I think... If he really needs more smooth app-start - he will be able to have it with this feature =) You give the opportunity - admin decides whether to use it, you see?

About second issue. What's the point of having app workers being idle?

I don't know what's the point to have workers idle forever. But it was your team who made this feature =)
It's just the same thoughts about opportunities. You've given a way to start non-dying workers through this config param - then it's rather obvious to give a way of pre-spawning this forever-living workers.

This all is not a kind of some important feature, it's just my thoughts I've decided to share with you. I've decided, maybe they can be useful somehow =)

[oktocat]

Oh, I missed that somehow.

But it was your team who made this feature

Frankly, it looks more like a bug than a feature : ) I guess, we'll fix it in the nearest future.
Yes, @noxiouz ?

[kobolog]

Hi.

Immortal workers feature was added for some reason, but nobody remembers what reason it was, so we'll probably drop it, yeah. We are planning to rework the scheduler, and the current implementation ideas for that new scheduler do not go well with pre-spawned or immortal workers, mainly because we plan to migrate workers around the cluster to utilize resources more effectively, which kind of kills the 'always ready' part of the benefits of having a pre-spawned worker pool, and I don't see any other benefits anyway.