gisce/autoworker

Spawn RQ Workers automatically

autoworker's Issues

Config workers from Environment AUTOWORKER_XXX

Exception is raised if max_procs > MAX_PROCS

The code that manages the max processes fails if max_procs > MAX_PROCS:

autoworker/autoworker/__init__.py

Lines 63 to 68 in a91d8ce

	if max_procs is None:
	self.max_procs = MAX_PROCS
	elif 1 <= max_procs < MAX_PROCS + 1:
	self.max_procs = max_procs
	else:
	raise ValueError('Max procs {} not supported'.format(max_procs))

But this is an strange behaviour, because:

• The variable is named max_procs, not min_procs. With something like min_procs, it would be logical to fail if the processor can't allocate that minimum processes. But if I say "i want 4 processes maximum" and there is only 2 processes available, just use 2 processes, it respects my 4 max_procs parameter.
• Also, the MAX_PROCS constant depends on the CPU load, what difficults to define correctly max_procs from outside because the CPU load can change and make the algorithm fail.

So my proposal is to just log a warning if max_procs > MAX_PROCS (something like "Only 2 processes where used instead of the desired 4 maximum processes), and just use min(max_procs, MAX_PROCS).