Add worker auto-respawning with max tries

[rauchg]

• Option to ensure that if a worker dies it tries to respawn immediately ?
• Option to ensure number of active workers always stays at numWorkers or above ?
• Option to ensure that there's always at least one worker alive and otherwise respawn ?
• When respawning, if worker fails to listen within certain timeout, retry ?
• Have a retry threshold to avoid overload ?
• When respawning, if worker dies shortly after listening, consider it an unsuccessful respawn and try ? This means it would still be captured by the retry threshold and you can't get into an endless cycle of considering the worker spawned, then dying shortly after listening, then triggering reload, ad infinitum.

[malkomalko]

Hi Guillermo,

First of all... really loving up. Right now when a worker dies there is no respawn.. is that correct? Is there any current strategy at ensuring workers stay alive? Maybe listen for a worker failure and send a kill -s SIGUSR2 to the master pid?

[rauchg]

Right now I have nothing for worker reloading I believe. We can start with the simplest implementation, but it quickly can get out of control if you don't cover many of the things I mention in this ticket. eg: crazy recursive/cyclic reloads that bring the server down because of faulty code.

[rauchg]

At the same time, maybe that problem needs to be solved at a different layer of abstraction.

[arohter]

See my comments in #21

[kevireilly]

It seems this has been integrated and documented, but has not yet been translated to the npm repository. Just a heads up.

[cmawhorter]

I think merging #60 will close this out.

• Option to ensure that if a worker dies it tries to respawn immediately ? Already Fixed
• Option to ensure number of active workers always stays at numWorkers or above ? Already Fixed
• Option to ensure that there's always at least one worker alive and otherwise respawn ? *Fixed by Exponentially backed-off respawns with maximum attempts #60 *
• When respawning, if worker fails to listen within certain timeout, retry ? *Fixed by Exponentially backed-off respawns with maximum attempts #60 *
• Have a retry threshold to avoid overload ? *Fixed by Exponentially backed-off respawns with maximum attempts #60 *
• When respawning, if worker dies shortly after listening, consider it an unsuccessful respawn and try ? This means it would still be captured by the retry threshold and you can't get into an endless cycle of considering the worker spawned, then dying shortly after listening, then triggering reload, ad infinitum. *Fixed by Exponentially backed-off respawns with maximum attempts #60 *