A panicking extension can sometimes cause a double panic and bring down the db

[ethransom]

A panicking extension can sometimes cause a double panic and bring down the db

[ankitbhrdwj]

The reason the process was crashing with multiple panics was the following code in Rust panic handler -

if panics > 1 {
        util::dumb_print(format_args!("thread panicked while panicking. \
                                       aborting.\n"));
        unsafe { intrinsics::abort() }
}

https://github.com/rust-lang/rust/blob/master/src/libstd/panicking.rs

The current solution is to check the response for the generator executing with the help of catch_unwind in the container and if the std::thread::panicking() is true then put the scheduler in an infinite loop.

if let Err(_) = res {
    self.state = COMPLETED;
    if thread::panicking() {
        if let Some((req, res)) = self.tear() {
            req.free_packet();
            res.free_packet();
        }
        loop {}
    }
}

The monitoring core will realize that this thread is taking a lot of time in current extension and will put this thread on the bad core and the process will never crash.

However, the limitation with this approach is that after the /proc/sys/kernel/threads-max number of thread Splinter won't be able to spawn any more threads; value for which is 382024 on our current machine.