PlannerThreadedTerminationCondition locking problem causes RRT* planner to freeze while improving solution

[ompl]

PlannerThreadedTerminationCondition locking problem causes RRT* planner to freeze while improving solution

[mamoll]

Original comment by Antti Valli (Bitbucket: avalli, GitHub: avalli).

---

Forgot the stacktrace and now I can't attach it for some reason. So here it is as a comment.

Attaching to process 27473
(gdb) bt
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
#1  0x00007f7035417065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00007f7035416eba in __pthread_mutex_lock (mutex=0x1b83798)
    at pthread_mutex_lock.c:61
#3  0x00007f70311b2172 in boost::thread::interrupt() ()
   from /usr/lib/libboost_thread.so.1.46.1
#4  0x00007f703713eacf in ompl::base::PlannerThreadedTerminationCondition::stopEvalThread() () from /usr/local/lib/libompl.so.6
#5  0x00007f703713eb2f in ompl::base::PlannerThreadedTerminationCondition::~PlannerThreadedTerminationCondition() () from /usr/local/lib/libompl.so.6
#6  0x00007f70371426c3 in ompl::base::Planner::solve(double) ()
   from /usr/local/lib/libompl.so.6
#7  0x00007f70371de784 in ompl::geometric::SimpleSetup::solve(double) ()
   from /usr/local/lib/libompl.so.6
#8  0x0000000000445b57 in MotionSolver::improve (this=0x7ffff5e84630)
    at MotionSolver.cpp:327
#9  0x000000000042a6c5 in main (argc=1, argv=0x7ffff5e84af8) at PlannerTest.cpp:90
(gdb) frame 2
#2  0x00007f7035416eba in __pthread_mutex_lock (mutex=0x1b83798)
    at pthread_mutex_lock.c:61
61	pthread_mutex_lock.c: No such file or directory.
(gdb) print *mutex
$2 = {__data = {__lock = 2, __count = 0, __owner = 7237, __nusers = 1, __kind = 0, 
    __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, 
  __size = "\002\000\000\000\000\000\000\000E\034\000\000\001", '\000' <repeats 26 times>, __align = 2}
(gdb) info threads
  Id   Target Id         Frame 
  2    Thread 0x7f702d0ac700 (LWP 7237) "PlannerTest" __lll_lock_wait ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
* 1    Thread 0x7f7038224740 (LWP 27473) "PlannerTest" __lll_lock_wait ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
(gdb) thread 2
[Switching to thread 2 (Thread 0x7f702d0ac700 (LWP 7237))]
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
132	../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: No such file or directory.
(gdb) bt
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:132
#1  0x00007f7035417065 in _L_lock_858 () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00007f7035416eba in __pthread_mutex_lock (mutex=0x1b836f0)
    at pthread_mutex_lock.c:61
#3  0x00007f70311b24a4 in boost::this_thread::interruption_point() ()
   from /usr/lib/libboost_thread.so.1.46.1
#4  0x00007f70311b445e in boost::this_thread::sleep(boost::posix_time::ptime const&)
    () from /usr/lib/libboost_thread.so.1.46.1
#5  0x00007f703713fee6 in ompl::base::PlannerThreadedTerminationCondition::periodicEval() () from /usr/local/lib/libompl.so.6
#6  0x00007f70311b1ce9 in thread_proxy () from /usr/lib/libboost_thread.so.1.46.1
#7  0x00007f7037b31b74 in ?? () from /usr/lib/nvidia-current/libGL.so.1
#8  0x00007f7035414e9a in start_thread (arg=0x7f702d0ac700) at pthread_create.c:308
#9  0x00007f7035a19cbd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#10 0x0000000000000000 in ?? ()

[mamoll]

Original comment by Ioan Sucan (Bitbucket: isucan, GitHub: isucan).

---

What version of OMPL are you using? Does the problem happen for every execution of the algorithm? How is RRT* configured to improve the solution (how is the optimization objective set up)?
Are you able to send a bit of code that exhibits this problem?

[mamoll]

Original comment by Antti Valli (Bitbucket: avalli, GitHub: avalli).

---

The problem first appeared with ompl-0.11.1 and updating to 0.12.2 did not resolve it. The problem randomly appears for some executions of the algorithm, but often enough that it affects every longer benchmarking run that I have done. BTRRT* does not appear to suffer from this problem. The planner is set up with ompl::geometric::SimpleSetup and delay_cc is set to 1. The configuration space is 6 dimensional CompoundStateSpace, with SO2 and RealVector statespaces as components. The solve method of simple setup is called repeatedly until a specified minimum improvement time has been used.

[mamoll]

Original comment by Ioan Sucan (Bitbucket: isucan, GitHub: isucan).

---

Sorry for the delayed reply -- somehow I did not catch your answer. I suspect this could be a numerical precision issue. This has been fixed in the source code. Would it be possible for you to try this using the latest source code?

[mamoll]

Original comment by Mark Moll (Bitbucket: mamoll, GitHub: mamoll).

---

See 7ea0c0172dc1.

[mamoll]

Original comment by Ioan Sucan (Bitbucket: isucan, GitHub: isucan).

---

I can confirm this is an issue with the termination condition. I will commit a fix shortly

[mamoll]

Original comment by Ioan Sucan (Bitbucket: isucan, GitHub: isucan).

---

this should be now fixed