Comments (5)
cburroughs
commented on July 24, 2024
we think/hope that this is fixed by #66
from monitor-core.
cburroughs
commented on July 24, 2024
Unfortunately, our hopes have been dashed. We are currently seeing this about once per week (which I think ends up being something like 1 in 201,600 polls gone bad).
After much digging with pstack/strace I found what I think was a suspicious blocked thread in the stack trace, but some errant debugging command killed gmetad and I lost it in a less buffer. If there is anything besides pstrack I should capture next time please let me know.
from monitor-core.
cburroughs
commented on July 24, 2024
So two clues. We got the following error at about the same time as the thread got 'stuck'
Apr 28 23:59:02 asu0d /usr/sbin/gmetad[15101]: Process XML (IAD Tiggr): XML_ParseBuffer() error at line 48299: no element found
I've verified with xmllint that all of the responses are currently valid, and a 2nd gmetad polling this set of gmond's did not have the same error (although I can't rule out some transient error at this time). I've attachd some stack traces below to see how things change over time, but I am hopeful the XML parsing error is the 'smoking gun'.
Stack 0:
Thread 15 (Thread 0x410c9940 (LWP 15102)):
#0 0x0000003653a0db9b in accept () from /lib64/libpthread.so.0
#1 0x0000000000405488 in pthread_attr_setdetachstate ()
#2 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#3 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 14 (Thread 0x41aca940 (LWP 15103)):
#0 0x0000003653a0d8fb in write () from /lib64/libpthread.so.0
#1 0x000000000040486f in pthread_attr_setdetachstate ()
#2 0x0000000000404f6e in pthread_attr_setdetachstate ()
#3 0x0000000000404c2d in pthread_attr_setdetachstate ()
#4 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#5 0x0000000000404c71 in pthread_attr_setdetachstate ()
#6 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#7 0x0000000000404c71 in pthread_attr_setdetachstate ()
#8 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#9 0x0000000000404b60 in pthread_attr_setdetachstate ()
#10 0x0000000000404a3d in pthread_attr_setdetachstate ()
#11 0x0000000000405588 in pthread_attr_setdetachstate ()
#12 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#13 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 13 (Thread 0x424cb940 (LWP 15104)):
#0 0x0000003653a0db9b in accept () from /lib64/libpthread.so.0
#1 0x0000000000405324 in pthread_attr_setdetachstate ()
#2 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#3 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 12 (Thread 0x42ecc940 (LWP 15105)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 11 (Thread 0x438cd940 (LWP 15106)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 10 (Thread 0x442ce940 (LWP 15107)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 9 (Thread 0x44ccf940 (LWP 15108)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x456d0940 (LWP 15109)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x460d1940 (LWP 15110)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x46ad2940 (LWP 15111)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000406d9e in pthread_attr_setdetachstate ()
#4 0x000000365be09bc9 in ?? () from /lib64/libexpat.so.0
#5 0x000000365be0ab44 in ?? () from /lib64/libexpat.so.0
#6 0x000000365be0b66a in ?? () from /lib64/libexpat.so.0
#7 0x000000365be0cc4b in ?? () from /lib64/libexpat.so.0
#8 0x000000365be03ef1 in XML_ParseBuffer () from /lib64/libexpat.so.0
#9 0x0000000000405920 in pthread_attr_setdetachstate ()
#10 0x0000000000404522 in pthread_attr_setdetachstate ()
#11 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#12 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x474d3940 (LWP 15113)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x47ed4940 (LWP 15114)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x488d5940 (LWP 15115)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x492d6940 (LWP 15116)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x00000000004091b7 in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x2abacc9d42f0 (LWP 15101)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000403348 in pthread_attr_setdetachstate ()
#4 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#5 0x00000000004030ca in pthread_attr_setdetachstate ()
#6 0x000000365321d9b4 in __libc_start_main () from /lib64/libc.so.6
#7 0x0000000000402b29 in pthread_attr_setdetachstate ()
#8 0x00007ffffb3e34b8 in ?? ()
#9 0x0000000000000000 in ?? ()
Stack 1:
Thread 15 (Thread 0x410c9940 (LWP 15102)):
#0 0x0000003653a0db9b in accept () from /lib64/libpthread.so.0
#1 0x0000000000405488 in pthread_attr_setdetachstate ()
#2 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#3 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 14 (Thread 0x41aca940 (LWP 15103)):
#0 0x0000003653a0d8fb in write () from /lib64/libpthread.so.0
#1 0x000000000040486f in pthread_attr_setdetachstate ()
#2 0x0000000000404f6e in pthread_attr_setdetachstate ()
#3 0x0000000000404c2d in pthread_attr_setdetachstate ()
#4 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#5 0x0000000000404c71 in pthread_attr_setdetachstate ()
#6 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#7 0x0000000000404c71 in pthread_attr_setdetachstate ()
#8 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#9 0x0000000000404b60 in pthread_attr_setdetachstate ()
#10 0x0000000000404a3d in pthread_attr_setdetachstate ()
#11 0x0000000000405588 in pthread_attr_setdetachstate ()
#12 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#13 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 13 (Thread 0x424cb940 (LWP 15104)):
#0 0x0000003653a0db9b in accept () from /lib64/libpthread.so.0
#1 0x0000000000405324 in pthread_attr_setdetachstate ()
#2 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#3 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 12 (Thread 0x42ecc940 (LWP 15105)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 11 (Thread 0x438cd940 (LWP 15106)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 10 (Thread 0x442ce940 (LWP 15107)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 9 (Thread 0x44ccf940 (LWP 15108)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x456d0940 (LWP 15109)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x460d1940 (LWP 15110)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x46ad2940 (LWP 15111)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000406d9e in pthread_attr_setdetachstate ()
#4 0x000000365be09bc9 in ?? () from /lib64/libexpat.so.0
#5 0x000000365be0ab44 in ?? () from /lib64/libexpat.so.0
#6 0x000000365be0b66a in ?? () from /lib64/libexpat.so.0
#7 0x000000365be0cc4b in ?? () from /lib64/libexpat.so.0
#8 0x000000365be03ef1 in XML_ParseBuffer () from /lib64/libexpat.so.0
#9 0x0000000000405920 in pthread_attr_setdetachstate ()
#10 0x0000000000404522 in pthread_attr_setdetachstate ()
#11 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#12 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x474d3940 (LWP 15113)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x47ed4940 (LWP 15114)):
#0 0x0000003653a0aee9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x0000003aa320c68b in pthread_rdwr_wlock_np () from /usr/lib64/libganglia-3.5.0.so.0
#2 0x0000003aa320ba3c in hash_insert () from /usr/lib64/libganglia-3.5.0.so.0
#3 0x0000000000405ab5 in pthread_attr_setdetachstate ()
#4 0x000000365be09e85 in ?? () from /lib64/libexpat.so.0
#5 0x000000365be0ab44 in ?? () from /lib64/libexpat.so.0
#6 0x000000365be0b66a in ?? () from /lib64/libexpat.so.0
#7 0x000000365be0cc4b in ?? () from /lib64/libexpat.so.0
#8 0x000000365be03ef1 in XML_ParseBuffer () from /lib64/libexpat.so.0
#9 0x0000000000405920 in pthread_attr_setdetachstate ()
#10 0x0000000000404522 in pthread_attr_setdetachstate ()
#11 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#12 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x488d5940 (LWP 15115)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x492d6940 (LWP 15116)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x00000000004091b7 in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x2abacc9d42f0 (LWP 15101)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000403348 in pthread_attr_setdetachstate ()
#4 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#5 0x00000000004030ca in pthread_attr_setdetachstate ()
#6 0x000000365321d9b4 in __libc_start_main () from /lib64/libc.so.6
#7 0x0000000000402b29 in pthread_attr_setdetachstate ()
#8 0x00007ffffb3e34b8 in ?? ()
#9 0x0000000000000000 in ?? ()
Stack 2:
Thread 15 (Thread 0x410c9940 (LWP 15102)):
#0 0x0000003653a0d8fb in write () from /lib64/libpthread.so.0
#1 0x000000000040486f in pthread_attr_setdetachstate ()
#2 0x0000000000404f6e in pthread_attr_setdetachstate ()
#3 0x0000000000404c2d in pthread_attr_setdetachstate ()
#4 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#5 0x0000000000404c71 in pthread_attr_setdetachstate ()
#6 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#7 0x0000000000404c71 in pthread_attr_setdetachstate ()
#8 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#9 0x0000000000404b60 in pthread_attr_setdetachstate ()
#10 0x0000000000404a3d in pthread_attr_setdetachstate ()
#11 0x0000000000405588 in pthread_attr_setdetachstate ()
#12 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#13 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 14 (Thread 0x41aca940 (LWP 15103)):
#0 0x0000003653a0db9b in accept () from /lib64/libpthread.so.0
#1 0x0000000000405488 in pthread_attr_setdetachstate ()
#2 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#3 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 13 (Thread 0x424cb940 (LWP 15104)):
#0 0x0000003653a0db9b in accept () from /lib64/libpthread.so.0
#1 0x0000000000405324 in pthread_attr_setdetachstate ()
#2 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#3 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 12 (Thread 0x42ecc940 (LWP 15105)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 11 (Thread 0x438cd940 (LWP 15106)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 10 (Thread 0x442ce940 (LWP 15107)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x000000000040530f in pthread_attr_setdetachstate ()
#4 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#5 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 9 (Thread 0x44ccf940 (LWP 15108)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 8 (Thread 0x456d0940 (LWP 15109)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x460d1940 (LWP 15110)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x46ad2940 (LWP 15111)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000406d9e in pthread_attr_setdetachstate ()
#4 0x000000365be09bc9 in ?? () from /lib64/libexpat.so.0
#5 0x000000365be0ab44 in ?? () from /lib64/libexpat.so.0
#6 0x000000365be0b66a in ?? () from /lib64/libexpat.so.0
#7 0x000000365be0cc4b in ?? () from /lib64/libexpat.so.0
#8 0x000000365be03ef1 in XML_ParseBuffer () from /lib64/libexpat.so.0
#9 0x0000000000405920 in pthread_attr_setdetachstate ()
#10 0x0000000000404522 in pthread_attr_setdetachstate ()
#11 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#12 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x474d3940 (LWP 15113)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x47ed4940 (LWP 15114)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x488d5940 (LWP 15115)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x000000000040440e in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x492d6940 (LWP 15116)):
#0 0x00000036532cd7b2 in select () from /lib64/libc.so.6
#1 0x00000038b1c1f915 in apr_sleep () from /usr/lib64/libapr-1.so.0
#2 0x00000000004091b7 in pthread_attr_setdetachstate ()
#3 0x0000003653a0673d in start_thread () from /lib64/libpthread.so.0
#4 0x00000036532d456d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x2abacc9d42f0 (LWP 15101)):
#0 0x0000003653a0d524 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003653a08e1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003653a08cdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000403348 in pthread_attr_setdetachstate ()
#4 0x0000003aa320b558 in hash_foreach () from /usr/lib64/libganglia-3.5.0.so.0
#5 0x00000000004030ca in pthread_attr_setdetachstate ()
#6 0x000000365321d9b4 in __libc_start_main () from /lib64/libc.so.6
#7 0x0000000000402b29 in pthread_attr_setdetachstate ()
#8 0x00007ffffb3e34b8 in ?? ()
#9 0x0000000000000000 in ?? ()
from monitor-core.
cburroughs
commented on July 24, 2024
Possibly related: http://www.mail-archive.com/[email protected]/msg05063.html
from monitor-core.
cburroughs
commented on July 24, 2024
Possibly related: https://sourceforge.net/p/ganglia/mailman/message/34851974/
from monitor-core.
Related Issues (20)
- Does the gmond support dynamic python module updating?
- error on the wiki page for value_treshhold
- ganglia-config should not return *all* CFLAGS/Fully support alternate SunRPC implementation like libtirpc
- bus error is occuring on gmetad on RHEL7 (built from "master")
- Need unique names for many small clusters in unicast send-only configuration
- with gmond 3.7.1 ，cannot create udp client if band_hostname is on.
- Ganglia Missing Metrics for Some Nodes
- Links in Wiki broken
- Migrate LGTM.com installation from OAuth to GitHub App
- does it plan to have a new release?
- Anybody built a working RPM for CentOS 8? HOT 1
- Small values bytes_in/bytes_out after week work
- memory leak
- Questions about abnormalities in the passive close of gmond HOT 1
- gmetad 3.7.2 service causes IO hangs HOT 1
- gmond/ganglia not starting with systemd HOT 2
- python_modules not installed
- trouble，what can i do? thank you for help! HOT 1
- undeclared functions in libmetrics/darwin/metrics.c
- default conf file missing in ganglia-monitor 3.7.2-6 (in Ubuntu 22)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from monitor-core.