segfault on startup when approximately10 or more iOS devices connected

[libimobiledevice]

segfault on startup when approximately10 or more iOS devices connected

[RossBencina]

free(): invalid pointer issue could be related to double-free fixed by this PR:
#41

possibly related to these issues:
#40
#39

[RossBencina]

I have retested the latest git HEAD (of all of usbmuxd, libusbmuxd, libplist, libimobiledevice) with the following PR's applied:
https://github.com/libimobiledevice/usbmuxd/pull/41/files
https://github.com/libimobiledevice/libplist/pull/64/files
https://github.com/libimobiledevice/libplist/pull/39/files

These do not resolve the issues.

During 5 test runs, with 9 - 27 devices connected, usbmuxd always crashed during startup. There were 4 distinct errors:

• 2 x segfault inside libc.so.6 (no additional info in stack trace)
• Memory heap corruption detected by glibc (buffer overrun in heap block?)
• segfault in xmlCharEncOutFunc() called from preflight/lockdown plist handling
• segfault in xmlCleanupCharEncodingHandlers() called from preflight/lockdown plist handling

I'll post the stack traces below. I have the output of usbmuxd -vvv that lead up to the crashes if anyone is interested.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x41e52470 (LWP 4019)]
0x400b225c in ?? () from /lib/arm-linux-gnueabi/libc.so.6
(gdb) bt
#0  0x400b225c in ?? () from /lib/arm-linux-gnueabi/libc.so.6
#1  0x00000000 in ?? ()

*** glibc detected *** /usr/local/sbin/usbmuxd: corrupted double-linked list: 0x00029d50 ***
Program received signal SIGABRT, Aborted.
0x400c5a98 in raise () from /lib/arm-linux-gnueabi/libc.so.6
(gdb) bt
#0  0x400c5a98 in raise () from /lib/arm-linux-gnueabi/libc.so.6
#1  0x400c9a08 in abort () from /lib/arm-linux-gnueabi/libc.so.6
#2  0x400c9a08 in abort () from /lib/arm-linux-gnueabi/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x427ff470 (LWP 4069)]
0x00000002 in ?? ()
(gdb) bt
#0  0x00000002 in ?? ()
#1  0x403cfd20 in xmlCharEncOutFunc ()
   from /usr/lib/arm-linux-gnueabi/libxml2.so.2
#2  0x403ffb5c in xmlOutputBufferFlush ()
   from /usr/lib/arm-linux-gnueabi/libxml2.so.2
#3  0x404ac668 in xmlDocDumpFormatMemoryEnc ()
   from /usr/lib/arm-linux-gnueabi/libxml2.so.2
#4  0x404ac73c in xmlDocDumpMemory ()
   from /usr/lib/arm-linux-gnueabi/libxml2.so.2
#5  0x4006d1b4 in plist_to_xml (plist=0x297e0, plist_xml=0x427fed38, 
    length=0x427fed3c) at xplist.c:543
#6  0x4004fbe0 in internal_plist_send (client=0x29ce0, plist=<optimized out>, 
    binary=<optimized out>) at property_list_service.c:119
#7  0x40050c78 in lockdownd_send (client=<optimized out>, 
    plist=<optimized out>) at lockdown.c:374
#8  0x40051a48 in lockdownd_set_value (client=0x29198, domain=<optimized out>, 
    key=<optimized out>, value=<optimized out>) at lockdown.c:501
#9  0x0000f6a8 in lockdownd_set_untrusted_host_buid (lockdown=0x29198)
    at preflight.c:67
#10 0x0000f81c in preflight_worker_handle_device_add (userdata=0x26830)
    at preflight.c:219
#11 0x40080b08 in start_thread () from /lib/arm-linux-gnueabi/libpthread.so.0
#12 0x40169b7c in ?? () from /lib/arm-linux-gnueabi/libc.so.6
Cannot access memory at address 0x0
#13 0x40169b7c in ?? () from /lib/arm-linux-gnueabi/libc.so.6
Cannot access memory at address 0x0
---Type <return> to continue, or q <return> to quit---
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x40e52470 (LWP 3893)]
0x403cee24 in xmlCleanupCharEncodingHandlers ()
   from /usr/lib/arm-linux-gnueabi/libxml2.so.2
(gdb) bt
#0  0x403cee24 in xmlCleanupCharEncodingHandlers ()
   from /usr/lib/arm-linux-gnueabi/libxml2.so.2
#1  0x4006d2b8 in plist_from_xml (plist_xml=<optimized out>, length=266, 
    plist=0x40e51d34) at xplist.c:589
#2  0x000119b4 in plist_read_from_filename (plist=0x40e51d34, 
    filename=<optimized out>) at utils.c:272
#3  0x00011b40 in internal_get_value (
    config_file=0x4df01f18 "/var/lib/lockdown/SystemConfiguration.plist", 
    key=0x154b4 "SystemBUID", value=0x40e51d54) at conf.c:288
#4  0x00011df0 in config_get_value (value=0x40e51d4c, key=0x154b4 "SystemBUID")
    at conf.c:310
#5  config_get_system_buid (system_buid=0x40e51d7c) at conf.c:368
#6  0x0000f678 in lockdownd_set_untrusted_host_buid (lockdown=0x4df01870)
    at preflight.c:65
#7  0x0000f81c in preflight_worker_handle_device_add (userdata=0x25f78)
    at preflight.c:219
#8  0x40080b08 in start_thread () from /lib/arm-linux-gnueabi/libpthread.so.0
#9  0x40169b7c in ?? () from /lib/arm-linux-gnueabi/libc.so.6
Cannot access memory at address 0x0
#10 0x40169b7c in ?? () from /lib/arm-linux-gnueabi/libc.so.6
Cannot access memory at address 0x0
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

[fxfactorial]

I am also experiencing this with regular frequently. I also see several PRs for this issue which haven't been accepted. Is there something that can be done to get this rolling?

[fxfactorial]

@RossBencina along the way building my tool I found how to get usbmuxd to work with over 20 devices, check README here https://github.com/onlinemediagroup/ocaml-usbmux

[RossBencina]

@fxfactorial interesting. We found that most crashes occurred while trying to compute new pairing certificates (we run on low-powered devices where this is slow, maybe that also causes multi-thread issues). So we rolled back to a more stable version and disabled generating new pairings by making the pairing record directory forced read-only. That "solved" most of the problems for us.

[hosijyun]

It's seen that libxml2 is not thread safe.
If there are many threads(or devices) invoke plist_to_xml and plist_from_xml at the same time, segment fault error may throw.
So please edit xplist.c and make that functions to be thread safe(e.g. Add mutex).

[fxfactorial]

@hosijyun Here, I forked it and removed the memory operations. Stable with 30 devices. Need to compile this and then recompile usbmuxd. https://github.com/onlinemediagroup/libplist

[RossBencina]

@fxfactorial It is good that you found the bug. But leaking memory is not a fix.

[fxfactorial]

@RossBencina leaking like 1MB of memory once at program startup is not a big deal to me. Having a program that works matters more.

[RossBencina]

@fxfactorial I guess it depends on your definition of "at program startup". We have 512 MB RAM total, and a new client program is launched each time a device is connected.

[qmfrederik]

@fxfactorial @RossBencina Based on the discussion, it seems that libimobiledevice/libplist#73 fixed this issue.

Would be great to know if this fix also helps you guys!

[nikias]

@fxfactorial @RossBencina can you please let us know if the libplist changes helped to resolve these issues?

[RossBencina]

@nikias I'm not currently working on that project, but if I ever do again I'll definitely do the test and get back to you.

[fxfactorial]

I might get around to it, just I disabled USB 3, worked fine with usb2

Sent from my iPhone

On Sep 15, 2016, at 9:42 AM, Ross Bencina [email protected] wrote:

@nikias I'm not currently working on that project, but if I ever do again I'll definitely do the test and get back to you.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[nikias]

AFAIK this is not a problem anymore.