Deserializing a large array of nested objects gives "zend_mm_heap corrupted"

[msgpack]

Deserializing a large array of nested objects gives "zend_mm_heap corrupted"

[bkw]

Here is a test case that is consistantly giving segfaults on my osx 10.7.4, macports php 5.3.15.

 https://gist.github.com/3699796

Can anybody confirm this?

[laruence]

I can not get segfault, but:
PHP Warning: msgpack Parse error in /tmp/1.php on line 7
[Wed Sep 12 14:55:03 2012] Script: '/tmp/1.php'
/home/huixinchen/opensource/github/msgpack-php/msgpack_unpack.c(469) : Freeing 0x16CCAC08 (24 bytes), script=/tmp/1.php
[Wed Sep 12 14:55:03 2012] Script: '/tmp/1.php'
/home/huixinchen/opensource/github/msgpack-php/msgpack_unpack.c(477) : Freeing 0x16A33950 (8 bytes), script=/tmp/1.php
=== Total 2 memory leaks detected ===

[Dexus]

Are you fixing the memory leaks? i have the same problem with an array bigger than 40 objects and more than 456466 Chars of payload.

[laruence]

I can confirm the segfault now, digging

[laruence]

@advect could you please look at this too?

[laruence]

I got a fix:
$ git diff
diff --git a/msgpack_unpack.h b/msgpack_unpack.h
index 2ffa790..1221c7d 100644
--- a/msgpack_unpack.h
+++ b/msgpack_unpack.h
@@ -4,7 +4,7 @@

#include "ext/standard/php_var.h"

-#define MSGPACK_EMBED_STACK_SIZE 1024
+#define MSGPACK_EMBED_STACK_SIZE 2048

#include "msgpack/unpack_define.h"

but I need more time to confirm it, maybe we should use heap memory for that stack

@Dexus @bkw could you please try with this patch? thanks

[bkw]

Hey Laurence, thanks for the patch! Unfortunately, is does not seem to do what You expected:

The test script gives me:
Warning: [msgpack] (php_msgpack_unserialize) Parse error in […]/tests/bug002.php on line 11

And my own test above still segfaults:

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000003644099df9

VM Regions Near 0x3644099df9:
    MALLOC_LARGE           0000000122599000-0000000122c99000 [ 7168K] rw-/rwx SM=PRV  
--> 
    MALLOC_TINY            00007fa2a2c00000-00007fa2a2f00000 [ 3072K] rw-/rwx SM=PRV  

Application Specific Information:
objc[24468]: garbage collection is OFF

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   php                             0x000000010ede2761 gc_zval_possible_root + 113
1   php                             0x000000010edd3353 zend_hash_destroy + 67
2   php                             0x000000010ede5b24 zend_objects_free_object_storage + 52
3   php                             0x000000010ede9401 zend_objects_store_del_ref_by_handle_ex + 401
4   php                             0x000000010ede94ba zend_objects_store_del_ref + 26
5   php                             0x000000010edbb3da _zval_ptr_dtor + 74
6   msgpack.so                      0x00000001120bbdc9 msgpack_unserialize_var_destroy + 89 (msgpack_unpack.c:304)
7   msgpack.so                      0x00000001120b6e90 php_msgpack_unserialize + 192 (msgpack.c:273)
8   msgpack.so                      0x00000001120b5fa5 zif_msgpack_unserialize + 85 (msgpack.c:334)
9   php                             0x000000010ee2d7f9 zend_do_fcall_common_helper_SPEC + 1081
10  php                             0x000000010ee2def1 execute + 609
11  php                             0x000000010edc6888 zend_execute_scripts + 440
12  php                             0x000000010ed72782 php_execute_script + 722
13  php                             0x000000010ee44f01 main + 4529
14  php                             0x000000010ebff624 start + 52

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000122959038  rbx: 0x0000000122a719f8  rcx: 0x0000003521740dc0  rdx: 0x00007fff6e7fbed0
  rdi: 0x0000000122a719f8  rsi: 0x0000000000000000  rbp: 0x00007fff6e7f1c40  rsp: 0x00007fff6e7f1c30
   r8: 0x0000000000000000   r9: 0x0000000000000000  r10: 0x0000000000000481  r11: 0x00000001120bbd70
  r12: 0x0000000000000000  r13: 0x00007fff6e7fc178  r14: 0x0000000122a72138  r15: 0x0000000122a721b8
  rip: 0x000000010ede2761  rfl: 0x0000000000000206  cr2: 0x0000003644099df9
Logical CPU: 0

[advect]

https://github.com/kjdev/php-ext-msgpacki

msgpacki is this no problem.
might be good.

[bkw]

php-msgpacki looks interesting, but it doesn't compile on osx yet. I'll give it a shot.

[advect]

It's a shame that

[laruence]

@advect it's better if there is only one msgpack php extension, so maybe we should contact with the msgpacki guy, merge our works together..

[asiminiceanu]

Hello,

The test bug002.phpt for this issue makes the RPM build impossible.
Can this test be marked as SKIP until the problem is resolved?

SKIP unpack of template converter: multiple class/variable: class unpacker (array) [tests/136b.phpt] reason: tests in PHP 5.1 or older
XFAIL Bug #2 (Deserializing a large array of nested objects gives "zend_mm_heap corrupted") [tests/bug002.phpt] 
PASS Bug #6 (bug with incorrect packing of mixed arrays) [tests/bug006.phpt] 
=====================================================================
TIME END 2012-12-14 11:50:34

=====================================================================
TEST RESULT SUMMARY
---------------------------------------------------------------------
Exts skipped    :    0
Exts tested     :   30
---------------------------------------------------------------------

Number of tests :  191               113
Tests skipped   :   78 ( 40.8%) --------
Tests warned    :    0 (  0.0%) (  0.0%)
Tests failed    :    0 (  0.0%) (  0.0%)
Expected fail   :    1 (  0.5%) (  0.9%)
Tests passed    :  112 ( 58.6%) ( 99.1%)
---------------------------------------------------------------------
Time taken      :    5 seconds
=====================================================================

=====================================================================
EXPECTED FAILED TEST SUMMARY
---------------------------------------------------------------------
Bug #2 (Deserializing a large array of nested objects gives "zend_mm_heap corrupted") [tests/bug002.phpt]
=====================================================================
+ unset NO_INTERACTION REPORT_EXIT_STATUS
++ find tests -name '*.diff' -type f -print
+ '[' -n tests/bug002.diff ']'
+ exit 1
error: Bad exit status from /var/tmp/rpm-tmp.85923 (%check)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.85923 (%check)

[laruence]

hey, maybe you can remove that from the rpm package?

thanks

[asiminiceanu]

Done. Check out the pull request #9