segmentation fault in lj_vm_growstack_f

[openresty]

segmentation fault in lj_vm_growstack_f

[agentzh]

@tokers LuaJIT is not multi-thread safe in general. I know very little about your setup, maybe it's using multiple OS threads. Not sure. Your backtrace looks like a stack corruption or heap corruptions. Such things are not usually debuggable without a reproducible environment.

[tokers]

@agentzh

There is only one OS thread in our service.

We used the Mozilla rr recorded the crash environment, but only get a few clues.

   │0x7f0c1a6aadbe <lj_vm_growstack_f+11>   mov    %edx,0x10(%rbp)                                                                                                                  │
   │0x7f0c1a6aadc1 <lj_vm_growstack_f+14>   mov    %eax,0x18(%rbp)                                                                                                                  │
   │0x7f0c1a6aadc4 <lj_vm_growstack_f+17>   mov    %ebx,0x1c(%rsp)                                                                                                                  │
   │0x7f0c1a6aadc8 <lj_vm_growstack_f+21>   mov    %ecx,%esi                                                                                                                        │
   │0x7f0c1a6aadca <lj_vm_growstack_f+23>   mov    %ebp,%edi                                                                                                                        │
   │0x7f0c1a6aadcc <lj_vm_growstack_f+25>   callq  0x7f0c1a6b4470 <lj_state_growstack>                                                                                              │
   │0x7f0c1a6aadd1 <lj_vm_growstack_f+30>   mov    0x10(%rbp),%edx                                                                                                                  │
   │0x7f0c1a6aadd4 <lj_vm_growstack_f+33>   mov    0x18(%rbp),%eax                                                                                                                  │
   │0x7f0c1a6aadd7 <lj_vm_growstack_f+36>   mov    -0x8(%rdx),%ebp                                                                                                                  │
   │0x7f0c1a6aadda <lj_vm_growstack_f+39>   sub    %edx,%eax                                                                                                                        │
   │0x7f0c1a6aaddc <lj_vm_growstack_f+41>   shr    $0x3,%eax                                                                                                                        │
   │0x7f0c1a6aaddf <lj_vm_growstack_f+44>   add    $0x1,%eax                                                                                                                        │
  >| 0x7f0c1a6aade2 <lj_vm_growstack_f+47>   mov    0x10(%rbp),%ebx                       

The %edx was stored in %rbp + 0x10 before calling lj_state_growstack. After lj_state_growstack was called, address %rbp + 0x10 isn't accessable. We set a watch point at this address:

(rr) p/x $rbp + 0x10
$19 = 0x411d2730
(rr) watch *(int *) 0x411d2730
Hardware watchpoint 8: *(int *) 0x411d2730
(rr) continue
Continuing.

Hardware watchpoint 8: *(int *) 0x411d2730

Old value = 1092445696
New value = 1081358768
resizestack (L=0x411d2720, n=192) at lj_state.c:76

(rr) p L
$70 = (lua_State *) 0x411d2720
(rr) p &L->base
$71 = (TValue **) 0x411d2730
(rr) p delta
$72 = -11086928
(rr) p st
$82 = (TValue *) 0x4073fbd8
(rr) p oldst
$83 = (TValue *) 0x411d2828
(rr) p/x L->stack
$84 = {ptr32 = 0x4073fbd8}
(rr) p oldsize
$85 = 98
(rr) p realsize
$86 = 198

BTW, here is the registers information when 0x411d2730 was changed:

rax            0x0      0
rbx            0xc6     198
rcx            0xffffffffff56d3b0       -11086928
rdx            0x400cc3b8       1074578360
rsi            0x0      0
rdi            0x411d2818       1092429848
rbp            0xc6     0xc6
rsp            0x1eee210        0x1eee210
r8             0x4073fbd8       1081342936
r9             0x4      4
r10            0x4073fbd8       1081342936
r11            0x8      8
r12            0x411d2720       1092429600
r13            0x411d2828       1092429864
r14            0xc0     192
r15            0x40655c28       1080384552
rip            0x7f67d9374061   0x7f67d9374061 <resizestack+145>
eflags         0x203    [ CF IF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0
fs_base        0x7f67da076740   0x7f67da076740
gs_base        0x0      0x0

[tokers]

@agentzh

By the way, someone replied this problem in the luajit maillist: https://www.freelists.org/post/luajit/segmentation-fault-in-lj-vm-growstack-f,1.

[tokers]

@agentzh
Seems that the problem was fixed after we added some tricks. Thanks a lot!