Segfault in mapper.rb on Ruby 2.0.0

[dwo]

Segfault in mapper.rb on Ruby 2.0.0

[apotonick]

Could you try upgrading to representable 1.8 and check if that helps?

My first thought is that the lambda block passed to the :if option is faulty. If that is true, the segfault with 1.8 should then come from the uber gem (which is responsible for executing the lambda blocks).

[dwo]

I saw this happen two more times this weekend, but the first line in "Control frame information" these times was https://github.com/apotonick/representable/blob/v1.7.7/lib/representable/binding.rb#L78

I have upgraded to 1.8.1 to see if it still happens, and added some more logging, so I might actually be able to reproduce it sometime this week. Thanks!

[apotonick]

I looked through your viagogo code for a bit but couldn't find a suspicious line. It must be a faulty lambda block somewhere that accesses global memory. Do you use :if, :getter, etc. somewhere?

[dwo]

Apologies about the long update today, and thank you for your patience and help.

The only lambdas I see in the viagogo code are for the date parsing: https://github.com/songkick/viagogo.rb/blob/0.0.1.sk.2/lib/viagogo/event.rb#L16-L27

I am fetching the events_by_category_id in threads, but consolidating the results (arrays of Viagogo::Events that use Representable) into a shared array in a Mutex#synchronize block.

In pseudocode:

THREADS = 4
threads, results, mutex = [], [], Mutex.new

big_array_of_category_ids.each_slice(THREADS) do |category_id_slice|
  category_id_slice.each do |category_id|
    threads << Thread.new do
      page_of_events = viagogo_client.events_by_category_id(category_id)
      mutex.synchronize { results.concat(page_of_events) }
    end
    threads.each(&:join)
    # do things with results here
  end
end

I got one segfault yesterday that had a slightly different control frame trace which suggested that perhaps the segfault was happening in stdlib's OpenStruct, not representable:

c:0118 p:0039 s:0540 E:0004c0 METHOD .../ruby-2.0.0-p353/lib/ruby/2.0.0/ostruct.rb:170
c:0117 p:0075 s:0534 e:000531 METHOD .../ruby-2.0.0-p353/lib/ruby/2.0.0/ostruct.rb:183
c:0116 p:0014 s:0525 e:000524 BLOCK  .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:82
c:0115 p:0025 s:0523 e:000522 METHOD .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:106
c:0114 p:0011 s:0516 e:000515 METHOD .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:81
c:0113 p:0009 s:0512 e:000511 BLOCK  .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:43
c:0112 p:0054 s:0509 e:000508 METHOD .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:67
c:0111 p:0009 s:0504 e:000503 BLOCK  .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:42
c:0110 p:0025 s:0502 e:000501 METHOD .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:106
c:0109 p:0011 s:0495 e:000494 METHOD .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:41
c:0108 p:0010 s:0491 e:000490 METHOD .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/mapper.rb:64
c:0107 p:0028 s:0486 e:000485 METHOD .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/mapper.rb:37
c:0106 p:0028 s:0480 e:000479 METHOD .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/readable_writeable.rb:5
c:0105 p:0013 s:0474 e:000473 BLOCK  .../vendor/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/mapper.rb:17 [FINISH]
c:0104 p:---- s:0471 e:000470 CFUNC  :each

I also got another segfault on 1.8.1, with the control frame information pointing at uber as you predicted.

c:0053 p:0013 s:0240 e:000236 METHOD .../shared/bundle/ruby/2.0.0/gems/uber-0.0.4/lib/uber/options.rb:77
c:0052 p:0026 s:0232 e:000231 METHOD .../shared/bundle/ruby/2.0.0/gems/uber-0.0.4/lib/uber/options.rb:69
c:0051 p:0026 s:0228 e:000227 METHOD .../shared/bundle/ruby/2.0.0/gems/uber-0.0.4/lib/uber/options.rb:58
c:0050 p:0088 s:0223 e:000222 METHOD .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:11
c:0049 p:0011 s:0216 e:000215 METHOD .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:81
c:0048 p:0009 s:0212 e:000211 BLOCK  .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:43
c:0047 p:0054 s:0209 e:000208 METHOD .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:67
c:0046 p:0009 s:0204 e:000203 BLOCK  .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:42
c:0045 p:0025 s:0202 e:000201 METHOD .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:10
c:0044 p:0011 s:0195 e:000194 METHOD .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/binding.rb:41
c:0043 p:0010 s:0191 e:000190 METHOD .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/mapper.rb:64
c:0042 p:0028 s:0186 e:000185 METHOD .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/mapper.rb:37
c:0041 p:0028 s:0180 e:000179 METHOD .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/readable_writ
c:0040 p:0013 s:0174 e:000173 BLOCK  .../shared/bundle/ruby/2.0.0/gems/representable-1.8.1/lib/representable/mapper.rb:17 [FINISH]
c:0039 p:---- s:0171 e:000170 CFUNC  :each                                       

I should be able to replay the same request(s) that happened around the segfault today to see if I can reproduce this. I'll let you know.

[apotonick]

Cool! I mean... terrible! But good news that you we're getting closer.

I am pretty pretty sure it's one of your lambdas. The fact that the segfault now comes from uber reinforces that suspicion. Also, the control frame shows that the segfault happens when the :setter is executed.

Being able to reproduce that would be great - I'd love to see if any lambda can make it crash in a Thread.

[dwo]

I have been rinsing this for all it is worth today (trying to run it with 100 threads under various conditions), but the segmentation fault has not reared its ugly head. I'm going to try and get a core dump that I can inspect with gdb.

[dwo]

I haven't seen this happen again, so perhaps representable 1.8 and uber 0.0.4 are less likely to segfault.

If I see it again and capture a core dump, I'll either re-open this issue or raise a new one on uber. Thanks again for your time!