Giter Site home page Giter Site logo

rubyomr-preview / ruby Goto Github PK

View Code? Open in Web Editor NEW

This project forked from ruby/ruby

68.0 68.0 9.0 135.32 MB

The Ruby+OMR Preview

License: Other

Ruby 57.54% Makefile 0.78% C 38.67% Perl 0.08% Python 0.01% Scheme 0.01% Perl 6 0.01% HTML 0.07% C++ 1.27% Shell 0.03% Assembly 0.01% CSS 0.05% JavaScript 0.06% Emacs Lisp 0.40% Yacc 0.85% XSLT 0.05% Scilab 0.01% ApacheConf 0.01% Batchfile 0.03% GDB 0.10%
compiler jit ruby

ruby's Introduction

Introducing the "Ruby + OMR Technology Preview"

This technology preview showcases how the Eclipse OMR project can be integrated into the Ruby VM. To do this work, we have now created Ruby + OMR as a Ruby fork. Our intention is to use this fork to contribute Ruby + OMR code changes back into Ruby.

Our current Ruby + OMR changes are based on Ruby 2.2 and located in the ruby_2_2_omr branch. You can build everything from source, or, use the docker images below.

Going forward, we will be working in this fork to update to the master branch with the goal of submitting a pull request against Ruby. We are open to feedback at any point on how best to structure the changes we've made or on how to best propose these changes into Ruby.

Because we have used some components in the Ruby + OMR Technology Preview that have not yet been open sourced via the Eclipse OMR project (for example, the Ruby JIT), we cannot yet include souce code changes for these components. But you can continue to try them out via our docker images in this project, and we welcome your feedback!

Our sincerest hope is that this preview helps us to work with the existing Ruby community to integrate those parts of Eclipse OMR that the Ruby community finds beneficial.

Our lawyers are keen to tell you that any reference to the Invoice in the License means that you are authorised to use one copy of the downloaded image.

If you can't wait to get started, you can go directly to the Quick Start Guide

Otherwise, you may be wondering…

What is “OMR” ?

The Eclipse OMR project is an open source community growing around reusable and easily consumable core components (like thread library, platform abstraction library, garbage collector, etc.) for building all kinds of language runtimes, from Java to Ruby to Smalltalk and beyond. Many of the components were contributed by IBM from the IBM J9 Java Virtual Machine (JVM), an enterprise class JVM implementation representing hundreds of person years of development creating scalable, high performance runtime technology. But going forward, this open project is accepting contributions from anyone.

The OMR components have 3 features that distinguish the Eclipse OMR project from other projects that aim to reuse technology for building language runtimes: 1) OMR has no language semantics, 2) OMR is not a language runtime, and 3) OMR components can be consumed by any language runtime. The first two features distinguish OMR from efforts to implement languages on top of a mature language runtime like the Java Virtual Machine (by implementing in Java) or the CoreCLR project (by implementing via CLR bytecodes). Both these other efforts require mapping a language's semantics into the semantics of another language (either Java or CLR). In contrast, OMR components can be brought into an existing language runtime with its own bytecode set and bytecode semantics (or abstract syntax tree semantics). OMR can therefore be used to build any kind of language runtime, free from the influence of Java or another language's semantics.

One of the most important motivators for the Eclipse OMR project is the increasing importance of cloud computing platforms, where the polyglot of programming languages must all cooperate seamlessly in what can really be a diverse environment. Language runtimes must be monitored and managed adaptively to adjust to the current operating conditions in the cloud platform. But not all runtimes have the same kinds of responsiveness built in because most language runtimes today actually share very little technology. can hurt or even sabotage longer term success. But every language runtime community needs to go through this process, and as existing runtimes become more and more sophisticated, the path to success for new languages becomes even more difficult.

By introducing shareable core technology components for building all kinds of language runtimes, the Eclipse OMR project hopes to both make the process easier for new languages to bootstrap themselves as well as to provide mechanisms for existing runtimes to fill gaps or to reduce maintenance costs so that communities can focus more of their efforts on the opportunities and problems that are specific to their language.

To learn more about the Eclipse OMR project, please visit us at the Eclipse OMR Github page.

So what’s in this Ruby + OMR preview?

To make sure the Eclipse OMR technology really could work in runtimes other than Java, the OMR team has been working to create several proof points to use the technology with different language runtimes (like Ruby!). This technology preview release represents the current state of our Ruby proof point, with new GC, JIT compiler, and method profiling capabilities. It's not a toy: we've got it running Rails applications. We cannot claim it is ready for use in production, but we think it's good enough to be able to meet the goals outlined at the beginning of this document.

We presented talks about our Ruby + OMR proof point at Ruby Kaigi 2015:

Building Ruby + OMR

The Eclipse OMR project strives to make it very easy to integrate components into a language runtime.

The Ruby VM needed the following high level sets of changes to incorporate the OMR technology:

  1. configure.in changed to add OMRDIR and OMRGLUE options which default to assume omr in ruby's top directory
  2. makefile.in changed to configure, build, and clean up OMR and OMR Ruby Glue under appropriate targets
  3. makefile.in changed to build and link against OMR and Ruby glue objects and static libraries
  4. various changes to files to invoke the methods in the Ruby glue and OMR project as appropriate

Once these changes were made, the Ruby + OMR Technology Preview release can be built via:

$ git clone https://github.com/rubyomr-preview/ruby.git --branch ruby_2_2_omr --recursive 
$ cd ruby
$ autoconf
$ ./configure SPEC=<specname> --with-omr-jit
$ make
$ make install

Since the Ruby + OMR code has only been tested on Linux x86-64, Linux PPC-LE-64, Linux PPC-BE-64 and Linux 390-64 the acceptable values for <specname> are:

1. linux_x86-64
2. linux_ppc-64_le_gcc
3. linux_ppc-64
4. linux_390-64

Building with this simple sequence of commands will download the latest version of the Ruby + OMR Technology Preview and Eclipse OMR, build them together and install them onto your system. That's it!

There are more configuration options than the simple set listed above, which is just a high level description. Search for "OMR" and hopefully you'll find the tags we used self explanatory. If not, feel free to ask questions!

To make it even easier for people to try out the Ruby + OMR Technology Preview without having to clone repositories or build, we also have three docker images available that come with the Ruby + OMR Technology Preview already built and pre-installed. Docker images make it easy to avoid fussing with platform specifics. The images have a preinstalled Ruby 2.2.x with built-in OMR technology. Also included is a monitoring agent which can be used with IBM Health Centre to visualize Ruby method profiles and garbage collection performance while your Ruby application is running.

The OMR technology itself is included only as part of the prebuilt binaries which you can access by simply running ruby (/usr/local/bin/ruby). Unfortunately, not all of the OMR project is available in the open (we're working on it really hard). The real place to look for the most up to date source code is at The Ruby + Eclipse OMR Ruby repo.

Not all of the OMR technologies are active by default in this version of Ruby. Environment variables activate those technologies that are not on by default, like the method profiling support when you connect to ruby with IBM Health Center (also included in the docker image: see the User's Guide!) or to turn the JIT compiler on. For example, to activate the JIT compiler technology, you'll need to set OMR_JIT_OPTIONS="-Xjit" which turns on the JIT where it will compile methods that are invoked more than 1000 times. OMR_JIT_OPTIONS="-Xjit:count=N" adjusts the invocation count before JIT compilation so that you can play around a bit.

For full details on how to activate each OMR component and the various configuration options available in this technology preview, please see the User's Guide in the Wiki !

Quick Start Guide

The following files are in this project:

  • This README.md file
  • A LICENSE directory describing in excruciating detail just how little you can rely on in this technology preview

You can also find the User's Guide in our wiki.

To start using the Ruby + OMR Technology Preview, please follow the directions for the platform you're using: Linux on X86, Linux on Z, or Linux on OpenPOWER. The Linux X86 Docker image has been updated to the latest version and we will be updating the Linux on Z and Linux on OpenPOWER soon.

Linux on X86, 64-bit

  1. If you do not already have docker installed, follow these directions to get started:

    Installing Docker For Linux on x86

  2. Download the rubyomrpreview docker image from Box.com with the command:

     $ wget https://ibm.box.com/shared/static/sy7pgu7pqbyht9j3g3tvko77ska1k8x1.tgz -O rubyomrpreview-x86_64.tgz
    
  3. Load the docker image locally:

     $ docker load -i rubyomrpreview-x86_64.tgz
    
  4. Run the docker image (you can omit the -p 1883:1883 if you won't be using Health Centre or if you won't be running the message broker--see the User's Guide-- inside the container):

     $ docker run -p 1883:1883 -it rubyomrpreview/rubyomrpreview /bin/bash
    
  5. Verify you can successfully run Ruby + OMR Technology Preview:

     root@d2ae8cf89313:/# ruby --version
     ruby 2.2.5p285 (Eclipse OMR Preview r1) (2016-03-29) [x86_64-linux]
    
  6. Play to your heart's content!

Linux on Z, 64-bit

  1. If you do not already have docker installed, follow these directions to get started:

    Installing Docker For Linux on Z

  2. Download the rubyomrpreview docker image from Box.com with the command:

     $ wget https://ibm.box.com/shared/static/1bzikt7fmfdejpvp4zqglnss64tcwls2.tgz -O rubyomrpreview-s390x.tgz
    
  3. Load the docker image locally:

     $ docker load -i rubyomrpreview-s390x.tgz
    
  4. Run the docker image (you can omit the -p 1883:1883 if you won't be using Health Centre or if you won't be running the message broker--see the User's Guide-- inside the container):

     $ docker run -p 1883:1883 -it rubyomrpreview/rubyomrpreview /bin/bash
    
  5. Verify you can successfully run Ruby + OMR Technology Preview:

     bash-4.2# ruby --version
     ruby 2.2.3p97 (OMR Preview r1)(2015-04-14) [s390x-linux]
    
  6. Play to your heart's content!

Linux on OpenPOWER, 64-bit

Installing Docker for Linux on OpenPOWER

  1. Download the rubyomrpreview docker image from Box.com with the command:

     $ wget https://ibm.box.com/shared/static/hdqgfvvnnq1idf6qvlsv9r0rw5dzwbhc.tgz -O rubyomrpreview-powerpc64le.tgz
    
  2. Load the docker image locally:

     $ docker load -i rubyomrpreview-powerpc64le.tgz
    
  3. Run the docker image (you can omit the -p 1883:1883 if you won't be using Health Centre or if you won't be running the message broker--see the User's Guide-- inside the container):

     $ docker run -p 1883:1883 -it rubyomrpreview/rubyomrpreview /bin/bash
    
  4. Verify you can successfully run Ruby + OMR Technology Preview:

     root@7305793b79e9:/# ruby --version
     ruby 2.2.3p97 (OMR Preview r1) (2015-04-14) [powerpc64le-linux]
    
  5. Play to your heart's content!

To see how to use the various OMR technologies, please look for Tracing, Garbage Collector, and Just In Time (JIT) compiler sections in the User’s Guide in the Wiki !

We would love to hear your feedback!

As we said above, we’re making this technology preview openly available so that everyone can try it out and let us know what's good and what's not so good. We welcome all feedback!

We’re excited to finally get into the open with this project, and look forward to hearing what you think!

We’re going to use the issue tracking associated with the rubyomr-preview github project to track the feedback, so if you’d like to tell or ask us anything, please open an issue

ruby's People

Contributors

akr avatar authornari avatar ayumin avatar drbrain avatar duerst avatar eban avatar eregon avatar hsbt avatar k-tsj avatar knu avatar ko1 avatar kou avatar mame avatar marcandre avatar matzbot avatar mrkn avatar nagachika avatar nobu avatar nurse avatar rhenium avatar shugo avatar shyouhei avatar sorah avatar suketa avatar tarui avatar tenderlove avatar tmm1 avatar unak avatar yugui avatar znz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ruby's Issues

TestProcess#test_deadlock_by_signal_at_forking

$ LD_LIBRARY_PATH=$PWD OMR_JIT_OPTIONS=-Xjit:count=0 make test-all TESTS=test/ruby/test_process.rb
...
[ 30/131] TestProcess#test_deadlock_by_signal_at_forking = 81.14 s                        
  1) Error:
TestProcess#test_deadlock_by_signal_at_forking:
Timeout::Error: execution of assert_separately expired
pid 645 killed by SIGTERM (signal 15)
...

MemorySegment crash compiling lib/rexml/parsers/treeparser.rb:17:parse

Thread 1 "ruby" received signal SIGSEGV, Segmentation fault.
0x00007ffff64be07c in TR::MemorySegment::remaining (this=0x0) at /home/magaudet/open/ruby_24/omr/compiler/env/MemorySegment.hpp:76
76	      return _size - _allocated;
(gdb) where
#0  0x00007ffff64be07c in TR::MemorySegment::remaining (this=0x0) at /home/magaudet/open/ruby_24/omr/compiler/env/MemorySegment.hpp:76
#1  0x00007ffff64bde94 in TR::Region::allocate (this=0x7fffffff7700, size=0, hint=0x0) at /home/magaudet/open/ruby_24/omr/compiler/env/Region.cpp:61
#2  0x00007ffff6457d22 in TR_Memory::allocateHeapMemory (this=0x7fffffff74e0, requestedSize=0, ot=TR_MemoryBase::Array) at /home/magaudet/open/ruby_24/omr/compiler/env/TRMemory.cpp:333
#3  0x00007ffff6457ff2 in TR_Memory::allocateMemory (this=0x7fffffff74e0, size=0, kind=(unknown: 0), ot=TR_MemoryBase::Array) at /home/magaudet/open/ruby_24/omr/compiler/env/TRMemory.cpp:379
#4  0x00007ffff642726e in TR_Array<TR::Node*>::copy (this=0x7fffffff7e60, other=...) at /home/magaudet/open/ruby_24/omr/compiler/infra/Array.hpp:216
#5  0x00007ffff642711b in TR_Array<TR::Node*>::operator= (this=0x7fffffff7e60, other=...) at /home/magaudet/open/ruby_24/omr/compiler/infra/Array.hpp:85
#6  0x00007ffff64270cd in TR_Stack<TR::Node*>::operator= (this=0x7fffffff7e60, other=...) at /home/magaudet/open/ruby_24/omr/compiler/infra/Stack.hpp:34
#7  0x00007ffff6427024 in TR_ByteCodeIteratorWithState<unsigned long, 93ul, TR_RubyByteCodeIterator, TR::Node*>::setupBBStartContext (this=0x555556292218, index=650)
    at /home/magaudet/open/ruby_24/omr/compiler/ilgen/ByteCodeIteratorWithState.hpp:249
#8  0x00007ffff64330b3 in TR_ByteCodeIteratorWithState<unsigned long, 93ul, TR_RubyByteCodeIterator, TR::Node*>::findNextByteCodeToGen (this=0x555556292218)
    at /home/magaudet/open/ruby_24/omr/compiler/ilgen/ByteCodeIteratorWithState.hpp:274
#9  0x00007ffff642f8cb in RubyIlGenerator::getinlinecache (this=0x555556292210, offset=4, ic=0x555556009750) at /home/magaudet/open/ruby_24/rbjitglue/ruby/ilgen/RubyIlGenerator.cpp:2194
#10 0x00007ffff642bd29 in RubyIlGenerator::indexedWalker (this=0x555556292210, startIndex=0, firstIndex=@0x7fffffff7d98: 0, lastIndex=@0x7fffffff7d94: 783)
    at /home/magaudet/open/ruby_24/rbjitglue/ruby/ilgen/RubyIlGenerator.cpp:915
#11 0x00007ffff6429a18 in RubyIlGenerator::walker (this=0x555556292210) at /home/magaudet/open/ruby_24/rbjitglue/ruby/ilgen/RubyIlGenerator.cpp:705
#12 0x00007ffff64298f8 in RubyIlGenerator::genILInternal (this=0x555556292210) at /home/magaudet/open/ruby_24/rbjitglue/ruby/ilgen/RubyIlGenerator.cpp:668
#13 0x00007ffff6427fbf in RubyIlGenerator::genIL (this=0x555556292210) at /home/magaudet/open/ruby_24/rbjitglue/ruby/ilgen/RubyIlGenerator.cpp:198
#14 0x00007ffff644bd13 in OMR::ResolvedMethodSymbol::genIL (this=0x5555562752e0, fe=0x7ffff6b47720 <jitInit(rb_vm_struct*, char*)::fe>, comp=0x7fffffffa140, symRefTab=0x7fffffff9880, customRequest=...)
    at /home/magaudet/open/ruby_24/omr/compiler/il/symbol/OMRResolvedMethodSymbol.cpp:1215
#15 0x00007ffff64ad26d in OMR::Compilation::compile (this=0x7fffffffa140) at /home/magaudet/open/ruby_24/omr/compiler/compile/OMRCompilation.cpp:828
#16 0x00007ffff643b22c in compileMethodFromDetails (omrVMThread=0x0, details=..., hotness=cold, rc=@0x7fffffffb98c: 1) at /home/magaudet/open/ruby_24/omr/compiler/control/CompileMethod.cpp:328
#17 0x00007ffff643abc9 in compileMethod (omrVMThread=0x0, compilee=..., hotness=cold, rc=@0x7fffffffb98c: 1) at /home/magaudet/open/ruby_24/omr/compiler/control/CompileMethod.cpp:227
#18 0x00007ffff643dbd5 in compileRubyISeq (iseq=0x555555e36780, name=0x5555561894f0 "/home/magaudet/open/ruby_24/lib/rexml/parsers/treeparser.rb:17:parse", optLevel=cold)
    at /home/magaudet/open/ruby_24/rbjitglue/ruby/control/RubyJit.cpp:281
#19 0x00007ffff643e02d in jit_compile (iseq=0x555555e36780) at /home/magaudet/open/ruby_24/rbjitglue/ruby/control/RubyJit.cpp:386
#20 0x00005555556d138d in vm_jit_compile (iseq=0x555555e36780, vm=<optimized out>) at vm_insnhelper.c:2848

Reproduction:

OMR_JIT_OPTIONS='-Xjit:count=0,limit={/home/magaudet/open/ruby_24/lib/rexml/parsers/treeparser.rb:17:parse}' LD_LIBRARY_PATH=$PWD ./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems "./test/runner.rb" --ruby="./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems" test/rexml

Ensure better YARV bytecode test coverage

It would be fantastic if there was a test file that

  1. Created all the opcodes in the YARV bytecode set
  2. Ensured all branches are taken in the YARV bytecode set.

It would likely consist of a total opcode set, that then ran again after redefining a bunch of the basic operations.

Another extra bonus would be verifying that the tests actually do generate all bytecodes.

Rework Ruby Calls to look more like OMR calls.

Today, a Ruby call looks like this in trees:

n102n     astorei  sp[#614  sp +8] [flags 0x607 0x200 ]                                    
n101n       aloadi  cfp[#613  cfp +48] [flags 0x607 0x200 ]                                
n100n         aload  <parm 0 Lrb_thread_t;>[#611  Parm] [flags 0x40010107 0x0 ]            
n99n        aladd                                                                          
n98n          aload  <temp slot 2>[#623  Auto] [flags 0x7 0x0 ]                            
n97n          lconst 16                                                                    
n104n     lstorei  stackSlot[#627  stackSlot +8] [flags 0x604 0x200 ]                      
n103n       aload  <temp slot 2>[#623  Auto] [flags 0x7 0x0 ]                              
n61n        ==>lcall
n105n     lstorei  stackSlot[#628  stackSlot] [flags 0x604 0x200 ]                         
n103n       ==>aload
n23n        ==>lloadi
n113n     astorei  pc[#615  pc] [flags 0x607 0x200 ]                                       
n112n       aloadi  cfp[#613  cfp +48] [flags 0x607 0x200 ]                                
n111n         aload  <parm 0 Lrb_thread_t;>[#611  Parm] [flags 0x40010107 0x0 ]            
n110n       aconst 0x55ec92d80eb8                                                          
n287n     treetop                                                                          
n286n       lstore  <temp slot 3>[#629  Auto] [flags 0x4 0x0 ]                             
n23n          ==>lloadi
n114n     treetop                                                                          
n109n       lcall  vm_send_without_block[#286  helper Method] [flags 0x400 0x0 ] ()        
n108n         aload  <parm 0 Lrb_thread_t;>[#611  Parm] [flags 0x40010107 0x0 ]            
n107n         aconst 0x55ec92d7fe40                                                        
n106n         aconst 0x55ec92d810b0                                                        
n23n          ==>lloadi

There's a lot going on there!

  1. Rematerialization of the stack pointer to the appropriate height (n102n)
  2. Rematerialization of the YARV stack (n104n, n105n)
  3. Rematerialization of the program counter (n113n)
  4. Call out to the vm_send_without_block helper.

That's a lot of trees for a single bytecode:

0022: 30 opt_send_without_block      puts (mid:15329) <argc:1 flag:0x14 (VM_CALL_FCALL|VM_CALL_ARGS_SIMPLE|0x0)>, E:94474564276400

A lot of the OMR compiler machinery would be a lot happier if calls looked more like how they do in other languages (hypothetically like):

acall mid:15329 puts 
  aload <temp slot 2>

I'm not sure how hard this would be, nor what the right way to go about this would be. An initial thought would be to keep code-generation looking similar, but to have high-level calls until a (mandatory opt) lowering pass.

Move JIT Glue into the Ruby Repo?

When we initially pushed out the Ruby+OMR JIT compiler, I made the choice to keep the 'JIT Glue' in a separate repo in order to emphasize the relatively low impact of the JIT compiler on the Ruby interpreter when running -without- a JIT.

However, it's not clear to me that the distinction continues to hold: It may make more sense to pull the rbjitglue submodule code into the VM directly; The glue is tied to the VM fairly tightly, so it makes sense for them to live in the same repo.

Looking for feedback here.

If I did this, I would likely only do it for the ruby_2_4_omr branch.

Build Break

ruby_2_4_omr_preliminary won't build after 9729ad8

Only noticed this because my dirty work tree preserved rbjitglue/ruby/Makefile

Can fix this by moving ruby.mk to Makefile in that dir. Will submit PR in a bit :)

Implement smarter compilation control

Today the JIT uses invocation counts to decide when to compile a method. This can be a problem where a method consists of a long-running loop -- it would be beneficial to compile this early, but the counts will wait a long time.

There are other approaches for compilation control. One that might be worth considering is stack sampling.

no superclass method in REXML tests @ warm

94462bf

magaudet@ubuntu1604VM:~/open/ruby_Inliner (detached*)*$ OMR_JIT_OPTIONS=count=3,optLevel=warm LD_LIBRARY_PATH=$PWD make test-all TESTS=test/rexml
<snip>
Run options: "--ruby=./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/

# Running tests:

[361/425] REXMLTests::Tester#test_element = 0.03 s                                                                                                                
  1) Error:
REXMLTests::Tester#test_element:
NoMethodError: super: no superclass method `parent=' for "1) He's a great guy!":REXML::Text
    /home/magaudet/open/ruby_Inliner/lib/rexml/text.rb:125:in `parent='
    /home/magaudet/open/ruby_Inliner/lib/rexml/parent.rb:142:in `replace_child'
    /home/magaudet/open/ruby_Inliner/lib/rexml/child.rb:30:in `replace_with'
    /home/magaudet/open/ruby_Inliner/lib/rexml/element.rb:506:in `text='
    /home/magaudet/open/ruby_Inliner/test/rexml/test_core.rb:519:in `test_element'

[365/425] REXMLTests::Tester#test_empty_doc = 0.00 s           
  2) Error:
REXMLTests::Tester#test_empty_doc:
NoMethodError: super: no superclass method `encoding_updated' for #<REXML::IOSource:0x0055a90e427340>

<snip>

Correctly (re)build the JIT object.

Currently, when you type make, the VM is built, but not the JIT object (librbjit.so).

Similarly, once the JIT is built, typing make librbjit.so never rebuilds the JIT, so what I've been doing is rm librbjit.so; make librbjit.so which is sort of exhausting 😫

Proper On Stack Replacement

Currently we manage JIT to Interperter transitions by restoring all interpreter state. We do this because JIT frames exist on the C-stack, along side interpreter frames, which is a problem when Ruby's exception handling story is implemented partially through setjmp and longjmp

To improve performance, it would be nice if the JIT only had to restore interpreter state when it absolutely had to. In order to do this, we need to implement proper on-stack replacement and connect it together with the exception handling system.

Before longjmping past a JIT frame, it needs to be given an opportunity to restore state it may have privatized, or we need to get the JIT frames off of the regular C call stack, and manage them ourselves.

Implement missing YARV bytecodes

Currently, a number of YARV bytecodes remain unimplemented in the Ruby JIT.

The list (as of dd6dae0) is:

  • getclassvariable
  • setclassvariable
  • putiseq
  • freezestring
  • reverse (see below)
  • checkkeyword
  • defineclass
  • opt_str_freeze
  • opt_newarray_max
  • opt_newarray_min
  • branchnil
  • getinlinecache
  • setinlinecache
  • once
  • opt_case_dispatch
  • opt_call_c_function
  • bitblt

`gem install syck` fails to build native extensions

Reproduce via:

  • run the preview docker image
  • run gem install syck

Observe the following output:

Fetching: syck-1.2.0.gem (100%)
Building native extensions.  This could take a while...
ERROR:  Error installing syck:
	ERROR: Failed to build gem native extension.

Digging into the compile error the rb_omr_markstate_t definition is missing:

In file included from syck.h:20:0,
                 from gram.y:16:
/usr/local/include/ruby-2.2.0/ruby/st.h:180:27: error: unknown type name 'rb_omr_markstate_t'
 void rb_omr_mark_st_table(rb_omr_markstate_t ms, st_table *table);

bundler tries to trap SIGABRT

Anything that runs through bundle exec fails with the following error ArgumentError: can't trap reserved signal: SIGABRT

As seen in here bundler considers only SEGV BUS ILL FPE VTALRM KILL STOP as reserved signals and then it traps all the signlas except those.

Additionaly, when I added ABRT to reserved signals it still continued to fail with the same error until I also added IOT to the same list. Only then it passed only to fail on nokogiri, but that is a thing for another issue

why is SIGABRT reserved in ruby-omr but not reserved in original ruby?

Recompile code @ Warm

With the option -JenableRubyTieredCompilation we will recompile methods. However, it appears we recompile at the same default opt level.

Add JIT command line option

Today we pass options to the JIT using OMR_JIT_OPTIONS environment variable.

Would be really nice to have a command line option.

make reconfig fails.

$ make -f common.mk BASERUBY=ruby MAKEDIRS='mkdir -p' srcdir=. update-config_files
common.mk:727: warning: overriding commands for target `/.timestamp/..time'
common.mk:717: warning: ignoring old commands for target `/.timestamp/..time'
downloading config.guess ... done
downloading config.sub ... done
$ autoconf
$ SPEC=linux_x86-64 ./configure -C --with-omr-jit --disable-install-doc --with-gcc=$CC 
$cp -pr config.status .ext/include config_1st
$ make reconfig
configure: loading cache config.cache
configure: error: `RBJITGLUE_DIR' was not set in the previous run
configure: error: `RELATIVE_OMR_DIR' was not set in the previous run
configure: error: in `/home/travis/build/mgaudet/ruby':
configure: error: changes in the environment can compromise the build
configure: error: run `make distclean' and/or `rm config.cache' and start over

Play nice with tailcalls

We've known for some time that the JIT + tailcall optimization in the RubyVM do not get along. Historically, this hasn't been a big deal, as tail calls weren't a particularly common occurrence.

Until 6fd18ca at which point they became part of the implementation of forwardable.rb, which isn't exactly uncommon :)

So, better handling of tailcalls has jumped in priority.

Restart work on the Ruby Inliner

We've got the bones of an inliner for Ruby, however, it's currently disabled because it needs updates for Ruby 2.4

I've got some of that work put together already.

Hook up preliminary Travis testing

This repo needs travis testing for the ruby_2_4_preliminary branch to avoid backsliding.

Currently, running make test should be the bar.

SEGV running test-all in rb_iseq_mark

[15938/17478] TestSyntax#test_keyword_rest/home/magaudet/open/ruby_Inliner/test/ruby/test_syntax.rb:120: [BUG] Segmentation fault at 0x000000000103e3
ruby 2.4.0p0 (2016-12-24 revision 57163) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0027 p:---- s:0156 e:000155 CFUNC  - :eval
c:0026 p:0010 s:0151 e:000150 BLOCK  - /home/magaudet/open/ruby_Inliner/test/ruby/test_syntax.rb:120 [FINISH]

/home/magaudet/open/ruby_Inliner/ruby(rb_vm_bugreport+0x53e) [0x5559befe88fe] vm_dump.c:685
/home/magaudet/open/ruby_Inliner/ruby(rb_bug_context+0xd1) [0x5559befdd901] error.c:426
/home/magaudet/open/ruby_Inliner/ruby(sigsegv+0x3e) [0x5559beed3ffe] signal.c:907
/lib/x86_64-linux-gnu/libpthread.so.0 [0x7efce50313d0]
/home/magaudet/open/ruby_Inliner/ruby(gc_mark_ptr+0x4e) [0x5559bedfe06e] gc.c:1111
/home/magaudet/open/ruby_Inliner/ruby(rb_iseq_mark+0xf5) [0x5559bee29295] iseq.c:128
/home/magaudet/open/ruby_Inliner/ruby(gc_marks_rest+0x74) [0x5559bee02094] gc.c:4629

According to backtrace, crash is here:

      RUBY_MARK_UNLESS_NULL(compile_data->mark_ary); 

lib/rss/parser.rb:163:parse being compiled causes a failure

magaudet@ubuntu1604VM:~/open/ruby_24 (ruby_2_4_omr_preliminary_contribution)*$ OMR_JIT_OPTIONS='-Xjit:count=0,limit={*parser.rb:163:parse*}' LD_LIBRARY_PATH=$PWD ./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems "./test/runner.rb" --ruby="./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems" --excludes-dir=./test/excludes test/rss
Run options: "--ruby=./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems" --excludes-dir=./test/excludes

# Running tests:

[ 22/305] RSS::TestAtomCore#test_to_xml = 0.01 s                       
  1) Error:
RSS::TestAtomCore#test_to_xml:
LocalJumpError: no block given
    /home/magaudet/open/ruby_24/lib/rss/parser.rb:80:in `parse'
    /home/magaudet/open/ruby_24/test/rss/test_atom.rb:220:in `test_to_xml'

<further failures snipped> 
/home/magaudet/open/ruby_24/test/lib/minitest/unit.rb:934: [BUG] vm_call_cfunc - cfp consistency error
ruby 2.4.0dev (2016-11-17 ruby_2_4_omr_p.. 56821) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0019 p:---- s:0108 e:000107 CFUNC  - :map
c:0018 p:0129 s:0104 e:000103 METHOD - /home/magaudet/open/ruby_24/test/lib/minitest/unit.rb:934
c:0017 p:0054 s:0092 e:000091 METHOD - /home/magaudet/open/ruby_24/test/lib/test/unit.rb:914
c:0016 p:0016 s:0085 e:000083 BLOCK  - /home/magaudet/open/ruby_24/test/lib/test/unit.rb:494 [FINISH]
c:0015 p:---- s:0079 e:000078 CFUNC  - :each
c:0014 p:0073 s:0075 e:000074 METHOD - /home/magaudet/open/ruby_24/test/lib/test/unit.rb:492
c:0013 p:0014 s:0068 e:000067 METHOD - /home/magaudet/open/ruby_24/test/lib/test/unit.rb:530
c:0012 p:0144 s:0061 e:000060 METHOD - /home/magaudet/open/ruby_24/test/lib/minitest/unit.rb:885
c:0011 p:0010 s:0050 e:000049 METHOD - /home/magaudet/open/ruby_24/test/lib/minitest/unit.rb:1096
c:0010 p:0010 s:0046 e:000045 BLOCK  - /home/magaudet/open/ruby_24/test/lib/minitest/unit.rb:1083 [FINISH]
c:0009 p:---- s:0042 e:000041 CFUNC  - :each
c:0008 p:0057 s:0038 e:000037 METHOD - /home/magaudet/open/ruby_24/test/lib/minitest/unit.rb:1082
c:0007 p:0020 s:0033 e:000032 METHOD - /home/magaudet/open/ruby_24/test/lib/minitest/unit.rb:1070
c:0006 p:0012 s:0028 e:000027 METHOD - /home/magaudet/open/ruby_24/test/lib/test/unit.rb:682
c:0005 p:0022 s:0022 e:000021 METHOD - /home/magaudet/open/ruby_24/test/lib/test/unit.rb:33
c:0004 p:0045 s:0017 e:000016 METHOD - /home/magaudet/open/ruby_24/test/lib/test/unit.rb:991
c:0003 p:0015 s:0013 e:000012 METHOD - /home/magaudet/open/ruby_24/test/lib/test/unit.rb:995
c:0002 p:0304 s:0008 E:001d38 EVAL   - ./test/runner.rb:40 [FINISH]
c:0001 p:0000 s:0003 E:000d40 (none) - [FINISH]

Investigate unsetting VM_FRAME_FLAG_JITTED on fallback to VM path

In some circumstances, a JIT body may decide it cannot handle what the interpreter wishes it to do. In those cases, we fall back to the interpreter by invoking vm_exec.

When we do that, in 2.2 we would unset VM_FRAME_FLAG_JITTED, but in the course of a forward port to Ruby 2.4 this was disabled.

This actually needs to be investigated, and given some thought. The comment says that there is only a RAS ('Reliability, Availability, Servicability') impact... however, during the investigation of resuscitating the inliner it became clear that in some circumstances that flag changes other behaviour in the VM.

While I don't have an issue that I can directly point at this, we need to decide what to do about this flag.

function 'vm_opt_call_c_function_jit' should not return a value

The callback bodies in vm_jit.inc are automatically generated by VmJitBodyGenerator in tool/instruction.rb

Unfortunately, that generator doesn't handle exceptions. A previous pass has attempted to map THROW_EXCEPTION to assert(0); However, that line is commented out and I don't recall why at the moment.

Probably just should stop emitting callbacks with impossible semantics.

Get an overview of status of 2.4 JIT

Currently, when running make test-all, due to #9, we fail relatively early.

It would be nice to trick test-all into continuing, to get a feeling for how many other crashers there are.

Unsafe Ruby Code

What I'd love is to be able to write ruby implementations of core ruby methods, that can be used to replace C calls for the purposes of inlining.

One way would be to provide an 'unsafe' module similar to sun.misc.unsafe in Java, in order to allow the JIT compiler to generate high quality code for these functions.

Fiddle may provide a good base for this.

Programatically compile a RubyVM::InstructionSequence

A script like this:

#!/usr/bin/ruby

puts RubyVM::InstructionSequence.compile_file(ARGV[0]).disasm

is able to dump all the bytecodes of functions defined in a file:

== catch table
| catch type: redo   st: 0002 ed: 0029 sp: 0000 cont: 0002
| catch type: next   st: 0002 ed: 0029 sp: 0000 cont: 0029
|------------------------------------------------------------------------
local table (size: 3, argc: 2 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
[ 3] k<Arg>     [ 2] v<Arg>     
0000 trace            256                                             (  64)
0002 trace            1
0004 putnil           
0005 getlocal_OP__WC__1 8
0007 getlocal_OP__WC__0 3
0009 getinstancevariable :@build_context, <is:0>
0012 opt_send_without_block <callinfo!mid:entities, argc:0, ARGS_SIMPLE>, <callcache>
0015 getlocal_OP__WC__0 3
0017 opt_aref         <callinfo!mid:[], argc:1, ARGS_SIMPLE>, <callcache>
0020 opt_send_without_block <callinfo!mid:value, argc:0, ARGS_SIMPLE>, <callcache>
0023 setn             3
0025 opt_aset         <callinfo!mid:[]=, argc:2, ARGS_SIMPLE>, <callcache>
0028 pop              
0029 trace            512
0031 leave            

It would be nice for problem determination purposes if we were able to also trigger compiles of all the bytecodes in a file (though, this would be devoid of profile information, so passes dependent on it such as inlining would be blind)

IMO, the way to do this would involve a couple of steps:

  • Create RubyVM::InstructionSequence#children that would use the same sort of logic as iseqw_disasm to return an array of all the child iseqs reachable from a given InstructionSequence.
  • Create RubyVM::JIT::compile_iseq that takes an iseq as the argument.

Optionally emit a warning when code cache is full

When running a large suite of tests, I always have a small pit in my stomach about half way through:

At this point, are we still testing the JITted code, or have we filled up the code cache, and are just bypassing compiles?

It would be nice to optionally emit a warning when methods stop getting compiled due to code-cache saturation.

Provide mechanism to override C methods with equivalent ruby versions

To experiment with inlining, it would be worth considering providing a uniform mechanism by which a pure-ruby implementation of a core C method could be picked up. I am imaging a special prelude module that would check a directory for 'bootstrap' overrides -- to borrow and abuse a term from Java.

Intermittent failure in TestException#test_stackoverflow

So far, I've seen hangs, and segfaults occurring in TestException#test_stackoverflow running 94462bf with

LD_LIBRARY_PATH=$PWD OMR_JIT_OPTIONS=count=0 make test-all TESTS=test/ruby/test_exception.rb

In the hang case, I attached with GDB, and found SDR spewed out more than my termincal could take: Here's a snippet:

c:9844 p:0015 s:39426 E:001650 METHOD J /home/magaudet/open/ruby_Inliner/test/ruby/test_exception.rb:570 [FINISH]
c:9843 p:0015 s:39422 E:0015f0 METHOD J /home/magaudet/open/ruby_Inliner/test/ruby/test_exception.rb:570 [FINISH]
c:9842 p:0015 s:39418 E:001590 METHOD J /home/magaudet/open/ruby_Inliner/test/ruby/test_exception.rb:570 [FINISH]
c:9841 p:0015 s:39414 E:001530 METHOD J /home/magaudet/open/ruby_Inliner/test/ruby/test_exception.rb:570 [FINISH]
c:9840 p:0015 s:39410 E:0014d0 METHOD J /home/magaudet/open/ruby_Inliner/test/ruby/test_exception.rb:570 [FINISH]
c:9839 p:0015 s:39406 E:001470 METHOD J /home/magaudet/open/ruby_Inliner/test/ruby/test_exception.rb:570 [FINISH]

In the segfault case, the core file isn't providing a lot of information. ruby_current_thread is null, and the IP looks bogus.

Missing output

# test_from_prime_division.rb
require 'prime'

def assert_equal expected, actual, msg=""
   if expected == actual  
      puts caller_locations[0] 
   else 
      puts "error! #{caller_locations[0]}" 
      puts "#{expected} != #{actual}" 
      raise "error"
   end
end

class Foo 
# The first 100 prime numbers
  PRIMES = [
    2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37,
    41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83,
    89, 97, 101, 103, 107, 109, 113, 127, 131,
    137, 139, 149, 151, 157, 163, 167, 173, 179,
    181, 191, 193, 197, 199, 211, 223, 227, 229,
    233, 239, 241, 251, 257, 263, 269, 271, 277,
    281, 283, 293, 307, 311, 313, 317, 331, 337,
    347, 349, 353, 359, 367, 373, 379, 383, 389,
    397, 401, 409, 419, 421, 431, 433, 439, 443,
    449, 457, 461, 463, 467, 479, 487, 491, 499,
    503, 509, 521, 523, 541,
  ]

  def test_from_prime_division
    assert_equal PRIMES.inject(&:*),  Integer.from_prime_division(PRIMES.map{|p| [p,1]})
    assert_equal(-PRIMES.inject(&:*), Integer.from_prime_division([[-1, 1]] + PRIMES.map{|p| [p,1]}))
  end

end

Foo.new.test_from_prime_division

We see that any output is lost here!

$ OMR_JIT_OPTIONS=-Xjit:count=0,limit={*prime.rb:172:int_from_prime_division*} LD_LIBRARY_PATH=$PWD  ./ruby --disable-gems -Ilib test_from_prime_division.rb
$ ./ruby --disable-gems -Ilib test_from_prime_division.rb 
test_from_prime_division.rb:30:in `test_from_prime_division'
test_from_prime_division.rb:31:in `test_from_prime_division'

Unexpected object allocation

#
# test_AREF_fstring_key.rb  
# 
def assert_equal expected, actual, msg=""
   if expected == actual  
      puts caller_locations[0] 
   else 
      puts "error! #{caller_locations[0]}" 
      puts "#{expected} != #{actual}" 
      raise "error"
   end

end


def test_AREF_fstring_key
  h = {"abc" => 1}
  before = GC.stat(:total_allocated_objects)
  5.times{ h["abc"] }
  assert_equal before, GC.stat(:total_allocated_objects)
end

test_AREF_fstring_key
$ OMR_JIT_OPTIONS=-Xjit:count=0 LD_LIBRARY_PATH=$PWD ./ruby --disable-gems test_AREF_fstring_key.rb 
error! test_AREF_fstring_key.rb:17:in `test_AREF_fstring_key'
7313 != 7314
test_AREF_fstring_key.rb:7:in `assert_equal': error (RuntimeError)
	from test_AREF_fstring_key.rb:17:in `test_AREF_fstring_key'
	from test_AREF_fstring_key.rb:20:in `<main>'
$ ruby --disable-gems test_AREF_fstring_key.rb 
test_AREF_fstring_key.rb:17:in `test_AREF_fstring_key'

Improve debuggability in trace logs

During ilgen, we know a lot about a call from the bytecode:

0021: 30 opt_send_without_block      end_with? (mid:6783) <argc:1 flag:0x10 (VM_CALL_ARGS_SIMPLE|0x0)>, E:93825003299064

By the time it becomes trees, we've obfuscated the information making it difficult to actually figure out what's happening:

n106n     treetop                                                                      
n101n       lcall  vm_send_without_block[#286  helper Method] [flags 0x400 0x0 ] ()    
n100n         aload  <parm 0 Lrb_thread_t;>[#611  Parm] [flags 0x40010107 0x0 ]        
n99n          aconst 0x555555fe2000                                                    
n98n          aconst 0x555555fe20f8                                                    
n80n          ==>lloadi

It would be really nice to know what that call is trying to resolve.

Improve aliasing of callbacks

Currently we assume a very conservative aliasing on pretty much all callbacks.

This is sufficient for correctness, but it would be nice to loosen this restriction to minimize rematerialization overhead where possible.

me->def != NULL, but not dereferenceable (@warm)

I'm seeing intermittent crashes in a number of scenarios when running with optLevel=warm regarding the method entry definition being non-null, but also not dereferenceable. The stack traces are always something like this:

/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(rb_vm_bugreport+0x53e) [0x55aa84ab19ce] vm_dump.c:685
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(rb_bug_context+0xd1) [0x55aa84aa69d1] error.c:426
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(sigsegv+0x3e) [0x55aa8499cffe] signal.c:907
/lib/x86_64-linux-gnu/libpthread.so.0 [0x7fc2d9f263d0]
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_InlinerBase::checkInlineableWithoutInitialCalleeSymbol(TR_CallSite*, TR::Compilation*)+0x36d) [0x7fc2d82f66b5] /home/magaudet/open/ruby_Inliner/rbjitglue/ruby/optimizer/RubyInliner.cpp:260
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_InlinerBase::getSymbolAndFindInlineTargets(TR_CallStack*, TR_CallSite*, bool)+0x2c8) [0x7fc2d832bcfc] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/Inliner.cpp:3824
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_DumbInliner::analyzeCallSite(TR_CallStack*, TR::TreeTop*, TR::Node*, TR::Node*)+0x11d) [0x7fc2d83210ad] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/Inliner.cpp:1287
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_DumbInliner::inlineCallTargets(TR::ResolvedMethodSymbol*, TR_CallStack*, TR_InnerPreexistenceInfo*)+0x551) [0x7fc2d8320df3] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/Inliner.cpp:1249
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_InlinerBase::performInlining(TR::ResolvedMethodSymbol*)+0xe4) [0x7fc2d831d8e0] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/Inliner.cpp:419
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(Ruby::TrivialInliner::perform()+0x10f) [0x7fc2d82f7245] /home/magaudet/open/ruby_Inliner/rbjitglue/ruby/optimizer/RubyTrivialInliner.cpp:48
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(OMR::Optimizer::performOptimization(OptimizationStrategy const*, int, int, int)+0x2a2d) [0x7fc2d839244d] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/OMROptimizer.cpp:2007
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(OMR::Optimizer::optimize()+0x88b) [0x7fc2d838f14d] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/OMROptimizer.cpp:1106
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(OMR::Compilation::performOptimizations()+0x139) [0x7fc2d81f99ab] /home/magaudet/open/ruby_Inliner/omr/compiler/compile/OMRCompilation.cpp:1093
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(OMR::Compilation::compile()+0x92b) [0x7fc2d81f92bd] /home/magaudet/open/ruby_Inliner/omr/compiler/compile/OMRCompilation.cpp:882
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(compileMethodFromDetails(OMR_VMThread*, TR::IlGeneratorMethodDetails&, TR_Hotness, int&)+0x618) [0x7fc2d8185776] /home/magaudet/open/ruby_Inliner/omr/compiler/control/CompileMethod.cpp:340
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(compileMethod(OMR_VMThread*, TR_ResolvedMethod&, TR_Hotness, int&)+0x59) [0x7fc2d8185113] /home/magaudet/open/ruby_Inliner/omr/compiler/control/CompileMethod.cpp:235
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(compileRubyISeq(rb_iseq_struct*, char const*, TR_Hotness)+0x7a) [0x7fc2d81881fa] /home/magaudet/open/ruby_Inliner/rbjitglue/ruby/control/RubyJit.cpp:281
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(jit_compile+0x3f8) [0x7fc2d8188652] /home/magaudet/open/ruby_Inliner/rbjitglue/ruby/control/RubyJit.cpp:386
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(vm_jit.isra.101+0x2d) [0x55aa84a07aed] vm_insnhelper.c:2851
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(vm_exec+0x85e) [0x55aa84a18dfe] vm.c:1729
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(vm_send_without_block+0x8e) [0x55aa84a1aeae] vm_insnhelper.c:2971
[0x7fc2cf8faa78]/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(rb_vm_bugreport+0x53e) [0x55aa84ab19ce] vm_dump.c:685
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(rb_bug_context+0xd1) [0x55aa84aa69d1] error.c:426
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(sigsegv+0x3e) [0x55aa8499cffe] signal.c:907
/lib/x86_64-linux-gnu/libpthread.so.0 [0x7fc2d9f263d0]
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_InlinerBase::checkInlineableWithoutInitialCalleeSymbol(TR_CallSite*, TR::Compilation*)+0x36d) [0x7fc2d82f66b5] /home/magaudet/open/ruby_Inliner/rbjitglue/ruby/optimizer/RubyInliner.cpp:260
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_InlinerBase::getSymbolAndFindInlineTargets(TR_CallStack*, TR_CallSite*, bool)+0x2c8) [0x7fc2d832bcfc] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/Inliner.cpp:3824
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_DumbInliner::analyzeCallSite(TR_CallStack*, TR::TreeTop*, TR::Node*, TR::Node*)+0x11d) [0x7fc2d83210ad] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/Inliner.cpp:1287
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_DumbInliner::inlineCallTargets(TR::ResolvedMethodSymbol*, TR_CallStack*, TR_InnerPreexistenceInfo*)+0x551) [0x7fc2d8320df3] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/Inliner.cpp:1249
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(TR_InlinerBase::performInlining(TR::ResolvedMethodSymbol*)+0xe4) [0x7fc2d831d8e0] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/Inliner.cpp:419
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(Ruby::TrivialInliner::perform()+0x10f) [0x7fc2d82f7245] /home/magaudet/open/ruby_Inliner/rbjitglue/ruby/optimizer/RubyTrivialInliner.cpp:48
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(OMR::Optimizer::performOptimization(OptimizationStrategy const*, int, int, int)+0x2a2d) [0x7fc2d839244d] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/OMROptimizer.cpp:2007
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(OMR::Optimizer::optimize()+0x88b) [0x7fc2d838f14d] /home/magaudet/open/ruby_Inliner/omr/compiler/optimizer/OMROptimizer.cpp:1106
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(OMR::Compilation::performOptimizations()+0x139) [0x7fc2d81f99ab] /home/magaudet/open/ruby_Inliner/omr/compiler/compile/OMRCompilation.cpp:1093
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(OMR::Compilation::compile()+0x92b) [0x7fc2d81f92bd] /home/magaudet/open/ruby_Inliner/omr/compiler/compile/OMRCompilation.cpp:882
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(compileMethodFromDetails(OMR_VMThread*, TR::IlGeneratorMethodDetails&, TR_Hotness, int&)+0x618) [0x7fc2d8185776] /home/magaudet/open/ruby_Inliner/omr/compiler/control/CompileMethod.cpp:340
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(compileMethod(OMR_VMThread*, TR_ResolvedMethod&, TR_Hotness, int&)+0x59) [0x7fc2d8185113] /home/magaudet/open/ruby_Inliner/omr/compiler/control/CompileMethod.cpp:235
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(compileRubyISeq(rb_iseq_struct*, char const*, TR_Hotness)+0x7a) [0x7fc2d81881fa] /home/magaudet/open/ruby_Inliner/rbjitglue/ruby/control/RubyJit.cpp:281
/home/magaudet/.rbenv/versions/ruby_2_4_omr/lib/librbjit.so(jit_compile+0x3f8) [0x7fc2d8188652] /home/magaudet/open/ruby_Inliner/rbjitglue/ruby/control/RubyJit.cpp:386
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(vm_jit.isra.101+0x2d) [0x55aa84a07aed] vm_insnhelper.c:2851
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(vm_exec+0x85e) [0x55aa84a18dfe] vm.c:1729
/home/magaudet/.rbenv/versions/ruby_2_4_omr/bin/ruby(vm_send_without_block+0x8e) [0x55aa84a1aeae] vm_insnhelper.c:2971
[0x7fc2cf8faa78]

We are peeking inside the call cache at this point, to see if we can't find ourselves an inlinable target.

The rough outline of the code is something like this:

   CALL_CACHE cc = <load call cache>;
   const rb_callable_method_entry_t *me = cc->me;
   if(!me)
      {
      // abort 
      }

   if (!me->def)
      {
      // abort 
      }

   switch (METHOD_ENTRY_VISI(cc->me))
      {
      case METHOD_VISI_PUBLIC:
         break; // OK for inlining.
      case METHOD_VISI_PRIVATE:
         if (ci->flag & VM_CALL_FCALL)
            break; // Ok for inlining
      default:
         //abort 
      }

   switch(me->def->type) // Crashes here de-referencing me->def 
      {
      // ...
      }

So it seems there's something missing here.

One thing I'm considering is that I'm attempting to inline a collected method -- and so really, what I've been inspecting is garbage.

Fix getLocalName

The function getLocalName seems to always return "?".

You can find the definition here;

Put ilgen trace under a trace option

While useful when things are going wrong, a trace log riddled with

        [48] writing pending to stack slot 0 (N = 0x55bfd99dae18)
Rematerializing SP, adding 1
RubyIlGenerator:: Generating bytecode 51 opt_gt into block 0x55bfd99c5890. Stack height is 2
RubyIlGenerator:: Generating bytecode 54 branchunless into block 0x55bfd99c5890. Stack height is 1

where you don't want it is frustrating ;) Should be under a more specific trace option just like traceInlining.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.