Giter Site home page Giter Site logo

artichoke / ferrocarril Goto Github PK

View Code? Open in Web Editor NEW
62.0 6.0 2.0 12.15 MB

๐Ÿš† Experiments to embed Ruby on Rails in Rust with mruby

Home Page: https://artichoke.github.io/ferrocarril/mruby/

License: MIT License

Ruby 33.82% Rust 64.27% HTML 0.19% C++ 0.11% C 1.03% Shell 0.58%
artichoke rack unicorn sinatra ruby rust

ferrocarril's Introduction

ferrocarril

CircleCI

Archived: ferrocarril has been superseded by Artichoke.

ferrocarril aims to embed a Ruby on Rails web application that talks to an external MySQL database in Rust and serve the app with Rocket.

ferrocarril means railway in Spanish and sounds like ferrous which means containing iron.

hubris

The hubris crate runs a Sinatra::Base echo server with Nemesis.

Usage

HUBRIS_LOG=info cargo run --bin hubris

Then, open http://localhost:8000 on your browser.

Features

  • Utilizes a Rust implenentation of Regexp and MatchData.
  • Utilizes a stack-based synchronous implementation of Thread.
  • Utilizes a complete implementation of Ruby 2.6.3 String API that uses a hybrid of mruby C, Ruby, and Rust.
  • Utilizes implementations of Ruby standard library packages customized for mruby, including delegate, forwardable, json, monitor, ostruct, set, strscan, and uri.
  • Utilizes patched versions of Sinatra and its dependencies.

foolsgold

The foolsgold crate is an early attempt to achieve the goal of a Rust-backed Ruby web application.

Usage

FOOLSGOLD_LOG=info cargo run --bin foolsgold

Then, open http://localhost:8000 on your browser.

Features

REPL

Crate mruby-bin provides an rirb executable that is an IRB shell and REPL for the mruby interpreter in this workspace. rirb aims to load every extension to mruby made by this workspace in addition to all gems in the gems crate.

Usage

cargo run --bin rirb

Contributing

There is a lot to build! If you'd like to help out, take a look at the open issues. Tickets that are tagged with good first issue might be a good introduction to the codebase.

Setup

A ferrocarril development environment has several dependencies. Setup your development environment.

Code Overview

To familiarize yourself with the code in this workspace, consider reviewing these source files:

File Purpose Learning Objective
manual.rs mruby crate integration test Define Rust-backed Ruby sources
extn::regexp Regexp implementation Implement a Ruby Core class with a mix of Rust and Ruby
nemesis.rs nemesis crate Ruby runtime Define a Gem
ffi_tests.rs C API test suite Manipulate an interpreter with the C API
repl.rs REPL loop for rirb Eval code on an interpreter and handle errors

Known Missing Features

Core

mruby does not implement all Ruby 2.6 core classes.

Required classes include (at least):

  • File
  • IO
  • Regexp

Standard Library

mruby does not implement any of the Ruby 2.6 standard library.

See the stdlib tracking ticket (GH-8) for more details.

Gems

Rails requires lots of gems. This workspace maintains a registry of vendored gems. To support the goal of running Rails, this crate identifies dependencies, vendors the gem sources, patches gems so they parse on mruby, reimplements C extensions in Rust, and runs the tests for each gem.

ferrocarril's People

Contributors

felipecsl avatar lopopolo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

felipecsl isgasho

ferrocarril's Issues

Infallible converters

Some converters are infallible. For example, immediate objects like nil, true, false, and Fixnum do not require heap allocation or FFI type coercion so they cannot fail.

Introduce a new trait in the convert package called FromMrb<T> which returns Self as opposed to TryFromMrb<T> which returns Result<Self, MrbError>.

It should be easy to derive a TryFromMrb<T> implementation for all types that implement FromMrb<T>.

Update rust-embed to pull in workspace fix

v4.5.0 of rust-embed was just released which includes my PR pyrossh/rust-embed#67 to add env var interpolation support to folder paths on embed derives.

In a workspace, rust-embed folder paths should be relative to CARGO_MANIFEST_DIR because doc tests run relative to the crate root whereas invoking cargo from the workspace root is relative to the workspace root.

ferrorcarril turns off doctests to work around this past limitation in a number of crates:

https://github.com/lopopolo/ferrocarril/blob/66e5d361ae6e9c19b89a149d18ae0e7c71b92ff0/mruby-gems/Cargo.toml#L7-L13

This hack should be undone in:

  • foolsgold
  • mruby
  • mruby-gems
  • nemesis

After pulling in this fix, test to see if it is possible to not require the debug-embed feature to speed up build times and ruby source iteration speed in debug builds.

Calling Kernel#require in a rescue block after a failed require panics

The following code results in an 'already borrowed: BorrowMutError'

def fail
  begin
    require 'foo'
  rescue LoadError
    require 'forwardable'
  end
end
fail
thread 'main' panicked at 'already borrowed: BorrowMutError', src/libcore/result.rs:999:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /Users/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /Users/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:198
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:212
   6: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:475
   7: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:382
   8: rust_begin_unwind
             at src/libstd/panicking.rs:309
   9: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  10: core::result::unwrap_failed
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libcore/macros.rs:18
  11: core::result::Result<T,E>::expect
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libcore/result.rs:827
  12: core::cell::RefCell<T>::borrow_mut
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libcore/cell.rs:871
  13: <alloc::rc::Rc<core::cell::RefCell<mruby::state::State>> as mruby::eval::MrbEval>::push_context
             at mruby/src/eval.rs:153
  14: mruby::extn::core::kernel::Kernel::require_impl
             at mruby/src/extn/core/kernel.rs:113
  15: mruby::extn::core::kernel::Kernel::require
             at mruby/src/extn/core/kernel.rs:139
  16: mrb_vm_exec
             at ./mruby-sys/vendor/mruby-c078758/src/vm.c:1466
  17: mrb_vm_run
             at ./mruby-sys/vendor/mruby-c078758/src/vm.c:974
  18: mrb_top_run
             at ./mruby-sys/vendor/mruby-c078758/src/vm.c:2862
  19: mrb_load_exec
             at ./mruby-sys/vendor/mruby-c078758/mrbgems/mruby-compiler/core/parse.y:6192
  20: mrb_load_nstring_cxt
             at ./mruby-sys/vendor/mruby-c078758/mrbgems/mruby-compiler/core/parse.y:6214
  21: <alloc::rc::Rc<core::cell::RefCell<mruby::state::State>> as mruby::eval::MrbEval>::eval
             at mruby/src/eval.rs:108
  22: mruby_bin::repl::run
             at ./mruby-bin/src/repl.rs:121
  23: rirb::main
             at mruby-bin/src/bin/rirb.rs:8
  24: std::rt::lang_start::{{closure}}
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libstd/rt.rs:64
  25: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:49
  26: std::panicking::try::do_call
             at src/libstd/panicking.rs:294
  27: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:85
  28: std::panicking::try
             at src/libstd/panicking.rs:273
  29: std::panic::catch_unwind
             at src/libstd/panic.rs:388
  30: std::rt::lang_start_internal
             at src/libstd/rt.rs:48
  31: std::rt::lang_start
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libstd/rt.rs:64
  32: rirb::main
Illegal instruction: 4

Extract the Rocket web application server into its own crate

To make these pieces reusable for the foolsgold crate and a future Rails crate, extract the thin-shaped bits into their own crate. I like the name nemesis.

The public interface for nemesis should take a FnMut(Mrb) -> Value where Value is a Rack "app".

From https://rack.github.io/:

A Rack app is an object that responds to the call method, taking the environment hash as a parameter, and returning an Array with three elements:

  • The HTTP response code
  • A Hash of headers
  • The response body, which must respond to each

Nemesis will probably use TryFromMrb converters to generate an env hash and call mruby::sys::mrb_funcall to pass the hash to the call method on the app.

Implement Clone for Value

If a Value's ruby_type is mrb_vtype::MRB_TT_DATA, we need to clone the Rc smart pointer that it wraps. Otherwise, clone the mrb_value directly.

Implement a prelude module for mruby crate

A common pattern in Rust is a prelude module which re-exports common items from a crate to allow easier usability with a glob import.

There are a metric boatload of use directives required to use the mruby crate.

https://github.com/lopopolo/ferrocarril/blob/2144cf230360e18937664393b4f0e245718386a1/foolsgold/src/foolsgold.rs#L1-L12

Implement an mruby::prelude module to make this easier and clean things up. At a minimum it should expose the interpreter, value, converter traits, errors, and all of the traits the interpreter implements. Also rather than importing individual macros, just use:

#[macro_use]
extern crate mruby;

Make MrbFile::require fallible

There is currently a lot of unwrapping going on in MrbFile implementations. Make require return Result<(), MrbError> to propagate these errors to the HTTP layer.

Complete the implementation of nemesis::request::Request::to_env

The keys in the environment hash are required by the Rack spec: https://www.rubydoc.info/github/rack/rack/file/SPEC#label-The+Environment

This implementation is incomplete:

  • TODO: Set SCRIPT_NAME from Rocket mount path.
  • TODO: Set SERVER_NAME instead of hardcoding 'localhost'.
  • TODO: Set SERVER_PORT instead of hardcoding it to 8000.
  • TODO: Set HTTP_VERSION instead of hardcoding it to '1.1'.
  • TODO: Set RACK_URL_SCHEME instead of hardcoding it to 'http'.
  • TODO: Set RACK_INPUT and RACK_ERRORS once IO is implemented. See GH-9.
  • TODO: RUN_ONCE should be true if in shared nothing execution mode

Document minimum supported Rust version

mruby* crates require 1.34.0 because of the TryFrom dependency.

foolsgold and nemesis depend on Rocket, so they require nightly.

Add something like this to CircleCI build:

cargo +1.34.0 test -p mruby -p mruby-bin -p mruby-gems -p mruby-sys -p mruby-vfs

Run Rack test suite in tests for mruby-rack

Rack's tests depend only on Ruby Core and bacon. Bacon is a 350-line RSpec clone with no dependencies.

Running Rack's tests will ensure we have ported it (there are some patches we'll need to write to get it to parse) to mruby correctly.

Document developer environment setup

  • install Rust toolchain with rustup
  • install rustfmt
  • install clippy
  • install node
  • yarn install
  • linting locally
  • running tests
  • updating Rust deps with Cargo
  • updating JavaScript deps with yarn

RustBackedValue::try_into_ruby holds a borrow while executing downstream Rust code

In MatchData#eql?:

.thread 'main' panicked at 'already borrowed: BorrowMutError', src/libcore/result.rs:999:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /Users/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /Users/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:198
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:212
   6: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:475
   7: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:382
   8: rust_begin_unwind
             at src/libstd/panicking.rs:309
   9: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  10: core::result::unwrap_failed
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libcore/macros.rs:18
  11: core::result::Result<T,E>::expect
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libcore/result.rs:827
  12: core::cell::RefCell<T>::borrow_mut
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libcore/cell.rs:871
  13: mruby::value::ValueLike::funcall
             at mruby/src/value/mod.rs:119
  14: mruby::value::Value::itself
             at mruby/src/value/mod.rs:362
  15: mruby::extn::core::regexp::enc::parse
             at mruby/src/extn/core/regexp/enc.rs:48
  16: mruby::extn::core::regexp::initialize::Args::extract
             at mruby/src/extn/core/regexp/initialize.rs:57
  17: mruby::extn::core::regexp::Regexp::initialize
             at mruby/src/extn/core/regexp/mod.rs:194
  18: mrb_funcall_with_block
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:522
  19: mrb_funcall_argv
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:539
  20: mrb_obj_new
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/class.c:1512
  21: mruby::convert::object::RustBackedValue::try_into_ruby
             at mruby/src/convert/object.rs:76
  22: mruby::extn::core::matchdata::regexp::method
             at mruby/src/extn/core/matchdata/regexp.rs:15
  23: mruby::extn::core::matchdata::MatchData::regexp
             at mruby/src/extn/core/matchdata/mod.rs:263
  24: mrb_vm_exec
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:1441
  25: mrb_vm_run
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:949
  26: mrb_top_run
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:2844
  27: mrb_load_exec
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/mrbgems/mruby-compiler/core/parse.y:6321
  28: mrb_load_nstring_cxt
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/mrbgems/mruby-compiler/core/parse.y:6343
  29: <alloc::rc::Rc<core::cell::RefCell<mruby::state::State>> as mruby::eval::MrbEval>::eval::run_protected
             at mruby/src/eval.rs:115
  30: mrb_protect
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/mrbgems/mruby-error/src/exception.c:16
  31: <alloc::rc::Rc<core::cell::RefCell<mruby::state::State>> as mruby::eval::MrbEval>::eval
             at mruby/src/eval.rs:155
  32: <alloc::rc::Rc<core::cell::RefCell<mruby::state::State>> as mruby::eval::MrbEval>::eval_with_context
             at mruby/src/eval.rs:190
  33: mruby::extn::core::kernel::Kernel::require_impl
             at mruby/src/extn/core/kernel.rs:129
  34: mruby::extn::core::kernel::Kernel::require
             at mruby/src/extn/core/kernel.rs:166
  35: mrb_vm_exec
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:1441
  36: mrb_vm_run
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:949
  37: mrb_run
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:2823
  38: mrb_funcall_with_block
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:529
  39: mrb_funcall_argv
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/src/vm.c:539
  40: mruby::value::ValueLike::funcall::run_protected
             at mruby/src/value/mod.rs:76
  41: mrb_protect
             at /Users/lopopolo/dev/repos/ferrocarril/target/debug/build/mruby-sys-a21763ba2a043d0b/out/mruby-1685c45/mrbgems/mruby-error/src/exception.c:16
  42: mruby::value::ValueLike::funcall
             at /Users/lopopolo/dev/repos/ferrocarril/mruby/src/value/mod.rs:126
  43: spec_runner::mspec::Runner::run
             at spec-runner/src/mspec.rs:58
  44: spec_runner::main
             at spec-runner/src/main.rs:37
  45: std::rt::lang_start::{{closure}}
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libstd/rt.rs:64
  46: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:49
  47: std::panicking::try::do_call
             at src/libstd/panicking.rs:294
  48: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:85
  49: std::panicking::try
             at src/libstd/panicking.rs:273
  50: std::panic::catch_unwind
             at src/libstd/panic.rs:388
  51: std::rt::lang_start_internal
             at src/libstd/rt.rs:48
  52: std::rt::lang_start
             at /rustc/d132f544f9d74e3cc047ef211e57eae60b78e5c5/src/libstd/rt.rs:64
  53: spec_runner::main
Illegal instruction: 4

Register global Rust objects in the State

Rust implementations of Ruby classes may want to access Rust state that lasts the life of the mruby interpreter. The way we do this now is with a static that is referenced in Rust functions that get embedded in the interpreter.

We may wish to share state that is not static. For example, when implementing the SQLite gem, we may want to load a fixture database and make it accessible to the interpreter.

To do this, I think we can define a module in Ruby and store globals within it as ivars by wrapping them in an Arc<Mutex<_>> via an MRB_TT_DATA type. We would need to expose mechanisms for retrieving the global on the interpreter instance.

Implement Rust wrappers around Value

This is a tracking ticket for implementing structs that wrap a Value with a single Ruby type, e.g. a struct that wraps mrb_vtype::MRB_TT_HASH, mrb_vtype::MRB_TT_ARRAY, or mrb_vtype::MRB_TT_DATA.

In order to implement Clone safely (see GH-3), Values with Ruby::Data Ruby type need to know their underlying T, but Value can't be generic on T. Wrapping Value and calling those wrappers from consumers of the mruby crate enables application code to check their invariants about the types of objects they are getting back from the mruby VM.

At a minimum, each wrapper should implement:

  • new(Value) -> Result<Self,MrbError>
  • Clone::clone
  • funcall(self, method: String, args: &[Value]) -> Result<Value, MrbError> (we might be able to move this to a ValueLike trait and provide a default impl).
  • funcall_with_block(self, method: String, args: &[Value], blk: Proc) -> Result<Value, MrbError> (out of scope).

These wrappers are also a convenient place to implement the methods exposed by the mruby VM on Rust types, e.g. String#sub.

This might be a means to implement a Proc wrapper in terms of Rust closures, but that is out of scope for this tracking ticket.

Child tickets:

document mruby crate

I really like the Snafu docs:

It makes sense to have one guide module per trait exposed on the interpreter, one for Value, one for converters.

Realistically I think it makes sense to hold off on this until the mruby API is stabilized.

Typo in mruby crate README and docs

The ValueLike trait exposes a funcall interface which can call Ruby functions on a Value using a String function name and a Vec or arguments.

Should be a Vec of arguments.

Optimize String#scan

rruby is a one-off mruby executor I built for this benchmark. It loads the string directly from disk because it is too large for mruby to gen code for.

$ cat bench.rb | ./target/release/rruby
Email: 5542.66ms - 92
URI: 132680.38ms - 5301
IP: 302.4ms - 5
$ ruby regexp.rb
Email: 296.16ms - 92
URI: 258.18ms - 5301
IP: 48.69ms - 5

The String#scan implementation uses Ranges to index into the remainder of the String which is expensive. This can probably be done with a position parameter passed to Regexp#match, which would let the implementation live in pure Rust.

rruby flamegraph:

Screen Shot 2019-06-27 at 5 12 40 PM

Maybe implement String#scan in Rust but I don't think we need to.

Make the same optimization for StringScanner.

Implement Value::funcall

Implement a safe funcall interface on Value aroundmrb_funcall_argv. Should play nice with GH-43.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.