Eval2C

Eval2C is a class that allows the compilation of Ruby code to a C extension at runtime. The compiled C extension will be required automatically and the compiled code will be available as a Proc.

It is easy to integrate Eval2C into existing scripts or libraries to improve performance. It is also pretty simple to provide fallback code for when Ruby2CExtension is not available.

Sections: Basic Usage, Options, Implementation Details, Using Eval2C Only When It Is Available.

Basic Usage

Here is an example:

require "ruby2cext/eval2c" 

$e2c = Ruby2CExtension::Eval2C.new

$e2c.toplevel_eval("puts 'hello'") # prints hello

First you need to create an Eval2C instance, this instance basically stores the options/configuration. In the above example no options were given, so the defaults are used. The available options are explained below.

The last line of the example does what the method name suggests: the string of Ruby code is evaluated at the toplevel (i.e. not inside any class scope). To be more precise, that last line is equivalent to this Ruby code:

$some_global_var = proc { puts 'hello' }
$some_global_var.call
The proc is compiled to a C extension, that C expension is then required and finally the proc is called.

The implementation of toplevel_eval is actually pretty simple:

def toplevel_eval(code_str)
  compile_to_proc(code_str).call
end

So the main work is done by compile_to_proc, which can also be used directly:

$e2c.compile_to_proc("|a,b| a+b").call(2, 3) # => 5

There are some things to note: The proc won’t have access to local variables and it will be defined at toplevel, which is important for constant lookup:

SOME_CONST = :toplevel

class A
  SOME_CONST = :A
  $e2c.compile_to_proc("SOME_CONST").call # => :toplevel, not :A!
end

Eval2C also offers equivalents to Ruby’s module_eval/class_eval and instance_eval:

class A
  $e2c.module_eval(self, %{
    def initialize(x)
      @x = x
    end
    def foo(y)
      @x + y
    end
  })
end

A.new(5).foo(6) # => 11

$e2c.instance_eval("4321", "reverse") # => "1234" 

Their implementations also use compile_to_proc and they are very similar to the implementation of toplevel_eval:

def module_eval(mod, code_str)
  mod.module_eval(&compile_to_proc(code_str))
end
alias :class_eval :module_eval

def instance_eval(object, code_str)
  object.instance_eval(&compile_to_proc(code_str))
end

With module_eval/class_eval it is possible to selectively compile some performance critical methods to C and leave the rest in Ruby.

But there is one more thing: compile_methods.

Defining the methods using module_eval as in the last example, has at least one downside: it won’t play nice with syntax-highlighting text editors, because the code of the methods is actually in a string. It would be much nicer to def the methods the normal way and then just compile them afterwards.

Thanks to compile_methods this is possible:

class A
  def initialize(x)
    @x = x
  end
  def foo(y)
    @x + y
  end

  $e2c.compile_methods(self, :initialize, :foo)
end

A.new(5).foo(6) # => 11

compile_methods will grab the methods’ node trees using RubyNode, compile them to C and then replace the old methods with the C versions. The visibility of the old methods will be preserved.

There are some limitations: compile_methods only works with methods defined using def and the methods must not need a cref (for more informations on what crefs are please see the respective section in limitations). This mainly means that constants must be prefixed with a double colon: Array needs a cref, while ::Array does not.

Options

Eval2C is configured with an option hash, the available options are:

Some examples:

Eval2C.new(:path => ".", :plugins => {})
Eval2C.new(:plugins => {:require_include=>[".", "../lib"], :optimizations=>:all})
Eval2C.new(:plugins => {:warnings=>true, :optimizations=>{:builtin_methods=>true}})
Eval2C.new(:prefix => "my_lib_name", :logger => Logger.new(STDOUT))

Implementation Details

The compile_to_proc method works as follows:

  1. The name of the extension is determined from the given prefix combined with the SHA1 digest of the Ruby code and some extra data (Ruby version, Ruby2CExtension version and plugin options). So the code determines the extension name and if the same code is compiled with the same version and settings, the name will be the same. This allows very simple and efficient caching of the compiled extensions.
  2. From the extension name a global variable name is constructed. The compiled proc will be stored in that global variable by the generated C extension.
  3. If the extension with the generated name already exists, then we are basically done and can skip this step. Otherwise the Ruby proc code is translated to C and compiled to a C extension in the specified path.
  4. The generated or already existing C extension is required.
  5. The global variable that now contains the compiled proc is read and the result is returned.

The compile_methods method works similar.

As mentioned above, this scheme allows very efficient caching of the C extensions. Basically each extension is only compiled once when the program/library is executed for the first time. In later runs the extension will already exist and just be required.

Adding the Ruby version and the Ruby2CExtension version to the digest avoids problems if multiple versions of Ruby or Ruby2CExtension are installed and used in parallel and it also guarantees that the extensions are automatically recompiled if Ruby or Ruby2CExtension are updated.

Using Eval2C Only When It Is Available

Having Ruby2CExtension as a hard dependency just for some speedup might not be acceptable for many projects, in particular since Ruby2CExtension requires that a C compiler is available at runtime.

An alternative that avoids the hard dependency is to use Ruby2CExtension only if it is installed anyway and otherwise just execute the Ruby code as usual. This works works very well if Eval2C and compile_methods is used, for example:

begin
  require "ruby2cext/eval2c" 
  $e2c = Ruby2CExtension::Eval2C.new
rescue LoadError
  $e2c = nil
end

class A
  def initialize(x)
    @x = x
  end
  def foo(y)
    @x + y
  end

  $e2c && $e2c.compile_methods(self, :initialize, :foo)
end

So if Ruby2CExtension is available, then the methods will be compiled and fast, otherwise the Ruby code will just work as usual (but it might be a bit slower of course).