-
Notifications
You must be signed in to change notification settings - Fork 10
Benchmarks
Here is a preliminary report:
Each program was run on MacOS 11.1 using machine with Apple M1 (4+4 cores, 16GB RAM).
All data were compared with CRuby3.0.0.
Optcarrot is a NES emulator, and a semi-official benchmark program for Ruby, on https://github.com/mame/optcarrot.
You can see the results of optcarrot benchmark for ruruby and other Ruby implementations here.
benchmark | CRuby | ruruby | rate |
---|---|---|---|
optcarrot | 56.09 ± 0.13 fps | 34.80 ± 0.08 fps | x 1.61 |
optcarrot --opt | 130.53 ± 0.82 fps | 101.85 ± 1.08 fps | x 1.28 |
* With --opt option, optcarrot rewrites its own code for optimization.
Frames per seconds are shown.
The higher, the better.
benchmark | CRuby | ruruby | rate |
---|---|---|---|
accessor_get.rb | 0.42 ± 0.00 s | 0.43 ± 0.00 s | x 1.02 |
accessor_set.rb | 0.48 ± 0.00 s | 0.53 ± 0.00 s | x 1.10 |
ivar_get.rb | 0.52 ± 0.00 s | 0.63 ± 0.00 s | x 1.21 |
ivar_set.rb | 0.52 ± 0.01 s | 0.66 ± 0.01 s | x 1.25 |
loop_times.rb | 0.72 ± 0.01 s | 0.60 ± 0.00 s | x 0.84 |
loop_for.rb | 0.78 ± 0.01 s | 0.71 ± 0.00 s | x 0.91 |
loop_whileloop.rb | 0.52 ± 0.00 s | 0.48 ± 0.01 s | x 0.93 |
so_concatenate.rb | 0.51 ± 0.01 s | 0.48 ± 0.01 s | x 0.94 |
string_scan_str.rb | 0.63 ± 0.00 s | 0.88 ± 0.01 s | x 1.40 |
string_scan_re.rb | 0.95 ± 0.00 s | 0.87 ± 0.01 s | x 0.92 |
fiber_allocate.rb | 0.59 ± 0.01 s | 0.41 ± 0.04 s | x 0.69 |
fiber_switch.rb | 0.64 ± 0.00 s | 0.52 ± 0.01 s | x 0.82 |
so_mandelbrot.rb | 1.18 ± 0.04 s | 1.36 ± 0.01 s | x 1.15 |
app_mandelbrot.rb | 0.84 ± 0.00 s | 0.95 ± 0.00 s | x 1.13 |
app_fibo.rb | 0.54 ± 0.01 s | 0.68 ± 0.00 s | x 1.25 |
app_aobench.rb | 5.89 ± 0.01 s | 9.74 ± 0.03 s | x 1.65 |
so_nbody.rb | 0.76 ± 0.01 s | 0.94 ± 0.00 s | x 1.22 |
Mean and SD of execution time (executed 8 times) in seconds are shown.
The lower, the better.
These programs are available on https://github.com/ruby/ruby/blob/master/benchmark.
The characteristics of the benchmark programs are described below.
Simply execute a times
method with block repeatedly.
Simply execute a for-loop repeatedly.
Simply execute a while-loop repeatedly.
Join small strings repeatedly.
A lot of arythmetic calculations with local variables in for-loop, producing almost no objects.
Produce and consume a lot of objects, and execute numerous method calls.
A huge number of recursive method calls.
Produce and consume a lot of objects, executing a huge number of floating point calculations and method calls.
To investigate memory consumption, maximum resident set size was measured using gnu-time. Memory consumption of ruruby is around 20%-25% as compared with CRuby.
benchmark | CRuby | ruruby | rate |
---|---|---|---|
simple_call.rb | 26.7M | 3.5M | x 0.13 |
accessor_get.rb | 26.6M | 3.5M | x 0.13 |
accessor_set.rb | 26.7M | 3.5M | x 0.13 |
ivar_get.rb | 26.6M | 3.5M | x 0.13 |
ivar_set.rb | 26.6M | 3.5M | x 0.13 |
loop_times.rb | 26.7M | 3.5M | x 0.13 |
loop_for.rb | 26.8M | 3.5M | x 0.13 |
loop_whileloop.rb | 26.7M | 3.5M | x 0.13 |
so_concatenate.rb | 75.8M | 27.3M | x 0.36 |
string_scan_str.rb | 28.2M | 5.7M | x 0.20 |
string_scan_re.rb | 28.5M | 5.5M | x 0.19 |
fiber_allocate.rb | 209.7M | 229.8M | x 1.10 |
fiber_switch.rb | 26.7M | 3.5M | x 0.13 |
so_mandelbrot.rb | 27.2M | 4.5M | x 0.16 |
app_mandelbrot.rb | 27.1M | 4.5M | x 0.16 |
app_fibo.rb | 26.7M | 3.5M | x 0.13 |
app_aobench.rb | 27.3M | 5.8M | x 0.21 |
so_nbody.rb | 26.8M | 3.7M | x 0.14 |
optcarrot | 100.9M | 65.2M | x 0.65 |
optcarrot --opt | 117.2M | 95.1M | x 0.81 |