This is an early work-in-progress project geared at replacing the existing CRuby parser. Its aims are threefold:
- Portability - we want the ability to use this parser in other projects, implementations, and tools.
- Error tolerance - we want this parser to be able to recover from as many syntax errors as possible.
- Maintainability - we want this to be a long-standing projects with good hygiene. This means tutorials, examples, documentation, clean code, good test coverage, etc.
This link is where you can find the design document for the project. It is also a work-in-progress, but should give you a good sense of the overall goals and motivations.
There are many parsers that have been built before in various stages of upkeep. Below is a list of the ones I have read through and found useful:
- jruby/jruby
- lib-ruby-parser/lib-ruby-parser
- natalie-lang/natalie_parser
- oracle/truffleruby
- ruby/ruby
- seattlerb/ruby_parser
- sisshiki1969/ruruby
- sorbet/sorbet
- whitequark/parser
There are also a couple of tools that define node shapes for every kind of node in the Ruby syntax tree. I've taken inspiration from those tools as well. They include most of the parsers above, as well as:
- ruby-syntax-tree/syntax_tree
- ruby/ruby/node.h (
RubyVM::AST)
The repository contains the infrastructure for both a shared library (librubyparser) and a native Ruby extension. The shared library has no bindings to Ruby itself, and so can be used by other C libraries. The native Ruby extension links against ruby.h, and so is suitable in the context of Ruby.
.
├── Makefile configuration to compile the shared library and native tests
├── Rakefile configuration to compile the native extension and run the Ruby tests
├── bin
│ ├── template generates code from the nodes and tokens configured by config.yml
│ └── templates directory containing all of the various templates
├── config.yml specification for tokens and nodes in the tree
├── ext
│ └── yarp
│ └── extension.c the native extension that interacts with librubyparser
├── lib
│ ├── yarp support files for the Ruby library
│ └── yarp.rb main entrypoint into the Ruby library
├── src
│ ├── yarp.c main entrypoint into the shared library
│ └── yarp.h main header file for the shared library
├── test Ruby tests for the Ruby library
└── test-native C tests for the shared library
See the CONTRIBUTING.md file for more information. We additionally have documentation about the overall design of the project as well as various subtopics.