February 2010

Introducing SexpBuilder

I’m a dreamer. When I see something like this:

def process_call(exp)
  # exp[1] => reciever.
  # exp[2] => method
  # exp[3] => (args)
  exp[3] = process(exp[3])
  if text?(exp)
    s(:parkaby, :text, exp[3])
  elsif tag = force_tag_call?(exp) || tag_call?(exp)
    s(:parkaby, :tag, *tag)
  else
    exp
  end
end

I just write down what I want it to look like:

rule :tag_call do
  # Forced tag call
  s(:call,
   s(:call, nil, :tag, s(:arglist)),
   wild % :name,
   args % :args) |

  # or regular tag call
  s(:call,
  nil,
  name % :name,
  args % :args)
end

rule :text do
  s(:call, nil,      :text, s(:arglist, wild % :content)) |
  s(:call, s(:self), :<<,   s(:arglist, wild % :content))
end

Then I implement it.

Okay. The above examples aren’t quite equal, so let’s have a look at what I’m really trying to solve.

Matching complex Sexp in Parkaby

Parkaby is my little experiment to make a super-duper-freaky-fast Markaby replacement by parsing the source and “compiling” it. Ultimately, a template like:

h1 "Hello World!"
p "Welcome #{@user}"

Should be compiled into:

"<h1>Hello World!</h1><p>Welcome #{@user}</p>"

In Parkaby, this involes two steps: a processor which figures out what should be considered HTML-tags and what should be considered regular method calls, and a generator which compiles it into Ruby. An interesting aspect of Parkaby is that the processor actually is quite complex. For instance, it has to figure out that div.post.clearfix.main!(:style => "display:none") should be compiled to <div class="post clearfix" id="main" style => "display:none"></div>.

My original approach was to use SexpProcessor:

def process_call(exp)
  # exp[1] => reciever.
  # exp[2] => method
  # exp[3] => (args)
  exp[3] = process(exp[3])
  if text?(exp)
    s(:parkaby, :text, exp[3])
  elsif tag = force_tag_call?(exp) || tag_call?(exp)
    s(:parkaby, :tag, *tag)
  else
    exp
  end
end

Every time it finds a method call, it checks if it’s a text-node or a HTML-tag and then turns it into a parkaby-sexp, or it just leaves it alone. The code ended up quite messy, and that’s before I even tried implementing CSS-proxy (tag.klass.klass.klass.id!): Parkaby before SexpBuilder.

Just thinking of implementing CSS-proxy got me a light headache, so I quickly realized that something had to be done.

Adam Sanderson to the rescue!

Ka-poof! Adam Sanderson writes SexpPath.

SexpPath is a DSL for matching Sexp. Let’s say we want to match text "Hello" and self << "Hello":

# text "Hello"
sexp1 = s(:call, nil, :text, s(:arglist, s(:str, "Hello")))
# self << "Hello"
sexp2 = s(:call, s(:self), :<<, s(:arglist, s(:str, "Hello")))

# the old approach:
def like_text?(exp)
  rec_meth = exp[1..2].to_a
  rec_meth == [[:self], :<<] || rec_meth == [nil, :text]
end

def text?(exp)
  like_text?(exp) and
  exp[3].length == 2
end

# the SexpPath approach:
query = Q? do
  s(:call, nil,      :text, s(:arglist, wild % :content)) |
  s(:call, s(:self), :<<,   s(:arglist, wild % :content))
end

Isn’t that pretty? The vertical-bar means “or”, `wild` matches everything and the percent sign captures the value in a Hash. Exactly what I want.

SexpBuilder

gem install sexp_builder

Well, SexpPath alone wasn’t enough to solve my problem – I had to introduce a new library. SexpBuilder is a more complete solution to easily match and replace complex Sexp. Very much like a parser, you define matchers and rules, and by combining them with a rewriter it was breeze to implement CSS-proxy.

Have a look at http://dojo.rubyforge.org/ for documentation, and Parkaby::Processor if you want a full example. There’s also Andand.rb for a more traditional example/demo.

What about Parkaby?

Well, at the moment, Parkaby is resting. In order to fully support it on other platforms than MRI, I’ll have to start yet another project. And that’s not happening until I release a few other projects I’m working on (like SexpBuilder).