Save 37% off PRO during our Black Friday Sale! »

Writing DSLs with Parslet - NYC.rb

Writing DSLs with Parslet - NYC.rb

A well-designed DSL improves programmer productivity and communication with domain experts. The Ruby community has produced a number of very popular external DSLs—Coffeescript, HAML, SASS, and Cucumber to name a few.

Parslet makes it easy to write these kinds of DSLs in pure Ruby. In this talk you’ll learn the basics, feel out the limitations of several approaches and find some common solutions. In no time, you’ll have the power to make a great new DSL, slurp in obscure file formats, modify and contribute to other people’s grammars (like Gherkin, TOML, or JSON), or even write your own programming language!

Presented by Jason Garber at Wicked Good Ruby Conf and NYC.rb. https://www.promptworks.com

99e2a6afab542ba98a9f1d1cae6c9670?s=128

PromptWorks

April 08, 2014
Tweet

Transcript

  1. Writing DSLs with Parslet Jason Garber N Y C .

    r b A p r i l 8 , 2 0 1 4
  2. None
  3. TDD TATFT Agile! Scrum Pair Programming VIM Pomodoro Continuous Integration

    Continuous Delivery
  4. None
  5. Parsing

  6. DSLS Domain-Specific Languages

  7. <?xml version="1.0"?> <configuration><configSections><sectionGr
 oup name="userSettings" type="System.Configuration.UserSettings
 Group, System, Version=2.0.0.0, Culture=neutral,

    PublicKeyToken
 =b77a5c561934e089"><section name="MSDNSampleSettings.My.MySetti
 ngs" type="System.Configuration.ClientSettingsSection, System, 
 Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e0
 89" allowExeDefinition="MachineToLocalUser" requirePermission="
 false"/></sectionGroup></configSections> ... <userSettings><MSD
 NSampleSettings.My.MySettings><setting name="Setting" serialize
 As="String"><value>SomeDefaultValue</value></setting></MSDNSamp
 leSettings.My.MySettings></userSettings> </configuration>
  8. </xml>

  9. I DSLs

  10. RssReader::Application.routes.draw do devise_for :users ! resources :feeds, only: [:edit, :update,

    :show, :index] post '/feeds', to: 'feeds#create', as: 'create_feed' resources :users do get 'page/:page', action: :index, on: :collection end resources :posts do member do put :update_category end end get '/my_profile', to: 'users#my_profile', as: :my_profile root to: "home#home" end
  11. describe Stack do context "stack with one item" do let(:stack)

    { a_stack_with_one_item } ! context "when popped" do before { stack.pop } it { should be_empty } end end end
  12. click_on "Sign Up" fill_in "Email", with: account[:email] fill_in "Password", with:

    account[:password] fill_in "Confirmation", with: account[:password_confirmation] fill_in "Name", with: account[:name] select account[:birthyear].to_s, from: "Year born" check "Terms" click_on "I'm ready to join!" current_path.should eq "/accounts/#{account.id}/dashboard" page.should have_content "Dashboard"
  13. desc 'Generate markup and stylesheets and open browser preview' task

    :preview => FileList['*.html'] + FileList['*.css'] do |t| sh 'open -g index.html' end rule '.html' => '.haml' do |t| puts "Rebuilding #{t.name}" sh "haml #{t.source} #{t.name}" end rule '.css' => lambda { |cssfile| source(cssfile) } do |t| puts "Rebuilding #{t.name}" sh "sass #{t.source} #{t.name}" end
  14. get '/' do @posts = Post.all(:order => [:id.desc], :limit =>

    20) erb :index end ! get '/post/new' do erb :new end ! get '/post/:id' do @post = Post.get(params[:id]) erb :post end ! post '/post/create' do post = Post.new(params) status 201 redirect "/post/#{post.id}" end
  15. rule(:table) do (str("table") >> attributes?.as(:attributes) >> str(".\n")).maybe >> table_row.repeat(1).as(:content) >>

    block_end end rule(:table_row) do table_row_attributes >> table_row_content >> end_table_row end rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe } rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) } rule(:end_table_row) { str("|") >> (block_end.present? | (str("\n"))) } rule(:table_header) do str("|_. ") >> table_content.as(:content) end rule(:table_data) do str("|") >> str("\n").absent? >> td_attributes? >> table_content.as(:content) end
  16. Internal DSLs Fluent Interfaces

  17. EXTERNAL DSLS

  18. upstream puma { server unix:///tmp/sockets/puma.sock fail_timeout=0; } server { listen

    80 default deferred; server_name promptworks.com www.promptworks.com; root /srv/promptworks/public; charset utf-8; if (-f $document_root/system/maintenance.html) { return 503; } error_page 503 @maintenance; location @maintenance { rewrite ^(.*)$ /system/maintenance.html last; break; } }
  19. <([A-Z][A-Z0-9]*)\b[^>]*>(.*?)<\/\1> ! ! \b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3} (?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b

  20. SELECT DISTINCT sc1.id FROM ( SELECT DATE_ADD(entry_point.start_date, INTERVAL (digits_1.digit *

    10 + d (1 << (DAYOFWEEK(DATE_ADD(entry_point.start_date, INTERVAL (digits_1.digi FROM (SELECT CAST('2012-03-01' AS date) AS start_date) AS entry_point INNER JOIN (SELECT 0 AS digit UNION SELECT 1 UNION SELECT 2 UNION SELECT ON (digits_1.digit * 10 + digits_2.digit) <= (DATEDIFF(DATE_ADD(entry_poi ) AS md INNER JOIN schedules AS sc1 INNER JOIN negative_link_influences AS neg on (sc1.call_type_id = neg.call_type_id or neg.call_type_id = -1) WHERE sc1.schedule_on = md.date AND neg.affected_shift = 0;
  21. BEGIN { TOTAL=0 } { TOTAL = TOTAL + $1

    } END { print TOTAL/NR } {print "<li><a href=\"" $1 "\">" $1 "</a></li>"}
  22. ! doctype html html head title Test body - unless

    items.empty? ol - items.each do |item| li = item - else p No items
  23. Feature: Searching music As a User I want to be

    able to search music So I can play it Background: Given I am logged in Scenario: Search music Given I am on the search screen When I search for "mix" Then I should see the mixtape
  24. class erlang { file { "/etc/apt/sources.list.d/esl-erlang.list": ensure => present, owner

    => root, content => 'deb http://example.com/debian precise contrib'; } exec { "apt-update": command => "/usr/bin/apt-get update", refreshonly => true; } package { "esl-erlang": ensure => installed, require => Exec['apt-update', 'import-key']; } } include erlang
  25. Heroku buildpack: Ruby ====================== This is a [Heroku buildpack](http://devcenter.heroku.com/article It

    uses [Bundler](http://gembundler.com) for dependency manageme Usage ----- ### Ruby Example Usage: $ heroku create --stack cedar --buildpack https://github.com $ git push heroku master
  26. title = "T??? Example" ! [owner] name = "Tom Preston-Werner"

    organization = "GitHub" bio = "GitHub Cofounder & CEO\nLikes tater tots and beer." dob = 1979-05-27T07:32:00Z # First class dates? Why not? ! [database] server = "192.168.1.1" ports = [ 8001, 8001, 8002 ] connection_max = 5000 enabled = true
  27. global= { \time 4/4 \key c \major \tempo "Allegro" 4

    = 132 \set Score.skipBars = ##t } ! Violinone = \new Voice { \relative c''{ \set Staff.midiInstrument = #"violin" \set tupletSpannerDuration = #(ly:make-moment 1 4) ! % Jason solo R1 * 9 r2 r4 \times 2/3 { c4( b8)
  28. gsave 1 0.5 scale 70 100 48 0 360 arc

    fill grestore /Helvetica-Bold 14 selectfont 1.0 setgray 29 45 moveto (Hello, world!) show showpage
  29. .accordion { li { border-top: 1px solid #e2e4e6; &:first-child {

    border-top-color: transparent; } } a { @include rem(padding, 5px 10px 6px); &.icon { padding-left: 40px; position: relative; } &.icon img { @include rem(left, 10px); margin-top: -2px; position: absolute; } } }
  30. None
  31. None
  32. h1. Give Textile a try! ! A *simple* paragraph with

    a line break, some _emphasis_ and a "link":http://redcloth.org ! * an item * and another ! # one # two # three
  33. h1. Give Textile a try! ! A *simple* paragraph with

    a line break, some _emphasis_ and a "link":http://redcloth.org ! * an item * and another ! # one # two # three
  34. None
  35. A_HLGN = /(?:\<(?!>)|\<\>|\=|[()]+)/ A_VLGN = /[\-^~]/ C_CLAS = '(?:\([^)]+\))' C_LNGE

    = '(?:\[[^\]]+\])' C_STYL = '(?:\{[^}]+\})' S_CSPN = '(?:\\\\\d+)' S_RSPN = '(?:/\d+)' A = "(?:#{A_HLGN}?#{A_VLGN}?|#{A_VLGN}?#{A_HLGN}?)" S = "(?:#{S_CSPN}?#{S_RSPN}|#{S_RSPN}?#{S_CSPN}?)" C = "(?:#{C_CLAS}?#{C_STYL}?#{C_LNGE}?|#{C_STYL}? #{C_LNGE}?#{C_CLAS}?|#{C_LNGE}?#{C_STYL}?#{C_CLAS}?)" ! PUNCT = Regexp::quote( '!"#$%&\'*+,-./:;=?@\\^_`|~' ) HYPERLINK = '(\S+?)([^\w\s/;=\?]*?)(\s|$)'
  36. def links( text ) text.gsub!( / ([\s\[{(]|[#{PUNCT}])? # $pre "

    # start (#{C}) # $atts ([^"]+?) # $text \s? (?:\(([^)]+?)\)(?="))? # $title ": (\S+?) # $url (\/)? # $slash ([^\w\/;]*?) # $post (?=\s|\.[#{PUNCT}]+|$) /x ) do |m| pre,atts,text,title,url,slash,post = $~[1..7] ! url = check_refs( url ) ! atts = pba( atts ) atts << " title=\"#{ title }\"" if title atts = shelve( atts ) if atts ! "#{ pre }<a href=\"#{ url }#{ slash }\"#{ atts }>" + "#{ text }</a>#{ post }" end end
  37. “Some people, when confronted with a problem, think ‘I know,

    I'll use regular expressions.’
 Now they have two problems.” — Jamie Zawinski
  38. None
  39. %%{ machine superredcloth_scan; include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl"; action extend { extend

    = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); } # blocks notextile_tag_start = "<notextile>" ; notextile_tag_end = "</notextile>" CRLF? ; notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ; pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ; pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"... block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ; pre_tag := |* pre_tag_end { CAT(block); DONE(block); fgoto main; }; default => esc_pre; *|; notextile_tag := |* notextile_tag_end { DONE(block); fgoto main; }; default => cat; *|; }%%
  40. void rb_str_cat_escaped(VALUE str, char *ts, char *te, unsigned int opts);

    void rb_str_cat_escaped_for_preformatted(VALUE str, char *ts, char *te, unsigned int opts); VALUE superredcloth_inline(VALUE, char *, char *, VALUE); VALUE superredcloth_inline2(VALUE, VALUE, VALUE); ! #define CAT(H) rb_str_cat(H, ts, te-ts) #define CLEAR(H) H = rb_str_new2("") #define INLINE(H, T) rb_str_append(H, rb_funcall(rb_formatter, rb_intern(#T), 1, regs)) ! VALUE superredcloth_transform(rb_formatter, p, pe, refs) VALUE rb_formatter; char *p, *pe; VALUE refs; { if (RSTRING(block)->len > 0) { ADD_BLOCK(); } ! if ( NIL_P(refs) && rb_funcall(refs_found, rb_intern("empty?"), 0) == Qfalse ) { return superredcloth_transform(rb_formatter, orig_p, orig_pe, refs_found); } else { rb_funcall(rb_formatter, rb_intern("after_transform"), 1, html); return html; } }
  41. None
  42. None
  43. %%{ machine superredcloth_scan; include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl"; action extend { extend

    = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); } # blocks notextile_tag_start = "<notextile>" ; notextile_tag_end = "</notextile>" CRLF? ; notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ; pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ; pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"... block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ; pre_tag := |* pre_tag_end { CAT(block); DONE(block); fgoto main; }; default => esc_pre; *|; notextile_tag := |* notextile_tag_end { DONE(block); fgoto main; }; default => cat; *|; }%%
  44. %%{ machine superredcloth_scan; include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl"; action extend { extend

    = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); } # blocks notextile_tag_start = "<notextile>" ; notextile_tag_end = "</notextile>" CRLF? ; notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; pre_tag_start = "<pre" [^>]* ">" (space* "<code>")? ; pre_tag_end = ("</code>" space*)? "</pre>" CRLF? ; pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"... block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ; pre_tag := |* pre_tag_end { CAT(block); DONE(block); fgoto main; }; default => esc_pre; *|; notextile_tag := |* notextile_tag_end { DONE(block); fgoto main; }; default => cat; *|; }%% /* * redcloth_scan.c.rl * * Copyright (C) 2009 Jason Garber */ #define redcloth_scan_c ! #define RSTRING_NOT_MODIFIED #include <ruby.h> #include "redcloth.h" ! VALUE mRedCloth, super_ParseError, super_RedCloth, super_HTML, super_LATEX; VALUE SYM_escape_preformatted, SYM_escape_attributes; ! #line 23 "ext/redcloth_scan/redcloth_scan.c" static const unsigned char _redcloth_scan_actions[] = { 0, 1, 0, 1, 2, 1, 3, 1, 4, 1, 5, 1, 6, 1, 7, 1, 9, 1, 10, 1, 11, 1, 12, 1, 13, 1, 14, 1, 15, 1, 16, 1, 17, 1, 18, 1, 19, 1, 20, 1, 21, 1, 22, 1, 23, 1, 24, 1, 25, 1, 26, 1, 27, 1, 28, 1, 29, 1, 30, 1, 34, 1, 35, 1, 36, 1, 38, 1, 40, 1, 42, 1, 43, 1, 44, 1, 45, 1, 48, 1, 57, 1, 58, 1, 59, 1, 60, 1, 61, 1, 62, 1, 63, 1, 64, 1, 65, 1, 66, 1, 70, 1, 73, 1,
  45. None
  46. None
  47. grammar Arithmetic rule additive multitive ( '+' multitive )* end

    ! rule multitive primary ( [*/%] primary )* end ! rule primary '(' additive ')' / number end ! rule number '-'? [1-9] [0-9]* end end
  48. require 'parslet' ! class MiniParser < Parslet::Parser rule(:integer) { match('[0-9]').repeat(1)

    } root(:integer) end ! MiniParser.new.parse("1324321") # => "1324321"@0
  49. Parslet::Atoms str('NYC.rb')

  50. Parslet::Atoms str('NYC.rb').parse('NYC.rb')

  51. Parslet::Atoms str('NYC.rb').parse('NYC.rb') # => "NYC.rb"@0

  52. Parslet::Atoms str('NYC.rb').parse('NYC.rb') # => "NYC.rb"@0 ! match('[0-9a-f]')

  53. Parslet::Atoms str('NYC.rb').parse('NYC.rb') # => "NYC.rb"@0 ! match('[0-9a-f]') ! any

  54. Operators str('jim') >> str('weirich') ! str('Ruby') | str('Elixir') ! match('[Nn]')

    >> str('ew') | match('[Yy]') >> str('ork City')
  55. Repetition str('foo').repeat str('foo').repeat(1) str('foo').repeat(1,3) str('foo').repeat(0, nil) str('foo').maybe

  56. Presence str('Java') >> str('Script').present? ! ! str('0').repeat(1).absent? >> match('[\d]').repeat(1)

  57. 1. Create a grammar
 What should be legal syntax? 2.

    Annotate the grammar:
 What is important data? 3. Create a transformation:
 How do I want to work with that data?
  58. 1. Create a grammar
 What should be legal syntax? 2.

    Annotate the grammar:
 What is important data? 3. Create a transformation:
 How do I want to work with that data?
  59. Capture str('Madison Sq').parse('Madison Sq') # => "Madison Sq"@0 ! str('Madison

    Sq').as(:park).parse('Madison Sq') # => {:park=>"Madison Sq"@0}
  60. Capture str('a').repeat.as(:b) # => {:b=>"aaa"@0} str('a').as(:b).repeat # => [{:b=>"a"@0}, {:b=>"a"@1},

    {:b=>"a"@2}] str('a').as(:a) >> str('b').as(:b) >> str('c') # => {:a=>"a"@0, :b=>"b"@1}
  61. rule(:table) do (str("table") >> attributes?.as(:attributes) >> str(".\n")).maybe >> table_row.repeat(1).as(:content) >>

    block_end end rule(:table_row) do table_row_attributes >> table_row_content >> end_table_row end rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe } rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) } rule(:end_table_row) { str("|") >> (block_end.present? | (str("\n"))) } rule(:table_header) do str("|_. ") >> table_content.as(:content) end rule(:table_data) do str("|") >> str("\n").absent? >> td_attributes? >> table_content.as(:content) end
  62. table(#prices).
 | Adults | $5 |
 | Children | $2

    | <table id="prices"> <tr> <td>Adults</td> <td>$5</td> </tr> <tr> <td>Children</td> <td>$2</td> </tr> </table>
  63. rule(:table) do (str("table") >> attributes?.as(:attributes) >> str(".\n")).maybe >> table_row.repeat(1).as(:content) >>

    block_end end rule(:table_row) do table_row_attributes >> table_row_content >> end_table_row end rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe } rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) } rule(:end_table_row) { str("|") >> (block_end.present? | (str("\n"))) } rule(:table_header) do str("|_. ") >> table_content.as(:content) end rule(:table_data) do str("|") >> str("\n").absent? >> td_attributes? >> table_content.as(:content) end
  64. def parenthesized(atom) str('(') >> atom >> str(')') end ! parenthesized(match['\d']).parse("(5)")

  65. class HtmlTag < Parslet::Parser root(:tag) rule(:tag) { open_tag | close_tag

    | self_closing_tag | comment_tag } rule(:open_tag) { str("<") >> tag_name >> attributes? >> str(">") } rule(:close_tag) { str("</") >> tag_name >> str(">") } rule(:tag_name) { match("[A-Za-z_:]") >> name_char.repeat } ... end ! class BlockHtmlTag < HtmlTag rule(:tag_name) do inline_tag_name.absent? >> any_tag_name end ! rule(:inline_tag_name) do INLINE_TAGS.map {|name| str(name) }.reduce(:|) end end
  66. > HtmlTag.new.open_tag.methods => [:name, :block, :try, :parslet, :to_s_inner, :parse, :apply,

    :cached?, ...] ! > HtmlTag.new.open_tag.parse("<blockquote>") => "<blockquote>"@0 ! > HtmlTag.new.open_tag.parse("</blockquote>") ! Parslet::ParseFailed: Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2. from /Users/jasongarber/.rvm/gems/ruby-2.0.0-p247/gems/parslet-1.5.0/lib/parslet/ cause.rb:63:in `raise' from /Users/jasongarber/.rvm/gems/ruby-2.0.0-p247/gems/parslet-1.5.0/lib/parslet/ atoms/base.rb:46:in `parse' from (irb):8 from /Users/jasongarber/.rvm/rubies/ruby-2.0.0-p247/bin/irb:16:in `<main>'
  67. begin RedClothParslet::Parser::BlockHtmlTag.new.tag.parse("<img>") rescue Parslet::ParseFailed => failure puts failure.cause.ascii_tree end !

    Expected one of [OPEN_TAG, CLOSE_TAG, SELF_CLOSING_TAG, COMMENT_TAG] at line 1 char 1. |- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2. | `- Failed to match sequence ((!INLINE_TAG_NAME / &(INLINE_TAG_NAME NAME_CHAR{1, })) ANY_TAG_NAME) at line 1 char 2. | `- Expected one of [!INLINE_TAG_NAME, &(INLINE_TAG_NAME NAME_CHAR{1, })] at line 1 char 2. | |- Input should not start with INLINE_TAG_NAME at line 1 char 2. | `- Input should start with INLINE_TAG_NAME NAME_CHAR{1, } at line 1 char 2. |- Failed to match sequence ('</' TAG_NAME '>') at line 1 char 1. | `- Expected "</", but got "<i" at line 1 char 1. |- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? SPACES? '/' '>') at line 1 char 2. | `- Failed to match sequence ((!INLINE_TAG_NAME / &(INLINE_TAG_NAME NAME_CHAR{1, })) ANY_TAG_NAME) at line 1 char 2. | `- Expected one of [!INLINE_TAG_NAME, &(INLINE_TAG_NAME NAME_CHAR{1, })] at line 1 char 2. | |- Input should not start with INLINE_TAG_NAME at line 1 char 2. | `- Input should start with INLINE_TAG_NAME NAME_CHAR{1, } at line 1 char 2. `- Failed to match sequence ('<!--' (!COMMENT_TAG_END .){0, } COMMENT_TAG_END) at line 1 char 1. `- Expected "<!--", but got "<img" at line 1 char 1.
  68. I Objects

  69. describe RedClothParslet::Parser::HtmlTag do it { should parse("<div>") } it {

    should parse("<hr />") } it { should parse("</div>") } it { should parse("<!-- an HTML comment -->") } ! describe "attribute" do subject { described_class.new.attribute } it { should parse(" class='awesome'") } it { should_not parse(' 9kittens="cute"') } end end
  70. $ rspec spec/parser/html_tag_spec.rb ! 1) RedClothParslet::Parser::HtmlTag tag Failure/Error: it {

    should parse("</div>") } expected TAG to be able to parse "</div>" Expected one of [OPEN_TAG, CLOSE_TAG, SELF_CLOSING_TAG, COMMENT_TAG] at line 1 char 1. |- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2. | `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2. | `- Failed to match [A-Za-z_:] at line 1 char 2. |- Failed to match sequence ('<' TAG_NAME '>') at line 1 char 2. | `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2. | `- Failed to match [A-Za-z_:] at line 1 char 2. |- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? SPACES? '/' '>') at line 1 char 2. | `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2. | `- Failed to match [A-Za-z_:] at line 1 char 2. `- Failed to match sequence ('<!--' (!COMMENT_TAG_END .){0, } COMMENT_TAG_END) at line 1 char 1. `- Expected "<!--", but got "</di" at line 1 char 1. # ./spec/parser/html_tag_spec.rb:9:in `block (3 levels) in <top (required)>' ! 13/13: 100% |==========================================| Time: 00:00:00 ! Finished in 0.01879 seconds 13 examples, 1 failure
  71. 1. Create a grammar
 What should be legal syntax? 2.

    Annotate the grammar:
 What is important data? 3. Create a transformation:
 How do I want to work with that data?
  72. Transformations tree = {left: {int: '1'}, op: '+', right: {int:

    '2'}} class T < Parslet::Transform rule(int: simple(:x)) { Integer(x) } end ! T.new.apply(tree) # => {:left=>1, :op=>"+", :right=>2}
  73. Transformations tree = {left: {int: '1'}, op: '+', right: {int:

    '2'}} ! class T < Parslet::Transform rule(int: simple(:x)) { Integer(x) } rule(op: '+', left: simple(:l), right: simple(:r)) { l + r } end T.new.apply(tree) # => 3
  74. Transformations rule(:content => subtree(:c), :attributes => subtree(:a)) do |dict| {:content

    => dict[:c], :opts => RedCloth::Ast::Attributes.new(dict[:a])} end ! rule(:table => subtree(:a)) do RedCloth::Ast::Table.new(a[:content], a[:opts]) end ! rule(:bq => subtree(:a)) do RedCloth::Ast::Blockquote.new(a[:content], a[:opts]) end
  75. Who you callin’ slow?

  76. class ErbParser < Parslet::Parser rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }

    ! rule(:expression) { (str('=') >> ruby).as(:expression) } rule(:comment) { (str('#') >> ruby).as(:comment) } rule(:code) { ruby.as(:code) } rule(:erb) { expression | comment | code } ! rule(:erb_with_tags) { str('<%') >> erb >> str('%>') } rule(:text) { (str('<%').absent? >> any).repeat(1) } ! rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t root(:text_with_ruby) end
  77. class ErbParser < Parslet::Parser rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }

    ! rule(:expression) { (str('=') >> ruby).as(:expression) } rule(:comment) { (str('#') >> ruby).as(:comment) } rule(:code) { ruby.as(:code) } rule(:erb) { expression | comment | code } ! rule(:erb_with_tags) { str('<%') >> erb >> str('%>') } rule(:text) { (str('<%').absent? >> any).repeat(1) } ! rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t root(:text_with_ruby) end class rule( ! rule( rule( rule( rule( ! rule( rule( ! rule( root( end
  78. class ErbParser < Parslet::Parser rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }

    ! rule(:expression) { (str('=') >> ruby).as(:expression) } rule(:comment) { (str('#') >> ruby).as(:comment) } rule(:code) { ruby.as(:code) } rule(:erb) { expression | comment | code } ! rule(:erb_with_tags) { str('<%') >> erb >> str('%>') } rule(:text) { (str('<%').absent? >> any).repeat(1) } ! rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t root(:text_with_ruby) end class rule( ! rule( rule( rule( rule( ! rule( rule( ! rule( root( end include Parslet::Accelerator optimized = apply( parser, rule( (str(:x).absent? >> any).repeat ) { GobbleUp.new(x, 0) } )
  79. Custom Atoms • Parse a limited set of natural-language queries

    • “Who makes the best cheesesteaks in NYC?” • EngTagger: a corpus-trained, probabilistic tagger • Custom Parslet::Atom
  80. rule(:question) do interrogative.maybe.as(:int) >> verb_phrase.maybe.as(:verb) >> superlative_phrase.maybe.as(:sup) >> subject.as(:subj) >>

    prepositional_phrase.repeat.as(:preps) >> sentence_end.repeat.as(:punct) end ! rule(:verb_phrase) { (verb_present | verb_past | verb_future).as(:vp) } rule(:verb_present) { word(:VBZ, :VB).as(:present) } rule(:verb_past) { word(:VBD).as(:past) } rule(:verb_future) { word(:MD).as(:future) >> word(:VB).maybe.as(:infinitive) }
  81. Other Crazy Uses •User-supplied formulas / logic •Logs •Streaming text

    •The Right Reverend and Right Honourable the Lord Bishop of London Richard John Carew Chartres
  82. Who will write the next awesome DSL?

  83. Who will write the next awesome DSL!. You

  84. Conclusions • DSLs make life better • internal_dsl > external_dsl

    if 
 internal_dsl.practicable? • Keep your parser clean, unit test • “Situational awareness!”
  85. The State of RedCloth •Needs lots of love + less

    neglectful maintainer •Finish the rewrite •Merge + release RedCloth 5.0
  86. @jasongarber @promptworks jason@promptworks.com Rate Me: http://spkr8.com/t/30651

  87. SpeakerRate Next time, post a link to my page