Slide 1
Slide 1 text
Writing DSLs
with Parslet
Jason Garber
W i c k e d G o o d R u b y C o n f
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
TDD
Slide 5
Slide 5 text
TDD
TATFT
Slide 6
Slide 6 text
TDD
TATFT
Agile!
Scrum
Pair
Programming
VIM
Pomodoro
Continuous
Integration
Continuous Delivery
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
Parsing
Slide 9
Slide 9 text
DSLS
Domain-Specific Languages
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
... SomeDefaultValue
Slide 12
Slide 12 text
Slide 13
Slide 13 text
I DSLs
Slide 14
Slide 14 text
RssReader::Application.routes.draw do
devise_for :users
resources :feeds, only: [:edit, :update, :show, :index]
post '/feeds', to: 'feeds#create', as: 'create_feed'
resources :users do
get 'page/:page', action: :index, on: :collection
end
resources :posts do
member do
put :update_category
end
end
get '/my_profile', to: 'users#my_profile', as: :my_profile
root to: "home#home"
end
Slide 15
Slide 15 text
describe Stack do
context "stack with one item" do
let(:stack) { a_stack_with_one_item }
context "when popped" do
before { stack.pop }
it { should be_empty }
end
end
end
Slide 16
Slide 16 text
click_on "Sign Up"
fill_in "Email", with: account[:email]
fill_in "Password", with: account[:password]
fill_in "Confirmation", with:
account[:password_confirmation]
fill_in "Name", with: account[:name]
select account[:birthyear].to_s, from: "Year born"
check "Terms"
click_on "I'm ready to join!"
current_path.should eq "/accounts/#{account.id}/dashboard"
page.should have_content "Dashboard"
Slide 17
Slide 17 text
desc 'Generate markup and stylesheets and open browser preview'
task :preview => FileList['*.html'] + FileList['*.css'] do |t|
sh 'open -g index.html'
end
rule '.html' => '.haml' do |t|
puts "Rebuilding #{t.name}"
sh "haml #{t.source} #{t.name}"
end
rule '.css' => lambda { |cssfile| source(cssfile) } do |t|
puts "Rebuilding #{t.name}"
sh "sass #{t.source} #{t.name}"
end
Slide 18
Slide 18 text
get '/' do
@posts = Post.all(:order => [:id.desc], :limit => 20)
erb :index
end
get '/post/new' do
erb :new
end
get '/post/:id' do
@post = Post.get(params[:id])
erb :post
end
post '/post/create' do
post = Post.new(params)
status 201
redirect "/post/#{post.id}"
end
Slide 19
Slide 19 text
rule(:table) do
(str("table") >> attributes?.as(:attributes) >> str(".\n")).maybe >>
table_row.repeat(1).as(:content) >> block_end
end
rule(:table_row) do
table_row_attributes >> table_row_content >> end_table_row
end
rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe }
rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) }
rule(:end_table_row) { str("|") >> (block_end.present? | (str("\n"))) }
rule(:table_header) do
str("|_. ") >> table_content.as(:content)
end
rule(:table_data) do
str("|") >> str("\n").absent? >> td_attributes? >> table_content.as(:content)
end
Slide 20
Slide 20 text
Internal DSLs
Fluent Interfaces
Slide 21
Slide 21 text
EXTERNAL
DSLS
Slide 22
Slide 22 text
upstream puma {
server unix:///tmp/sockets/puma.sock fail_timeout=0;
}
server {
listen 80 default deferred;
server_name promptworks.com www.promptworks.com;
root /srv/promptworks/public;
charset utf-8;
if (-f $document_root/system/maintenance.html) {
return 503;
}
error_page 503 @maintenance;
location @maintenance {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
}
Slide 23
Slide 23 text
<([A-Z][A-Z0-9]*)\b[^>]*>(.*?)<\/\1>
\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}
(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
Slide 24
Slide 24 text
SELECT DISTINCT sc1.id
FROM (
SELECT DATE_ADD(entry_point.start_date, INTERVAL (digits_1.digit * 10 + d
(1 << (DAYOFWEEK(DATE_ADD(entry_point.start_date, INTERVAL (digits_1.digi
FROM (SELECT CAST('2012-03-01' AS date) AS start_date) AS entry_point
INNER JOIN (SELECT 0 AS digit UNION SELECT 1 UNION SELECT 2 UNION SELECT
ON (digits_1.digit * 10 + digits_2.digit) <= (DATEDIFF(DATE_ADD(entry_poi
) AS md
INNER JOIN schedules AS sc1
INNER JOIN negative_link_influences AS neg
on (sc1.call_type_id = neg.call_type_id or neg.call_type_id = -1)
WHERE sc1.schedule_on = md.date
AND neg.affected_shift = 0;
Slide 25
Slide 25 text
BEGIN { TOTAL=0 }
{ TOTAL = TOTAL + $1 }
END { print TOTAL/NR }
{print ""
$1 " "}
Slide 26
Slide 26 text
! doctype html
html
head
title Test
body
- unless items.empty?
ol
- items.each do |item|
li
= item
- else
p No items
Slide 27
Slide 27 text
Feature: Searching music
As a User
I want to be able to search music
So I can play it
Background:
Given I am logged in
Scenario: Search music
Given I am on the search screen
When I search for "mix"
Then I should see the mixtape
Slide 28
Slide 28 text
class erlang {
file { "/etc/apt/sources.list.d/esl-erlang.list":
ensure => present,
owner => root,
content => 'deb http://example.com/debian precise contrib';
}
exec { "apt-update":
command => "/usr/bin/apt-get update",
refreshonly => true;
}
package { "esl-erlang":
ensure => installed,
require => Exec['apt-update', 'import-key'];
}
}
include erlang
Slide 29
Slide 29 text
Heroku buildpack: Ruby
======================
This is a [Heroku buildpack](http://devcenter.heroku.com/article
It uses [Bundler](http://gembundler.com) for dependency manageme
Usage
-----
### Ruby
Example Usage:
$ heroku create --stack cedar --buildpack https://github.com
$ git push heroku master
Slide 30
Slide 30 text
title = "T??? Example"
[owner]
name = "Tom Preston-Werner"
organization = "GitHub"
bio = "GitHub Cofounder & CEO\nLikes tater tots and
beer."
dob = 1979-05-27T07:32:00Z # First class dates? Why
not?
[database]
server = "192.168.1.1"
ports = [ 8001, 8001, 8002 ]
connection_max = 5000
enabled = true
Slide 31
Slide 31 text
global= {
\time 4/4
\key c \major
\tempo "Allegro" 4 = 132
\set Score.skipBars = ##t
}
Violinone = \new Voice { \relative c''{
\set Staff.midiInstrument = #"violin"
\set tupletSpannerDuration = #(ly:make-moment 1 4)
% Jason solo
R1 * 9
r2 r4 \times 2/3 { c4( b8)
Slide 32
Slide 32 text
gsave
1 0.5 scale
70 100 48 0 360 arc
fill
grestore
/Helvetica-Bold 14 selectfont
1.0 setgray
29 45 moveto
(Hello, world!) show
showpage
Slide 33
Slide 33 text
.accordion {
li {
border-top: 1px solid #e2e4e6;
&:first-child { border-top-color: transparent; }
}
a {
@include rem(padding, 5px 10px 6px);
&.icon {
padding-left: 40px;
position: relative;
}
&.icon img {
@include rem(left, 10px);
margin-top: -2px;
position: absolute;
}
}
}
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
h1. Give Textile a try!
A *simple* paragraph with a line break,
some _emphasis_ and a "link":http://redcloth.org
* an item
* and another
# one
# two
# three
Slide 37
Slide 37 text
h1. Give Textile a try!
A *simple* paragraph with a line break,
some _emphasis_ and a "link":http://redcloth.org
* an item
* and another
# one
# two
# three
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
A_HLGN = /(?:\<(?!>)|\<\>|\=|[()]+)/
A_VLGN = /[\-^~]/
C_CLAS = '(?:\([^)]+\))'
C_LNGE = '(?:\[[^\]]+\])'
C_STYL = '(?:\{[^}]+\})'
S_CSPN = '(?:\\\\\d+)'
S_RSPN = '(?:/\d+)'
A = "(?:#{A_HLGN}?#{A_VLGN}?|#{A_VLGN}?#{A_HLGN}?)"
S = "(?:#{S_CSPN}?#{S_RSPN}|#{S_RSPN}?#{S_CSPN}?)"
C = "(?:#{C_CLAS}?#{C_STYL}?#{C_LNGE}?|#{C_STYL}?
#{C_LNGE}?#{C_CLAS}?|#{C_LNGE}?#{C_STYL}?#{C_CLAS}?)"
PUNCT = Regexp::quote( '!"#$%&\'*+,-./:;=?@\\^_`|~' )
HYPERLINK = '(\S+?)([^\w\s/;=\?]*?)(\s|$)'
Slide 40
Slide 40 text
def links( text )
text.gsub!( /
([\s\[{(]|[#{PUNCT}])? # $pre
" # start
(#{C}) # $atts
([^"]+?) # $text
\s?
(?:\(([^)]+?)\)(?="))? # $title
":
(\S+?) # $url
(\/)? # $slash
([^\w\/;]*?) # $post
(?=\s|\.[#{PUNCT}]+|$)
/x ) do |m|
pre,atts,text,title,url,slash,post = $~[1..7]
url = check_refs( url )
atts = pba( atts )
atts << " title=\"#{ title }\"" if title
atts = shelve( atts ) if atts
"#{ pre }" +
"#{ text }#{ post }"
end
end
Slide 41
Slide 41 text
“Some people, when confronted
with a problem, think ‘I know, I'll
use regular expressions.’
Now they have two problems.”
— Jamie Zawinski
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
%%{
machine superredcloth_scan;
include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl";
action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); }
# blocks
notextile_tag_start = "" ;
notextile_tag_end = "" CRLF? ;
notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
pre_tag_start = "
]* ">" (space* "")? ;
pre_tag_end = ("" space*)? "" CRLF? ;
pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"...
block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ;
pre_tag := |*
pre_tag_end { CAT(block); DONE(block); fgoto main; };
default => esc_pre;
*|;
notextile_tag := |*
notextile_tag_end { DONE(block); fgoto main; };
default => cat;
*|;
}%%
Slide 44
Slide 44 text
void rb_str_cat_escaped(VALUE str, char *ts, char *te, unsigned int opts);
void rb_str_cat_escaped_for_preformatted(VALUE str, char *ts, char *te, unsigned int opts);
VALUE superredcloth_inline(VALUE, char *, char *, VALUE);
VALUE superredcloth_inline2(VALUE, VALUE, VALUE);
#define CAT(H) rb_str_cat(H, ts, te-ts)
#define CLEAR(H) H = rb_str_new2("")
#define INLINE(H, T) rb_str_append(H, rb_funcall(rb_formatter, rb_intern(#T), 1, regs))
VALUE superredcloth_transform(rb_formatter, p, pe, refs)
VALUE rb_formatter;
char *p, *pe;
VALUE refs;
{
if (RSTRING(block)->len > 0)
{
ADD_BLOCK();
}
if ( NIL_P(refs) && rb_funcall(refs_found, rb_intern("empty?"), 0) == Qfalse ) {
return superredcloth_transform(rb_formatter, orig_p, orig_pe, refs_found);
} else {
rb_funcall(rb_formatter, rb_intern("after_transform"), 1, html);
return html;
}
}
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
No content
Slide 47
Slide 47 text
%%{
machine superredcloth_scan;
include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl";
action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); }
# blocks
notextile_tag_start = "" ;
notextile_tag_end = "" CRLF? ;
notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
pre_tag_start = "
]* ">" (space* "")? ;
pre_tag_end = ("" space*)? "" CRLF? ;
pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"...
block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ;
pre_tag := |*
pre_tag_end { CAT(block); DONE(block); fgoto main; };
default => esc_pre;
*|;
notextile_tag := |*
notextile_tag_end { DONE(block); fgoto main; };
default => cat;
*|;
}%%
Slide 48
Slide 48 text
%%{
machine superredcloth_scan;
include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl";
action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); }
# blocks
notextile_tag_start = "" ;
notextile_tag_end = "" CRLF? ;
notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
pre_tag_start = "
]* ">" (space* "")? ;
pre_tag_end = ("" space*)? "" CRLF? ;
pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"...
block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ;
pre_tag := |*
pre_tag_end { CAT(block); DONE(block); fgoto main; };
default => esc_pre;
*|;
notextile_tag := |*
notextile_tag_end { DONE(block); fgoto main; };
default => cat;
*|;
}%%
Slide 49
Slide 49 text
%%{
machine superredcloth_scan;
include superredcloth_common "ext/superredcloth_scan/superredcloth_common.rl";
action extend { extend = rb_hash_aref(regs, ID2SYM(rb_intern("type"))); }
# blocks
notextile_tag_start = "" ;
notextile_tag_end = "" CRLF? ;
notextile_block_start = ( "notextile" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
pre_tag_start = "
]* ">" (space* "")? ;
pre_tag_end = ("" space*)? "" CRLF? ;
pre_block_start = ( "pre" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
bc_start = ( "bc" >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ; btype = ( "p" | "h1"...
block_start = ( btype >A %{ STORE(type) } A C :> "." ( "." %extend | "" ) " "+ ) ;
next_block_start = ( btype A C :> "."+ " " ) >A @{ p = reg - 1; } ;
pre_tag := |*
pre_tag_end { CAT(block); DONE(block); fgoto main; };
default => esc_pre;
*|;
notextile_tag := |*
notextile_tag_end { DONE(block); fgoto main; };
default => cat;
*|;
}%%
/*
* redcloth_scan.c.rl
*
* Copyright (C) 2009 Jason Garber
*/
#define redcloth_scan_c
#define RSTRING_NOT_MODIFIED
#include
#include "redcloth.h"
VALUE mRedCloth, super_ParseError, super_RedCloth, super_HTML, super_LATEX;
VALUE SYM_escape_preformatted, SYM_escape_attributes;
#line 23 "ext/redcloth_scan/redcloth_scan.c"
static const unsigned char _redcloth_scan_actions[] = {
0, 1, 0, 1, 2, 1, 3, 1,
4, 1, 5, 1, 6, 1, 7, 1,
9, 1, 10, 1, 11, 1, 12, 1,
13, 1, 14, 1, 15, 1, 16, 1,
17, 1, 18, 1, 19, 1, 20, 1,
21, 1, 22, 1, 23, 1, 24, 1,
25, 1, 26, 1, 27, 1, 28, 1,
29, 1, 30, 1, 34, 1, 35, 1,
36, 1, 38, 1, 40, 1, 42, 1,
43, 1, 44, 1, 45, 1, 48, 1,
57, 1, 58, 1, 59, 1, 60, 1,
61, 1, 62, 1, 63, 1, 64, 1,
65, 1, 66, 1, 70, 1, 73, 1,
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
grammar Arithmetic
rule additive
multitive ( '+' multitive )*
end
rule multitive
primary ( [*/%] primary )*
end
rule primary
'(' additive ')' / number
end
rule number
'-'? [1-9] [0-9]*
end
end
Slide 54
Slide 54 text
require 'parslet'
class MiniParser < Parslet::Parser
rule(:integer) { match('[0-9]').repeat(1) }
root(:integer)
end
MiniParser.new.parse("1324321")
# => "1324321"@0
Slide 55
Slide 55 text
Parslet::Atoms
str('Boston')
Slide 56
Slide 56 text
Parslet::Atoms
str('Boston').parse('Boston')
Slide 57
Slide 57 text
Parslet::Atoms
str('Boston').parse('Boston')
# => "Boston"@0
Slide 58
Slide 58 text
Parslet::Atoms
str('Boston').parse('Boston')
# => "Boston"@0
match('[0-9a-f]')
Slide 59
Slide 59 text
Parslet::Atoms
str('Boston').parse('Boston')
# => "Boston"@0
match('[0-9a-f]')
any
Slide 60
Slide 60 text
Operators
str('Wicked') >> str('Good')
str('Ruby') | str('Elixir')
match('[Bb]') >> str('oston') |
match('[Mm]') >> str('assachusetts')
Slide 61
Slide 61 text
Repetition
str('foo').repeat
str('foo').repeat(1)
str('foo').repeat(1,3)
str('foo').repeat(0, nil)
str('foo').maybe
Slide 62
Slide 62 text
Presence
str('Java') >> str('Script').present?
str('0').repeat(1).absent? >>
match('[\d]').repeat(1)
Slide 63
Slide 63 text
1. Create a grammar
What should be legal syntax?
2. Annotate the grammar:
What is important data?
3. Create a transformation:
How do I want to work with that data?
Slide 64
Slide 64 text
1. Create a grammar
What should be legal syntax?
2. Annotate the grammar:
What is important data?
3. Create a transformation:
How do I want to work with that data?
Slide 65
Slide 65 text
Capture
str('Common').parse('Common')
# => "Common"@0
str('Common').as(:park).parse('Common')
# => {:park=>"Common"@0}
Slide 66
Slide 66 text
Capture
str('a').repeat.as(:b)
# => {:b=>"aaa"@0}
str('a').as(:b).repeat
# => [{:b=>"a"@0}, {:b=>"a"@1}, {:b=>"a"@2}]
str('a').as(:a) >> str('b').as(:b) >> str('c')
# => {:a=>"a"@0, :b=>"b"@1}
Slide 67
Slide 67 text
rule(:table) do
(str("table") >> attributes?.as(:attributes) >> str(".\n")).maybe >>
table_row.repeat(1).as(:content) >> block_end
end
rule(:table_row) do
table_row_attributes >> table_row_content >> end_table_row
end
rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe }
rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) }
rule(:end_table_row) { str("|") >> (block_end.present? | (str("\n"))) }
rule(:table_header) do
str("|_. ") >> table_content.as(:content)
end
rule(:table_data) do
str("|") >> str("\n").absent? >> td_attributes? >> table_content.as(:content)
end
Slide 68
Slide 68 text
table(#prices).
| Adults | $5 |
| Children | $2 |
Adults
$5
Children
$2
Slide 69
Slide 69 text
rule(:table) do
(str("table") >> attributes?.as(:attributes) >> str(".\n")).maybe >>
table_row.repeat(1).as(:content) >> block_end
end
rule(:table_row) do
table_row_attributes >> table_row_content >> end_table_row
end
rule(:table_row_attributes) { (attributes?.as(:attributes) >> str(". ")).maybe }
rule(:table_row_content) { (table_header | table_data).repeat(1).as(:content) }
rule(:end_table_row) { str("|") >> (block_end.present? | (str("\n"))) }
rule(:table_header) do
str("|_. ") >> table_content.as(:content)
end
rule(:table_data) do
str("|") >> str("\n").absent? >> td_attributes? >> table_content.as(:content)
end
Slide 70
Slide 70 text
def parenthesized(atom)
str('(') >> atom >> str(')')
end
parenthesized(match['\d']).parse("(500)")
Slide 71
Slide 71 text
class HtmlTag < Parslet::Parser
root(:tag)
rule(:tag) { open_tag | close_tag | self_closing_tag | comment_tag }
rule(:open_tag) { str("<") >> tag_name >> attributes? >> str(">") }
rule(:close_tag) { str("") >> tag_name >> str(">") }
rule(:tag_name) { match("[A-Za-z_:]") >> name_char.repeat }
...
end
class BlockHtmlTag < HtmlTag
rule(:tag_name) do
inline_tag_name.absent? >> any_tag_name
end
rule(:inline_tag_name) do
INLINE_TAGS.map {|name| str(name) }.reduce(:|)
end
end
Slide 72
Slide 72 text
> HtmlTag.new.open_tag.methods
=> [:name, :block, :try, :parslet, :to_s_inner, :parse, :apply, :cached?, ...]
> HtmlTag.new.open_tag.parse("
") => ""@0 > HtmlTag.new.open_tag.parse("") Parslet::ParseFailed: Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2. from /Users/jasongarber/.rvm/gems/ruby-2.0.0-p247/gems/parslet-1.5.0/lib/parslet/ cause.rb:63:in `raise' from /Users/jasongarber/.rvm/gems/ruby-2.0.0-p247/gems/parslet-1.5.0/lib/parslet/ atoms/base.rb:46:in `parse' from (irb):8 from /Users/jasongarber/.rvm/rubies/ruby-2.0.0-p247/bin/irb:16:in `'
Slide 73
Slide 73 text
begin
RedClothParslet::Parser::BlockHtmlTag.new.tag.parse("![]()
")
rescue Parslet::ParseFailed => failure
puts failure.cause.ascii_tree
end
Expected one of [OPEN_TAG, CLOSE_TAG, SELF_CLOSING_TAG, COMMENT_TAG] at line 1 char 1.
|- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2.
| `- Failed to match sequence ((!INLINE_TAG_NAME / &(INLINE_TAG_NAME NAME_CHAR{1, })) ANY_TAG_NAME)
at line 1 char 2.
| `- Expected one of [!INLINE_TAG_NAME, &(INLINE_TAG_NAME NAME_CHAR{1, })] at line 1 char 2.
| |- Input should not start with INLINE_TAG_NAME at line 1 char 2.
| `- Input should start with INLINE_TAG_NAME NAME_CHAR{1, } at line 1 char 2.
|- Failed to match sequence ('' TAG_NAME '>') at line 1 char 1.
| `- Expected "", but got "') at line 1 char 2.
| `- Failed to match sequence ((!INLINE_TAG_NAME / &(INLINE_TAG_NAME NAME_CHAR{1, })) ANY_TAG_NAME)
at line 1 char 2.
| `- Expected one of [!INLINE_TAG_NAME, &(INLINE_TAG_NAME NAME_CHAR{1, })] at line 1 char 2.
| |- Input should not start with INLINE_TAG_NAME at line 1 char 2.
| `- Input should start with INLINE_TAG_NAME NAME_CHAR{1, } at line 1 char 2.
`- Failed to match sequence ('
Slide 74
Slide 74 text
I Objects
Slide 75
Slide 75 text
describe RedClothParslet::Parser::HtmlTag do
it { should parse("
") }
it { should parse("
") } it { should parse("
") }
it { should parse("") }
describe "attribute" do
subject { described_class.new.attribute }
it { should parse(" class='awesome'") }
it { should_not parse(' 9kittens="cute"') }
end
end
") } it { should parse("
Slide 76
Slide 76 text
$ rspec spec/parser/html_tag_spec.rb
1) RedClothParslet::Parser::HtmlTag tag
Failure/Error: it { should parse("") }
expected TAG to be able to parse ""
Expected one of [OPEN_TAG, CLOSE_TAG, SELF_CLOSING_TAG, COMMENT_TAG] at line 1 char 1.
|- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? '>') at line 1 char 2.
| `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2.
| `- Failed to match [A-Za-z_:] at line 1 char 2.
|- Failed to match sequence ('<' TAG_NAME '>') at line 1 char 2.
| `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2.
| `- Failed to match [A-Za-z_:] at line 1 char 2.
|- Failed to match sequence ('<' TAG_NAME ATTRIBUTES? SPACES? '/' '>') at line 1 char 2.
| `- Failed to match sequence ([A-Za-z_:] NAME_CHAR{0, }) at line 1 char 2.
| `- Failed to match [A-Za-z_:] at line 1 char 2.
`- Failed to match sequence ('
Slide 77
Slide 77 text
1. Create a grammar
What should be legal syntax?
2. Annotate the grammar:
What is important data?
3. Create a transformation:
How do I want to work with that data?
Slide 78
Slide 78 text
Transformations
tree = {left: {int: '1'},
op: '+',
right: {int: '2'}}
class T < Parslet::Transform
rule(int: simple(:x)) { Integer(x) }
end
T.new.apply(tree)
# => {:left=>1, :op=>"+", :right=>2}
Slide 79
Slide 79 text
Transformations
tree = {left: {int: '1'},
op: '+',
right: {int: '2'}}
class T < Parslet::Transform
rule(int: simple(:x)) { Integer(x) }
rule(op: '+', left: simple(:l),
right: simple(:r)) { l + r }
end
T.new.apply(tree) # => 3
Slide 80
Slide 80 text
Transformations
rule(:content => subtree(:c), :attributes => subtree(:a)) do |dict|
{:content => dict[:c], :opts => RedCloth::Ast::Attributes.new(dict[:a])}
end
rule(:table => subtree(:a)) do
RedCloth::Ast::Table.new(a[:content], a[:opts])
end
rule(:bq => subtree(:a)) do
RedCloth::Ast::Blockquote.new(a[:content], a[:opts])
end
Slide 81
Slide 81 text
Who you
callin’ slow?
Slide 82
Slide 82 text
class ErbParser < Parslet::Parser
rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
rule(:expression) { (str('=') >> ruby).as(:expression) }
rule(:comment) { (str('#') >> ruby).as(:comment) }
rule(:code) { ruby.as(:code) }
rule(:erb) { expression | comment | code }
rule(:erb_with_tags) { str('<%') >> erb >> str('%>') }
rule(:text) { (str('<%').absent? >> any).repeat(1) }
rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t
root(:text_with_ruby)
end
Slide 83
Slide 83 text
class ErbParser < Parslet::Parser
rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
rule(:expression) { (str('=') >> ruby).as(:expression) }
rule(:comment) { (str('#') >> ruby).as(:comment) }
rule(:code) { ruby.as(:code) }
rule(:erb) { expression | comment | code }
rule(:erb_with_tags) { str('<%') >> erb >> str('%>') }
rule(:text) { (str('<%').absent? >> any).repeat(1) }
rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t
root(:text_with_ruby)
end
class ErbParser < Parslet::Parser
rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
rule(:expression) { (str('=') >> ruby).as(:expression) }
rule(:comment) { (str('#') >> ruby).as(:comment) }
rule(:code) { ruby.as(:code) }
rule(:erb) { expression | comment | code }
rule(:erb_with_tags) { str('<%') >> erb >> str('%>') }
rule(:text) { (str('<%').absent? >> any).repeat(1) }
rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t
root(:text_with_ruby)
end
Slide 84
Slide 84 text
class ErbParser < Parslet::Parser
rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
rule(:expression) { (str('=') >> ruby).as(:expression) }
rule(:comment) { (str('#') >> ruby).as(:comment) }
rule(:code) { ruby.as(:code) }
rule(:erb) { expression | comment | code }
rule(:erb_with_tags) { str('<%') >> erb >> str('%>') }
rule(:text) { (str('<%').absent? >> any).repeat(1) }
rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t
root(:text_with_ruby)
end
class ErbParser < Parslet::Parser
rule(:ruby) { (str('%>').absent? >> any).repeat.as(:ruby) }
rule(:expression) { (str('=') >> ruby).as(:expression) }
rule(:comment) { (str('#') >> ruby).as(:comment) }
rule(:code) { ruby.as(:code) }
rule(:erb) { expression | comment | code }
rule(:erb_with_tags) { str('<%') >> erb >> str('%>') }
rule(:text) { (str('<%').absent? >> any).repeat(1) }
rule(:text_with_ruby) { (text.as(:text) | erb_with_tags).repeat.as(:t
root(:text_with_ruby)
end
include Parslet::Accelerator
optimized = apply( parser,
rule( (str(:x).absent? >> any).repeat )
{ GobbleUp.new(x, 0) }
)
Slide 85
Slide 85 text
Custom Atoms
• Parse a limited set of natural-language queries
• “Who makes the best cheesesteaks in Boston?”
• EngTagger: a corpus-trained, probabilistic tagger
• Custom Parslet::Atom
Slide 86
Slide 86 text
rule(:question) do
interrogative.maybe.as(:int) >>
verb_phrase.maybe.as(:verb) >>
superlative_phrase.maybe.as(:sup) >>
subject.as(:subj) >>
prepositional_phrase.repeat.as(:preps) >>
sentence_end.repeat.as(:punct)
end
rule(:verb_phrase) { (verb_present | verb_past | verb_future).as(:vp) }
rule(:verb_present) { word(:VBZ, :VB).as(:present) }
rule(:verb_past) { word(:VBD).as(:past) }
rule(:verb_future) { word(:MD).as(:future) >> word(:VB).maybe.as(:infinitive) }
Slide 87
Slide 87 text
Other Crazy Uses
Slide 88
Slide 88 text
Other Crazy Uses
•User-supplied formulas / logic
Slide 89
Slide 89 text
Other Crazy Uses
•User-supplied formulas / logic
•Logs
Slide 90
Slide 90 text
Other Crazy Uses
•User-supplied formulas / logic
•Logs
•Streaming text
Slide 91
Slide 91 text
Other Crazy Uses
•User-supplied formulas / logic
•Logs
•Streaming text
•The Right Reverend and Right Honourable the Lord
Bishop of London Richard John Carew Chartres
Slide 92
Slide 92 text
Who will write the next
awesome DSL?
Slide 93
Slide 93 text
Who will write the next
awesome DSL!.
You
Slide 94
Slide 94 text
Conclusions
Slide 95
Slide 95 text
Conclusions
• DSLs make life better
Slide 96
Slide 96 text
Conclusions
• DSLs make life better
• internal_dsl > external_dsl if
internal_dsl.practicable?
Slide 97
Slide 97 text
Conclusions
• DSLs make life better
• internal_dsl > external_dsl if
internal_dsl.practicable?
• Keep your parser clean
Slide 98
Slide 98 text
Conclusions
• DSLs make life better
• internal_dsl > external_dsl if
internal_dsl.practicable?
• Keep your parser clean
• “Situational awareness!”
Slide 99
Slide 99 text
@jasongarber @promptworks
jason@promptworks.com