Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Pattern Matching
Search
Aaron Patterson
May 11, 2011
Programming
170
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Pattern Matching
MagRails 2011
Aaron Patterson
May 11, 2011
Transcript
150 slides
Pattern Matching
omg
Steak Holder
Aaron Patterson
@tenderlove
Ruby core team
Rails core team
! WARNING !
Stage II: Advanced Beginner
0. Excited to be in England 1. English country, I
have accent. 2. Fear and guilt 3. Fear of plugs and outlets 4. Can't stop thinking of strange 5. Romance Ebi
Manchester Agile Rails
None
None
None
None
! WARNING !
Graphviz http://graphviz.org
world hello greeting
digraph nfa { rankdir=LR; world [shape = doublecircle]; hello [shape
= circle]; hello -> world [label="greeting"]; } graph.dot
digraph nfa { rankdir=LR; world [shape = doublecircle]; hello [shape
= circle]; goodbye [shape = circle]; hello -> world [label="greeting"]; goodbye -> world } graph.dot
world hello greeting goodbye
2 3 . 4 / /articles(.:format) 5 6 /articles/new(.:format) 0
1 / articles (?-mix:[^./?]+) new
Pattern Matching Move between theory and practice
Journey Yes, it's named after the '70s rock sensation
What is a router? Provides 1: URL generation 2: Path
Recognition 3: Route parsing
Why a new router?
Maintenance
0 700 1400 2100 2800 Journey Journey-2 Rack-Mount LOC
Known algorithms
References • Compilers: Principles, Techniques, & Tools (Aho, Lam, Sethi,
Ullman) • Intro to Formal Languages and Automata (Linz)
Predictability
CPU Time
Memory Usage
Current Performance url generation path recognition route parsing rack-mount Journey
Patterns
Why patterns matter
Regular Expressions /om(g)!/
Rails Routes
resource :articles
/articles(.:format) /articles/new(.:format) /articles/:id/edit(.:format) /articles/:id(.:format)
Parens don't capture
:whatever => /([^./?]+)/
/articles(.:format) /\/articles(?:\.([^.\/?]+))?$/
/\/articles(?:.([^.\/?]+))?$/ /\/articles\/new(?:.([^.\/?]+))?$/ /\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/ /\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/ GET /articles/new 200 OK!
/\/articles(?:.([^.\/?]+))?$/ /\/articles\/new(?:.([^.\/?]+))?$/ /\/articles\/([^.\/?]+)\/edit(?:.([^.\/?]+))?$/ /\/articles\/([^.\/?]+)(?:.([^.\/?]+))?$/ GET /foos/new 404 Not Found
How long does this take?
r = routes.length
x = regexp compare
O(r ⨉ x)
Can we do better?
Finite State Machines Parse Trees Automata
Parse Trees
Parsing regexp
(a|b)*abb
◦ ◦ b ◦ b a * a b ()
| Describe node types
Parsing rails routes
Journey::Parser parser = Journey::Parser.new ast = parser.parse '/articles(.:format)' puts ast.to_dot
◦ ◦ () / articles ◦ . :format
To String! parser = Journey::Parser.new ast = parser.parse '/articles(.:format)' puts
ast.to_s puts ast.to_regexp
OR AST parser = Journey::Parser.new trees = [ '/articles(.:format)', '/articles/new(.:format)',
].map { |s| parser.parse s } ast = Journey::Nodes::Or.new trees puts ast.to_dot
| ◦ ◦ ◦ () / articles ◦ . :format
◦ () ◦ new ◦ / / articles ◦ . :format
SECRET FEATURE parser = Journey::Parser.new ast = parser.parse '/articles|books(.:format)' SUPER
SECRET!
Automata
(a|b)*abb
3 0 b 2 a b 1 a a b
b a Double circle is acceptance state
baabb
3 0 b 2 a b 1 a a b
b a
3 0 b 2 a b 1 a a b
b a b
3 0 b 2 a b 1 a a b
b a ba
3 0 b 2 a b 1 a a b
b a baa
3 0 b 2 a b 1 a a b
b a baabb
aab 3 0 b 2 a b 1 a a
b b a
Deterministic Finite Automaton
Only one path
Storage class DFA attr_reader :table def initialize @table = Hash.new
{ |h, from| h[from] = {} } @table[0]['a'] = 1 @table[0]['b'] = 0 @table[1]['a'] = 1 @table[1]['b'] = 2 @table[2]['a'] = 1 @table[2]['b'] = 3 @table[3]['b'] = 0 @table[3]['a'] = 2 end end
FSM Simulation
class Simulator def initialize(dfa) @dfa = dfa end def simulate(symbols)
state = 0 until symbols.empty? state = @dfa.move(state, symbols.shift) end state end end
class DFA ... def move(from, symbol) @table[from][symbol] end end move
function
irb> sim = Simulator.new(DFA.new) => #<Simulator:0x007f95929a82e0 ...> irb> sim.simulate %w{
b a a b b } => 3 irb> sim.simulate %w{ a a b } => 2
Time: O(n) n = string.length
Space: S + T states.length + transitions.length
Nondeterministic Finite Automaton
Has nil edges
Can't tell direction
Simulation of NFA
6 0 2 ε 4 ε 3 a 5 b
ε ε a|b
6 0 2 ε 4 ε 3 a 5 b
ε ε nil-closure
6 0 2 ε 4 ε 3 a ε 5
b ε a
Storage class NFA def initialize @table = Hash.new { |h,
from| h[from] = Hash.new { |i,sym| i[sym] = [] } } @table[0][nil] << 2 @table[0][nil] << 4 @table[2]['a'] << 3 @table[4]['b'] << 5 @table[3][nil] << 6 @table[5][nil] << 6 end def nil_closure(states) states.map { |s| @table[s][nil] }.flatten end end
FSM Simulation class Simulator def initialize(nfa) @nfa = nfa end
def simulate(symbols) states = @nfa.nil_closure([0]) until symbols.empty? next_s = @nfa.move(states, symbols.shift) states = @nfa.nil_closure(next_s) end states end end
Move function class NFA ... def move(states, symbol) states.map {
|s| @table[s][symbol] }.flatten end end
irb> sim = Simulator.new(NFA.new) => #<Simulator:0x007faa188a5f88 ...> irb> sim.simulate %w{
a } => [6] irb> sim.simulate %w{ b } => [6] irb> sim.simulate %w{ b b } => [] irb> sim.simulate %w{ c } => []
Time: O(r ⨉ x) r = operators.length, x = string.length
NFA Construction
◦ ◦ () / articles ◦ . :format /articles(.:format)
cat nodes 1 0 /
/articles 2 0 1 / articles
Optional 3 0 ε 1 ε 2 ????? ε
6 0 1 / 2 articles ε 3 ε 4
. 5 :format ε
parser = Journey::Parser.new ast = parser.parse '/articles(.:format)' nfa = Journey::NFA::Builder.new
ast tt = nfa.transition_table puts tt.to_dot Journey::NFA
Converting NFA to DFA
Eliminate nil transitions
Collapse duplicate edges
6 0 1 / 2 articles ε 3 ε 4
. 5 :format ε /articles(.:format)
2 3 . 4 0 1 / articles :format /articles(.:format)
CODES! parser = Journey::Parser.new ast = parser.parse '/articles(.:format)' nfa =
Journey::NFA::Builder.new ast tt = nfa.transition_table.generalized_table puts tt.to_dot
SHORTER CODES! parser = Journey::Parser.new ast = parser.parse '/articles(.:format)' dfa
= Journey::GTG::Builder.new ast tt = dfa.transition_table puts tt.to_dot
ANY NFA converts to DFA
O(r ⨉ x) => O(x) r = operations.length, x =
string.length
Converting Automata to Regexp
2 3 . 4 0 1 / articles :format /articles(.:format)
2 3 . 4 0 /articles :format /articles(.:format)
/articles(.:format) 2 4 .:format 0 /articles
/articles(.:format) 4 0 /articles(.:format)?
Generalized Transition Graph
/users(.:format) /users/new(.:format) /users/:id/edit(.:format) /users/:id(.:format) resource :users
2 3 . 4 / 5 6 8 . 7
9 / 10 . 11 12 14 . 13 15 0 1 / users (?-mix:[^./?]+) new (?-mix:[^./?]+) (?-mix:[^./?]+) edit (?-mix:[^./?]+) (?-mix:[^./?]+) resource :users
The Plan?
Combine All Routes
Produce DFA
Simulate in O(n) Time
The Future
JS Simulation
None
Rails Console irb> File.open('out.html', 'wb') { |f| irb* f.write( irb*
Wot::Application.routes.router.visualizer irb> )} => 69074
Table => JSON parser = Journey::Parser.new ast = parser.parse '/articles(.:format)'
dfa = Journey::GTG::Builder.new ast tt = dfa.transition_table puts tt.to_json
Table => SVG parser = Journey::Parser.new ast = parser.parse '/articles(.:format)'
dfa = Journey::GTG::Builder.new ast tt = dfa.transition_table puts tt.to_svg
JS Tokenizer function tokenize(input, callback) { while(input.length > 0) {
callback(input.match(/^[\/\.\?]|[^\/\.\?]+/)[0]); input = input.replace(/^[\/\.\?]|[^\/\.\?]+/, ''); } }
JS Simulator tokenize(input, function(token) { var new_states = []; for(var
key in states) { var state = states[key]; if(string_states[state] && string_states[state][token]) { var new_state = string_states[state][token]; highlight_edge(state, new_state); highlight_state(new_state); new_states.push(new_state); } } if(new_states.length == 0) { return; } states = new_states; });
routes.rb marshalling
Test suggestions
Test coverage
Usage Heat Maps
????
Open Questions
Is our GTG deterministic?
/users/new|/users/:id 4 5 0 1 / 2 users 3 /
new (?-mix:[^./?]+)
"new" =~ /new|[^.\/?]+/
Can we make it deterministic?
L1 = {new} L2 = {[^./?]+}
/users/new|/users/:id 4 5 0 1 / 2 users 3 /
L1 L2 - L1
Is it worth our effort?
REGEX AST NFA DFA
DFA => YACC
DFA => RACC
DFA => Ragel
Is it worth our effort?
Thanks Tatsuya Ono!
Thank You!!
<3<3<3<3<3
More Decks by Aaron Patterson
See All by Aaron Patterson
Faster FFI for Ruby
tenderlove
0
120
RubyKaigi 2025: Class New, A New Approach
tenderlove
0
320
RubyKaigi Dev Meeting 2025
tenderlove
1
6.6k
Speeding up Instance Variables in Ruby 3.3
tenderlove
2
620
[Feature #20425] Speeding up delegate methods
tenderlove
3
410
RailsConf 2023
tenderlove
30
1.6k
Don't @ Me! Faster Instance Variables with Object Shapes
tenderlove
1
590
RailsConf 2022 Keynote
tenderlove
2
730
Some Assembly Required
tenderlove
1
680
Other Decks in Programming
See All in Programming
エラーチャンネルのボールを拾い続けて鍛えるエンジニアの地力 / phpcon-niigata2026
meihei3
1
380
[DroidKnights 2026] Workflow AX 적용기 — 자동 정책 정리부터 E2E까지
dayeon
0
100
Beyond Chatbots: Agentic UI with Open Standards @W-JAX 2026, München
manfredsteyer
PRO
0
110
手の動かし方がわからなかった私が個人開発者になるまで
usagi0920_pochi
0
220
なぜ私たちは「現時点では」KMPの導入を見送ったのか
chigichan24
1
290
AIコーディング時代のKotest ユニットテストから APIレベルの仕様固定化テストへ
ooga04
0
190
コード生成時代のAPI定義を支えるRequest Spec移行
pvcresin
0
200
どうしても Eloquent でリポジトリパターンをやりたいんだ #phpcon_niigata / phpcon-niigata-2026
shogogg
0
560
止まらない比較を止める — 余帰納法で無限ストリームを比べる
tarao
0
190
[DroidKnights 2026] Workflow AX: From Auto-Documented Policies to E2E Testing
raeminkang
0
580
Distance field of Julia set
kinakomoti321
0
330
AIが生成したコードは読むべきか?抽象化設計とコードの品質保証 / abstraction-design-and-quality
minodriven
16
9.3k
Featured
See All Featured
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
130k
Prompt Engineering for Job Search
mfonobong
0
590
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
25
2.2k
How to Build an AI Search Optimization Roadmap - Criteria and Steps to Take #SEOIRL
aleyda
1
2.3k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
360
31k
Agile Leadership in an Agile Organization
kimpetersen
PRO
0
310
What Being in a Rock Band Can Teach Us About Real World SEO
427marketing
0
1.1k
Designing Powerful Visuals for Engaging Learning
tmiket
1
660
Visual Storytelling: How to be a Superhuman Communicator
reverentgeek
2
750
Building Applications with DynamoDB
mza
96
7.3k
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
akashhashmi
0
550
The B2B funnel & how to create a winning content strategy
katarinadahlin
PRO
1
610