The Evolution of Path::Dispatcher

A new-and-improved version of my YAPC::NA talk directly below. This talk has become a case study in Moosey design, describing how the needs of Path::Dispatcher's users have influenced its evolution. Too many talks present a topic as though the solution sprung fully-formed from the designer's brain in an instant, ignoring the interesting details of how a system is improved over time. Maybe next conference I'll present The Evolution of 'The Evolution of Path::Dispatcher', a talk on how to evolve talks after presenting them. :)

Transcript

1. The Evolution of Path::Dispatcher Shawn M Moore @sartak ਐԽ 1 1 2010೥10݄15೔༵ۚ೔ Presented 2010-10-16, Oookayama, Tokyo Japan, YAPC::Asia 2010
2. αʔλοΫ •http://sartak.org • ձࣾ͸@obraͷBest Practical SolutionsͰ͢ • ΞϝϦΧͷϘετϯʹॅΜͰ͍·͢ • 28೔·Ͱ೔ຊʹ଺ࡏ͓͖ͯ͠·͢ʈʈ 2 2 2010೥10݄15೔༵ۚ೔ A little bit about me. I work for Best Practical with Jesse Vincent. I live in Boston. I’ll be here in Japan until the 28th.
3. Jifty::Dispatcher • Jifty’s URI dispatcher • Web͚ͩͰ͢ Jiftyͱ͍͏WAFͷJifty::Dispatcher͔Β ΠϯεϐϨʔγϣϯΛड͚·ͨ͠ 3 3 2010೥10݄15೔༵ۚ೔ Path::Dispatcher drew inspiration from the Jifty web framework’s URI dispatcher, Jifty::Dispatcher. We’ll see later that the syntax is pretty similar even though I’ve grown to dislike it.
4. Jifty::DispatcherͷAPI on '/tag/*' => run { dispatch “/list/not/complete/tag/$1"; }; on qr{^/news/(\d+)} => run { my $news = load_news($1) or abort 404; set news => $news; show '/news/item'; }; 4 4 2010೥10݄15೔༵ۚ೔ This is what Jifty::Dispatcher looks like.
5. Jifty::DispatcherͷAPI on '/tag/ *' => run { dispatch “/list/not/complete/tag/ $1”; }; /tag/೔ຊޠ -> /list/not/complete/tag/೔ຊޠ ύλʔϯ͕ग़དྷͯɺ $1,$2౳ʹ਺஋Λஔ͖·͢ 5 5 2010೥10݄15೔༵ۚ೔ You’ll notice we let you use patterns, and we populate the number variables with their values.
6. Jifty::DispatcherͷAPI under ticket => run { on create => run { ... }; on update => run { ... }; on delete => run { ... }; }; /ticket/create /ticket/update /ticket/delete ͖Ε͍΍εϐʔυͷͨΊʹɺ underͰrule͕ूΊΒΕ·͢ 6 6 2010೥10݄15೔༵ۚ೔ You can use “under” to collect rules together, for cleanliness, readability, and efficiency.
7. Simple Defects (sd) • Prophet͸෼ࢄܕσʔλϕʔεͰ͢ • όάͷͨΊʹɺSD͸ProphetΛ޿͛·͢ • ୯७ͳ୺຤ͷΠϯλϑΣʔε͕͋Γ·ͨ͠ • @obra͕ʮվྑͯ͠Լ͍͞ʯͱݴ͍·ͨ͠ 7 7 2010೥10݄15೔༵ۚ೔ Prophet is a distributed property (key-value) database. SD is a distributed bug tracker built on top of Prophet. For a while it had a pretty simplistic command-line interface. But then Jesse asked me to build a better one.
8. use Jifty::Dispatcher; on '/tag/*' => run { dispatch "/list/not/complete/tag/$1"; }; on qr{^/news/(\d+)} => run { my $news = load_news($1) or abort 404; set news => $news; show '/news/item'; }; ·ΔͰ͜ΕΛ࢖͍͍ͨͰͨ͠ʂ ͔͠͠web͚ͩͳͷͰ͢ 8 8 2010೥10݄15೔༵ۚ೔ What I wanted to use was something like this. But remember that Jifty::Dispatcher is web only and we’re writing a command-line interface.
9. $ sd '/ticket/update/10 ?summary=crikey &due=today' ͔͠΋ɺೖྗ͢Δ͜ͱ͸໘౗͍Ͱ͢Ͷ 9 9 2010೥10݄15೔༵ۚ೔ And on top of that, who wants to type URLs into their command-line?
10. $ sd ticket update 10 --summary=crikey --due=today ͜ͷํ͕ྑ͍Ͱ͢Ͷ 10 10 2010೥10݄15೔༵ۚ೔ We want our users to type something like this instead.
11. ͔ͩΒ Path::Dispatcher͸ ੜ·ΕͨͷͰͨ͠ 11 11 2010೥10݄15೔༵ۚ೔ And so, Path::Dispatcher was born.
12. Path::DispatcherͷAPI on RULE => sub { CODE }; Jifty::DispatcherΈ͍ͨͳAPIΛ࡞Γ·ͨ͠ɻ յΕ͍ͯͳ͍෺Λ௚͠·ͤΜʂ 12 12 2010೥10݄15೔༵ۚ೔ The basic API looks like this. It’s pretty much the same as Jifty::Dispatcher’s. If it ain’t broke don’t fix it.
13. API on qr{^ /ticket /update /(\d+)$} => sub { # $1: 10 }; $ sd /ticket /update /10 ͜Ε͸ɺ/ Λೖྗ͢ΔඞཁͳͷͰɺ ·ͩ໘౗͍Ͱ͢Ͷ 13 13 2010೥10݄15೔༵ۚ೔ We can’t have exactly the same API because requiring the user to type slashes is pretty junk.
14. API on ‘ticket’, ‘update’, qr{^(\d+)$} => sub { # $1: ticket # $2: update # $3: 10 }; $ sd ticket update 10 API͕͜Εʹม͑ΒΕΔͳΒ… 14 14 2010೥10݄15೔༵ۚ೔ If we can change the API to something like this...
15. API POST /ticket/update/10 $ sd ticket update 10 ҰͭͷdispatcherͷΈ webͰ΋୺຤Ͱ΋ԿͰ΋ग़དྷ·͢ 15 15 2010೥10݄15೔༵ۚ೔ ...then a single dispatcher can handle the web, CLI, and anywhere else.
16. σβΠϯ • Moose͕େ޷͖Ͱ͢ • Any::Mooseʹ଱͑·͢ • OOP͕σβΠϯʹਁಁ͠·ͨ͠ 16 16 2010೥10݄15೔༵ۚ೔ I love Moose so Path::Dispatcher uses it. My boss doesn’t, so I tolerate Any::Moose instead, even though it makes the code a little uglier for native traits which Mouse lacks AFAIK. But either way, Moosey OOP definitely pervades the design.
17. σβΠϯ • Path::Dispatcher • PD::Path • PD::Rule, ::Regex, ::Sequence, ::Eq౳ • PD::Match • PD::Dispatch ͜ΜͳOOPͷσβΠϯ͸ૉ੖Β͍͠ͱࢥ͍·͢ 17 17 2010೥10݄15೔༵ۚ೔ These are some of the classes that make up Path::Dispatcher. As you can see, I like OOP. This kind of design has definitely made PD very flexible.
18. @nothingmuch͞Μ͕ executionΛdispatchͱ੾Γ཭ͯ͠ߟ͑ͯͶ ͱఏҊ͠·ͨ͠ 18 18 2010೥10݄15೔༵ۚ೔ I was talking to Yuval Kogman about Path::Dispatcher and he suggested separating execution from dispatch.
19. on ‘͠·ͬͨ’ => sub { die ‘ϚδʹϚδͰʁʂ’ }; my $d = $dispatcher->dispatch(‘͠·ͬͨ’); # Path::Dispatcher::Dispatch return 404 unless $d->has_matches; say(($d->matches)[0]->rule->string); # ͠·ͬͨ $d->run; # ࢮ executionΛdispatchͱ੾Γ཭͢Δ͜ͱͰɺ matchΛಎ࡯ग़དྷ·͢ 19 19 2010೥10݄15೔༵ۚ೔ By separating execution from dispatch, we can glean more insight into how the match went down. Like here, we inspect the matches, which keep track of which rules they matched, which themselves have metadata, and so on. Only once we execute the dispatch does the codeblock, which will throw an exception, runs.
20. ࢥ͍ग़ͤ·͔͢ ೋͭͷdispatcher͕͋Δ͕ɺࠞ߹͍ͨ͠Ͱ͢ɻ SDͰ΋୭͔ͷΞϓϦͰ΋ ProphetͷdispatcherΛͨͩͰ࠶ར༻ग़དྷ͍ͨͰ͢ • όάͷͨΊʹɺSD͸ProphetΛ޿͛·͢ 20 20 2010೥10݄15೔༵ۚ೔ Do you remember how I said SD is built on Prophet? That means there are two dispatchers and we want to be able to use them together. We want SD, as well as anyone else’s app to reuse the commands we defined in Prophet for free.
21. API package App::SD::CLI::Dispatcher; . . . redispatch_to ‘Prophet::CLI::Dispatcher’; . . . API͸ํ๏ΑΓָͰͨ͠Α ํ๏͸໌ന͡Ό͋Γ·ͤΜͰͨ͠ 21 21 2010೥10݄15೔༵ۚ೔ Here’s what the API for that looks like. That’s the easy part. The tricky part is how to make this actually work.
22. @nothingmuch͞Μ͕ PD::Rule::Dispatch͕ਅ͙ͬ͢ʹͨ͘͞ΜPD::MatchΛੜΜͰͶ ͱఏҊ͠·ͨ͠ 22 22 2010೥10݄15೔༵ۚ೔ Yuval was staying with me in Boston so I asked him how he’d do it. He suggested making PD::Rule::Dispatch just return many PD::Rule::Match objects directly. He was designing KiokuDB at the time and I wish I could have been as much help as he was for me!
23. Layering for my $plugin (plugins()) { redispatch_to $plugin->dispatcher; } redispatch_toͰ޿͛Δ͜ͱ͕ग़དྷ·͢Ͷ 23 23 2010೥10݄15೔༵ۚ೔ You can do all sorts of things with redispatch_to. It’s a pretty neat trick.
24. SD͸webΠϯλϑΣʔε΋͋Γ·͢ 24 24 2010೥10݄15೔༵ۚ೔ SD also has a web interface. It’s pretty.
25. SDͷweb on qr{^/GET/ticket/(\d+)$} => sub { say ticket($1)->dump; }; on qr{^/POST/ticket/(\d+)$} => sub { ticket($1)->update(??); }; GETͱPOSTͷ͚͡ΊΛ͚ͭΔ͜ͱ͕؊ཁͰ͢ Ͱ΋ɺ͜ͷAPI͸ͻͲ͍ʂ 25 25 2010೥10݄15೔༵ۚ೔ The web interface is powered by Path::Dispatcher. But we were running into a problem of how to distinguish GETs from POSTs.
26. PD::Rule::Metadata under ‘ticket’, qr{^\d+$} => sub { on { method => ‘GET’ } => sub { ... }; on { method => ‘POST’ } => sub { ... }; }; $path = Path::Dispatcher::Path->new( path => $uri, metadata => { method => $method }, ); $dispatcher->run($path); PD::Rule::MetadataͰͨ͘͞Μύϥϝʔλʔ͕࢖͑·͢ 26 26 2010೥10݄15೔༵ۚ೔ With Path::Dispatcher::Rule::Metadata, you can use many parameters in your rules, like request method.
27. PD::Rule::Metadata sub get { return { method => ‘GET’ } } sub post { return { method => ‘UNDER’ } } under ‘ticket’, qr{^\d+$} => sub { on get, sub { ... }; on post, sub { ... }; }; ͜ͷํ͕ྑ͍Ͱ͔͢ʁ 27 27 2010೥10݄15೔༵ۚ೔ Is this any better?
28. PD::Rule::Metadata sub method { my ($method, @on) = @_; under { method => $method } => sub { on @on; } } sub get { method(‘get’, @_) } sub post { method(‘post’, @_) } get ‘ticket’, qr{^\d+$} => sub { ... }; post ‘ticket’, qr{^\d+$} => sub { ... }; ͜ͷํ͕ྑ͍Ͱ͔͢ʁ 28 28 2010೥10݄15೔༵ۚ೔ How about this?
29. PD::Rule::Metadata Path::Dispatcher::Path->new( rule => $ENV{REQUEST_URI}, metadata => \%ENV, ); 29 29 2010೥10݄15೔༵ۚ೔ By the way, Path::Dispatcher is fully PSGI compatible with like no effort.
30. PD::Rule::Metadata Path::Dispatcher::Path->new( rule => $ENV{REQUEST_URI}, metadata => \%ENV, ); PSGI compatible 29 29 2010೥10݄15೔༵ۚ೔ By the way, Path::Dispatcher is fully PSGI compatible with like no effort.
31. PD::Rule::Metadata Path::Dispatcher::Path->new_from_psgi; ·ͩ࢖͑·ͤΜ Ͱ΋ɺྑ͍ͱࢥ͍·͢ 30 30 2010೥10݄15೔༵ۚ೔ Should I implement this? I don’t see why not.
32. tabͰॻ͘ $ sd a<tab> about alias aliases attachment @obra΋ࢲ΋tabͰॻ͘͜ͱ͕޷͖Ͱ͢ ඞཁ͸໌നͰ͢Ͷ 31 31 2010೥10݄15೔༵ۚ೔ Jesse and I like tab completion. Obviously we needed to add it.
33. tabͰॻ͘ $ sd he<tab> $ sd help <tab> about authors environment log settings alias clone find publish summary-format aliases comment help pull sync attach comments history push ticket attachment config init search ticket.summary-format attachments copying intro server ticket_summary_format author env list setting tickets ଟ෼ͨ͘͞ΜdispatcherͷruleͷϨϕϧ͕͋Γ·͢ 32 32 2010೥10݄15೔༵ۚ೔ There might be many levels of dispatcher rules.
34. PD::Rule::Enum sub complete { my ($self, $path) = @_; return grep { my $partial = substr($_, 0, length($path)); $partial eq $path } $self->enum; } tabͷͨΊͷίʔυΛॻ͘͜ͱ͸೉͘͠ͳ͔ͬͨͰ͢ ͜ͷίʔυ͸Path::Dispatcherͷ಺෦ͳͷͰɺ ॻ͔ͳ͍ͰԼ͍͞ 33 33 2010೥10݄15೔༵ۚ೔ Writing the code to support tab completion was pretty straightforward. This is internal, so when you’re using Path::Dispatcher you generally don’t have to care about writing this kind of thing.
35. PD::Rule::Enum on enum(‘perl’, ‘python’, ‘php’, ‘ruby’) => sub { say “$1͕େ޷͖ʂ”; }; $ daisuki p<tab> perl python php $ daisuki pe<tab> $ daisuki perl perl͕େ޷͖ʂ ͜Μͳίʔυ͕ඞཁ͚ͩͰ͢ 34 34 2010೥10݄15೔༵ۚ೔ This is all the code you need for tab completion.
36. PDͷruleͷclass Alternation Always Chain CodeRef Dispatch Empty Enum Eq Intersection Metadata Regex Sequence Tokens Under 14ͭͷruleͷclass͕͋Γ·͢ ҉͍ruleΛ࢖͍·ͤΜ 35 35 2010೥10݄15೔༵ۚ೔ Path::Dispatcher has 14 rule types built in. I’ve greyed out the ones I don’t use much.
37. PDͷruleͷclass SD::CLI::Dispatcher::Rule::Record ΞοϓͷruleͷclassΛ࡞Γ·͠ΐ͏ʂ Alternation Always Chain CodeRef Dispatch Empty Enum Eq Intersection Metadata Regex Sequence Tokens Under 36 36 2010೥10݄15೔༵ۚ೔ Why not create our own rule classes?
38. SD::Rule::Record sub complete { my ($self, $path) = @_; map { $_->id } $self->database->lookup_prefix($path); } ͜Ε͚ͩͰɺtabͰidΛॻ͚·͢ 37 37 2010೥10݄15೔༵ۚ೔ With basically this much code we can use tab completion on SD record IDs. Pretty nifty eh?
39. SD::Rule::Record sub match { my ($self, $path) = @_; $self->database->record_exists($path); } matchͷํ͕؆୯Ͱ͢Ͷ 38 38 2010೥10݄15೔༵ۚ೔ match is even easier than complete! Domain-specific rules can get as complicated as you need of course. But they can also be pretty simple and efficient.
40. $1, $2౳ my $re = join '', map { "(\Q$_\E)" } @_; my $str = join '', @_; ($str, $re) = ("x", "x") if length($str) == 0; $str =~ qr{^$re$}; $code->(); executionΛdispatchͱ੾Γ཭ͨ͠ͷͰɺ $1,$2౳ʹ਺஋Λஔ͘͜ͱ͸ɺ໘౗͍Ͱ͢Α 39 39 2010೥10݄15೔༵ۚ೔ It turns out that, because we separate dispatch from execution, populating $1, $2, etc. is tricky.
41. $1, $2౳ my $re = join '', map { "(\Q$_\E)" } @_; my $str = join '', @_; ($str, $re) = ("x", "x") if length($str) == 0; $str =~ qr{^$re$}; $code->(); YAPC::NAͰ͜ͷͻͲ͍ίʔυͷઆ໌͍ͯ͠Δ࣌͸ɺ @chromatic͞Μ͕ʮglob assignmentग़དྷ·ͤΜ͔ʁʯ 40 40 2010೥10݄15೔༵ۚ೔ When I was explaining this awful code at YAPC::NA, chromatic asked if you could use glob assignment.
42. $1, $2౳ my $i = 0; no strict 'refs'; *{ ++$i } = \$_ for @_; $code->(); ͜ͷίʔυͷํ͕͖Ε͍ͳͷʹɺ જࡏੑͷ໰୊͕͋Γ·͢ 41 41 2010೥10݄15೔༵ۚ೔ Though this code is nicer, it has a subtle problem in it.
43. $1, $2౳ “YAPC-Europe” =~ /YAPC-(\w+)/; $dispatcher->run($1); say $1; # Europe͡Όͳ͍Ͱ͠ΐ͏ localΛ࢖͑ͳ͍͔Βɺ glob assignmentΛ࢖͏͜ͱ͕޷͖͡Ό͋Γ·ͤΜ 42 42 2010೥10݄15೔༵ۚ೔ Because the glob assignment doesn’t use local, I don’t like it very much. It could cause action at a distance.
44. $1, $2౳ local *{ ++$i } = \$_ for @_; $code->(); for (@_) { local *{ ++$i } = \$_; } $code->(); ͜ΕΛॻ͜͏ͱ͠·ͨ͠ $_ͳͷͰɺperlͰ͸ɺҧ͍·ͤΜΑʂ 43 43 2010೥10݄15೔༵ۚ೔ I tried to write this but in Perl these are actually the same! Because of $_, Perl has to rewrite the former code into the latter, which breaks uses of local like this.
45. $1, $2౳ { local $global = $x if $x; $code->(); } ͜ͷίʔυ͸࢖͑·͢Ͷ 44 44 2010೥10݄15೔༵ۚ೔ Have you used code like this before? It does work.
46. $1, $2౳ { local $global = $x for $x; $code->(); } ͜ͷίʔυ͸ବ໨Ͱ͢ʂ 45 45 2010೥10݄15೔༵ۚ೔ But this does not!
47. $1, $2౳ on ‘ticket’, ‘update’, qr{^\d+} => sub { my $match = shift; my $id = $match->pos(3); my $ticket = ticket($id); ... }; ͜ͷίʔυ͸ԿΑΓ΋ྑ͍ͱࢥ͍·͢ 46 46 2010೥10݄15೔༵ۚ೔ I’m pretty sure this is the best, and only safe way, to do this.
48. $1, $2౳ on ‘ticket’, ‘update’, named(id => qr{^\d+}) => sub { my $match = shift; my $id = $match->named(‘id'); my $ticket = ticket($id); ... }; ͔͠΋ɺnamed captureΛ࢖͑Δͭ΋ΓͰ͢ 47 47 2010೥10݄15೔༵ۚ೔ What’s more is you’ll be able to use named captures pretty easily.
49. ༗Γ೉͏ޚ࠲͍·ͨ͠ 48 48 2010೥10݄15೔༵ۚ೔ Thank you!