Pattern matching - New feature in Ruby 2.7

https://rubykaigi.org/2019/presentations/k_tsj.html

Transcript

1. PATTERN MATCHING NEW FEATURE IN RUBY 2.7 Kazuki Tsujimoto Nomura Research Institute, LTD.
2. SELF INTRODUCTION • Twitter: @k_tsj • GitHub: k-tsj • CRuby committer • Proposer of pattern matching • power_assert gem author
3. TL;DR • Pattern matching has already been committed in trunk as experimental feature • The specification is still under discussion • Please try it and give us feedback
4. AGENDA • What is pattern matching? • Specification • Design
5. WHAT IS PATTERN MATCHING?
6. WHAT IS PATTERN MATCHING? • Definition • "Pattern matching consists of specifying patterns to which some data should conform and then checking to see if it does and deconstructing the data according to those patterns" • Learn You a Haskell for Great Good! (Miran Lipovaca) • For Rubyist • case/when + multiple assignment
7. WHAT IS PATTERN MATCHING? • We extend case expressions for pattern matching (case/in) case [0, [1, 2, 3]] in [a, [b, *c]] p a #=> 0 p b #=> 1 p c #=> [2, 3] end
8. WHAT IS PATTERN MATCHING? • Unlike multiple assignment, pattern matching also checks the object structure case [0, [1, 2, 3]] in [a] :unreachable in [0, [a, 2, b]] p a #=> 1 p b #=> 3 end
9. WHAT IS PATTERN MATCHING? • Pattern matching also supports Hash case {a: 0, b: 1} in {a: 0, x: 1} :unreachable in {a: 0, b: var} p var #=> 1 end
10. EXAMPLE • It is useful
 when you handle JSON data { "name": "Alice", "age": 30, "children": [ { "name": "Bob", "age": 2 } ] } case JSON.parse(json, symbolize_names: true) in {name: "Alice", children: [{name: "Bob", age: age}]} p age # => 2 end
11. EXAMPLE • With pattern matching person = JSON.parse(json, symbolize_names: true) if person[:name] == "Alice" children = person[:children] if children.length == 1 && children[0][:name] == "Bob" p children[0][:age] #=> 2 end end case JSON.parse(json, symbolize_names: true) in {name: "Alice", children: [{name: "Bob", age: age}]} p age # => 2 end • Without pattern matching
12. SPECIFICATION
13. SYNTAX • The patterns are run in sequence until the first one that matches • If no pattern matches, the else clause is executed • If no pattern matches and no else clause, NoMatchingPatternError exception is raised case expr in pattern [if|unless condition] ... in pattern [if|unless condition] ... else ... end
14. SYNTAX • The pattern may also be followed by a guard • The guard expression is evaluated if the preceding pattern matches case [0, 1] in [a, b] unless a == b :reachable end
15. PATTERN • Value pattern • Variable pattern • Alternative pattern • As pattern • Array pattern • Hash pattern
16. VALUE PATTERN • A value pattern matches an object such that
 pattern === object pat: literal | Constant case 0 in 0 in -1..1 in Integer end
17. VARIABLE PATTERN • A variable pattern matches any value and binds the variable name to that value pat: var case 0 in a p a #=> 0 end
18. VARIABLE PATTERN • You can use _ to drop values • It is same as the idiomatic way of multiple assignment case [0, 1] in [_, _] :reachable end
19. VARIABLE PATTERN • A variable pattern always binds the variable even if you have outer variable with same name a = 0 case 1 in a p a #=> 1 end
20. VARIABLE PATTERN • When you want to pattern match against an existing variable's value, use ^ • It is same as the pin operator in Elixir a = 0 case 1 in ^a # means `in 0` :unreachable end #=> NoMatchingPatternError
21. ALTERNATIVE PATTERN • An alternative pattern matches if any of patterns matches pat: pat | pat | ... case 0 in 0 | 1 | 2 :reachable end
22. AS PATTERN • An as pattern binds the variable to the value if the pat matches pat: pat => var case 0 in Integer => a a #=> 0 end
23. AS PATTERN • It is useful for complex objects case [0, [1, 2]] in [0, [1, _] => a] p a #=> [1, 2] end
24. ARRAY PATTERN • Despite its name, array patterns are not solely for array objects • An array pattern matches if: • Constant === object returns true • The object has a #deconstruct method that returns Array • The result of applying the nested pattern to object.deconstruct is true pat: Constant(pat, ..., *var, pat, ...) | Constant[pat, ..., *var, pat, ...] | [pat, ..., *var, pat, ...] # Syntactic sugar for BasicObject(...)
25. ARRAY PATTERN • An array pattern with Array class Array def deconstruct self end end case [0, 1, 2] in Array(0, *a, 2) in Object[0, *a, 2] in [0, *a, 2] in 0, *a, 2 # You can omit brackets end p a #=> [1]
26. ARRAY PATTERN • An array pattern with Struct class Struct alias deconstruct to_a end Color = Struct.new(:r, :g, :b) p Color[0, 10, 20].deconstruct #=> [0, 10, 20]
27. ARRAY PATTERN • An array pattern with Struct case color in Color[0, 0, 0] puts "Black" in Color[255, 0, 0] puts "Red" in Color[r, g ,b] puts "#{r}, #{g}, #{b}" end
28. ARRAY PATTERN • An array pattern with RubyVM::AbstractSyntaxTree::Node class RubyVM::AbstractSyntaxTree::Node def deconstruct [type, *children, [first_lineno, first_column, last_lineno, last_column]] end end ast = RubyVM::AbstractSyntaxTree.parse('1 + 1') p ast.type #=> :SCOPE p ast.children #=> [[], nil, #<RubyVM::AbstractSyntaxTree::Node:OPCALL@1:0-1:5>] p ast.deconstruct #=> [:SCOPE, [], nil, #<RubyVM::AbstractSyntaxTree::Node:OPCALL@1:0-1:5>, [1, 0, 1, 5]]
29. ARRAY PATTERN • Implement Power Assert by using RubyVM::AbstractSyntaxTree • It needs AST of assert block Failure: assert { 3.times.to_a.include?(3) } | | | | | false | [0, 1, 2] #<Enumerator: 3:times>
30. ARRAY PATTERN node = RubyVM::AbstractSyntaxTree.parse('assert { 3.times.to_a.include?(3) }') pp node #=> (SCOPE@1:0-1:35 # tbl: [] # args: nil # body: # (ITER@1:0-1:35 (FCALL@1:0-1:6 :assert nil) # (SCOPE@1:7-1:35 # tbl: [] # args: nil # body: # (CALL@1:9-1:33 # (CALL@1:9-1:21 (CALL@1:9-1:16 (LIT@1:9-1:10 3) :times nil) :to_a # nil) :include? (ARRAY@1:31-1:32 (LIT@1:31-1:32 3) nil)))))
31. ARRAY PATTERN case node in :SCOPE, _, _, [:ITER, [:FCALL, :assert, _, _], body, _], _ pp body #=> (SCOPE@1:7-1:35 # tbl: [] # args: nil # body: # (CALL@1:9-1:33 # (CALL@1:9-1:21 (CALL@1:9-1:16 (LIT@1:9-1:10 3) :times nil) :to_a nil) # :include? (ARRAY@1:31-1:32 (LIT@1:31-1:32 3) nil))) end
32. HASH PATTERN • Hash patterns are not solely for hash objects • A hash pattern matches if: • Constant === object returns true • The object has a #deconstruct_keys method that returns Hash • The result of applying the nested pattern to object.deconstruct_keys(keys) is true pat: Constant(id: pat, id:, ..., **var) | Constant[id: pat, id:, ..., **var] | {id:, id: pat, **var} # Syntactic sugar for BasicObject(...)
33. HASH PATTERN • A hash pattern with Hash class Hash def deconstruct_keys(keys) self end end case {a: 0, b: 1} in Hash(a: a, b: 1) in Object[a: a] in {a: a} in {a: a, **rest} p rest #=> {b: 1} end
34. HASH PATTERN • You can omit braces too • a: is syntactic sugar for a: a case {a: 0, b: 1} in a:, b: p a #=> 0 p b #=> 1 end
35. HASH PATTERN • keys argument provides hint information for efficient implementation • If we implement deconstruct_keys imprudently, the results might be very inefficient class Time def deconstruct_keys(keys) { year: year, month: month, asctime: asctime, ctime: ctime, ..., yday: yday, zone: zone } end end case Time.now in year: p year #=> 2019 end
36. HASH PATTERN • keys refers to an array containing keys specified in the pattern • You can ignore any keys that are not included in keys • If **rest is specified in the pattern, nil is passed instead • In such case, you must return all key-value pairs class Time VALID_KEYS = %i(year month ...) def deconstruct_keys(keys) if keys (VALID_KEYS & keys).each_with_object({}) do |k, h| h[k] = send(k) end else {year: year, month: month, ...} end end end now = Time.now case now in year: # Calls now.deconstruct_keys([:year]), # receives {year: 2019} end
37. DESIGN
38. HISTORY • 2012/02/04: Published PoC of pattern-match gem • 2012/03/03: Published pattern-match gem • 2012/09/15: Sapporo RubyKaigi 2012 - "Pattern Matching in Ruby" • 2016/05/28: Tokyo RubyKaigi 11 - "Re: Pattern Matching in Ruby"
39. HISTORY • 2012/02/04: Published PoC of pattern-match gem • https://gist.github.com/k-tsj/1734716 match(["foo-bar@example.com"]) do pattern(Array.(EMail.(/(\w+)-(\w+)/.(firstname, 'bar') :: name, domain))) do p [firstname, name, domain] # => ["foo", "foo-bar", "example.com"] end end
40. HISTORY • 2012/03/03: Published pattern-match gem • https://github.com/k-tsj/pattern-match match([Set[0, 1, 2], Set[3, 4]]) do with(_[Set.(a, b), Set.(c)], guard { a + b * c == 2 } ) do [a, b, c] #=> [2, 0, 3] end end
41. HISTORY • 2016/09/08: RubyKaigi 2016 - Discussed about pattern matching with Matz • 2016/09/25: Published Prototype (1) • 2017/09/20: RubyKaigi 2017 - "Pattern Matching in Ruby" by @yotii23 • 2017/09/20: Published Prototype (2) • 2018/06/04: "Toward Ruby 3, there is no progress on pattern matching..." by @_ko1 • 2018/07/15: Feature #14912: Introduce pattern matching syntax
42. HISTORY • 2016/09/25: Published Prototype (1) • https://gist.github.com/k-tsj/ 5e8a3167d654b04dac9231c329387928 case {addresses: ['foo-bar@example.com', 'baz-bar@example.com']} => {addresses: [*head, EMail(name && /(\w+)-(\w+)/(firstname, “bar”), domain)]} p name #=> "baz-bar" p firstname #=> "baz" p domain #=> "example.com" end
43. HISTORY • 2017/09/20: Published Prototype (2) • https://github.com/k-tsj/ruby/tree/prototype-pattern- matching2 class C def deconstruct [0, 1, [2, 1], x: 3, y: 4, z: 5] end end case C.new => =(0, *, =(b && Integer, _), x: 3, y:, **z) p b #=> 2 p y #=> 4 p z #=> {z: 5} end
44. POLICY • Keep compatibility • Be Ruby-ish
45. KEEP COMPATIBILITY • Compatibility is very important • Pattern matching introduces the following changes to Ruby • Syntax • case/in • Namespace • NoMatchingPatternError • deconstruct, deconstruct_keys • We should be careful about their effects
46. KEEP COMPATIBILITY: SYNTAX • Do not define new reserved words • If we introduce new match expressions, it breaks the following code • We have no choice but to extend the case expression match = foo.match?(bar)
47. KEEP COMPATIBILITY: SYNTAX • What should we use instead of the when keyword? • It should be suitable for pattern matching • It must not be placed at the beginning of the expression
48. KEEP COMPATIBILITY: SYNTAX • Ruby has the for expression • Legacy loop syntax • It contains an in keyword that meets the requirements for i in [0, 1, 2] p i end
49. KEEP COMPATIBILITY: NAMESPACE • We cannot maintain 100% compatibility • NoMatchingPatternError • If NoMatchingPatternError has already been used, it will not work • deconstruct, deconstruct_keys • No effect will result if pattern matching is not used • Still, it is preferable to choose a name that is not currently used
50. KEEP COMPATIBILITY: NAMESPACE • Investigate code of all gems by using gem-codesearch • NoMatchingPatternError: 0 gems • deconstruct: 8 gems • deconstruct_keys: 0 gems • We need better names
51. BE RUBY-ISH • Pattern matching is a widely used feature in statically typed functional programming languages • Ruby is a dynamically typed object oriented language • It is necessary to combine these essences efficiently • Powerful Array, Hash support • Encourage duck typing
52. BE RUBY-ISH: POWERFUL ARRAY, HASH SUPPORT • Array and Hash are very important data structures for Ruby • e.g. multiple assignment, keyword arguments • Provide array patterns, hash patterns • Brackets and braces are optional • id: : syntactic sugar for id: id
53. BE RUBY-ISH: POWERFUL ARRAY, HASH SUPPORT • By default, an array pattern is an exact match and a Hash pattern is a subset match • Because these designations fit the major use cases Exact match Subset match Array pattern case [0, 1] in [a] :unreachable end case [0, 1] in [a, *] :reachable end Hash pattern case {a: 0, b: 1} in {a: 0, **rest} if rest.empty? :unreachable end case {a: 0, b: 1} in {a: 0} :reachable end
54. BE RUBY-ISH: POWERFUL ARRAY, HASH SUPPORT • What should {} match? def match_with_empty_hash_pattern(obj) case obj in {} :matched end end
55. BE RUBY-ISH: POWERFUL ARRAY, HASH SUPPORT • If it is a subset match, it matches any hash object match_with_empty_hash_pattern({a: 0, b: 1}) #=> :matched match_with_empty_hash_pattern({a: 0}) #=> :matched match_with_empty_hash_pattern({}) #=> :matched match_with_empty_hash_pattern({a: 0, b: 1}) #=> NoMatchingPatternError match_with_empty_hash_pattern({a: 0}) #=> NoMatchingPatternError match_with_empty_hash_pattern({}) #=> :matched • However, a user might expect {} to match only an empty hash object • Matz selects the latter
56. BE RUBY-ISH: ENCOURAGE DUCK TYPING • Should not allow users to write code to check that the object is an instance of a specific class • Provide succinct syntax for duck typing case time in year:, month: ... in Time(year:, month:) ... end
57. BE RUBY-ISH: ENCOURAGE DUCK TYPING • Should not allow users to write code to check that the object is an instance of a specific class • Do not provide a way to match against arguments # Syntax NG def m(a, b) in Integer, String ... end
58. FUTURE WORKS • Scope • Non-symbol keys for hash patterns • Extending the pin operator • Misc
59. FUTURE WORKS: SCOPE • Current scope is the same as case/when expressions • Do not create a new scope • You can see the value of a variable even if it failed to match • It is obviously strange, so this behavior is going to be changed • What should we do? • Return nil • Raise NameError case [0, 1] in x, y unless y == 1 :unreachable in x, z :reachable end p y #=> 1
60. FUTURE WORKS: NON-SYMBOL KEYS FOR HASH PATTERNS • Hash patterns only take symbols as keys • Should it allow non-symbol keys? • It will have a strong impact on syntax
61. FUTURE WORKS: NON-SYMBOL KEYS FOR HASH PATTERNS • How do we write such a hash pattern? => ? • How do we write an as pattern? • Should we introduce & (and pattern) instead? • It seems an and pattern is too powerful to be used instead of an as pattern in Integer & i in {'key' => value}
62. FUTURE WORKS: EXTENDING THE PIN OPERATOR • Complex expressions (e.g. method calling) are not allowed as pattern syntax # Synatax NG case time in Time.now.. end
63. FUTURE WORKS: EXTENDING THE PIN OPERATOR • Use the pin operator or guard in such cases # Synatax OK now = Time.now case time in ^now.. in t if now <= t end
64. FUTURE WORKS: EXTENDING THE PIN OPERATOR • If it is used frequently, it might be better to extend the pin operator in order to take an expression (^(expr)) case time in ^(Time.now).. end
65. FUTURE WORKS: MISC • Define deconstruct, deconstruct_keys for builtin/ standard library • Few classes have deconstruct/deconstruct_keys now (Array, Hash, Struct) • Destructuring assignment • Performance
66. HOW TO TRY AND GIVE FEEDBACK • Setup build environment • rbenv/ruby-build Wiki • Suggested build environment • https://github.com/rbenv/ruby-build/wiki • Compile and install $ git clone https://github.com/ruby/ruby.git $ cd ruby $ autoconf $ ./configure —prefix=/usr/local/ruby-trunk $ make && sudo make install
67. HOW TO TRY AND GIVE FEEDBACK • Give feedback • Feature #14912: Introduce pattern matching syntax • https://bugs.ruby-lang.org/issues/14912 • Foyer, Party, Twitter(#rubykaigi), etc
68. SPECIAL THANKS • Koichi Sasada (@_ko1) • Yusuke Endoh (@mametter)