Pattern matching - New feature in Ruby 2.7

Transcript

1. PATTERN MATCHING NEW FEATURE IN RUBY 2.7 Kazuki Tsujimoto Nomura

   Research Institute, LTD.

2. SELF INTRODUCTION • Twitter: @k_tsj • GitHub: k-tsj • CRuby

   committer • Proposer of pattern matching • power_assert gem author

3. TL;DR • Pattern matching has already been committed in trunk

   as experimental feature • The specification is still under discussion • Please try it and give us feedback

4. AGENDA • What is pattern matching? • Specification • Design

5. WHAT IS PATTERN MATCHING?

6. WHAT IS PATTERN MATCHING? • Definition • "Pattern matching consists

   of specifying patterns to which some data should conform and then checking to see if it does and deconstructing the data according to those patterns" • Learn You a Haskell for Great Good! (Miran Lipovaca) • For Rubyist • case/when + multiple assignment

7. WHAT IS PATTERN MATCHING? • We extend case expressions for

   pattern matching (case/in) case [0, [1, 2, 3]] in [a, [b, *c]] p a #=> 0 p b #=> 1 p c #=> [2, 3] end

8. WHAT IS PATTERN MATCHING? • Unlike multiple assignment, pattern matching

   also checks the object structure case [0, [1, 2, 3]] in [a] :unreachable in [0, [a, 2, b]] p a #=> 1 p b #=> 3 end

9. WHAT IS PATTERN MATCHING? • Pattern matching also supports Hash

   case {a: 0, b: 1} in {a: 0, x: 1} :unreachable in {a: 0, b: var} p var #=> 1 end

10. EXAMPLE • It is useful
 when you handle JSON data

    { "name": "Alice", "age": 30, "children": [ { "name": "Bob", "age": 2 } ] } case JSON.parse(json, symbolize_names: true) in {name: "Alice", children: [{name: "Bob", age: age}]} p age # => 2 end

11. EXAMPLE • With pattern matching person = JSON.parse(json, symbolize_names: true)

    if person[:name] == "Alice" children = person[:children] if children.length == 1 && children[0][:name] == "Bob" p children[0][:age] #=> 2 end end case JSON.parse(json, symbolize_names: true) in {name: "Alice", children: [{name: "Bob", age: age}]} p age # => 2 end • Without pattern matching

12. SPECIFICATION

13. SYNTAX • The patterns are run in sequence until the

    first one that matches • If no pattern matches, the else clause is executed • If no pattern matches and no else clause, NoMatchingPatternError exception is raised case expr in pattern [if|unless condition] ... in pattern [if|unless condition] ... else ... end

14. SYNTAX • The pattern may also be followed by a

    guard • The guard expression is evaluated if the preceding pattern matches case [0, 1] in [a, b] unless a == b :reachable end

15. PATTERN • Value pattern • Variable pattern • Alternative pattern

    • As pattern • Array pattern • Hash pattern

16. VALUE PATTERN • A value pattern matches an object such

    that
 pattern === object pat: literal | Constant case 0 in 0 in -1..1 in Integer end

17. VARIABLE PATTERN • A variable pattern matches any value and

    binds the variable name to that value pat: var case 0 in a p a #=> 0 end

18. VARIABLE PATTERN • You can use _ to drop values

    • It is same as the idiomatic way of multiple assignment case [0, 1] in [_, _] :reachable end

19. VARIABLE PATTERN • A variable pattern always binds the variable

    even if you have outer variable with same name a = 0 case 1 in a p a #=> 1 end

20. VARIABLE PATTERN • When you want to pattern match against

    an existing variable's value, use ^ • It is same as the pin operator in Elixir a = 0 case 1 in ^a # means `in 0` :unreachable end #=> NoMatchingPatternError

21. ALTERNATIVE PATTERN • An alternative pattern matches if any of

    patterns matches pat: pat | pat | ... case 0 in 0 | 1 | 2 :reachable end

22. AS PATTERN • An as pattern binds the variable to

    the value if the pat matches pat: pat => var case 0 in Integer => a a #=> 0 end

23. AS PATTERN • It is useful for complex objects case

    [0, [1, 2]] in [0, [1, _] => a] p a #=> [1, 2] end

24. ARRAY PATTERN • Despite its name, array patterns are not

    solely for array objects • An array pattern matches if: • Constant === object returns true • The object has a #deconstruct method that returns Array • The result of applying the nested pattern to object.deconstruct is true pat: Constant(pat, ..., *var, pat, ...) | Constant[pat, ..., *var, pat, ...] | [pat, ..., *var, pat, ...] # Syntactic sugar for BasicObject(...)

25. ARRAY PATTERN • An array pattern with Array class Array

    def deconstruct self end end case [0, 1, 2] in Array(0, *a, 2) in Object[0, *a, 2] in [0, *a, 2] in 0, *a, 2 # You can omit brackets end p a #=> [1]

26. ARRAY PATTERN • An array pattern with Struct class Struct

    alias deconstruct to_a end Color = Struct.new(:r, :g, :b) p Color[0, 10, 20].deconstruct #=> [0, 10, 20]

27. ARRAY PATTERN • An array pattern with Struct case color

    in Color[0, 0, 0] puts "Black" in Color[255, 0, 0] puts "Red" in Color[r, g ,b] puts "#{r}, #{g}, #{b}" end

28. ARRAY PATTERN • An array pattern with RubyVM::AbstractSyntaxTree::Node class RubyVM::AbstractSyntaxTree::Node

    def deconstruct [type, *children, [first_lineno, first_column, last_lineno, last_column]] end end ast = RubyVM::AbstractSyntaxTree.parse('1 + 1') p ast.type #=> :SCOPE p ast.children #=> [[], nil, #<RubyVM::AbstractSyntaxTree::Node:OPCALL@1:0-1:5>] p ast.deconstruct #=> [:SCOPE, [], nil, #<RubyVM::AbstractSyntaxTree::Node:OPCALL@1:0-1:5>, [1, 0, 1, 5]]

29. ARRAY PATTERN • Implement Power Assert by using RubyVM::AbstractSyntaxTree •

    It needs AST of assert block Failure: assert { 3.times.to_a.include?(3) } | | | | | false | [0, 1, 2] #<Enumerator: 3:times>

30. ARRAY PATTERN node = RubyVM::AbstractSyntaxTree.parse('assert { 3.times.to_a.include?(3) }') pp node

    #=> (SCOPE@1:0-1:35 # tbl: [] # args: nil # body: # (ITER@1:0-1:35 (FCALL@1:0-1:6 :assert nil) # (SCOPE@1:7-1:35 # tbl: [] # args: nil # body: # (CALL@1:9-1:33 # (CALL@1:9-1:21 (CALL@1:9-1:16 (LIT@1:9-1:10 3) :times nil) :to_a # nil) :include? (ARRAY@1:31-1:32 (LIT@1:31-1:32 3) nil)))))

31. ARRAY PATTERN case node in :SCOPE, _, _, [:ITER, [:FCALL,

    :assert, _, _], body, _], _ pp body #=> (SCOPE@1:7-1:35 # tbl: [] # args: nil # body: # (CALL@1:9-1:33 # (CALL@1:9-1:21 (CALL@1:9-1:16 (LIT@1:9-1:10 3) :times nil) :to_a nil) # :include? (ARRAY@1:31-1:32 (LIT@1:31-1:32 3) nil))) end

32. HASH PATTERN • Hash patterns are not solely for hash

    objects • A hash pattern matches if: • Constant === object returns true • The object has a #deconstruct_keys method that returns Hash • The result of applying the nested pattern to object.deconstruct_keys(keys) is true pat: Constant(id: pat, id:, ..., **var) | Constant[id: pat, id:, ..., **var] | {id:, id: pat, **var} # Syntactic sugar for BasicObject(...)

33. HASH PATTERN • A hash pattern with Hash class Hash

    def deconstruct_keys(keys) self end end case {a: 0, b: 1} in Hash(a: a, b: 1) in Object[a: a] in {a: a} in {a: a, **rest} p rest #=> {b: 1} end

34. HASH PATTERN • You can omit braces too • a:

    is syntactic sugar for a: a case {a: 0, b: 1} in a:, b: p a #=> 0 p b #=> 1 end

35. HASH PATTERN • keys argument provides hint information for efficient

    implementation • If we implement deconstruct_keys imprudently, the results might be very inefficient class Time def deconstruct_keys(keys) { year: year, month: month, asctime: asctime, ctime: ctime, ..., yday: yday, zone: zone } end end case Time.now in year: p year #=> 2019 end

36. HASH PATTERN • keys refers to an array containing keys

    specified in the pattern • You can ignore any keys that are not included in keys • If **rest is specified in the pattern, nil is passed instead • In such case, you must return all key-value pairs class Time VALID_KEYS = %i(year month ...) def deconstruct_keys(keys) if keys (VALID_KEYS & keys).each_with_object({}) do |k, h| h[k] = send(k) end else {year: year, month: month, ...} end end end now = Time.now case now in year: # Calls now.deconstruct_keys([:year]), # receives {year: 2019} end

37. DESIGN

38. HISTORY • 2012/02/04: Published PoC of pattern-match gem • 2012/03/03:

    Published pattern-match gem • 2012/09/15: Sapporo RubyKaigi 2012 - "Pattern Matching in Ruby" • 2016/05/28: Tokyo RubyKaigi 11 - "Re: Pattern Matching in Ruby"

39. HISTORY • 2012/02/04: Published PoC of pattern-match gem • https://gist.github.com/k-tsj/1734716

    match(["foo-bar@example.com"]) do pattern(Array.(EMail.(/(\w+)-(\w+)/.(firstname, 'bar') :: name, domain))) do p [firstname, name, domain] # => ["foo", "foo-bar", "example.com"] end end

40. HISTORY • 2012/03/03: Published pattern-match gem • https://github.com/k-tsj/pattern-match match([Set[0, 1,

    2], Set[3, 4]]) do with(_[Set.(a, b), Set.(c)], guard { a + b * c == 2 } ) do [a, b, c] #=> [2, 0, 3] end end

41. HISTORY • 2016/09/08: RubyKaigi 2016 - Discussed about pattern matching

    with Matz • 2016/09/25: Published Prototype (1) • 2017/09/20: RubyKaigi 2017 - "Pattern Matching in Ruby" by @yotii23 • 2017/09/20: Published Prototype (2) • 2018/06/04: "Toward Ruby 3, there is no progress on pattern matching..." by @_ko1 • 2018/07/15: Feature #14912: Introduce pattern matching syntax

42. HISTORY • 2016/09/25: Published Prototype (1) • https://gist.github.com/k-tsj/ 5e8a3167d654b04dac9231c329387928 case

    {addresses: ['foo-bar@example.com', 'baz-bar@example.com']} => {addresses: [*head, EMail(name && /(\w+)-(\w+)/(firstname, “bar”), domain)]} p name #=> "baz-bar" p firstname #=> "baz" p domain #=> "example.com" end

43. HISTORY • 2017/09/20: Published Prototype (2) • https://github.com/k-tsj/ruby/tree/prototype-pattern- matching2 class

    C def deconstruct [0, 1, [2, 1], x: 3, y: 4, z: 5] end end case C.new => =(0, *, =(b && Integer, _), x: 3, y:, **z) p b #=> 2 p y #=> 4 p z #=> {z: 5} end

44. POLICY • Keep compatibility • Be Ruby-ish

45. KEEP COMPATIBILITY • Compatibility is very important • Pattern matching

    introduces the following changes to Ruby • Syntax • case/in • Namespace • NoMatchingPatternError • deconstruct, deconstruct_keys • We should be careful about their effects

46. KEEP COMPATIBILITY: SYNTAX • Do not define new reserved words

    • If we introduce new match expressions, it breaks the following code • We have no choice but to extend the case expression match = foo.match?(bar)

47. KEEP COMPATIBILITY: SYNTAX • What should we use instead of

    the when keyword? • It should be suitable for pattern matching • It must not be placed at the beginning of the expression

48. KEEP COMPATIBILITY: SYNTAX • Ruby has the for expression •

    Legacy loop syntax • It contains an in keyword that meets the requirements for i in [0, 1, 2] p i end

49. KEEP COMPATIBILITY: NAMESPACE • We cannot maintain 100% compatibility •

    NoMatchingPatternError • If NoMatchingPatternError has already been used, it will not work • deconstruct, deconstruct_keys • No effect will result if pattern matching is not used • Still, it is preferable to choose a name that is not currently used

50. KEEP COMPATIBILITY: NAMESPACE • Investigate code of all gems by

    using gem-codesearch • NoMatchingPatternError: 0 gems • deconstruct: 8 gems • deconstruct_keys: 0 gems • We need better names

51. BE RUBY-ISH • Pattern matching is a widely used feature

    in statically typed functional programming languages • Ruby is a dynamically typed object oriented language • It is necessary to combine these essences efficiently • Powerful Array, Hash support • Encourage duck typing

52. BE RUBY-ISH: POWERFUL ARRAY, HASH SUPPORT • Array and Hash

    are very important data structures for Ruby • e.g. multiple assignment, keyword arguments • Provide array patterns, hash patterns • Brackets and braces are optional • id: : syntactic sugar for id: id

53. BE RUBY-ISH: POWERFUL ARRAY, HASH SUPPORT • By default, an

    array pattern is an exact match and a Hash pattern is a subset match • Because these designations fit the major use cases Exact match Subset match Array pattern case [0, 1] in [a] :unreachable end case [0, 1] in [a, *] :reachable end Hash pattern case {a: 0, b: 1} in {a: 0, **rest} if rest.empty? :unreachable end case {a: 0, b: 1} in {a: 0} :reachable end

54. BE RUBY-ISH: POWERFUL ARRAY, HASH SUPPORT • What should {}

    match? def match_with_empty_hash_pattern(obj) case obj in {} :matched end end

55. BE RUBY-ISH: POWERFUL ARRAY, HASH SUPPORT • If it is

    a subset match, it matches any hash object match_with_empty_hash_pattern({a: 0, b: 1}) #=> :matched match_with_empty_hash_pattern({a: 0}) #=> :matched match_with_empty_hash_pattern({}) #=> :matched match_with_empty_hash_pattern({a: 0, b: 1}) #=> NoMatchingPatternError match_with_empty_hash_pattern({a: 0}) #=> NoMatchingPatternError match_with_empty_hash_pattern({}) #=> :matched • However, a user might expect {} to match only an empty hash object • Matz selects the latter

56. BE RUBY-ISH: ENCOURAGE DUCK TYPING • Should not allow users

    to write code to check that the object is an instance of a specific class • Provide succinct syntax for duck typing case time in year:, month: ... in Time(year:, month:) ... end

57. BE RUBY-ISH: ENCOURAGE DUCK TYPING • Should not allow users

    to write code to check that the object is an instance of a specific class • Do not provide a way to match against arguments # Syntax NG def m(a, b) in Integer, String ... end

58. FUTURE WORKS • Scope • Non-symbol keys for hash patterns

    • Extending the pin operator • Misc

59. FUTURE WORKS: SCOPE • Current scope is the same as

    case/when expressions • Do not create a new scope • You can see the value of a variable even if it failed to match • It is obviously strange, so this behavior is going to be changed • What should we do? • Return nil • Raise NameError case [0, 1] in x, y unless y == 1 :unreachable in x, z :reachable end p y #=> 1

60. FUTURE WORKS: NON-SYMBOL KEYS FOR HASH PATTERNS • Hash patterns

    only take symbols as keys • Should it allow non-symbol keys? • It will have a strong impact on syntax

61. FUTURE WORKS: NON-SYMBOL KEYS FOR HASH PATTERNS • How do

    we write such a hash pattern? => ? • How do we write an as pattern? • Should we introduce & (and pattern) instead? • It seems an and pattern is too powerful to be used instead of an as pattern in Integer & i in {'key' => value}

62. FUTURE WORKS: EXTENDING THE PIN OPERATOR • Complex expressions (e.g.

    method calling) are not allowed as pattern syntax # Synatax NG case time in Time.now.. end

63. FUTURE WORKS: EXTENDING THE PIN OPERATOR • Use the pin

    operator or guard in such cases # Synatax OK now = Time.now case time in ^now.. in t if now <= t end

64. FUTURE WORKS: EXTENDING THE PIN OPERATOR • If it is

    used frequently, it might be better to extend the pin operator in order to take an expression (^(expr)) case time in ^(Time.now).. end

65. FUTURE WORKS: MISC • Define deconstruct, deconstruct_keys for builtin/ standard

    library • Few classes have deconstruct/deconstruct_keys now (Array, Hash, Struct) • Destructuring assignment • Performance

66. HOW TO TRY AND GIVE FEEDBACK • Setup build environment

    • rbenv/ruby-build Wiki • Suggested build environment • https://github.com/rbenv/ruby-build/wiki • Compile and install $ git clone https://github.com/ruby/ruby.git $ cd ruby $ autoconf $ ./configure —prefix=/usr/local/ruby-trunk $ make && sudo make install

67. HOW TO TRY AND GIVE FEEDBACK • Give feedback •

    Feature #14912: Introduce pattern matching syntax • https://bugs.ruby-lang.org/issues/14912 • Foyer, Party, Twitter(#rubykaigi), etc

68. SPECIAL THANKS • Koichi Sasada (@_ko1) • Yusuke Endoh (@mametter)