Pattern matching - New feature in Ruby 2.7

https://rubykaigi.org/2019/presentations/k_tsj.html

Transcript

1. PATTERN MATCHING
   NEW FEATURE IN RUBY 2.7
   Kazuki Tsujimoto
   Nomura Research Institute, LTD.
   

   View Slide

2. SELF INTRODUCTION
   • Twitter: @k_tsj
   • GitHub: k-tsj
   • CRuby committer
   • Proposer of pattern matching
   • power_assert gem author
   

   View Slide

3. TL;DR
   • Pattern matching has already been committed in trunk
   as experimental feature
   • The specification is still under discussion
   • Please try it and give us feedback
   

   View Slide

4. AGENDA
   • What is pattern matching?
   • Specification
   • Design
   

   View Slide

5. WHAT IS PATTERN
   MATCHING?
   

   View Slide

6. WHAT IS PATTERN MATCHING?
   • Definition
   • "Pattern matching consists of specifying patterns to
   which some data should conform and then checking
   to see if it does and deconstructing the data
   according to those patterns"
   • Learn You a Haskell for Great Good! (Miran Lipovaca)
   • For Rubyist
   • case/when + multiple assignment
   

   View Slide

7. WHAT IS PATTERN MATCHING?
   • We extend case expressions for pattern matching
   (case/in)
   case [0, [1, 2, 3]]
   in [a, [b, *c]]
   p a #=> 0
   p b #=> 1
   p c #=> [2, 3]
   end
   

   View Slide

8. WHAT IS PATTERN MATCHING?
   • Unlike multiple assignment, pattern matching also
   checks the object structure
   case [0, [1, 2, 3]]
   in [a]
   :unreachable
   in [0, [a, 2, b]]
   p a #=> 1
   p b #=> 3
   end
   

   View Slide

9. WHAT IS PATTERN MATCHING?
   • Pattern matching also supports Hash
   case {a: 0, b: 1}
   in {a: 0, x: 1}
   :unreachable
   in {a: 0, b: var}
   p var #=> 1
   end
   

   View Slide

10. EXAMPLE
    • It is useful

    when you handle JSON data
    {
    "name": "Alice",
    "age": 30,
    "children": [
    {
    "name": "Bob",
    "age": 2
    }
    ]
    }
    case JSON.parse(json, symbolize_names: true)
    in {name: "Alice", children: [{name: "Bob", age: age}]}
    p age # => 2
    end
    

    View Slide

11. EXAMPLE
    • With pattern matching
    person = JSON.parse(json, symbolize_names: true)
    if person[:name] == "Alice"
    children = person[:children]
    if children.length == 1 && children[0][:name] == "Bob"
    p children[0][:age] #=> 2
    end
    end
    case JSON.parse(json, symbolize_names: true)
    in {name: "Alice", children: [{name: "Bob", age: age}]}
    p age # => 2
    end
    • Without pattern matching
    

    View Slide

12. SPECIFICATION
    

    View Slide

13. SYNTAX
    • The patterns are run in sequence
    until the first one that matches
    • If no pattern matches, the else
    clause is executed
    • If no pattern matches and no
    else clause,
    NoMatchingPatternError
    exception is raised
    case expr
    in pattern [if|unless condition]
    ...
    in pattern [if|unless condition]
    ...
    else
    ...
    end
    

    View Slide

14. SYNTAX
    • The pattern may also be followed by a guard
    • The guard expression is evaluated if the preceding
    pattern matches
    case [0, 1]
    in [a, b] unless a == b
    :reachable
    end
    

    View Slide

15. PATTERN
    • Value pattern
    • Variable pattern
    • Alternative pattern
    • As pattern
    • Array pattern
    • Hash pattern
    

    View Slide

16. VALUE PATTERN
    • A value pattern matches an object such that

    pattern === object
    pat: literal
    | Constant
    case 0
    in 0
    in -1..1
    in Integer
    end
    

    View Slide

17. VARIABLE PATTERN
    • A variable pattern matches any value and binds the
    variable name to that value
    pat: var
    case 0
    in a
    p a #=> 0
    end
    

    View Slide

18. VARIABLE PATTERN
    • You can use _ to drop values
    • It is same as the idiomatic way of multiple
    assignment
    case [0, 1]
    in [_, _]
    :reachable
    end
    

    View Slide

19. VARIABLE PATTERN
    • A variable pattern always binds the variable even if
    you have outer variable with same name
    a = 0
    case 1
    in a
    p a #=> 1
    end
    

    View Slide

20. VARIABLE PATTERN
    • When you want to pattern match against an existing
    variable's value, use ^
    • It is same as the pin operator in Elixir
    a = 0
    case 1
    in ^a # means `in 0`
    :unreachable
    end #=> NoMatchingPatternError
    

    View Slide

21. ALTERNATIVE PATTERN
    • An alternative pattern matches if any of patterns
    matches
    pat: pat | pat | ...
    case 0
    in 0 | 1 | 2
    :reachable
    end
    

    View Slide

22. AS PATTERN
    • An as pattern binds the variable to the value if the pat
    matches
    pat: pat => var
    case 0
    in Integer => a
    a #=> 0
    end
    

    View Slide

23. AS PATTERN
    • It is useful for complex objects
    case [0, [1, 2]]
    in [0, [1, _] => a]
    p a #=> [1, 2]
    end
    

    View Slide

24. ARRAY PATTERN
    • Despite its name, array patterns are not solely for array objects
    • An array pattern matches if:
    • Constant === object returns true
    • The object has a #deconstruct method that returns Array
    • The result of applying the nested pattern to
    object.deconstruct is true
    pat: Constant(pat, ..., *var, pat, ...)
    | Constant[pat, ..., *var, pat, ...]
    | [pat, ..., *var, pat, ...] # Syntactic sugar for BasicObject(...)
    

    View Slide

25. ARRAY PATTERN
    • An array pattern with Array
    class Array
    def deconstruct
    self
    end
    end
    case [0, 1, 2]
    in Array(0, *a, 2)
    in Object[0, *a, 2]
    in [0, *a, 2]
    in 0, *a, 2 # You can omit brackets
    end
    p a #=> [1]
    

    View Slide

26. ARRAY PATTERN
    • An array pattern with Struct
    class Struct
    alias deconstruct to_a
    end
    Color = Struct.new(:r, :g, :b)
    p Color[0, 10, 20].deconstruct #=> [0, 10, 20]
    

    View Slide

27. ARRAY PATTERN
    • An array pattern with Struct
    case color
    in Color[0, 0, 0]
    puts "Black"
    in Color[255, 0, 0]
    puts "Red"
    in Color[r, g ,b]
    puts "#{r}, #{g}, #{b}"
    end
    

    View Slide

28. ARRAY PATTERN
    • An array pattern with
    RubyVM::AbstractSyntaxTree::Node
    class RubyVM::AbstractSyntaxTree::Node
    def deconstruct
    [type, *children, [first_lineno, first_column, last_lineno, last_column]]
    end
    end
    ast = RubyVM::AbstractSyntaxTree.parse('1 + 1')
    p ast.type #=> :SCOPE
    p ast.children #=> [[], nil, #]
    p ast.deconstruct #=> [:SCOPE, [], nil,
    #, [1, 0, 1, 5]]
    

    View Slide

29. ARRAY PATTERN
    • Implement Power Assert by using
    RubyVM::AbstractSyntaxTree
    • It needs AST of assert block
    Failure:
    assert { 3.times.to_a.include?(3) }
    | | |
    | | false
    | [0, 1, 2]
    #
    

    View Slide

30. ARRAY PATTERN
    node = RubyVM::AbstractSyntaxTree.parse('assert { 3.times.to_a.include?(3) }')
    pp node #=> (SCOPE@1:0-1:35
    # tbl: []
    # args: nil
    # body:
    # (ITER@1:0-1:35 (FCALL@1:0-1:6 :assert nil)
    # (SCOPE@1:7-1:35
    # tbl: []
    # args: nil
    # body:
    # (CALL@1:9-1:33
    # (CALL@1:9-1:21 (CALL@1:9-1:16 (LIT@1:9-1:10 3) :times nil) :to_a
    # nil) :include? (ARRAY@1:31-1:32 (LIT@1:31-1:32 3) nil)))))
    

    View Slide

31. ARRAY PATTERN
    case node
    in :SCOPE, _, _, [:ITER, [:FCALL, :assert, _, _], body, _], _
    pp body #=> (SCOPE@1:7-1:35
    # tbl: []
    # args: nil
    # body:
    # (CALL@1:9-1:33
    # (CALL@1:9-1:21 (CALL@1:9-1:16 (LIT@1:9-1:10 3) :times nil) :to_a nil)
    # :include? (ARRAY@1:31-1:32 (LIT@1:31-1:32 3) nil)))
    end
    

    View Slide

32. HASH PATTERN
    • Hash patterns are not solely for hash objects
    • A hash pattern matches if:
    • Constant === object returns true
    • The object has a #deconstruct_keys method that returns Hash
    • The result of applying the nested pattern to
    object.deconstruct_keys(keys) is true
    pat: Constant(id: pat, id:, ..., **var)
    | Constant[id: pat, id:, ..., **var]
    | {id:, id: pat, **var} # Syntactic sugar for BasicObject(...)
    

    View Slide

33. HASH PATTERN
    • A hash pattern with Hash
    class Hash
    def deconstruct_keys(keys)
    self
    end
    end
    case {a: 0, b: 1}
    in Hash(a: a, b: 1)
    in Object[a: a]
    in {a: a}
    in {a: a, **rest}
    p rest #=> {b: 1}
    end
    

    View Slide

34. HASH PATTERN
    • You can omit braces too
    • a: is syntactic sugar for a: a
    case {a: 0, b: 1}
    in a:, b:
    p a #=> 0
    p b #=> 1
    end
    

    View Slide

35. HASH PATTERN
    • keys argument provides hint
    information for efficient
    implementation
    • If we implement deconstruct_keys
    imprudently, the results might be
    very inefficient
    class Time
    def deconstruct_keys(keys)
    {
    year: year, month: month,
    asctime: asctime, ctime: ctime,
    ...,
    yday: yday, zone: zone
    }
    end
    end
    case Time.now
    in year:
    p year #=> 2019
    end
    

    View Slide

36. HASH PATTERN
    • keys refers to an array
    containing keys specified in the
    pattern
    • You can ignore any keys that
    are not included in keys
    • If **rest is specified in the
    pattern, nil is passed instead
    • In such case, you must return
    all key-value pairs
    class Time
    VALID_KEYS = %i(year month ...)
    def deconstruct_keys(keys)
    if keys
    (VALID_KEYS & keys).each_with_object({}) do |k, h|
    h[k] = send(k)
    end
    else
    {year: year, month: month, ...}
    end
    end
    end
    now = Time.now
    case now
    in year: # Calls now.deconstruct_keys([:year]),
    # receives {year: 2019}
    end
    

    View Slide

37. DESIGN
    

    View Slide

38. HISTORY
    • 2012/02/04: Published PoC of pattern-match gem
    • 2012/03/03: Published pattern-match gem
    • 2012/09/15: Sapporo RubyKaigi 2012 - "Pattern
    Matching in Ruby"
    • 2016/05/28: Tokyo RubyKaigi 11 - "Re: Pattern
    Matching in Ruby"
    

    View Slide

39. HISTORY
    • 2012/02/04: Published PoC of pattern-match gem
    • https://gist.github.com/k-tsj/1734716
    match(["foo-bar@example.com"]) do
    pattern(Array.(EMail.(/(\w+)-(\w+)/.(firstname, 'bar') :: name, domain))) do
    p [firstname, name, domain] # => ["foo", "foo-bar", "example.com"]
    end
    end
    

    View Slide

40. HISTORY
    • 2012/03/03: Published pattern-match gem
    • https://github.com/k-tsj/pattern-match
    match([Set[0, 1, 2], Set[3, 4]]) do
    with(_[Set.(a, b), Set.(c)], guard { a + b * c == 2 } ) do
    [a, b, c] #=> [2, 0, 3]
    end
    end
    

    View Slide

41. HISTORY
    • 2016/09/08: RubyKaigi 2016 - Discussed about pattern
    matching with Matz
    • 2016/09/25: Published Prototype (1)
    • 2017/09/20: RubyKaigi 2017 - "Pattern Matching in Ruby" by
    @yotii23
    • 2017/09/20: Published Prototype (2)
    • 2018/06/04: "Toward Ruby 3, there is no progress on pattern
    matching..." by @_ko1
    • 2018/07/15: Feature #14912: Introduce pattern matching syntax
    

    View Slide

42. HISTORY
    • 2016/09/25: Published Prototype (1)
    • https://gist.github.com/k-tsj/
    5e8a3167d654b04dac9231c329387928
    case {addresses: ['foo-bar@example.com', 'baz-bar@example.com']}
    => {addresses: [*head, EMail(name && /(\w+)-(\w+)/(firstname, “bar”), domain)]}
    p name #=> "baz-bar"
    p firstname #=> "baz"
    p domain #=> "example.com"
    end
    

    View Slide

43. HISTORY
    • 2017/09/20: Published Prototype (2)
    • https://github.com/k-tsj/ruby/tree/prototype-pattern-
    matching2
    class C
    def deconstruct
    [0, 1, [2, 1], x: 3, y: 4, z: 5]
    end
    end
    case C.new
    => =(0, *, =(b && Integer, _), x: 3, y:, **z)
    p b #=> 2
    p y #=> 4
    p z #=> {z: 5}
    end
    

    View Slide

44. POLICY
    • Keep compatibility
    • Be Ruby-ish
    

    View Slide

45. KEEP COMPATIBILITY
    • Compatibility is very important
    • Pattern matching introduces the following changes to Ruby
    • Syntax
    • case/in
    • Namespace
    • NoMatchingPatternError
    • deconstruct, deconstruct_keys
    • We should be careful about their effects
    

    View Slide

46. KEEP COMPATIBILITY: SYNTAX
    • Do not define new reserved words
    • If we introduce new match expressions, it breaks the
    following code
    • We have no choice but to extend the case expression
    match = foo.match?(bar)
    

    View Slide

47. KEEP COMPATIBILITY: SYNTAX
    • What should we use instead of the when keyword?
    • It should be suitable for pattern matching
    • It must not be placed at the beginning of the
    expression
    

    View Slide

48. KEEP COMPATIBILITY: SYNTAX
    • Ruby has the for expression
    • Legacy loop syntax
    • It contains an in keyword that meets the requirements
    for i in [0, 1, 2]
    p i
    end
    

    View Slide

49. KEEP COMPATIBILITY: NAMESPACE
    • We cannot maintain 100% compatibility
    • NoMatchingPatternError
    • If NoMatchingPatternError has already been used, it
    will not work
    • deconstruct, deconstruct_keys
    • No effect will result if pattern matching is not used
    • Still, it is preferable to choose a name that is not
    currently used
    

    View Slide

50. KEEP COMPATIBILITY: NAMESPACE
    • Investigate code of all gems by using gem-codesearch
    • NoMatchingPatternError: 0 gems
    • deconstruct: 8 gems
    • deconstruct_keys: 0 gems
    • We need better names
    

    View Slide

51. BE RUBY-ISH
    • Pattern matching is a widely used feature in statically
    typed functional programming languages
    • Ruby is a dynamically typed object oriented language
    • It is necessary to combine these essences efficiently
    • Powerful Array, Hash support
    • Encourage duck typing
    

    View Slide

52. BE RUBY-ISH:
    POWERFUL ARRAY, HASH SUPPORT
    • Array and Hash are very important data structures for
    Ruby
    • e.g. multiple assignment, keyword arguments
    • Provide array patterns, hash patterns
    • Brackets and braces are optional
    • id: : syntactic sugar for id: id
    

    View Slide

53. BE RUBY-ISH:
    POWERFUL ARRAY, HASH SUPPORT
    • By default, an array pattern is an exact match and a
    Hash pattern is a subset match
    • Because these designations fit the major use cases
    Exact match Subset match
    Array pattern
    case [0, 1]
    in [a]
    :unreachable
    end
    case [0, 1]
    in [a, *]
    :reachable
    end
    Hash pattern
    case {a: 0, b: 1}
    in {a: 0, **rest} if rest.empty?
    :unreachable
    end
    case {a: 0, b: 1}
    in {a: 0}
    :reachable
    end
    

    View Slide

54. BE RUBY-ISH:
    POWERFUL ARRAY, HASH SUPPORT
    • What should {} match?
    def match_with_empty_hash_pattern(obj)
    case obj
    in {}
    :matched
    end
    end
    

    View Slide

55. BE RUBY-ISH:
    POWERFUL ARRAY, HASH SUPPORT
    • If it is a subset match, it matches any hash object
    match_with_empty_hash_pattern({a: 0, b: 1}) #=> :matched
    match_with_empty_hash_pattern({a: 0}) #=> :matched
    match_with_empty_hash_pattern({}) #=> :matched
    match_with_empty_hash_pattern({a: 0, b: 1}) #=> NoMatchingPatternError
    match_with_empty_hash_pattern({a: 0}) #=> NoMatchingPatternError
    match_with_empty_hash_pattern({}) #=> :matched
    • However, a user might expect {} to match only an
    empty hash object
    • Matz selects the latter
    

    View Slide

56. BE RUBY-ISH:
    ENCOURAGE DUCK TYPING
    • Should not allow users to write code to check that the
    object is an instance of a specific class
    • Provide succinct syntax for duck typing
    case time
    in year:, month:
    ...
    in Time(year:, month:)
    ...
    end
    

    View Slide

57. BE RUBY-ISH:
    ENCOURAGE DUCK TYPING
    • Should not allow users to write code to check that the
    object is an instance of a specific class
    • Do not provide a way to match against arguments
    # Syntax NG
    def m(a, b)
    in Integer, String
    ...
    end
    

    View Slide

58. FUTURE WORKS
    • Scope
    • Non-symbol keys for hash patterns
    • Extending the pin operator
    • Misc
    

    View Slide

59. FUTURE WORKS: SCOPE
    • Current scope is the same as case/when
    expressions
    • Do not create a new scope
    • You can see the value of a variable even if
    it failed to match
    • It is obviously strange, so this behavior is
    going to be changed
    • What should we do?
    • Return nil
    • Raise NameError
    case [0, 1]
    in x, y unless y == 1
    :unreachable
    in x, z
    :reachable
    end
    p y #=> 1
    

    View Slide

60. FUTURE WORKS:
    NON-SYMBOL KEYS FOR HASH PATTERNS
    • Hash patterns only take symbols as keys
    • Should it allow non-symbol keys?
    • It will have a strong impact on syntax
    

    View Slide

61. FUTURE WORKS:
    NON-SYMBOL KEYS FOR HASH PATTERNS
    • How do we write such a hash pattern? => ?
    • How do we write an as pattern?
    • Should we introduce & (and pattern) instead?
    • It seems an and pattern is too powerful to be used instead of
    an as pattern
    in Integer & i
    in {'key' => value}
    

    View Slide

62. FUTURE WORKS:
    EXTENDING THE PIN OPERATOR
    • Complex expressions (e.g. method calling) are not
    allowed as pattern syntax
    # Synatax NG
    case time
    in Time.now..
    end
    

    View Slide

63. FUTURE WORKS:
    EXTENDING THE PIN OPERATOR
    • Use the pin operator or guard in such cases
    # Synatax OK
    now = Time.now
    case time
    in ^now..
    in t if now <= t
    end
    

    View Slide

64. FUTURE WORKS:
    EXTENDING THE PIN OPERATOR
    • If it is used frequently, it might be better to extend the
    pin operator in order to take an expression (^(expr))
    case time
    in ^(Time.now)..
    end
    

    View Slide

65. FUTURE WORKS: MISC
    • Define deconstruct, deconstruct_keys for builtin/
    standard library
    • Few classes have deconstruct/deconstruct_keys now
    (Array, Hash, Struct)
    • Destructuring assignment
    • Performance
    

    View Slide

66. HOW TO TRY AND GIVE FEEDBACK
    • Setup build environment
    • rbenv/ruby-build Wiki
    • Suggested build environment
    • https://github.com/rbenv/ruby-build/wiki
    • Compile and install
    $ git clone https://github.com/ruby/ruby.git
    $ cd ruby
    $ autoconf
    $ ./configure —prefix=/usr/local/ruby-trunk
    $ make && sudo make install
    

    View Slide

67. HOW TO TRY AND GIVE FEEDBACK
    • Give feedback
    • Feature #14912: Introduce pattern matching syntax
    • https://bugs.ruby-lang.org/issues/14912
    • Foyer, Party, Twitter(#rubykaigi), etc
    

    View Slide

68. SPECIAL THANKS
    • Koichi Sasada (@_ko1)
    • Yusuke Endoh (@mametter)
    

    View Slide