Pragmatic Monadic Programming in Ruby

Transcript

1. Pragmatic Monadic Programming in Ruby @joker1007 (Repro inc.) RubyKaigi 2019

2. self.inspect id: @joker1007 name: Tomohiro Hashidate Repro inc. CTO I

   familiar with ... Ruby/Rails Ruby Black Magic (TracePoint) Speaker of RubyKaigi2018, RubyConf2018

3. We provide a web service as ... Analytics of Mobile

   Apps, Web Apps. Marketing Automation. Of course, We're hiring!!

4. I am a member of an amazing Ruby community.

5. In ruby-2.6, Proc is very interesting!!

6. Composing proc Proc#>> Proc#<<

7. RubyVM::AST.of can receive a Proc pr = proc { puts

   :foo } ast = RubyVM::AbstractSyntaxTree.of(pr) pp ast (SCOPE@1:10-1:23 tbl: [] args: nil body: (FCALL@1:12-1:21 :puts (ARRAY@1:17-1:21 (LIT@1:17-1:21 :foo) nil)))

8. TracePoint#enable can receive a Proc pr = proc do puts

   :foo puts :bar end trace = TracePoint.new(:line) do |tp| p tp.lineno end trace.enable(target: pr) pr.call 2 foo 3 bar

9. To begin with, What is "Proc"?

10. "Proc" is object of Procedure In other words, Function object.

    pr = proc { puts :hello_proc } p pr #<Proc:0x00000000025fad50@/home/joker/slides/rubykaigi2019/proc.rb:1>

11. "Proc" is closure. def counter x = 0 proc {

    x += 1 } end c = counter counter.call # => 1 counter.call # => 2 Proc keeps the environment of the scope where it is created.

12. Method can receive a "Proc" as block def with_retry(exception, &block)

    block.call rescue exception retry end

13. In fact, Ruby ... has function object as First-class object

    can receive functions as return value can pass functions as method arguments In other words, Ruby has a factor of functional programming.

14. Some FP languages have a very interesting feature.

15. Monad In functional programming, a monad is a design pattern[1]

    that allows structuring programs generically while automating away boilerplate code needed by the program logic. Monads achieve this by providing their own data type, which represents a specific form of computation, along with one procedure to wrap values of any basic type within the monad (yielding a monadic value) and another to compose functions that output monadic values (called monadic functions). from Wikipedia https://en.wikipedia.org/wiki/Monad_(functional_programming).

16. It seems difficult!!

17. But, Monad is very simple and useful pattern actually. And

    syntax sugar is very important for monad.

18. I thinked. "I can implement true Monad syntax sugar by

    Ruby black magics."

19. Let's get down to the main topic.

20. Agenda Functor in Ruby Applicative Functor in Ruby Monad in

    Ruby Syntax sugar for monad Implement monadic syntax in Ruby Examples, DEMO Today, I will not explain mathematics. I will talk about only the programming technique. ここまでで5 〜6 分ぐらいだと良いなあ

21. Functor Functor ... is a container has a context for

    specific purpose. is a object that can be mapped by any function. The most popular functor in Ruby is "Array". [1,2,3].map { |i| i.to_s } In Haskell, map for Functor is called fmap .

22. Why is Functor useful Without functor and map, a container

    needs to implement all methods for any object that it may contain. Functor makes a container enable to collaborate with any methods.

23. Functor requires some laws functor.fmap(&:itself) == functor functor.fmap(&(proc_a >> proc_b))

    == functor.fmap(&proc_a).fmap(&proc_b) These laws ensure that the behaviors of a functor are proper.

24. Applicative Functor (1) If you want to process with more

    than two functor objects, But fmap cannot handle more than two functors. [1,2,3].map do |i| [5,6,7].map do |j| i + j end end This sample outputs nested array. Of course, we can use flatten and flat_map . But we have a more functional approach.

25. Applicative Functor (2) Applicative Functor can contain functions. And contained

    functions apply other objects contained by each functor. array_plus = [:+.to_proc] array_plus.ap([2, 3], [4]) # => [6, 7] array_plus.ap([2, 3], [5, 6, 7]) # => [7, 8, 9, 8, 9, 10] array_plus.ap([2, 3], []) # => []

26. Applicative Functor (3) def ap(*targets) curried = ->(*extracted) do fmap

    { |pr| pr.call(*extracted) } end.curry(targets.size) applied = targets.inject(pure(curried)) do |pured, t| pured.flat_map do |pr| t.fmap { |v| pr.call(v) } end end applied.flat_map(&:itself) end

27. Applicative also requires some laws But laws of Applicative Functor

    are more complicated than ones of Functor. Sorry, I omit explaining details.

28. What is difference between Applicative and Monad Applicative Functor cannot

    express multiple dependent effects. For example, a calculation that may fail depends on whether previous calculations succeeded or failed. In such cases, Monad is useful.

29. Monad Monad is a container like Functor and Applicative. In

    Haskell, Monad requires some implementations. class Monad m where (>>=) :: m a -> (a -> m b) -> m b (>>) :: m a -> m b -> m b return :: a -> m a fail :: String -> m a Especially, (>>=) is most important. it is called "bind operator".

30. What is >>= (bind operator) ? In the case of

    Array, Array#bind receives a function that receives an item contained by the array and outputs new array. Example. ["foo","bar"].bind { |s| s.split(//) } # => ["f","o","o","b","a","r"] In fact, it's flat_map

31. Monad in Scala Scala has a syntax sugar for monad.

    for { x <- Some(10) y <- functionMaybeFailed() } yield x + y // Return Some(10 + y) or None Scala transforms this code to flat_map style internally. Like below. Some(10).flatMap { x => functionMaybeFailed().flatMap { y => x + y } }

32. Monad is flat_map !

33. Syntax sugar for monad Some functional languages have syntax sugar

    for monad. Haskell has do-syntax, Scala has for-syntax. Because the main purpose of monad is a chain of contextual computation, and syntax sugar is very effective to use it more easily.

34. Nested flatMap is not readable Some(10).flatMap { x => functionA(x).flatMap

    { y => fuctionB.flatMap { z => z.process } } } The frequent appearance of flatMap is very noisy. { , } is the same. indent++.

35. Scala is hybrid paradigm language Ruby is similar to Scala

    in a sense I copied code transformation from Scala

36. Monad syntax in Ruby https://github.com/joker1007/monar calc = ->(*val) do val.monadic_eval

    do |x| a = x.odd? ? x : x / 2 y <<= [a + 14] z <<= [y, y + 1, y + 2] end end expect(calc.call(7, 8)).to \ eq([21, 22, 23, 18, 19, 20]) This code is valid syntax!! There is no warning.

37. My idea is very simple. a <<= <statement> transform to

    flat_map do |a| <statement> It's all. <<= is important discovery!!

38. <<= is assignment with operator a <<= foo equals a

    = a << foo This is valid ruby code. And Ruby treats a as assigned local variable! Moreover, most ruby programmers have not written such codes. In other words, I can change the behavior freely!!

39. Review the code transformation [1,2,3].monadic_eval do |i| j <<= [i,

    i*2] j.select(&:odd?) end transforms to [1,2,3].flat_map do |i| [i, i*2].flat_map do |j| j.select(&:odd?) end end

40. It is difficult to resolve nested do-end by method chain.

    OK, AST Transformation!! We already have RubyVM::AST.of . It is a new feature of Ruby-2.6 and very useful for handling AST.

41. Breakdown of my implementation Extract AST from given block by

    RubyVM::AST.of Detect a pattern like a <<= foo Extract fragments of source code Reconstruct source code Wrap into new proc (to cache reconstructed code) instance_eval new source code ここまでで20 分弱だと良いなあ

42. Detect the pattern def __is_bind_statement?(node) (node.type == :DASGN || node.type

    == :DASGN_CURR) && node.children[1].type == :CALL && node.children[1].children[1] == :<< end

43. Extract fragments of codes # @param source [Array<String>] lines of

    source code def extract_source(source, first_lineno, first_column, last_lineno, last_column) lines = source[(first_lineno - 1)..(last_lineno-1)] first_line = lines.shift last_line = lines.pop if last_line.nil? first_line[first_column..last_column] else first_line[first_column..-1] + lines.join + last_line[0..last_column] end end

44. Reconstruct codes def __transform_node(source, buf, node, last_stmt: false) if __is_bind_statement?(node)

    lvar = node.children[0] rhv = node.children[1].children[2] origin = Monad.extract_source( source, rhv.first_lineno, rhv.first_column, rhv.last_lineno, rhv.last_column).chomp buf[0].concat( "(#{origin}).flat_map do |#{lvar}|\n" ) buf[1] += 1 else buf[0].concat("(#{Monad.extract_source( source, node.first_lineno, node.first_column, node.last_lineno, node.last_column ).chomp})\n") end end

45. Wrap into a new proc gen = "proc { |#{caller_local_variables.map(&:to_s).join(",")}|

    begin; " + buf[0] + "rescue => ex; rescue_in_monad(ex); end; }\n" pr = instance_eval(gen, caller_location.path, caller_location.lineno) Monad.proc_cache["#{block_location[0]}:#{block_location[1]}"] = pr instance_eval outputs a proc object which contains transformed process. And I cached it to avoid source code transformation repeatedly.

46. Handling local variables Reconstructing codes lose local variables, because the

    environment contained by block is lost. Binding is required to handle local variables.

47. That is when TracePoint is effective! I found a technique

    to get Binding of a given block

48. proc_binding = nil trace = TracePoint.new(:line) do |tp| # this

    block is called Just before evaluating proc's first line proc_binding = tp.binding throw :escape end catch(:escape) do # In Ruby-2.6, TracePoint can limit tracking target. trace.enable(target: block) block.call ensure trace.disable end

49. I got a Binding of Proc!! And Binding has everything

    for black magic.

50. ast = RubyVM::AbstractSyntaxTree.of(block); # SCOPE args_tbl = ast.children[0] args_node =

    ast.children[1] caller_local_variables = proc_binding.local_variables - args_tbl gen = "proc { |#{caller_local_variables.map(&:to_s).join(",")}| ...." I got local_variables and copy into generated proc.

51. At last, instance_exec generates proc with local variables of the

    caller. instance_exec( *(caller_local_variables.map { |lvar| proc_binding.local_variable_get(lvar) }), &generated_pr )

52. Show powers of monad syntax by some examples. Maybe Either

    Future State ParserCombinator ここまでで30 分ぐらい

53. Maybe (Just) class Just include Monad include Monar::Maybe def initialize(value)

    @value = value; end def flat_map(&pr) pr.call(@value); end def monad_class Monar::Maybe; end end

54. Maybe (Nothing) class Nothing include Monad include Monar::Maybe def initialize(*value);

    end def flat_map(&pr) self; end def monad_class Monar::Maybe; end end

55. Maybe example key1 = "users/#{user.id}/posts/#{post.id}" key2 = "posts/#{post.id}/comments" # lookup_cache

    returns Maybe lookup_cache(key1).monadic_eval do |cached_post| cached_comments <<= lookup_cache(key2) # If lookup_cache(key2) fails, # below processes are not executed post = Oj.load(cached_post) comments = Oj.load(cached_comments) post.comments = comments pure post end # return Just(post) or Nothing

56. Either (Right) Ritht is the same as Just .

57. Either (Left) Left is the same as Nothing . But

    Left has a exception.

58. Either (helper function) def either(&pr) Monar::Either.right(pr.call) rescue => ex Monar::Either.left.new(ex)

    end

59. Either example either { Balance.find_by!(user: user_a) }.monadic_eval do |balance_a| _

    <<= balance_a.ensure_amount!(amount) # return Either balance_b <<= either { Balance.find_by!(user: user_b) } TransferMoneyService.new(from: user_a, to: user_b, amount: amount) .process # return Either end # return Right(result) or Left(exception) Either hides error handling behind monad syntax. And it has a high affinity with pattern matching. Pattern matching is very hot topic.

60. Future (extend concurrent-ruby) Concurrent::Promises::Future.include(Monad) class Concurrent::Promises::Future def flat_map(&pr) yield value!

    rescue => ex if rejected? self else Concurrent::Promises.rejected_future(ex) end end end

61. Future example Concurrent::Promises.make_future(5).monadic_eval do |x| a = x.odd? ? x

    : x / 2 b = Concurrent::Promises.future { sleep 2; 11 } c = Concurrent::Promises.future { sleep 1; 3 } d <<= b # wait b e <<= c # wait c pure(a + d + e) end # Return Future(19) like async syntax of JavaScript.

62. State (constructor) State has a proc that receives current state

    and outputs value and next state. class State include Monad # @param next_state [Proc (s -> [a, s])] def initialize(next_state) raise ArgumentError.new("need to respond to :call") unless next_state.respond_to?(:call) @next_state = next_state end end

63. State (flat_map) def flat_map(&pr0) self.class.new( proc do |s0| x, s1

    = run_state(s0) new_state_monad = pr0.call(x) new_state_monad.next_state.call(s1) end ) end run_state(s0) -> [a, s2] -> block.call(a) -> new state monad -> run_state(s1) These processes are wrapped by the new State class.

64. State (helper function) def get State.new(proc { |s| [s, s]

    }) end def put(st) State.new(proc { |_| [nil, st] }) end

65. State example state = State.pure(5).monadic_eval do |n| status <<= get

    # Return initial state _ <<= if status != :saved result = n * 10 put(:saved) # Update state else pure(nil) end pure(result) end val, st = state.run_state(:not_saved) # val == 50, st == :saved val, st = state.run_state(:saved) # val == nil, st == :saved

66. Mental model of State State handles 2 pipelines.

67. ParserCombinator Parser also contains a proc. that proc receives String

    and returns [[Object, String]] . Object is a result of parsing. String is remained characters. flat_map expresses parser combination. | expresses selective parser.

68. Parser#flat_map def flat_map(&pr) self.class.new( proc do |str0| result0 = run_parser(str0)

    # return Array result0.flat_map do |output, remained| next_parser = pr.call(output) next_parser.run_parser(remained) end end ) end Below processes is wrapped by Proc . Get result -> given proc processes result -> Get New Parser -> parse remained chars.

69. Parser#| def |(other) self.class.new( proc do |str| result0 = run_parser(str)

    result0.empty? other.run_parser(str) : result0 end ) end

70. Parser examples 1 def anychar new( proc do |string| string

    == "" ? [] : [[string[0], string[1..-1]]] end ) end # @param cond [Proc (Char -> TrueClass | FalseClass)] def satisfy(cond) anychar.flat_map do |char| if cond.call(char) new(proc { |string| [[char, string]] }) else new(proc { |string| [] }) end end end

71. Parser examples 2 def string(str) return pure "" if str

    == "" c, tail = str[0], str[1..-1] char(c).monadic_eval do |c1| cs <<= self.class.string(tail) pure [c1, *cs].join end end

72. Parser examples 3 def one_of(chars) satisfy(->(char) { chars.include?(char) }) end

    def many(parser, level = 1) combined = parser | pure(nil) combined.monadic_eval do |result| result2 <<= if result.nil? pure(nil) else self.class.many(parser, level + 1) end pure([result, result2].compact.flatten(level)) end end

73. DEMO (Arithmetic Parser)

74. Conclusion Monad is not difficult, It's flat_map The abstraction of

    monad is very powerful Syntax is very important I want other representations for nested block By this implementation, I recognized the fun of monadic programming again.

75. At last, I never recommend using this gem on production

    now!! If you are interested in TracePoint and AST, Examples of the gem's code are helpful for you, maybe. And, let's enjoy darkness programming!!