Building Serverless Ruby Bots @ Ruby Conf 2018

Want to build a Bot without the hassle of provisioning and managing servers? Amazon’s got a service for that and it’s called AWS Lambda - it executes your code only when needed, scales automatically and you pay only for the compute time you consume.

There’s one problem with Lambda - it doesn’t support Ruby! 
Let’s checkout multiple options on how to build a Ruby Bot and package it into an executable which you can run on any machine without the need to actually have Ruby installed.

Transcript

1. BUILDING
   Serverless RUBY
   BOTS
   

   View Slide

2. View Slide

3. NJUSKALO.HR
   

   View Slide

4. View Slide

5. View Slide

6. # Gemfile
   gem 'httparty'
   gem 'oga'
   gem 'actionmailer'
   gem 'rake'
   gem 'whenever'
   

   View Slide

7. View Slide

8. Serverless
   

   View Slide

9. AWS LAMBDA
   

   View Slide

10. LANGUAGE SUPPORT
    JAVA, NODE.JS, PYTHON ...
    

    View Slide

11. 29 0CT 2015
    

    View Slide

12. 16 JAN 2018
    

    View Slide

13. NO RUBY? :(
    

    View Slide

14. TRAVELING Ruby
    MRuby
    JRuby
    

    View Slide

15. TRAVELING RUBY
    

    View Slide

16. MRUBY
    

    View Slide

17. JRUBY
    

    View Slide

18. View Slide

19. SHRINK THE CODE
    BASE
    

    View Slide

20. GEMFILE
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'
    

    View Slide

21. USE NET/HTTP
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'
    

    View Slide

22. USE NET/HTTP
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'
    

    View Slide

23. USE LAMBDA SCHEDULER
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'
    

    View Slide

24. View Slide

25. View Slide

26. 
    ...
    13.02.2017.
    
    
    ...
    13.02.2017.
    
    
    ...
    14.02.2017.
    
    

    View Slide

27. REGEX IT IS.
    

    View Slide

28. RUBY BOT CODE
    

    View Slide

29. # main.rb
    require 'net/http'
    require 'time'
    class NewApartments
    attr_reader :url
    def initialize(url)
    @url = url
    @last_scrapped_at = Time.now - 5 * 60 # 5 minutes ago.
    end
    def any?
    response_body
    .scan(/datetime\="(.*)" /)
    .any? { |published_at| Time.parse(published_at) > last_scrapped_at }
    end
    private
    def response_body
    Net::HTTP.get(URI(url))
    end
    end
    

    View Slide

30. class MailgunEmailer
    def self.call
    Net::HTTP.post_form(
    URI("https://api:key-#{MAILGUN_API_KEY}@api.mailgun.net/v3/#{MAILGUN_DOMAIN}/messages"),
    from: "Apartment Bot ",
    to: RECIPIENT,
    subject: 'New Apartments found!',
    html: "Check new apartments"
    )
    end
    end
    

    View Slide

31. SEARCH_URL = 'http://iapi.njuskalo.hr?sort=new&categoryId=10920&locationId=2619'
    MailgunEmailer.call if NewApartments.new(SEARCH_URL).any?
    

    View Slide

32. IMPLEMENTING THE BOT
    WITH Traveling Ruby
    PACKAGED MRI RUNTIMES
    

    View Slide

33. DOWNLOAD THE RUNTIMES
    OS X FOR OUR DEV MACHINE
    LINUX X86_64 COMPATIBLE BINARY FOR AWS LAMBDA
    

    View Slide

34. curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz
    curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-osx.tar.gz
    

    View Slide

35. traveling-ruby-bot
    - ruby-linux-x86_64/
    - ruby-os-x/
    

    View Slide

36. traveling-ruby-bot
    - ruby-linux-x86_64
    - bin/
    - ruby
    - ...
    - ...
    - ruby-os-x
    - bin/
    - ruby
    - ...
    - ...
    

    View Slide

37. traveling-ruby-bot
    - ruby-linux-x86_64/
    - ruby-os-x/
    - main.rb
    

    View Slide

38. USUAL RUBY EXECUTION
    ruby main.rb
    

    View Slide

39. INVOKE WITH TRAVELING
    RUBY
    ruby-os-x/bin/ruby main.rb
    

    View Slide

40. UPLOADING THE BOT TO AWS
    Lambda
    

    View Slide

41. // lambda-function-wrapper.js
    var spawn = require('child_process').spawn;
    exports.handler = function(event, context) {
    var child = spawn('ruby-linux-x86_64/bin/ruby main.rb');
    child.stdout.on('data', function (data) { console.log("stdout:\n"+data); });
    child.stderr.on('data', function (data) { console.log("stderr:\n"+data); });
    child.on('close', function (code) { context.done(); });
    }
    

    View Slide

42. zip ruby-bot.zip ruby-linux-x86_64 main.rb lambda-function-wrapper.js
    

    View Slide

43. 1. CREATE A FUNCTION
    

    View Slide

44. 2. CREATE A TRIGGER
    

    View Slide

45. 3. CREATE A RULE FOR THE TRIGGER
    

    View Slide

46. 4. UPLOAD THE FUNCTION CODE
    

    View Slide

47. 5. SET A HANDLER
    

    View Slide

48. IMPLEMENTING THE BOT
    WITH mruby
    

    View Slide

49. TWO WAYS TO BUILD APPS
    WITH MRUBY:
    - DOWNLOAD MRUBY SOURCE DIRECTLY FROM GITHUB
    - MRUBY-CLI - PLATFORM FOR BUILDING NATIVE COMMAND LINE APPLICATIONS FOR LINUX, WINDOWS, AND OS
    X.
    

    View Slide

50. BUILDING DIRECTLY FROM MRUBY SOURCE
    

    View Slide

51. git clone https://github.com/mruby/mruby
    

    View Slide

52. mruby-bot
    - bin
    - build_config.rb
    - minirake
    

    View Slide

53. ./MINIRAKE
    

    View Slide

54. mruby-bot
    - bin/
    - mruby
    - mirb
    - build_config.rb
    - minirake
    

    View Slide

55. > echo "print 'hello world'" >> hello_world.rb
    

    View Slide

56. INVOKE WITH MRUBY
    bin/mruby hello_world.rb
    

    View Slide

57. > bin/mruby main.rb
    main.rb:1: undefined method 'require' for main (NoMethodError)
    

    View Slide

58. # build_config.rb
    conf.gem github: 'iij/mruby-env'
    conf.gem github: 'mattn/mruby-curl'
    conf.gem github: 'mattn/mruby-onig-regexp'
    

    View Slide

59. ./MINIRAKE
    

    View Slide

60. EXECUTE THE
    CODE
    bin/mruby main.rb
    

    View Slide

61. CROSS COMPILATION
    

    View Slide

62. IMPLEMENTING THE BOT
    WITH JRUBY
    

    View Slide

63. AWS LAMBDA JRUBY
    

    View Slide

64. jruby-bot
    - lib/
    - jruby.jar
    - src/
    - main/
    - resources/
    - main.rb
    - gradlew
    

    View Slide

65. PASTE CODE
    mv main.rb src/main/resources/main.rb
    

    View Slide

66. BUILD THE PROJECT
    ./gradlew build
    

    View Slide

67. UPLOAD THE BOT CODE
    

    View Slide

68. METRICS & NUMBERS
    

    View Slide

69. CODE SIZE
    Traveling Ruby 6.7 MB
    mruby 945 kB
    JRuby 24 MB
    

    View Slide

70. MEMORY CONSUMPTION
    Traveling Ruby 25 MB
    mruby 40 MB
    JRuby 150 MB
    

    View Slide

71. EXECUTION TIME (COLD START)
    Traveling Ruby ~3900 ms
    mruby ~3900 ms
    JRuby ~25000 ms
    

    View Slide

72. COLDSTARTS
    

    View Slide

73. EXECUTION TIME (WARMED UP)
    Traveling Ruby ~3300 ms
    mruby ~3300 ms
    JRuby ~1200 ms
    

    View Slide

74. YMMV
    

    View Slide

75. AWS LAMBDA PRICING
    

    View Slide

76. AWS LAMBDA PRICING
    THE NUMBER OF REQUESTS (I.E. NUMBER OF TIMES OUR FUNCTION IS INVOKED)
    THE SUM OF DURATIONS OUR FUNCTIONS TOOK TO EXECUTE (EXPRESSED IN GB-SECONDS)
    

    View Slide

77. FREE TIER
    1M REQUESTS & 400,000 GB-SECONDS PER MONTH
    

    View Slide

78. 1M TIMES PER MONTH
    ROUGLY EVERY 3 SECONDS
    

    View Slide

79. 400,000 GB-SECONDS /
    MONTH
    MEMORY SIZE * DURATION IN SECONDS
    

    View Slide

80. e.g. 128MB * 4s = 0.5GB-secons
    

    View Slide

81. IF WE RAN THE BOT EVERY
    MINUTE
    20.000 GB-SECONDS
    

    View Slide

82. CHOOSING THE RIGHT Ruby
    FOR THE JOB
    

    View Slide

83. Traveling Ruby
    LOW MEMORY CONSUMPTION / WELL
    KNOWN ENVIRONMENT
    SUPPORT?
    

    View Slide

84. MRuby
    LOW MEMORY CONSUMPTION /
    LIGHTWEIGHT
    LIBRARY SUPPORT / CROSS COMPILATION
    

    View Slide

85. JRuby
    SIMPLE PACKAGING* / FAST EXECUTION
    HIGH MEMORY CONSUMPTION / JAVA
    ERRORS
    

    View Slide

86. Alternatives
    

    View Slide

87. RUBY Packer
    RUBY Snap
    

    View Slide

88. Azure FUNCTIONS
    Google Cloud FUNCTIONS
    

    View Slide

89. View Slide

90. Lots of OPTIONS
    

    View Slide

91. Hacky OPTIONS
    

    View Slide

92. ..A MONTH AGO..
    

    View Slide

93. APACHE
    OpenWhisk
    

    View Slide

94. IBM Cloud Functions
    

    View Slide

95. IBM Cloud Functions
    

    View Slide

96. Inline editor
    

    View Slide

97. Serverless Framework
    SUPPORT
    

    View Slide

98. View Slide

99. serverless create --template openwhisk-ruby
    --path ruby_bot
    

    View Slide

100. - ruby_bot
     - handler.rb
     - serverless.yml
     

     View Slide

101. # serverless.yml
     service: ruby_bot
     provider:
     name: openwhisk
     runtime: ruby
     functions:
     ruby_bot:
     handler: handler.main
     plugins:
     - serverless-openwhisk
     

     View Slide

102. serverless deploy
     

     View Slide

103. JETS
     SERVERLESS FRAMEWORK FOR RUBY
     

     View Slide

104. Jets.application.routes.draw do
     get 'posts', to: 'posts#index'
     end
     class PostsController < ApplicationController
     def index
     render json: { hello: 'world', action: 'index' }
     end
     end
     

     View Slide

105. FaastRuby.io
     

     View Slide

106. Conclusion
     

     View Slide

107. SERVERLESS-RUBY.ORG
     

     View Slide

108. View Slide

109. DAMIR SVRTAN
     

     View Slide

110. View Slide

111. View Slide

112. tinyurl.com/netflix-ruby
     

     View Slide

113. DAMIR SVRTAN
     SENIOR SOFTWARE ENGINEER @ Netflix
     HIT ME UP @DamirSvrtan
     TINYURL.COM/NETFLIX-RUBY
     

     View Slide