Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Serverless Ruby Bots @ Ruby Conf 2018

DamirSvrtan
November 15, 2018

Building Serverless Ruby Bots @ Ruby Conf 2018

Want to build a Bot without the hassle of provisioning and managing servers? Amazon’s got a service for that and it’s called AWS Lambda - it executes your code only when needed, scales automatically and you pay only for the compute time you consume.

There’s one problem with Lambda - it doesn’t support Ruby! 
Let’s checkout multiple options on how to build a Ruby Bot and package it into an executable which you can run on any machine without the need to actually have Ruby installed.

DamirSvrtan

November 15, 2018
Tweet

More Decks by DamirSvrtan

Other Decks in Technology

Transcript

  1. BUILDING
    Serverless RUBY
    BOTS

    View Slide

  2. View Slide

  3. NJUSKALO.HR

    View Slide

  4. View Slide

  5. View Slide

  6. # Gemfile
    gem 'httparty'
    gem 'oga'
    gem 'actionmailer'
    gem 'rake'
    gem 'whenever'

    View Slide

  7. View Slide

  8. Serverless

    View Slide

  9. AWS LAMBDA

    View Slide

  10. LANGUAGE SUPPORT
    JAVA, NODE.JS, PYTHON ...

    View Slide

  11. 29 0CT 2015

    View Slide

  12. 16 JAN 2018

    View Slide

  13. NO RUBY? :(

    View Slide

  14. TRAVELING Ruby
    MRuby
    JRuby

    View Slide

  15. TRAVELING RUBY

    View Slide

  16. MRUBY

    View Slide

  17. JRUBY

    View Slide

  18. View Slide

  19. SHRINK THE CODE
    BASE

    View Slide

  20. GEMFILE
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'

    View Slide

  21. USE NET/HTTP
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'

    View Slide

  22. USE NET/HTTP
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'

    View Slide

  23. USE LAMBDA SCHEDULER
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'

    View Slide

  24. View Slide

  25. View Slide


  26. ...
    13.02.2017.


    ...
    13.02.2017.


    ...
    14.02.2017.

    View Slide

  27. REGEX IT IS.

    View Slide

  28. RUBY BOT CODE

    View Slide

  29. # main.rb
    require 'net/http'
    require 'time'
    class NewApartments
    attr_reader :url
    def initialize(url)
    @url = url
    @last_scrapped_at = Time.now - 5 * 60 # 5 minutes ago.
    end
    def any?
    response_body
    .scan(/datetime\="(.*)" /)
    .any? { |published_at| Time.parse(published_at) > last_scrapped_at }
    end
    private
    def response_body
    Net::HTTP.get(URI(url))
    end
    end

    View Slide

  30. class MailgunEmailer
    def self.call
    Net::HTTP.post_form(
    URI("https://api:key-#{MAILGUN_API_KEY}@api.mailgun.net/v3/#{MAILGUN_DOMAIN}/messages"),
    from: "Apartment Bot ",
    to: RECIPIENT,
    subject: 'New Apartments found!',
    html: "Check new apartments"
    )
    end
    end

    View Slide

  31. SEARCH_URL = 'http://iapi.njuskalo.hr?sort=new&categoryId=10920&locationId=2619'
    MailgunEmailer.call if NewApartments.new(SEARCH_URL).any?

    View Slide

  32. IMPLEMENTING THE BOT
    WITH Traveling Ruby
    PACKAGED MRI RUNTIMES

    View Slide

  33. DOWNLOAD THE RUNTIMES
    OS X FOR OUR DEV MACHINE
    LINUX X86_64 COMPATIBLE BINARY FOR AWS LAMBDA

    View Slide

  34. curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz
    curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-osx.tar.gz

    View Slide

  35. traveling-ruby-bot
    - ruby-linux-x86_64/
    - ruby-os-x/

    View Slide

  36. traveling-ruby-bot
    - ruby-linux-x86_64
    - bin/
    - ruby
    - ...
    - ...
    - ruby-os-x
    - bin/
    - ruby
    - ...
    - ...

    View Slide

  37. traveling-ruby-bot
    - ruby-linux-x86_64/
    - ruby-os-x/
    - main.rb

    View Slide

  38. USUAL RUBY EXECUTION
    ruby main.rb

    View Slide

  39. INVOKE WITH TRAVELING
    RUBY
    ruby-os-x/bin/ruby main.rb

    View Slide

  40. UPLOADING THE BOT TO AWS
    Lambda

    View Slide

  41. // lambda-function-wrapper.js
    var spawn = require('child_process').spawn;
    exports.handler = function(event, context) {
    var child = spawn('ruby-linux-x86_64/bin/ruby main.rb');
    child.stdout.on('data', function (data) { console.log("stdout:\n"+data); });
    child.stderr.on('data', function (data) { console.log("stderr:\n"+data); });
    child.on('close', function (code) { context.done(); });
    }

    View Slide

  42. zip ruby-bot.zip ruby-linux-x86_64 main.rb lambda-function-wrapper.js

    View Slide

  43. 1. CREATE A FUNCTION

    View Slide

  44. 2. CREATE A TRIGGER

    View Slide

  45. 3. CREATE A RULE FOR THE TRIGGER

    View Slide

  46. 4. UPLOAD THE FUNCTION CODE

    View Slide

  47. 5. SET A HANDLER

    View Slide

  48. IMPLEMENTING THE BOT
    WITH mruby

    View Slide

  49. TWO WAYS TO BUILD APPS
    WITH MRUBY:
    - DOWNLOAD MRUBY SOURCE DIRECTLY FROM GITHUB
    - MRUBY-CLI - PLATFORM FOR BUILDING NATIVE COMMAND LINE APPLICATIONS FOR LINUX, WINDOWS, AND OS
    X.

    View Slide

  50. BUILDING DIRECTLY FROM MRUBY SOURCE

    View Slide

  51. git clone https://github.com/mruby/mruby

    View Slide

  52. mruby-bot
    - bin
    - build_config.rb
    - minirake

    View Slide

  53. ./MINIRAKE

    View Slide

  54. mruby-bot
    - bin/
    - mruby
    - mirb
    - build_config.rb
    - minirake

    View Slide

  55. > echo "print 'hello world'" >> hello_world.rb

    View Slide

  56. INVOKE WITH MRUBY
    bin/mruby hello_world.rb

    View Slide

  57. > bin/mruby main.rb
    main.rb:1: undefined method 'require' for main (NoMethodError)

    View Slide

  58. # build_config.rb
    conf.gem github: 'iij/mruby-env'
    conf.gem github: 'mattn/mruby-curl'
    conf.gem github: 'mattn/mruby-onig-regexp'

    View Slide

  59. ./MINIRAKE

    View Slide

  60. EXECUTE THE
    CODE
    bin/mruby main.rb

    View Slide

  61. CROSS COMPILATION

    View Slide

  62. IMPLEMENTING THE BOT
    WITH JRUBY

    View Slide

  63. AWS LAMBDA JRUBY

    View Slide

  64. jruby-bot
    - lib/
    - jruby.jar
    - src/
    - main/
    - resources/
    - main.rb
    - gradlew

    View Slide

  65. PASTE CODE
    mv main.rb src/main/resources/main.rb

    View Slide

  66. BUILD THE PROJECT
    ./gradlew build

    View Slide

  67. UPLOAD THE BOT CODE

    View Slide

  68. METRICS & NUMBERS

    View Slide

  69. CODE SIZE
    Traveling Ruby 6.7 MB
    mruby 945 kB
    JRuby 24 MB

    View Slide

  70. MEMORY CONSUMPTION
    Traveling Ruby 25 MB
    mruby 40 MB
    JRuby 150 MB

    View Slide

  71. EXECUTION TIME (COLD START)
    Traveling Ruby ~3900 ms
    mruby ~3900 ms
    JRuby ~25000 ms

    View Slide

  72. COLDSTARTS

    View Slide

  73. EXECUTION TIME (WARMED UP)
    Traveling Ruby ~3300 ms
    mruby ~3300 ms
    JRuby ~1200 ms

    View Slide

  74. YMMV

    View Slide

  75. AWS LAMBDA PRICING

    View Slide

  76. AWS LAMBDA PRICING
    THE NUMBER OF REQUESTS (I.E. NUMBER OF TIMES OUR FUNCTION IS INVOKED)
    THE SUM OF DURATIONS OUR FUNCTIONS TOOK TO EXECUTE (EXPRESSED IN GB-SECONDS)

    View Slide

  77. FREE TIER
    1M REQUESTS & 400,000 GB-SECONDS PER MONTH

    View Slide

  78. 1M TIMES PER MONTH
    ROUGLY EVERY 3 SECONDS

    View Slide

  79. 400,000 GB-SECONDS /
    MONTH
    MEMORY SIZE * DURATION IN SECONDS

    View Slide

  80. e.g. 128MB * 4s = 0.5GB-secons

    View Slide

  81. IF WE RAN THE BOT EVERY
    MINUTE
    20.000 GB-SECONDS

    View Slide

  82. CHOOSING THE RIGHT Ruby
    FOR THE JOB

    View Slide

  83. Traveling Ruby
    LOW MEMORY CONSUMPTION / WELL
    KNOWN ENVIRONMENT
    SUPPORT?

    View Slide

  84. MRuby
    LOW MEMORY CONSUMPTION /
    LIGHTWEIGHT
    LIBRARY SUPPORT / CROSS COMPILATION

    View Slide

  85. JRuby
    SIMPLE PACKAGING* / FAST EXECUTION
    HIGH MEMORY CONSUMPTION / JAVA
    ERRORS

    View Slide

  86. Alternatives

    View Slide

  87. RUBY Packer
    RUBY Snap

    View Slide

  88. Azure FUNCTIONS
    Google Cloud FUNCTIONS

    View Slide

  89. View Slide

  90. Lots of OPTIONS

    View Slide

  91. Hacky OPTIONS

    View Slide

  92. ..A MONTH AGO..

    View Slide

  93. APACHE
    OpenWhisk

    View Slide

  94. IBM Cloud Functions

    View Slide

  95. IBM Cloud Functions

    View Slide

  96. Inline editor

    View Slide

  97. Serverless Framework
    SUPPORT

    View Slide

  98. View Slide

  99. serverless create --template openwhisk-ruby
    --path ruby_bot

    View Slide

  100. - ruby_bot
    - handler.rb
    - serverless.yml

    View Slide

  101. # serverless.yml
    service: ruby_bot
    provider:
    name: openwhisk
    runtime: ruby
    functions:
    ruby_bot:
    handler: handler.main
    plugins:
    - serverless-openwhisk

    View Slide

  102. serverless deploy

    View Slide

  103. JETS
    SERVERLESS FRAMEWORK FOR RUBY

    View Slide

  104. Jets.application.routes.draw do
    get 'posts', to: 'posts#index'
    end
    class PostsController < ApplicationController
    def index
    render json: { hello: 'world', action: 'index' }
    end
    end

    View Slide

  105. FaastRuby.io

    View Slide

  106. Conclusion

    View Slide

  107. SERVERLESS-RUBY.ORG

    View Slide

  108. View Slide

  109. DAMIR SVRTAN

    View Slide

  110. View Slide

  111. View Slide

  112. tinyurl.com/netflix-ruby

    View Slide

  113. DAMIR SVRTAN
    SENIOR SOFTWARE ENGINEER @ Netflix
    HIT ME UP @DamirSvrtan
    TINYURL.COM/NETFLIX-RUBY

    View Slide