Building Ruby Bots on AWS Lambda

Want to build a Ruby Bot without the hassle of provisioning and managing servers? Amazon's got a service for that and it's called AWS Lambda - it executes your code only when needed, scales automatically and you pay only for the compute time you consume.

There's one problem with Lambda - it doesn't support Ruby!
Let's checkout multiple options on how to build a Ruby Bot and package it into an executable which you can run on any machine without the need to actually have Ruby installed.

Transcript

1. Building Ruby Bots on
   AWS Lambda
   

   View Slide

2. njuskalo.hr
   

   View Slide

3. View Slide

4. # Gemfile
   gem 'httparty'
   gem 'oga'
   gem 'actionmailer'
   gem 'rake'
   gem 'whenever'
   

   View Slide

5. github.com/DamirSvrtan/apartment-
   finder
   

   View Slide

6. AWS Lambda
   

   View Slide

7. Language support
   Java, Node.js, Python
   

   View Slide

8. 29 0ct 2015
   

   View Slide

9. No Ruby? :(
   

   View Slide

10. Traveling Ruby
    MRuby
    JRuby
    

    View Slide

11. Traveling Ruby
    

    View Slide

12. mruby
    

    View Slide

13. JRuby
    

    View Slide

14. View Slide

15. Shrink The Code Base
    

    View Slide

16. Gemfile
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'
    

    View Slide

17. Use net/http
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'
    

    View Slide

18. Use net/http
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'
    

    View Slide

19. Use Lambda scheduler
    gem 'httparty'
    gem 'actionmailer'
    gem 'whenever'
    gem 'rake'
    gem 'oga'
    

    View Slide

20. There must be an API!
    

    View Slide

21. 1. Download the Android apk file.
    2. Decompile the apk file
    3. Reverse engineer the code.
    

    View Slide

22. Failure
    

    View Slide

23. 1. Install the iOS app
    2. Plug my mobile to a computer
    3. Use Charles to sniff traffic
    

    View Slide

24. They have an API!
    

    View Slide

25. An HTML API :(
    

    View Slide

26. View Slide

27. 14.02.2017.
    

    View Slide

28. Regex it is.
    

    View Slide

29. Ruby Bot Code
    

    View Slide

30. # main.rb
    require 'net/http'
    require 'time'
    class ApartmentPublishDates
    LAST_SCRAPPED_AT = Time.now - 1 * 60 # 1 minute ago.
    attr_reader :url
    def initialize(url)
    @url = url
    end
    def all
    response_body
    .scan(/datetime\="(.*)" /)
    .map(&:first)
    .map { |time| Time.parse(time) }
    end
    def recent
    all.select { |publish_date| publish_date > LAST_SCRAPPED_AT }
    end
    private
    def response_body
    Net::HTTP.get(URI(url))
    end
    end
    

    View Slide

31. class MailgunEmailer
    def self.call
    Net::HTTP.post_form(
    URI("https://api:key-#{MAILGUN_API_KEY}@api.mailgun.net/v3/#{MAILGUN_DOMAIN}/messages"),
    from: "Apartment Bot ",
    to: RECIPIENT,
    subject: 'New Apartments found!',
    html: "Check new apartments"
    )
    end
    end
    

    View Slide

32. SEARCH_URL = 'http://iapi.njuskalo.hr?sort=new&categoryId=10920&locationId=2619'.freeze
    RECIPIENT = ENV.fetch('RECIPIENT')
    MAILGUN_API_KEY = ENV.fetch('MAILGUN_API_KEY')
    MAILGUN_DOMAIN = ENV.fetch('MAILGUN_DOMAIN')
    MailgunEmailer.call if ApartmentPublishDates.new(SEARCH_URL).recent.any?
    

    View Slide

33. Implementing the Bot with
    Traveling Ruby
    Packaged MRI runtimes
    

    View Slide

34. Download the runtimes
    OS X for our dev machine
    Linux x86_64 compatible binary for AWS Lambda
    

    View Slide

35. curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz
    curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-osx.tar.gz
    

    View Slide

36. mkdir ruby-linux-x86_64
    tar -xzf traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz -C ruby-linux-x86_64
    mkdir ruby-os-x
    tar -xzf traveling-ruby-20141215-2.1.5-osx.tar.gz -C ruby-os-x
    

    View Slide

37. traveling-ruby-bot
    - ruby-linux-x86_64/
    - ruby-os-x/
    

    View Slide

38. traveling-ruby-bot
    - ruby-linux-x86_64
    - bin/
    - ruby
    - ...
    - ...
    - ruby-os-x
    - bin/
    - ruby
    - ...
    - ...
    

    View Slide

39. traveling-ruby-bot
    - ruby-linux-x86_64/
    - ruby-os-x/
    - main.rb
    

    View Slide

40. Execute the code
    ruby-os-x/bin/ruby main.rb
    

    View Slide

41. Uploading the bot to AWS Lambda
    

    View Slide

42. // lambda-function-wrapper.js
    var spawn = require('child_process').spawn;
    exports.handler = function(event, context) {
    var child = spawn('ruby-linux-x86_64/bin/ruby main.rb');
    child.stdout.on('data', function (data) { console.log("stdout:\n"+data); });
    child.stderr.on('data', function (data) { console.log("stderr:\n"+data); });
    child.on('close', function (code) { context.done(); });
    }
    

    View Slide

43. zip ruby-bot.zip ruby-linux-x86_64 main.rb lambda-function-wrapper.js
    

    View Slide

44. 1. Create a function
    

    View Slide

45. 2. Create a trigger
    

    View Slide

46. 3. Create a rule for the trigger
    

    View Slide

47. 4. Upload the function code
    

    View Slide

48. 5. Set a handler
    

    View Slide

49. Good Traveling Ruby Tutorials
    hello world app
    app with gems
    app with native extensions such as Nokogiri
    

    View Slide

50. Implementing the Bot with
    mruby
    

    View Slide

51. Two ways to build apps with
    MRuby:
    - download mruby source directly from Github
    - mruby-cli - platform for building native command line applications
    for Linux, Windows, and OS X.
    

    View Slide

52. Building directly from mruby source
    

    View Slide

53. git clone https://github.com/mruby/mruby
    

    View Slide

54. mruby-bot
    - bin
    - build_config.rb
    - minirake
    

    View Slide

55. ./minirake
    

    View Slide

56. mruby-bot
    - bin/
    - mruby
    - mirb
    - build_config.rb
    - minirake
    

    View Slide

57. > echo "puts 'hello world'" >> hello_world.rb
    > bin/mruby hello_world.rb
    hello world
    

    View Slide

58. > bin/ruby main.rb
    main.rb:1: undefined method 'require' for main (NoMethodError)
    

    View Slide

59. # build_config.rb
    conf.gem github: 'iij/mruby-env'
    conf.gem github: 'mattn/mruby-curl'
    conf.gem github: 'mattn/mruby-onig-regexp'
    

    View Slide

60. ./minirake
    

    View Slide

61. Execute the code
    bin/mruby main.rb
    

    View Slide

62. CROSS COMPILATION
    

    View Slide

63. zip ruby-bot.zip bin/mruby main.rb lambda-function-wrapper.js
    

    View Slide

64. Implementing the Bot with JRuby
    

    View Slide

65. Download the jruby-complete.jar from
    their official downloads page.
    

    View Slide

66. mv jruby-complete-9.1.7.0.jar ruby-bot.jar
    

    View Slide

67. cp main.rb ruby-bot.rb
    

    View Slide

68. jar -ufe ruby-bot.jar org.jruby.RubyBotMain ruby-bot.rb
    

    View Slide

69. Execute the code
    java -jar ruby-bot.jar
    

    View Slide

70. Metrics & Numbers
    

    View Slide

71. Code Size
    Traveling Ruby 6.7 MB
    mruby 945 kB
    JRuby 23.2 MB
    

    View Slide

72. Memory consumption
    Traveling Ruby 25 MB
    mruby 40 MB
    JRuby 150 MB
    

    View Slide

73. Execution Time
    Traveling Ruby ~3900 ms
    mruby ~3900 ms
    JRuby ~12000 ms
    

    View Slide

74. AWS Lambda Pricing
    

    View Slide

75. AWS Lambda Pricing
    the number of requests (i.e. number of times our function is
    invoked)
    the sum of durations our functions took to execute (expressed in
    GB-seconds)
    

    View Slide

76. Free Tier
    1M requests & 400,000 GB-seconds per month
    

    View Slide

77. 1M times per month
    Rougly every 3 seconds
    

    View Slide

78. 400,000 GB-seconds / month
    memory size * duration in seconds
    

    View Slide

79. e.g. 128MB * 4s = 0.5GB-second
    

    View Slide

80. If we ran the bot every minute
    48,960 GB-seconds
    

    View Slide

81. Choosing the right Ruby for the
    job
    

    View Slide

82. JRuby
    simple packaging / slow execution
    

    View Slide

83. MRuby
    lightweight / good execution / bad
    library support
    

    View Slide

84. Traveling Ruby
    good execution / well known
    environment
    

    View Slide

85. View Slide