Building Ruby Bots on AWS Lambda

5f81e2d2889d7642fd84c8b24db7ee17?s=47 DamirSvrtan
February 21, 2017

Building Ruby Bots on AWS Lambda

Want to build a Ruby Bot without the hassle of provisioning and managing servers? Amazon's got a service for that and it's called AWS Lambda - it executes your code only when needed, scales automatically and you pay only for the compute time you consume.

There's one problem with Lambda - it doesn't support Ruby!
Let's checkout multiple options on how to build a Ruby Bot and package it into an executable which you can run on any machine without the need to actually have Ruby installed.

5f81e2d2889d7642fd84c8b24db7ee17?s=128

DamirSvrtan

February 21, 2017
Tweet

Transcript

  1. Building Ruby Bots on AWS Lambda

  2. njuskalo.hr

  3. None
  4. # Gemfile gem 'httparty' gem 'oga' gem 'actionmailer' gem 'rake'

    gem 'whenever'
  5. github.com/DamirSvrtan/apartment- finder

  6. AWS Lambda

  7. Language support Java, Node.js, Python

  8. 29 0ct 2015

  9. No Ruby? :(

  10. Traveling Ruby MRuby JRuby

  11. Traveling Ruby

  12. mruby

  13. JRuby

  14. None
  15. Shrink The Code Base

  16. Gemfile gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem

    'oga'
  17. Use net/http gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake'

    gem 'oga'
  18. Use net/http gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake'

    gem 'oga'
  19. Use Lambda scheduler gem 'httparty' gem 'actionmailer' gem 'whenever' gem

    'rake' gem 'oga'
  20. There must be an API!

  21. 1. Download the Android apk file. 2. Decompile the apk

    file 3. Reverse engineer the code.
  22. Failure

  23. 1. Install the iOS app 2. Plug my mobile to

    a computer 3. Use Charles to sniff traffic
  24. They have an API!

  25. An HTML API :(

  26. None
  27. <time datetime="2017-02-14T23:44:18+01:00">14.02.2017.</time>

  28. Regex it is.

  29. Ruby Bot Code

  30. # main.rb require 'net/http' require 'time' class ApartmentPublishDates LAST_SCRAPPED_AT =

    Time.now - 1 * 60 # 1 minute ago. attr_reader :url def initialize(url) @url = url end def all response_body .scan(/datetime\="(.*)" /) .map(&:first) .map { |time| Time.parse(time) } end def recent all.select { |publish_date| publish_date > LAST_SCRAPPED_AT } end private def response_body Net::HTTP.get(URI(url)) end end
  31. class MailgunEmailer def self.call Net::HTTP.post_form( URI("https://api:key-#{MAILGUN_API_KEY}@api.mailgun.net/v3/#{MAILGUN_DOMAIN}/messages"), from: "Apartment Bot <postmaster@#{MAILGUN_DOMAIN}>",

    to: RECIPIENT, subject: 'New Apartments found!', html: "Check new <a href='#{SEARCH_URL}'>apartments</a>" ) end end
  32. SEARCH_URL = 'http://iapi.njuskalo.hr?sort=new&categoryId=10920&locationId=2619'.freeze RECIPIENT = ENV.fetch('RECIPIENT') MAILGUN_API_KEY = ENV.fetch('MAILGUN_API_KEY') MAILGUN_DOMAIN

    = ENV.fetch('MAILGUN_DOMAIN') MailgunEmailer.call if ApartmentPublishDates.new(SEARCH_URL).recent.any?
  33. Implementing the Bot with Traveling Ruby Packaged MRI runtimes

  34. Download the runtimes OS X for our dev machine Linux

    x86_64 compatible binary for AWS Lambda
  35. curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-osx.tar.gz

  36. mkdir ruby-linux-x86_64 tar -xzf traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz -C ruby-linux-x86_64 mkdir ruby-os-x tar

    -xzf traveling-ruby-20141215-2.1.5-osx.tar.gz -C ruby-os-x
  37. traveling-ruby-bot - ruby-linux-x86_64/ - ruby-os-x/

  38. traveling-ruby-bot - ruby-linux-x86_64 - bin/ - ruby - ... -

    ... - ruby-os-x - bin/ - ruby - ... - ...
  39. traveling-ruby-bot - ruby-linux-x86_64/ - ruby-os-x/ - main.rb

  40. Execute the code ruby-os-x/bin/ruby main.rb

  41. Uploading the bot to AWS Lambda

  42. // lambda-function-wrapper.js var spawn = require('child_process').spawn; exports.handler = function(event, context)

    { var child = spawn('ruby-linux-x86_64/bin/ruby main.rb'); child.stdout.on('data', function (data) { console.log("stdout:\n"+data); }); child.stderr.on('data', function (data) { console.log("stderr:\n"+data); }); child.on('close', function (code) { context.done(); }); }
  43. zip ruby-bot.zip ruby-linux-x86_64 main.rb lambda-function-wrapper.js

  44. 1. Create a function

  45. 2. Create a trigger

  46. 3. Create a rule for the trigger

  47. 4. Upload the function code

  48. 5. Set a handler

  49. Good Traveling Ruby Tutorials hello world app app with gems

    app with native extensions such as Nokogiri
  50. Implementing the Bot with mruby

  51. Two ways to build apps with MRuby: - download mruby

    source directly from Github - mruby-cli - platform for building native command line applications for Linux, Windows, and OS X.
  52. Building directly from mruby source

  53. git clone https://github.com/mruby/mruby

  54. mruby-bot - bin - build_config.rb - minirake

  55. ./minirake

  56. mruby-bot - bin/ - mruby - mirb - build_config.rb -

    minirake
  57. > echo "puts 'hello world'" >> hello_world.rb > bin/mruby hello_world.rb

    hello world
  58. > bin/ruby main.rb main.rb:1: undefined method 'require' for main (NoMethodError)

  59. # build_config.rb conf.gem github: 'iij/mruby-env' conf.gem github: 'mattn/mruby-curl' conf.gem github:

    'mattn/mruby-onig-regexp'
  60. ./minirake

  61. Execute the code bin/mruby main.rb

  62. CROSS COMPILATION

  63. zip ruby-bot.zip bin/mruby main.rb lambda-function-wrapper.js

  64. Implementing the Bot with JRuby

  65. Download the jruby-complete.jar from their official downloads page.

  66. mv jruby-complete-9.1.7.0.jar ruby-bot.jar

  67. cp main.rb ruby-bot.rb

  68. jar -ufe ruby-bot.jar org.jruby.RubyBotMain ruby-bot.rb

  69. Execute the code java -jar ruby-bot.jar

  70. Metrics & Numbers

  71. Code Size Traveling Ruby 6.7 MB mruby 945 kB JRuby

    23.2 MB
  72. Memory consumption Traveling Ruby 25 MB mruby 40 MB JRuby

    150 MB
  73. Execution Time Traveling Ruby ~3900 ms mruby ~3900 ms JRuby

    ~12000 ms
  74. AWS Lambda Pricing

  75. AWS Lambda Pricing the number of requests (i.e. number of

    times our function is invoked) the sum of durations our functions took to execute (expressed in GB-seconds)
  76. Free Tier 1M requests & 400,000 GB-seconds per month

  77. 1M times per month Rougly every 3 seconds

  78. 400,000 GB-seconds / month memory size * duration in seconds

  79. e.g. 128MB * 4s = 0.5GB-second

  80. If we ran the bot every minute 48,960 GB-seconds

  81. Choosing the right Ruby for the job

  82. JRuby simple packaging / slow execution

  83. MRuby lightweight / good execution / bad library support

  84. Traveling Ruby good execution / well known environment

  85. None