Building Ruby Bots on AWS Lambda

Want to build a Ruby Bot without the hassle of provisioning and managing servers? Amazon's got a service for that and it's called AWS Lambda - it executes your code only when needed, scales automatically and you pay only for the compute time you consume.

There's one problem with Lambda - it doesn't support Ruby!
Let's checkout multiple options on how to build a Ruby Bot and package it into an executable which you can run on any machine without the need to actually have Ruby installed.

Transcript

1. Building Ruby Bots on AWS Lambda
2. njuskalo.hr
4. # Gemfile gem 'httparty' gem 'oga' gem 'actionmailer' gem 'rake' gem 'whenever'
5. github.com/DamirSvrtan/apartment- finder
6. AWS Lambda
7. Language support Java, Node.js, Python
8. 29 0ct 2015
9. No Ruby? :(
10. Traveling Ruby MRuby JRuby
11. Traveling Ruby
12. mruby
13. JRuby
15. Shrink The Code Base
16. Gemfile gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem 'oga'
17. Use net/http gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem 'oga'
18. Use net/http gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem 'oga'
19. Use Lambda scheduler gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem 'oga'
20. There must be an API!
21. 1. Download the Android apk file. 2. Decompile the apk file 3. Reverse engineer the code.
22. Failure
23. 1. Install the iOS app 2. Plug my mobile to a computer 3. Use Charles to sniff traffic
24. They have an API!
25. An HTML API :(
27. <time datetime="2017-02-14T23:44:18+01:00">14.02.2017.</time>
28. Regex it is.
29. Ruby Bot Code
30. # main.rb require 'net/http' require 'time' class ApartmentPublishDates LAST_SCRAPPED_AT = Time.now - 1 * 60 # 1 minute ago. attr_reader :url def initialize(url) @url = url end def all response_body .scan(/datetime\="(.*)" /) .map(&:first) .map { |time| Time.parse(time) } end def recent all.select { |publish_date| publish_date > LAST_SCRAPPED_AT } end private def response_body Net::HTTP.get(URI(url)) end end
31. class MailgunEmailer def self.call Net::HTTP.post_form( URI("https://api:key-#{MAILGUN_API_KEY}@api.mailgun.net/v3/#{MAILGUN_DOMAIN}/messages"), from: "Apartment Bot <postmaster@#{MAILGUN_DOMAIN}>", to: RECIPIENT, subject: 'New Apartments found!', html: "Check new <a href='#{SEARCH_URL}'>apartments</a>" ) end end
32. SEARCH_URL = 'http://iapi.njuskalo.hr?sort=new&categoryId=10920&locationId=2619'.freeze RECIPIENT = ENV.fetch('RECIPIENT') MAILGUN_API_KEY = ENV.fetch('MAILGUN_API_KEY') MAILGUN_DOMAIN = ENV.fetch('MAILGUN_DOMAIN') MailgunEmailer.call if ApartmentPublishDates.new(SEARCH_URL).recent.any?
33. Implementing the Bot with Traveling Ruby Packaged MRI runtimes
34. Download the runtimes OS X for our dev machine Linux x86_64 compatible binary for AWS Lambda
35. curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-osx.tar.gz
36. mkdir ruby-linux-x86_64 tar -xzf traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz -C ruby-linux-x86_64 mkdir ruby-os-x tar -xzf traveling-ruby-20141215-2.1.5-osx.tar.gz -C ruby-os-x
37. traveling-ruby-bot - ruby-linux-x86_64/ - ruby-os-x/
38. traveling-ruby-bot - ruby-linux-x86_64 - bin/ - ruby - ... - ... - ruby-os-x - bin/ - ruby - ... - ...
39. traveling-ruby-bot - ruby-linux-x86_64/ - ruby-os-x/ - main.rb
40. Execute the code ruby-os-x/bin/ruby main.rb
41. Uploading the bot to AWS Lambda
42. // lambda-function-wrapper.js var spawn = require('child_process').spawn; exports.handler = function(event, context) { var child = spawn('ruby-linux-x86_64/bin/ruby main.rb'); child.stdout.on('data', function (data) { console.log("stdout:\n"+data); }); child.stderr.on('data', function (data) { console.log("stderr:\n"+data); }); child.on('close', function (code) { context.done(); }); }
43. zip ruby-bot.zip ruby-linux-x86_64 main.rb lambda-function-wrapper.js
44. 1. Create a function
45. 2. Create a trigger
46. 3. Create a rule for the trigger
47. 4. Upload the function code
48. 5. Set a handler
49. Good Traveling Ruby Tutorials hello world app app with gems app with native extensions such as Nokogiri
50. Implementing the Bot with mruby
51. Two ways to build apps with MRuby: - download mruby source directly from Github - mruby-cli - platform for building native command line applications for Linux, Windows, and OS X.
52. Building directly from mruby source
53. git clone https://github.com/mruby/mruby
54. mruby-bot - bin - build_config.rb - minirake
55. ./minirake
56. mruby-bot - bin/ - mruby - mirb - build_config.rb - minirake
57. > echo "puts 'hello world'" >> hello_world.rb > bin/mruby hello_world.rb hello world
58. > bin/ruby main.rb main.rb:1: undefined method 'require' for main (NoMethodError)
59. # build_config.rb conf.gem github: 'iij/mruby-env' conf.gem github: 'mattn/mruby-curl' conf.gem github: 'mattn/mruby-onig-regexp'
60. ./minirake
61. Execute the code bin/mruby main.rb
62. CROSS COMPILATION
63. zip ruby-bot.zip bin/mruby main.rb lambda-function-wrapper.js
64. Implementing the Bot with JRuby
65. Download the jruby-complete.jar from their official downloads page.
66. mv jruby-complete-9.1.7.0.jar ruby-bot.jar
67. cp main.rb ruby-bot.rb
68. jar -ufe ruby-bot.jar org.jruby.RubyBotMain ruby-bot.rb
69. Execute the code java -jar ruby-bot.jar
70. Metrics & Numbers
71. Code Size Traveling Ruby 6.7 MB mruby 945 kB JRuby 23.2 MB
72. Memory consumption Traveling Ruby 25 MB mruby 40 MB JRuby 150 MB
73. Execution Time Traveling Ruby ~3900 ms mruby ~3900 ms JRuby ~12000 ms
74. AWS Lambda Pricing
75. AWS Lambda Pricing the number of requests (i.e. number of times our function is invoked) the sum of durations our functions took to execute (expressed in GB-seconds)
76. Free Tier 1M requests & 400,000 GB-seconds per month
77. 1M times per month Rougly every 3 seconds
78. 400,000 GB-seconds / month memory size * duration in seconds
79. e.g. 128MB * 4s = 0.5GB-second
80. If we ran the bot every minute 48,960 GB-seconds
81. Choosing the right Ruby for the job
82. JRuby simple packaging / slow execution
83. MRuby lightweight / good execution / bad library support
84. Traveling Ruby good execution / well known environment