Slide 1

Slide 1 text

Building Ruby Bots on AWS Lambda

Slide 2

Slide 2 text

njuskalo.hr

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

# Gemfile gem 'httparty' gem 'oga' gem 'actionmailer' gem 'rake' gem 'whenever'

Slide 5

Slide 5 text

github.com/DamirSvrtan/apartment- finder

Slide 6

Slide 6 text

AWS Lambda

Slide 7

Slide 7 text

Language support Java, Node.js, Python

Slide 8

Slide 8 text

29 0ct 2015

Slide 9

Slide 9 text

No Ruby? :(

Slide 10

Slide 10 text

Traveling Ruby MRuby JRuby

Slide 11

Slide 11 text

Traveling Ruby

Slide 12

Slide 12 text

mruby

Slide 13

Slide 13 text

JRuby

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

Shrink The Code Base

Slide 16

Slide 16 text

Gemfile gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem 'oga'

Slide 17

Slide 17 text

Use net/http gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem 'oga'

Slide 18

Slide 18 text

Use net/http gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem 'oga'

Slide 19

Slide 19 text

Use Lambda scheduler gem 'httparty' gem 'actionmailer' gem 'whenever' gem 'rake' gem 'oga'

Slide 20

Slide 20 text

There must be an API!

Slide 21

Slide 21 text

1. Download the Android apk file. 2. Decompile the apk file 3. Reverse engineer the code.

Slide 22

Slide 22 text

Failure

Slide 23

Slide 23 text

1. Install the iOS app 2. Plug my mobile to a computer 3. Use Charles to sniff traffic

Slide 24

Slide 24 text

They have an API!

Slide 25

Slide 25 text

An HTML API :(

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Regex it is.

Slide 29

Slide 29 text

Ruby Bot Code

Slide 30

Slide 30 text

# main.rb require 'net/http' require 'time' class ApartmentPublishDates LAST_SCRAPPED_AT = Time.now - 1 * 60 # 1 minute ago. attr_reader :url def initialize(url) @url = url end def all response_body .scan(/datetime\="(.*)" /) .map(&:first) .map { |time| Time.parse(time) } end def recent all.select { |publish_date| publish_date > LAST_SCRAPPED_AT } end private def response_body Net::HTTP.get(URI(url)) end end

Slide 31

Slide 31 text

class MailgunEmailer def self.call Net::HTTP.post_form( URI("https://api:key-#{MAILGUN_API_KEY}@api.mailgun.net/v3/#{MAILGUN_DOMAIN}/messages"), from: "Apartment Bot ", to: RECIPIENT, subject: 'New Apartments found!', html: "Check new apartments" ) end end

Slide 32

Slide 32 text

SEARCH_URL = 'http://iapi.njuskalo.hr?sort=new&categoryId=10920&locationId=2619'.freeze RECIPIENT = ENV.fetch('RECIPIENT') MAILGUN_API_KEY = ENV.fetch('MAILGUN_API_KEY') MAILGUN_DOMAIN = ENV.fetch('MAILGUN_DOMAIN') MailgunEmailer.call if ApartmentPublishDates.new(SEARCH_URL).recent.any?

Slide 33

Slide 33 text

Implementing the Bot with Traveling Ruby Packaged MRI runtimes

Slide 34

Slide 34 text

Download the runtimes OS X for our dev machine Linux x86_64 compatible binary for AWS Lambda

Slide 35

Slide 35 text

curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz curl -O https://d6r77u77i8pq3.cloudfront.net/releases/traveling-ruby-20141215-2.1.5-osx.tar.gz

Slide 36

Slide 36 text

mkdir ruby-linux-x86_64 tar -xzf traveling-ruby-20141215-2.1.5-linux-x86_64.tar.gz -C ruby-linux-x86_64 mkdir ruby-os-x tar -xzf traveling-ruby-20141215-2.1.5-osx.tar.gz -C ruby-os-x

Slide 37

Slide 37 text

traveling-ruby-bot - ruby-linux-x86_64/ - ruby-os-x/

Slide 38

Slide 38 text

traveling-ruby-bot - ruby-linux-x86_64 - bin/ - ruby - ... - ... - ruby-os-x - bin/ - ruby - ... - ...

Slide 39

Slide 39 text

traveling-ruby-bot - ruby-linux-x86_64/ - ruby-os-x/ - main.rb

Slide 40

Slide 40 text

Execute the code ruby-os-x/bin/ruby main.rb

Slide 41

Slide 41 text

Uploading the bot to AWS Lambda

Slide 42

Slide 42 text

// lambda-function-wrapper.js var spawn = require('child_process').spawn; exports.handler = function(event, context) { var child = spawn('ruby-linux-x86_64/bin/ruby main.rb'); child.stdout.on('data', function (data) { console.log("stdout:\n"+data); }); child.stderr.on('data', function (data) { console.log("stderr:\n"+data); }); child.on('close', function (code) { context.done(); }); }

Slide 43

Slide 43 text

zip ruby-bot.zip ruby-linux-x86_64 main.rb lambda-function-wrapper.js

Slide 44

Slide 44 text

1. Create a function

Slide 45

Slide 45 text

2. Create a trigger

Slide 46

Slide 46 text

3. Create a rule for the trigger

Slide 47

Slide 47 text

4. Upload the function code

Slide 48

Slide 48 text

5. Set a handler

Slide 49

Slide 49 text

Good Traveling Ruby Tutorials hello world app app with gems app with native extensions such as Nokogiri

Slide 50

Slide 50 text

Implementing the Bot with mruby

Slide 51

Slide 51 text

Two ways to build apps with MRuby: - download mruby source directly from Github - mruby-cli - platform for building native command line applications for Linux, Windows, and OS X.

Slide 52

Slide 52 text

Building directly from mruby source

Slide 53

Slide 53 text

git clone https://github.com/mruby/mruby

Slide 54

Slide 54 text

mruby-bot - bin - build_config.rb - minirake

Slide 55

Slide 55 text

./minirake

Slide 56

Slide 56 text

mruby-bot - bin/ - mruby - mirb - build_config.rb - minirake

Slide 57

Slide 57 text

> echo "puts 'hello world'" >> hello_world.rb > bin/mruby hello_world.rb hello world

Slide 58

Slide 58 text

> bin/ruby main.rb main.rb:1: undefined method 'require' for main (NoMethodError)

Slide 59

Slide 59 text

# build_config.rb conf.gem github: 'iij/mruby-env' conf.gem github: 'mattn/mruby-curl' conf.gem github: 'mattn/mruby-onig-regexp'

Slide 60

Slide 60 text

./minirake

Slide 61

Slide 61 text

Execute the code bin/mruby main.rb

Slide 62

Slide 62 text

CROSS COMPILATION

Slide 63

Slide 63 text

zip ruby-bot.zip bin/mruby main.rb lambda-function-wrapper.js

Slide 64

Slide 64 text

Implementing the Bot with JRuby

Slide 65

Slide 65 text

Download the jruby-complete.jar from their official downloads page.

Slide 66

Slide 66 text

mv jruby-complete-9.1.7.0.jar ruby-bot.jar

Slide 67

Slide 67 text

cp main.rb ruby-bot.rb

Slide 68

Slide 68 text

jar -ufe ruby-bot.jar org.jruby.RubyBotMain ruby-bot.rb

Slide 69

Slide 69 text

Execute the code java -jar ruby-bot.jar

Slide 70

Slide 70 text

Metrics & Numbers

Slide 71

Slide 71 text

Code Size Traveling Ruby 6.7 MB mruby 945 kB JRuby 23.2 MB

Slide 72

Slide 72 text

Memory consumption Traveling Ruby 25 MB mruby 40 MB JRuby 150 MB

Slide 73

Slide 73 text

Execution Time Traveling Ruby ~3900 ms mruby ~3900 ms JRuby ~12000 ms

Slide 74

Slide 74 text

AWS Lambda Pricing

Slide 75

Slide 75 text

AWS Lambda Pricing the number of requests (i.e. number of times our function is invoked) the sum of durations our functions took to execute (expressed in GB-seconds)

Slide 76

Slide 76 text

Free Tier 1M requests & 400,000 GB-seconds per month

Slide 77

Slide 77 text

1M times per month Rougly every 3 seconds

Slide 78

Slide 78 text

400,000 GB-seconds / month memory size * duration in seconds

Slide 79

Slide 79 text

e.g. 128MB * 4s = 0.5GB-second

Slide 80

Slide 80 text

If we ran the bot every minute 48,960 GB-seconds

Slide 81

Slide 81 text

Choosing the right Ruby for the job

Slide 82

Slide 82 text

JRuby simple packaging / slow execution

Slide 83

Slide 83 text

MRuby lightweight / good execution / bad library support

Slide 84

Slide 84 text

Traveling Ruby good execution / well known environment

Slide 85

Slide 85 text

No content