Slide 1

Slide 1 text

Ruby on Google Cloud Platform 2016/07/14 Roppongi.rb #01 @deme0607

Slide 2

Slide 2 text

Naoki Shimizu @deme0607

Slide 3

Slide 3 text

Game Platform @DeNA SWET (Software Engineer in Test) End to End Test w/ RSpec, Selenium WebDriver Testing Application w/ Sinatra, Rails MERY @peroli Server Side Engineer Web & API App, Backend Server w/ Rails Managing Infrastructure w/ AWS

Slide 4

Slide 4 text

CXI Team @mercari CXI: Customer Experience Improvement Customer Support Tool w/ PHP & JavaScript API Server w/ PHP

Slide 5

Slide 5 text

No Ruby!!!

Slide 6

Slide 6 text

Motivation Write Ruby code Develop, Release, Operate services w/ Ruby

Slide 7

Slide 7 text

Problems outside of writing Ruby Setting up infrastructures app server, DB, load balancer, etc. Scalability lower initial cost is better don’t want to pay attention to scalability

Slide 8

Slide 8 text

https://cloudplatform.googleblog.com/2016/05/Ruby-on-Google-App-Engine-goes-betaruntime.html

Slide 9

Slide 9 text

https://cloudplatform.googleblog.com/2016/03/announcing-two-new-Cloud-Platform-Regions-and-10-more-to-come_22.html

Slide 10

Slide 10 text

https://speakerdeck.com/ttsuruoka/atutekai-fa-falseji-shu-golang-to-google-app-engine

Slide 11

Slide 11 text

Google App Engine Enable us to focus on developing application Enable us to focus on writing Ruby code → Enjoy Programming! Many GAE Experts in mercari!

Slide 12

Slide 12 text

Today’s Theme Ruby on Google App Engine GAE ∈ Google Cloud Platform † Not related to my works in mercari

Slide 13

Slide 13 text

Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE

Slide 14

Slide 14 text

Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE

Slide 15

Slide 15 text

Google App Engine Built-in services & APIs app server, load balancing, health check, logging datastore, memcache, pub/sub, user authentication API Automatic Scaling from zero to millions of users

Slide 16

Slide 16 text

Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE

Slide 17

Slide 17 text

GAE Environments #1. Standard Environment container instances running on Google's infrastructure Java 7, Python 2.7, Go and PHP #2. Flexible environment (beta) Docker containers on Google Compute Engine Initially named “Managed VMs” Java 8, Python 2.7 / 3.4, Go, Node.js and Ruby

Slide 18

Slide 18 text

GAE Environments #1. Standard Environment container instances running on Google's infrastructure Java 7, Python 2.7, Go and PHP #2. Flexible environment (beta) Docker containers on Google Compute Engine Initially named “Managed VMs” Java 8, Python 2.7 / 3.4, Go, Node.js and Ruby

Slide 19

Slide 19 text

GAE Flexible Environment Customisable runtime & OS w/ Dockerfile changing runtimes, installing binaries, etc Native Backend Threads & Processes High Performance Host VM we can choose any machine types of GCE https://cloud.google.com/appengine/docs/the-appengine-environments

Slide 20

Slide 20 text

runtime: ruby vm: true # application entorypoint entrypoint: bundle exec rackup -p 8080 -E production config.ru # if true, GAE sends multiple requests to the application threadsafe: true # GCE machine type is based on resources config resources: cpu: 4 memory_gb: 10 disk_size_gb: 10 app.yaml

Slide 21

Slide 21 text

root@hostname:~# docker ps CONTAINER ID IMAGE COMMAND PORTS NAMES 1b94d28fa5c2 gcr.io/google_appengine/mvm-agent "/usr/local/bin/gunic" sad_hawking 7a9d61dd9e67 gcr.io/google_appengine/nginx-proxy “/var/lib/nginx/bin/s" 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8443/tcp, 8090/tcp high_banach 9d5872fae34d us.gcr.io/pid/appengine/default.version@sha256:hash "/bin/sh -c 'GRPC_TRA" 8080/tcp gaeapp a0903dab94ad gcr.io/google_appengine/memcache-proxy "/home/memcache/memca" 11211/tcp memcache 1e13af1911e5 gcr.io/google_appengine/fluentd-logger "/opt/google-fluentd/" furious_bartik

Slide 22

Slide 22 text

Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE

Slide 23

Slide 23 text

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Slide 24

Slide 24 text

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Slide 25

Slide 25 text

GAE Ruby Docker Image Published in GitHub https://github.com/GoogleCloudPlatform/ruby-docker Optimised to basic Ruby (Rack) Application

Slide 26

Slide 26 text

Dockerfile # Install dependencies for Ruby # Also installs dependencies for the following common gems: # # gems dependencies # ------------------------------------------------------------------ # curb libcurl3, libcurl3-gnutls, libcurl4-openssl-dev # pg libpq-dev # rmagick libmagickwand-dev # nokogiri libxml2-dev, libxslt-dev # sqlite3 libsqlite3-dev # mysql2 libmysqlclient-dev

Slide 27

Slide 27 text

# Install rbenv ENV RBENV_ROOT /rbenv RUN git clone https://github.com/sstephenson/rbenv.git /rbenv && \ git clone https://github.com/sstephenson/ruby-build.git /rbenv/plugins/ruby-build ENV PATH /rbenv/shims:/rbenv/bin:$PATH # Preinstall ruby runtimes. # The LAST version in the list is set as the default. ENV PREINSTALLED_RUBY_VERSIONS 2.1.8 2.2.4 2.3.0 ENV RUBY_CONFIGURE_OPTS --disable-install-doc RUN for V in $PREINSTALLED_RUBY_VERSIONS; do \ rbenv install -v $V; \ rbenv rehash; \ rbenv global $V; \ done Dockerfile

Slide 28

Slide 28 text

Customise Image FROM gcr.io/google_appengine/ruby # install middleware # install another libraries # install another runtime

Slide 29

Slide 29 text

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Slide 30

Slide 30 text

Datastore Option A. Google Cloud SQL Managed MySQL (Like RDS on AWS) B. Run RDBMS on Google Compute Engine Manage MySQL/PostgreSQL etc. by our own C. Google Cloud Datastore zero-configuration, fully-managed, highly-scalable, non-relational database

Slide 31

Slide 31 text

Datastore Option A. Google Cloud SQL Managed MySQL (Like RDS on AWS) B. Run RDBMS on Google Compute Engine Manage MySQL/PostgreSQL etc. by our own C. Google Cloud Datastore zero-configuration, fully-managed, highly-scalable, non-relational database

Slide 32

Slide 32 text

Google Cloud Datastore Schemaless Database based on BigTable BigTable: KVS used in Google services https://cloud.google.com/appengine/docs/go/datastore/

Slide 33

Slide 33 text

Google Cloud Datastore Pros High scalability w/ high performance High availability Fully managed with no downtime Cons Limited ACID transaction No JOIN, GROUP BY, COUNT, etc. Not high performance BUT it wouldn’t be worse

Slide 34

Slide 34 text

Cloud Datastore w/ ActiveModel require ‘gcloud/datastore’ class Book include ActiveModel::Validations attr_accessor :id, :title validates :title, presence: true def self.dataset @dataset ||= Gcloud.datastore(ENV[‘DATASET_ID’]) end Schemaless → Validation & Definition by ActiveModel app/models/book.rb

Slide 35

Slide 35 text

Cloud Datastore w/ ActiveModel def save if valid? entity = Gcloud::Datastore::Entity.new entity.key = Gcloud::Datastore::Key.new ‘Book’, id entity[‘title’] = title Book.dataset.save entity self.id = entity.key.id true else false end end [Datastore] Entity 㲗 [Ruby] Object

Slide 36

Slide 36 text

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Slide 37

Slide 37 text

Logging if Dir.exist?(‘/var/log/app_engine/custom_logs’) config.logger = ActiveSupport::TaggedLogging.new( Logger.new(‘/var/log/app_engine/custom_logs/application.log’) ) end Automatically collected by Stackdriver config/environments/production.rb

Slide 38

Slide 38 text

Logging

Slide 39

Slide 39 text

Other Services Memcache Pub / Sub w/ ActiveJob Authenticating Users Data Storage

Slide 40

Slide 40 text

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Slide 41

Slide 41 text

Rack Server Option Unicorn (multi process) Puma (multi thread)

Slide 42

Slide 42 text

(In most case) Puma is primary option Maximum request timeout: 60mins Unicorn is susceptible to slow client attacks with long timeout https://devcenter.heroku.com/changelog-items/594

Slide 43

Slide 43 text

grpc bug in multi process environment grpc-ruby gem is used when accessing Datastore Current version(0.15.0) doesn’t work after fork Issue: https://github.com/grpc/grpc/issues/6577

Slide 44

Slide 44 text

Workaround # Gemfile source ‘https://rubygems.org' gem ‘rails’ gem ‘puma’ group :lazy_load do gem ‘gcloud’ end

Slide 45

Slide 45 text

Workaround # config/puma.rb on_worker_boot do require ‘gcloud’ require ‘gcloud/datastore’ end # config/unicorn.rb after_fork do |server, worker| require ‘gcloud’ require ‘gcloud/datastore’ end

Slide 46

Slide 46 text

Conclusion We can develop Ruby app in GAE smoothly Because of Ruby (&Rails) & GAE’s flexibility It’s not easy to use in critical environment (currently beta support) bug, backward compatibility, know-how, etc. BUT it has large potential Enable us to focus developing service = Enjoy Programming!

Slide 47

Slide 47 text

Thanks for listening! https://www.wantedly.com/companies/mercariapp- com/projects @deme0607