Ruby on Google Cloud Platform 2016/07/14 Roppongi.rb #01 @deme0607

Naoki Shimizu @deme0607

Game Platform @DeNA SWET (Software Engineer in Test) End to End Test w/ RSpec, Selenium WebDriver Testing Application w/ Sinatra, Rails MERY @peroli Server Side Engineer Web & API App, Backend Server w/ Rails Managing Infrastructure w/ AWS

CXI Team @mercari CXI: Customer Experience Improvement Customer Support Tool w/ PHP & JavaScript API Server w/ PHP

No Ruby!!!

Motivation Write Ruby code Develop, Release, Operate services w/ Ruby

Problems outside of writing Ruby Setting up infrastructures app server, DB, load balancer, etc. Scalability lower initial cost is better don’t want to pay attention to scalability

https://cloudplatform.googleblog.com/2016/05/Ruby-on-Google-App-Engine-goes-betaruntime.html

https://cloudplatform.googleblog.com/2016/03/announcing-two-new-Cloud-Platform-Regions-and-10-more-to-come_22.html

https://speakerdeck.com/ttsuruoka/atutekai-fa-falseji-shu-golang-to-google-app-engine

Google App Engine Enable us to focus on developing application Enable us to focus on writing Ruby code → Enjoy Programming! Many GAE Experts in mercari!

Today’s Theme Ruby on Google App Engine GAE ∈ Google Cloud Platform † Not related to my works in mercari

Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE

Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE

Google App Engine Built-in services & APIs app server, load balancing, health check, logging datastore, memcache, pub/sub, user authentication API Automatic Scaling from zero to millions of users

Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE

GAE Environments #1. Standard Environment container instances running on Google's infrastructure Java 7, Python 2.7, Go and PHP #2. Flexible environment (beta) Docker containers on Google Compute Engine Initially named “Managed VMs” Java 8, Python 2.7 / 3.4, Go, Node.js and Ruby

GAE Environments #1. Standard Environment container instances running on Google's infrastructure Java 7, Python 2.7, Go and PHP #2. Flexible environment (beta) Docker containers on Google Compute Engine Initially named “Managed VMs” Java 8, Python 2.7 / 3.4, Go, Node.js and Ruby

GAE Flexible Environment Customisable runtime & OS w/ Dockerfile changing runtimes, installing binaries, etc Native Backend Threads & Processes High Performance Host VM we can choose any machine types of GCE https://cloud.google.com/appengine/docs/the-appengine-environments

runtime: ruby vm: true # application entorypoint entrypoint: bundle exec rackup -p 8080 -E production config.ru # if true, GAE sends multiple requests to the application threadsafe: true # GCE machine type is based on resources config resources: cpu: 4 memory_gb: 10 disk_size_gb: 10 app.yaml

root@hostname:~# docker ps CONTAINER ID IMAGE COMMAND PORTS NAMES 1b94d28fa5c2 gcr.io/google_appengine/mvm-agent "/usr/local/bin/gunic" sad_hawking 7a9d61dd9e67 gcr.io/google_appengine/nginx-proxy “/var/lib/nginx/bin/s" 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8443/tcp, 8090/tcp high_banach 9d5872fae34d us.gcr.io/pid/appengine/default.version@sha256:hash "/bin/sh -c 'GRPC_TRA" 8080/tcp gaeapp a0903dab94ad gcr.io/google_appengine/memcache-proxy "/home/memcache/memca" 11211/tcp memcache 1e13af1911e5 gcr.io/google_appengine/fluentd-logger "/opt/google-fluentd/" furious_bartik

Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

GAE Ruby Docker Image Published in GitHub https://github.com/GoogleCloudPlatform/ruby-docker Optimised to basic Ruby (Rack) Application

Dockerfile # Install dependencies for Ruby # Also installs dependencies for the following common gems: # # gems dependencies # ------------------------------------------------------------------ # curb libcurl3, libcurl3-gnutls, libcurl4-openssl-dev # pg libpq-dev # rmagick libmagickwand-dev # nokogiri libxml2-dev, libxslt-dev # sqlite3 libsqlite3-dev # mysql2 libmysqlclient-dev

# Install rbenv ENV RBENV_ROOT /rbenv RUN git clone https://github.com/sstephenson/rbenv.git /rbenv && \ git clone https://github.com/sstephenson/ruby-build.git /rbenv/plugins/ruby-build ENV PATH /rbenv/shims:/rbenv/bin:$PATH # Preinstall ruby runtimes. # The LAST version in the list is set as the default. ENV PREINSTALLED_RUBY_VERSIONS 2.1.8 2.2.4 2.3.0 ENV RUBY_CONFIGURE_OPTS --disable-install-doc RUN for V in $PREINSTALLED_RUBY_VERSIONS; do \ rbenv install -v $V; \ rbenv rehash; \ rbenv global $V; \ done Dockerfile

Customise Image FROM gcr.io/google_appengine/ruby # install middleware # install another libraries # install another runtime

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Datastore Option A. Google Cloud SQL Managed MySQL (Like RDS on AWS) B. Run RDBMS on Google Compute Engine Manage MySQL/PostgreSQL etc. by our own C. Google Cloud Datastore zero-configuration, fully-managed, highly-scalable, non-relational database

Datastore Option A. Google Cloud SQL Managed MySQL (Like RDS on AWS) B. Run RDBMS on Google Compute Engine Manage MySQL/PostgreSQL etc. by our own C. Google Cloud Datastore zero-configuration, fully-managed, highly-scalable, non-relational database

Google Cloud Datastore Schemaless Database based on BigTable BigTable: KVS used in Google services https://cloud.google.com/appengine/docs/go/datastore/

Google Cloud Datastore Pros High scalability w/ high performance High availability Fully managed with no downtime Cons Limited ACID transaction No JOIN, GROUP BY, COUNT, etc. Not high performance BUT it wouldn’t be worse

Cloud Datastore w/ ActiveModel require ‘gcloud/datastore’ class Book include ActiveModel::Validations attr_accessor :id, :title validates :title, presence: true def self.dataset @dataset ||= Gcloud.datastore(ENV[‘DATASET_ID’]) end Schemaless → Validation & Definition by ActiveModel app/models/book.rb

Cloud Datastore w/ ActiveModel def save if valid? entity = Gcloud::Datastore::Entity.new entity.key = Gcloud::Datastore::Key.new ‘Book’, id entity[‘title’] = title Book.dataset.save entity self.id = entity.key.id true else false end end [Datastore] Entity 㲗 [Ruby] Object

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Logging if Dir.exist?(‘/var/log/app_engine/custom_logs’) config.logger = ActiveSupport::TaggedLogging.new( Logger.new(‘/var/log/app_engine/custom_logs/application.log’) ) end Automatically collected by Stackdriver config/environments/production.rb

Logging

Other Services Memcache Pub / Sub w/ ActiveJob Authenticating Users Data Storage

Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server

Rack Server Option Unicorn (multi process) Puma (multi thread)

(In most case) Puma is primary option Maximum request timeout: 60mins Unicorn is susceptible to slow client attacks with long timeout https://devcenter.heroku.com/changelog-items/594

grpc bug in multi process environment grpc-ruby gem is used when accessing Datastore Current version(0.15.0) doesn’t work after fork Issue: https://github.com/grpc/grpc/issues/6577

Workaround # Gemfile source ‘https://rubygems.org' gem ‘rails’ gem ‘puma’ group :lazy_load do gem ‘gcloud’ end

Workaround # config/puma.rb on_worker_boot do require ‘gcloud’ require ‘gcloud/datastore’ end # config/unicorn.rb after_fork do |server, worker| require ‘gcloud’ require ‘gcloud/datastore’ end

Conclusion We can develop Ruby app in GAE smoothly Because of Ruby (&Rails) & GAE’s flexibility It’s not easy to use in critical environment (currently beta support) bug, backward compatibility, know-how, etc. BUT it has large potential Enable us to focus developing service = Enjoy Programming!

Thanks for listening! https://www.wantedly.com/companies/mercariapp- com/projects @deme0607