Ruby on Google Cloud Platform

20160714 Roppongi.rb #1

Transcript

1. Ruby on Google Cloud Platform 2016/07/14 Roppongi.rb #01 @deme0607
2. Naoki Shimizu @deme0607
3. Game Platform @DeNA SWET (Software Engineer in Test) End to End Test w/ RSpec, Selenium WebDriver Testing Application w/ Sinatra, Rails MERY @peroli Server Side Engineer Web & API App, Backend Server w/ Rails Managing Infrastructure w/ AWS
4. CXI Team @mercari CXI: Customer Experience Improvement Customer Support Tool w/ PHP & JavaScript API Server w/ PHP
5. No Ruby!!!
6. Motivation Write Ruby code Develop, Release, Operate services w/ Ruby
7. Problems outside of writing Ruby Setting up infrastructures app server, DB, load balancer, etc. Scalability lower initial cost is better don’t want to pay attention to scalability
8. https://cloudplatform.googleblog.com/2016/05/Ruby-on-Google-App-Engine-goes-betaruntime.html
9. https://cloudplatform.googleblog.com/2016/03/announcing-two-new-Cloud-Platform-Regions-and-10-more-to-come_22.html
10. https://speakerdeck.com/ttsuruoka/atutekai-fa-falseji-shu-golang-to-google-app-engine
11. Google App Engine Enable us to focus on developing application Enable us to focus on writing Ruby code → Enjoy Programming! Many GAE Experts in mercari!
12. Today’s Theme Ruby on Google App Engine GAE ∈ Google Cloud Platform † Not related to my works in mercari
13. Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE
14. Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE
15. Google App Engine Built-in services & APIs app server, load balancing, health check, logging datastore, memcache, pub/sub, user authentication API Automatic Scaling from zero to millions of users
16. Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE
17. GAE Environments #1. Standard Environment container instances running on Google's infrastructure Java 7, Python 2.7, Go and PHP #2. Flexible environment (beta) Docker containers on Google Compute Engine Initially named “Managed VMs” Java 8, Python 2.7 / 3.4, Go, Node.js and Ruby
18. GAE Environments #1. Standard Environment container instances running on Google's infrastructure Java 7, Python 2.7, Go and PHP #2. Flexible environment (beta) Docker containers on Google Compute Engine Initially named “Managed VMs” Java 8, Python 2.7 / 3.4, Go, Node.js and Ruby
19. GAE Flexible Environment Customisable runtime & OS w/ Dockerfile changing runtimes, installing binaries, etc Native Backend Threads & Processes High Performance Host VM we can choose any machine types of GCE https://cloud.google.com/appengine/docs/the-appengine-environments
20. runtime: ruby vm: true # application entorypoint entrypoint: bundle exec rackup -p 8080 -E production config.ru # if true, GAE sends multiple requests to the application threadsafe: true # GCE machine type is based on resources config resources: cpu: 4 memory_gb: 10 disk_size_gb: 10 app.yaml
21. root@hostname:~# docker ps CONTAINER ID IMAGE COMMAND PORTS NAMES 1b94d28fa5c2 gcr.io/google_appengine/mvm-agent "/usr/local/bin/gunic" sad_hawking 7a9d61dd9e67 gcr.io/google_appengine/nginx-proxy “/var/lib/nginx/bin/s" 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8443/tcp, 8090/tcp high_banach 9d5872fae34d us.gcr.io/pid/appengine/default.version@sha256:hash "/bin/sh -c 'GRPC_TRA" 8080/tcp gaeapp a0903dab94ad gcr.io/google_appengine/memcache-proxy "/home/memcache/memca" 11211/tcp memcache 1e13af1911e5 gcr.io/google_appengine/fluentd-logger "/opt/google-fluentd/" furious_bartik
22. Agenda Google App Engine Google App Engine Flexible Environment Developing Ruby application on GAE
23. Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server
24. Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server
25. GAE Ruby Docker Image Published in GitHub https://github.com/GoogleCloudPlatform/ruby-docker Optimised to basic Ruby (Rack) Application
26. Dockerfile # Install dependencies for Ruby # Also installs dependencies for the following common gems: # # gems dependencies # ------------------------------------------------------------------ # curb libcurl3, libcurl3-gnutls, libcurl4-openssl-dev # pg libpq-dev # rmagick libmagickwand-dev # nokogiri libxml2-dev, libxslt-dev # sqlite3 libsqlite3-dev # mysql2 libmysqlclient-dev
27. # Install rbenv ENV RBENV_ROOT /rbenv RUN git clone https://github.com/sstephenson/rbenv.git /rbenv && \ git clone https://github.com/sstephenson/ruby-build.git /rbenv/plugins/ruby-build ENV PATH /rbenv/shims:/rbenv/bin:$PATH # Preinstall ruby runtimes. # The LAST version in the list is set as the default. ENV PREINSTALLED_RUBY_VERSIONS 2.1.8 2.2.4 2.3.0 ENV RUBY_CONFIGURE_OPTS --disable-install-doc RUN for V in $PREINSTALLED_RUBY_VERSIONS; do \ rbenv install -v $V; \ rbenv rehash; \ rbenv global $V; \ done Dockerfile
28. Customise Image FROM gcr.io/google_appengine/ruby # install middleware # install another libraries # install another runtime
29. Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server
30. Datastore Option A. Google Cloud SQL Managed MySQL (Like RDS on AWS) B. Run RDBMS on Google Compute Engine Manage MySQL/PostgreSQL etc. by our own C. Google Cloud Datastore zero-configuration, fully-managed, highly-scalable, non-relational database
31. Datastore Option A. Google Cloud SQL Managed MySQL (Like RDS on AWS) B. Run RDBMS on Google Compute Engine Manage MySQL/PostgreSQL etc. by our own C. Google Cloud Datastore zero-configuration, fully-managed, highly-scalable, non-relational database
32. Google Cloud Datastore Schemaless Database based on BigTable BigTable: KVS used in Google services https://cloud.google.com/appengine/docs/go/datastore/
33. Google Cloud Datastore Pros High scalability w/ high performance High availability Fully managed with no downtime Cons Limited ACID transaction No JOIN, GROUP BY, COUNT, etc. Not high performance BUT it wouldn’t be worse
34. Cloud Datastore w/ ActiveModel require ‘gcloud/datastore’ class Book include ActiveModel::Validations attr_accessor :id, :title validates :title, presence: true def self.dataset @dataset ||= Gcloud.datastore(ENV[‘DATASET_ID’]) end Schemaless → Validation & Definition by ActiveModel app/models/book.rb
35. Cloud Datastore w/ ActiveModel def save if valid? entity = Gcloud::Datastore::Entity.new entity.key = Gcloud::Datastore::Key.new ‘Book’, id entity[‘title’] = title Book.dataset.save entity self.id = entity.key.id true else false end end [Datastore] Entity 㲗 [Ruby] Object
36. Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server
37. Logging if Dir.exist?(‘/var/log/app_engine/custom_logs’) config.logger = ActiveSupport::TaggedLogging.new( Logger.new(‘/var/log/app_engine/custom_logs/application.log’) ) end Automatically collected by Stackdriver config/environments/production.rb
38. Logging
39. Other Services Memcache Pub / Sub w/ ActiveJob Authenticating Users Data Storage
40. Developing Ruby application on GAE GAE Ruby Docker Image Datastore Logging Rack Server
41. Rack Server Option Unicorn (multi process) Puma (multi thread)
42. (In most case) Puma is primary option Maximum request timeout: 60mins Unicorn is susceptible to slow client attacks with long timeout https://devcenter.heroku.com/changelog-items/594
43. grpc bug in multi process environment grpc-ruby gem is used when accessing Datastore Current version(0.15.0) doesn’t work after fork Issue: https://github.com/grpc/grpc/issues/6577
44. Workaround # Gemfile source ‘https://rubygems.org' gem ‘rails’ gem ‘puma’ group :lazy_load do gem ‘gcloud’ end
45. Workaround # config/puma.rb on_worker_boot do require ‘gcloud’ require ‘gcloud/datastore’ end # config/unicorn.rb after_fork do |server, worker| require ‘gcloud’ require ‘gcloud/datastore’ end
46. Conclusion We can develop Ruby app in GAE smoothly Because of Ruby (&Rails) & GAE’s flexibility It’s not easy to use in critical environment (currently beta support) bug, backward compatibility, know-how, etc. BUT it has large potential Enable us to focus developing service = Enjoy Programming!
47. Thanks for listening! https://www.wantedly.com/companies/mercariapp- com/projects @deme0607