Apps Without Ops - Web Development in Google Cloud Platform

Google Cloud Platform is a suite of cloud computing services that runs on the same infrastructure that Google uses internally for its end-user products. This talk will provide a brief overview of tools like App Engine, Cloud SQL, Crons, Cloud Shell, Storage and more to help web developers move toward an environment where they can focus on building applications, and spend less time worrying about dev-ops.

Transcript

1. APPS WITHOUT OPS Web Development in Google Cloud Platform
2. HOWDY My name is Chris Deaton I do web applications at NC State @dcdeaton || @flashadvocate 2 Chris Deaton | NC State University
3. 3 Chris Deaton | NC State University
4. 4 Chris Deaton | NC State University
5. 5 Chris Deaton | NC State University
6. WHAT IS GOOGLE CLOUD PLATFORM? • Google Cloud Platform is a suite of cloud computing services • Runs on the same infrastructure that Google uses internally for its end-user products, such as Google Search and YouTube • App Engine (PaaS), specifically the standard environment 6 Chris Deaton | NC State University
7. THE AGENDA • Why we went with GCP • Development in the cloud • Source code examples • Tips for the road 7 Chris Deaton | NC State University
8. BUT FIRST, YOU SHOULD KNOW • GAE Standard supports many languages... • Go, Java, .NET, Node.js, PHP, Python, Ruby • This presentation focuses on GCP, not GSuite • This is not a deep dive 8 Chris Deaton | NC State University
9. WHY WE WENT WITH GOOGLE CLOUD PLATFORM 9 Chris Deaton | NC State University
10. PROBLEMS WE WANTED TO SOLVE • Development platform dependence • The need for greater reliability, scalability • Remove Reduce the need for dev-ops • Inconsistent deployment strategies 10 Chris Deaton | NC State University
11. HOW GCP SOLVES THEM • Platform dependence: Develop on any machine with Shell, Editor • Scalability, reliability: Cloud SQL failover replicas, backups • Less dev-ops: Deploy code to GAE, and you're done • Deployment strategies: App engine versioning for dev-test-prod 11 Chris Deaton | NC State University
12. PLUS... LOTS OF COOL STUFF COMPUTE NETWORKING DATABASE SECURITY MANAGEMENT DEV TOOLS Compute Engine Load Balancing Cloud Storage Resource Manager Stackdriver Cloud SDK App Engine Cloud DNS Cloud SQL IAM Logging Cloud Shell Kubernetes Engine Private Cloud Cloud Datastore Security Scanner Error reporting Editor Cloud container Firebase Cloud KMS Cloud console Cloud Build App Engine 12 Chris Deaton | NC State University
13. COMPARISONS TO AWS • Long-term discounts (after 1 month vs AWS' 1 year) • Customizable virtual machine configurations • Full-scale encryption of all data and comms channels (thanks NSA!)2 • Pricing calculator: https:/ /cloud.google.com/products/calculator/ 2 Snowden documents reveal breach - https:/ /www.bbc.com/news/world-us-canada-24751821 13 Chris Deaton | NC State University
14. DEVELOPMENT IN THE CLOUD 14 Chris Deaton | NC State University
15. APP ENGINE • Supports a variety of runtimes, or you can bring your own • Read-only environment (except /tmp) • Easily versioned during deployment • Traffic splitting for a/b, testing • Standard, Flexible environments 15 Chris Deaton | NC State University
16. STACKDRIVER • Monitoring • Collect metrics, events, and metadata from GCP, AWS, etc • Monitoring agent (collectd) • Logging • Store, search, alert on log data and events (GCP, AWS) • Use a logging agent (fluentd) or logging libraries • Error reporting (30 day retention) 16 Chris Deaton | NC State University
17. CLOUD SHELL • Command-line access to cloud resources via the browser • Management of projects, storage, deployments • 5GB of persistent disk storage mounted as $HOME • Deleted after 120 days of inactivity 17 Chris Deaton | NC State University
18. 18 Chris Deaton | NC State University
19. CLOUD SDK • Set of development tools for Cloud Platform • Contains gcloud, gsutil, and bq • Includes local service emulators 19 Chris Deaton | NC State University
20. # Set the project of interest $ gcloud config set project hello-world-314 # Create a Cloud SQL instance $ gcloud sql instances create my-instance $ gcloud sql users set-password root % --instance my-instance --password [PASSWORD] $ gcloud sql connect my-instance --user=root # Create a Cloud Storage Bucket $ gsutil mb gs://my-bucket-555 $ gsutil cp test.dat gs://my-bucket-555 # Deploy to App engine $ gcloud app deploy --version=test --no-promote 20 Chris Deaton | NC State University
21. # Set the project of interest $ gcloud config set project hello-world-314 # Create a Cloud SQL instance $ gcloud sql instances create my-instance $ gcloud sql users set-password root % --instance my-instance --password [PASSWORD] $ gcloud sql connect my-instance --user=root # Create a Cloud Storage Bucket $ gsutil mb gs://my-bucket-555 $ gsutil cp test.dat gs://my-bucket-555 # Deploy to App engine $ gcloud app deploy --version=test --no-promote 21 Chris Deaton | NC State University
22. # Set the project of interest $ gcloud config set project hello-world-314 # Create a Cloud SQL instance $ gcloud sql instances create my-instance $ gcloud sql users set-password root % --instance my-instance --password [PASSWORD] $ gcloud sql connect my-instance --user=root # Create a Cloud Storage Bucket $ gsutil mb gs://my-bucket-555 $ gsutil cp test.dat gs://my-bucket-555 # Deploy to App engine $ gcloud app deploy --version=test --no-promote 22 Chris Deaton | NC State University
23. # Set the project of interest $ gcloud config set project hello-world-314 # Create a Cloud SQL instance $ gcloud sql instances create my-instance $ gcloud sql users set-password root % --instance my-instance --password [PASSWORD] $ gcloud sql connect my-instance --user=root # Create a Cloud Storage Bucket $ gsutil mb gs://my-bucket-555 $ gsutil cp test.dat gs://my-bucket-555 # Deploy to App engine $ gcloud app deploy --version=test --no-promote 23 Chris Deaton | NC State University
24. # Start a deployment instance of the application server $ dev_appserver.py ./ • https://8080-dot-2958221-dot-devshell.appspot.com INFO 2018-09-27 02:07:41,105 devappserver2.py:281] Skipping SDK update check. INFO 2018-09-27 02:07:41,202 api_server.py:275] Starting API server at: http://0.0.0.0:38391 INFO 2018-09-27 02:07:41,214 dispatcher.py:270] Starting module "default" running at: http://0.0.0.0:8080 INFO 2018-09-27 02:07:41,215 admin_server.py:152] Starting admin server at: http://0.0.0.0:8000 INFO 2018-09-27 02:07:42,938 instance.py:294] Instance PID: 373 24 Chris Deaton | NC State University
25. CLOUD EDITOR • Based on Orion, a web-based editor • Currently still beta (has been for a year or so) • Supports syntax-coloring, light and dark themes • Customization of overall appearance • Basic language linting 25 Chris Deaton | NC State University
26. 26 Chris Deaton | NC State University
27. CLOUD SQL • Fully managed database service • Supports MySQL and PostgreSQL • Scalable up to 10TB, 40,000 IOPS, 416GB RAM per instance • Proxy for direct connections via workstation 27 Chris Deaton | NC State University
28. 28 Chris Deaton | NC State University
29. 29 Icons made by Google - https:/ /cloud.google.com/icons/
30. CONFIGURING FOR GAE DEPLOYMENT • app.yaml • Provides runtime, routes, env variables • cron.yaml • Define cron schedules • 20 tasks for non-billed, 250 for billed • .gcloudignore 30 Chris Deaton | NC State University
31. # app.yaml runtime: php55 api_version: 1 handlers: - url: / script: home.php - url: /stylesheets static_dir: stylesheets - url: /(.*\.(gif|png|jpg))$ static_files: static/\1 upload: static/.*\.(gif|png|jpg)$ # admin = primitive role of project browser, editor, owner, viewer - url: /admin/.* script: admin.php login: admin 31 Chris Deaton | NC State University
32. # cron.yaml cron: - description: "daily summary job" url: /tasks/summary schedule: every 24 hours - description: "monday morning mailout" url: /mail/weekly schedule: every monday 09:00 timezone: Australia/NSW - description: "new daily summary job" url: /tasks/summary schedule: every 24 hours target: beta 32 Chris Deaton | NC State University
34. SOURCE CODE EXAMPLES 34 Chris Deaton | NC State University
35. 35 Chris Deaton | NC State University
36. # main.app # https://webapp2.readthedocs.io/en/latest/ import os from flask import Flask import redis app = Flask(__name__) redis_host = os.environ.get('REDIS_HOST', 'localhost') redis_port = int(os.environ.get('REDIS_PORT', 6379)) redis_password = os.environ.get('REDIS_PASSWORD', None) redis_client = redis.StrictRedis( host=redis_host, port=redis_port, password=redis_password) @app.route('/') def index(): value = redis_client.incr('counter', 1) return 'Value is {}'.format(value) 36 Chris Deaton | NC State University
37. # app.yaml # https://cloud.google.com/appengine/docs/standard/python/config/appref runtime: python37 env_variables: REDIS_HOST: your-redis-host REDIS_PORT: your-redis-port REDIS_PASSWORD: your-redis-password 37 Chris Deaton | NC State University
38. 38 Chris Deaton | NC State University
39. // index.php require 'vendor/autoload.php'; $app = new Slim\App(); $app->get('/', function ($request, $response) { $name = $request->getQueryParams()['name'] ?? 'World'; return $response->getBody()->write("Hello, {$name}!"); }); $app->run(); 39 Chris Deaton | NC State University
40. # app.yaml # PHP72 runtime requires application front-controller # All requests go to index.php or public/index.php runtime: php72 40 Chris Deaton | NC State University
41. 41 Chris Deaton | NC State University
42. // bootstrap/app.php // override default storage location with env attribute $app->useStoragePath( env('APP_STORAGE', base_path() . '/storage') ); 42 Chris Deaton | NC State University
43. // config/view.php return [ 'paths' => [ resource_path('views'), ], // store compiled view files in storage 'compiled' => storage_path(), ]; 43 Chris Deaton | NC State University
44. # app.yaml runtime: php72 # values normally stored in .env env_variables: APP_LOG: errorlog APP_KEY: YOUR_APP_KEY APP_STORAGE: /tmp CACHE_DRIVER: database SESSION_DRIVER: database DB_DATABASE: YOUR_DB_DATABASE DB_USERNAME: YOUR_DB_USERNAME DB_PASSWORD: YOUR_DB_PASSWORD DB_SOCKET: "/cloudsql/YOUR_CLOUDSQL_CONNECTION_NAME" 44 Chris Deaton | NC State University
45. TIPS FOR THE ROAD 45 Chris Deaton | NC State University
46. • For Laravel deployments, versioning app.yaml can be problematic • Check out beyondcode/laravel-credentials • Also consider Cloud KMS for managing app secrets • Working across GCP projects? • Don't forget permissions (IAM) and API access 46 Chris Deaton | NC State University
47. # PSR-3 Logging on GAE Standard /** * Logs written to stdout and stderr are automatically sent to * Stackdriver Logging for you, without needing to use * Stackdriver Logging library for PHP. */ use Google\Cloud\Logging\LoggingClient; // enable batch to prevent RPC latency $logger = LoggingClient::psrBatchLogger('app'); // $logging = new LoggingClient(); // $logger = $logging->psrLogger('app', ['batchEnabled' => true]); $logger->info('This will show up as log level INFO'); $logger->warning('This will show up as log level WARNING'); $logger->error('This will show up as log level ERROR'); 47 https:/ /github.com/GoogleCloudPlatform/php-docs-samples/tree/master/appengine/php72/errorreporting
48. USEFUL LINKS • GAE examples for PHP • https:/ /github.com/GoogleCloudPlatform/php-docs-samples • GAE examples for Python • https:/ /github.com/GoogleCloudPlatform/python-docs-samples • Google Cloud App Engine Docs • https:/ /cloud.google.com/appengine/docs • google/cloud PHP client library • https:/ /googleapis.github.io/google-cloud-php/ • GCP Live - Youtube Channel • https:/ /www.youtube.com/channel/UC7ZAfUgm7xIXzzRPltMnFzQ 48 Chris Deaton | NC State University