Cloud-agnostic Serverless built with GitLab @tnir - 2020-01-15 - Tech Play Serverless #2 @TECH PLAY SHIBUYA 

Disclaimer ● すべて個人の見解です。 ● The views and opinions represented in this presentation are personal to the author of each respective talk and do not represent the views or opinions of any organization unless explicitly stated. All content provided on this presentation is for informational purposes only. 

@tnir (Takuya Noguchi) ● DevOps Tech Lead, Japan Digital Design ● Core Team, GitLab ● GitLab SuperStar 2018/2019, Superhero 2019 ● Cloud Native Ambassador, CNCF ● Co-organizer, Docker Tokyo ● Founder & organizer, GitLab Tokyo ● OSS contributors of multiple projects ○ kubernetes/website, kubernetes-docs-ja, Django / django-ja, pandas, Mattermost, etc. 

DB1 DB2 API Gateway DB3 Suppose that DB 2 is not either RDS/Aurora (MySQL, PostgreSQL etc.) or DynamoDB. (cf. RDS Proxy (re:Invent 2019)) Connection pooling problem in this architecture (if with Lambda) 

Serverless 

「amazon lambdaの活用事例につい て...」(登壇依頼でのテーマ) ● AWS Lambda since 2014 ● 5+ years passed after re:Invent 2014 ● → Cloud-agnostic (full) DevOps solution (my personal mission) 

Serverless ≠ Lambda 

Multi-cloud (cloud-agnostic) ● Lambda looks nice！ ● →Cloud vendor lock-in？ ○ →e.g.) 2019/8 Service disruption at AWS Tokyo region ○ Multiple regions, compliance with laws etc. ○ Want to use Google Cloud Spanner (GCP) ○ (Full-managed) Cloud Run GA’ed (Jul 2019): https://medium.com/google-cloud-jp/cloud-run-ga-fb31378cd0a1 ● →cloud native ○ →Kubernetes？ ■ →Kubernetes specialist required, hard Ops ● →Extract the common code as framework!!! 2019/8のAWS Tokyo障害: https://aws.amazon.com/jp/message/56489/ 

https://s.cncf.io 

https://twitter.com/gitlab/status/1217133723818184704 

GitLab Serverless 

GitLab Serverless ● Released in Jan 2019 ○ https://www.publickey1.jp/blog/19/gitlab_serverlessgitlab_116knative.html ● Status: Alpha (as Jan 2020) ● Offerings: 1. Functions (TriggerMesh (Knative) / OpenFaaS) 2. Containers (TriggerMesh (Knative)) 3. AWS Lambda (with Serverless Framework) ← pre-defined runtime ← arbitrary runtime 

CI pipeline workflow of Lambda image: node:latest stages: - deploy production: stage: deploy before_script: - npm config set prefix /usr/local - npm install -g serverless script: - serverless deploy --stage production --verbose environment: production ↑ Add this to your .gitlab-ci.yml to run CI pipeline workflow. https://gitlab.com/gitlab-org/serverless/examples/serverless-framework-js/blob/c4bab3616b0ccea96c88d8a28a1ca934ff55e0f8/.gitlab-ci.yml with Serverless Framework 

CI pipeline workflow of Functions include: template: Serverless.gitlab-ci.yml functions:build: extends: .serverless:build:functions environment: production functions:deploy: extends: .serverless:deploy:functions environment: production ↑ Add this to your .gitlab-ci.yml to run CI pipeline workflow. https://gitlab.com/knative-examples/functions/blob/2741e54eb82f882179114590df72cb73074d1c48/.gitlab-ci.yml with Knative (thru gitlabktl) GitLab / OpenFaaS runtimes 

CI pipeline workflow of containers Include: template: Serverless.gitlab-ci.yml build: extends: .serverless:build:image deploy: extends: .serverless:deploy:image ↑ Add this to your .gitlab-ci.yml to run CI pipeline workflow. https://gitlab.com/knative-examples/knative-ruby-app/blob/ece26c9a98eb0c2cafb70c7904c9dbc35f0a0ded/.gitlab-ci.yml with Knative (thru gitlabktl) Your Dockerfile required 

Future of GitLab Serverless 1. Integrated with TriggerMesh’s Knative Lambda Runtime to emulate AWS Lambda on anywhere 2. Deeper Knative integration on GCP Anthos 3. (… Separate GitLab Serverless from GitLab itself) 

Crossplane integration ● Cloud-agnostic ≒ everything prepared by us ● → Use cloud (managed service) with (some) layer ● Crossplane (by Upbound) ○ Provides multiple-cloud seamless control plane. ○ Enables independent CockroachDB, Elasticsearch, MongoDB datastore layers (DBaaS) available. https://crossplane.io/ 

DevOps of “flexible” serverless with Crossplane 

Progressive delivery ● PD = Extension of continuous delivery (CD) with canaries ○ by LaunchDarkly (2018) and Jenkins X (2019) ● Automated with rolling back when key metrics are not fulfilled after deployed ● An optional way to easily introduce CD 

Observability 

Monitoring at one place 

Observability: with context with Jaeger 

Security 

https://blog.shiftleft.io/the-shiftleft-vision-68114e5f5efd 

Security - Application Security ● Application security on serverless is as important as appsec on non-serverless ○ 4 of top 6 attacks are web-application securities ● Higher responsibility on serverless with shared responsibility model ● For IAM/Networking, the same as cloud practice ● Developer UX (DX): key to widely spread ● → Unified & integrated platform is required 

User voice from around me ● “Hard to see the logs!” ○ https://gitlab.com/gitlab-org/security-products/tests/java-maven/-/jobs/402560464 ● “Where is the security dashboard?” 

Company-wide (app) security Will be released on Jan 22, 2020 

GitLab DAST, same as legacy apps 

Uncovered area ● Security check for IAM, bucket (S3, GCS) policies ○ → Use Terraform, Ansible, CFn, or whatever you want ● Capacity management ○ → “memory”-based optimization required; no practice (to me) ● Chaos engineering framework ○ → Use general chaos engineering tools 

まとめ ● クラウド非依存サーバレスへの挑戦 ● DevOpsのチームへのアダプション〜ツールの改善の ループを回している（DevOpsツールの継続的改善） ● サーバレスも考えることはたくさんなのに、実践からの 抽象化は難しい ● 日本語圏でのGitLabの利用情報が少ないので試して 共有していただけると嬉しい 

@tnir Takuya Noguchi Tw: @tn961ir ● https://github.com/tnir ● https://tnir.gitlab.io/ 

懇親会の後半にGitLabグッズをプレゼント(5名)