Pro Yearly is on sale from $80 to $50! »

Cloud-agnostic Serverless built with GitLab

8c5062689b241827e7532be41ff19e20?s=47 tnir
January 15, 2020

Cloud-agnostic Serverless built with GitLab

2020-01-15 Tech Play Serverless #2
TECH PLAY SHIBUYA

8c5062689b241827e7532be41ff19e20?s=128

tnir

January 15, 2020
Tweet

Transcript

  1. Cloud-agnostic Serverless built with GitLab @tnir - 2020-01-15 - Tech

    Play Serverless #2 @TECH PLAY SHIBUYA
  2. Disclaimer • すべて個人の見解です。 • The views and opinions represented in

    this presentation are personal to the author of each respective talk and do not represent the views or opinions of any organization unless explicitly stated. All content provided on this presentation is for informational purposes only.
  3. @tnir (Takuya Noguchi) • DevOps Tech Lead, Japan Digital Design

    • Core Team, GitLab • GitLab SuperStar 2018/2019, Superhero 2019 • Cloud Native Ambassador, CNCF • Co-organizer, Docker Tokyo • Founder & organizer, GitLab Tokyo • OSS contributors of multiple projects ◦ kubernetes/website, kubernetes-docs-ja, Django / django-ja, pandas, Mattermost, etc.
  4. DB1 DB2 API Gateway DB3 Suppose that DB 2 is

    not either RDS/Aurora (MySQL, PostgreSQL etc.) or DynamoDB. (cf. RDS Proxy (re:Invent 2019)) Connection pooling problem in this architecture (if with Lambda)
  5. Serverless

  6. 「amazon lambdaの活用事例につい て...」(登壇依頼でのテーマ) • AWS Lambda since 2014 • 5+

    years passed after re:Invent 2014 • → Cloud-agnostic (full) DevOps solution (my personal mission)
  7. Serverless ≠ Lambda

  8. Multi-cloud (cloud-agnostic) • Lambda looks nice! • →Cloud vendor lock-in?

    ◦ →e.g.) 2019/8 Service disruption at AWS Tokyo region ◦ Multiple regions, compliance with laws etc. ◦ Want to use Google Cloud Spanner (GCP) ◦ (Full-managed) Cloud Run GA’ed (Jul 2019): https://medium.com/google-cloud-jp/cloud-run-ga-fb31378cd0a1 • →cloud native ◦ →Kubernetes? ▪ →Kubernetes specialist required, hard Ops • →Extract the common code as framework!!! 2019/8のAWS Tokyo障害: https://aws.amazon.com/jp/message/56489/
  9. https://s.cncf.io

  10. https://twitter.com/gitlab/status/1217133723818184704

  11. GitLab Serverless

  12. GitLab Serverless • Released in Jan 2019 ◦ https://www.publickey1.jp/blog/19/gitlab_serverlessgitlab_116knative.html •

    Status: Alpha (as Jan 2020) • Offerings: 1. Functions (TriggerMesh (Knative) / OpenFaaS) 2. Containers (TriggerMesh (Knative)) 3. AWS Lambda (with Serverless Framework) ← pre-defined runtime ← arbitrary runtime
  13. CI pipeline workflow of Lambda image: node:latest stages: - deploy

    production: stage: deploy before_script: - npm config set prefix /usr/local - npm install -g serverless script: - serverless deploy --stage production --verbose environment: production ↑ Add this to your .gitlab-ci.yml to run CI pipeline workflow. https://gitlab.com/gitlab-org/serverless/examples/serverless-framework-js/blob/c4bab3616b0ccea96c88d8a28a1ca934ff55e0f8/.gitlab-ci.yml with Serverless Framework
  14. CI pipeline workflow of Functions include: template: Serverless.gitlab-ci.yml functions:build: extends:

    .serverless:build:functions environment: production functions:deploy: extends: .serverless:deploy:functions environment: production ↑ Add this to your .gitlab-ci.yml to run CI pipeline workflow. https://gitlab.com/knative-examples/functions/blob/2741e54eb82f882179114590df72cb73074d1c48/.gitlab-ci.yml with Knative (thru gitlabktl) GitLab / OpenFaaS runtimes
  15. CI pipeline workflow of containers Include: template: Serverless.gitlab-ci.yml build: extends:

    .serverless:build:image deploy: extends: .serverless:deploy:image ↑ Add this to your .gitlab-ci.yml to run CI pipeline workflow. https://gitlab.com/knative-examples/knative-ruby-app/blob/ece26c9a98eb0c2cafb70c7904c9dbc35f0a0ded/.gitlab-ci.yml with Knative (thru gitlabktl) Your Dockerfile required
  16. Future of GitLab Serverless 1. Integrated with TriggerMesh’s Knative Lambda

    Runtime to emulate AWS Lambda on anywhere 2. Deeper Knative integration on GCP Anthos 3. (… Separate GitLab Serverless from GitLab itself)
  17. Crossplane integration • Cloud-agnostic ≒ everything prepared by us •

    → Use cloud (managed service) with (some) layer • Crossplane (by Upbound) ◦ Provides multiple-cloud seamless control plane. ◦ Enables independent CockroachDB, Elasticsearch, MongoDB datastore layers (DBaaS) available. https://crossplane.io/
  18. DevOps of “flexible” serverless with Crossplane

  19. Progressive delivery • PD = Extension of continuous delivery (CD)

    with canaries ◦ by LaunchDarkly (2018) and Jenkins X (2019) • Automated with rolling back when key metrics are not fulfilled after deployed • An optional way to easily introduce CD
  20. Observability

  21. Monitoring at one place

  22. Observability: with context with Jaeger

  23. Security

  24. https://blog.shiftleft.io/the-shiftleft-vision-68114e5f5efd

  25. Security - Application Security • Application security on serverless is

    as important as appsec on non-serverless ◦ 4 of top 6 attacks are web-application securities • Higher responsibility on serverless with shared responsibility model • For IAM/Networking, the same as cloud practice • Developer UX (DX): key to widely spread • → Unified & integrated platform is required
  26. User voice from around me • “Hard to see the

    logs!” ◦ https://gitlab.com/gitlab-org/security-products/tests/java-maven/-/jobs/402560464 • “Where is the security dashboard?”
  27. Company-wide (app) security Will be released on Jan 22, 2020

  28. GitLab DAST, same as legacy apps

  29. Uncovered area • Security check for IAM, bucket (S3, GCS)

    policies ◦ → Use Terraform, Ansible, CFn, or whatever you want • Capacity management ◦ → “memory”-based optimization required; no practice (to me) • Chaos engineering framework ◦ → Use general chaos engineering tools
  30. まとめ • クラウド非依存サーバレスへの挑戦 • DevOpsのチームへのアダプション〜ツールの改善の ループを回している(DevOpsツールの継続的改善) • サーバレスも考えることはたくさんなのに、実践からの 抽象化は難しい •

    日本語圏でのGitLabの利用情報が少ないので試して 共有していただけると嬉しい
  31. @tnir Takuya Noguchi Tw: @tn961ir • https://github.com/tnir • https://tnir.gitlab.io/

  32. 懇親会の後半にGitLabグッズをプレゼント(5名)