CyberAgentのDocker/Kubernetes 〜全社横断で見る利用事例〜 / Docker-Kubernetes

サイバーエージェントの技術者（エンジニア・クリエイター）向けカンファレンス『CA BASE CAMP 2019』

CyberAgentのDocker/Kubernetes 〜全社横断で見る利用事例〜
青山 真也

Transcript

1. CyberAgent  Docker / Kubernetes
    
   [  ] @CA BASE CAMP 2019 2019-02-22 amsy810 @amsy810 Masaya Aoyama CyberAgent adtech studio
2. +*CyberAgent2,<=)!%8;E 07: GC   Q. A Core '&61   Q. (#$95?4 Q. 3F"#%   Q. VM 61 /.>  Q. Prod / Stg B@-D etc.
3. K Instagramable Kubernetes
4. 
     -D4 A C h N m 3 3 C3 A 3 # - C 8 D -D4 A C 3 - C cbe a O S T M kgli A 1D C kgli - - # Masaya Aoyama (@amsy810) Infrastructure Engineer   D 3C 3 2 3 A J 3 3 C3 A 3 K D 3C . CD 2 0A83 G A -D4 A C . CD 0A83 G A -D4 d 0A83 G A CA 4DC C 0 1C3 3 -D4 A C  KaaS   /  K8s 
   
5. 1. What is Kubernetes / Docker / Cloud Native? 2. Docker / Kubernetes at CyberAgent
    3. Cloud Native CI/CD with Kubernetes 4. Conclusion
6. What is Kubernetes / Docker / Cloud Native
7. CNCF and The Linux Foundation • Kubernetes The Linux Foundation 
   CNCF  •     
8. Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for everyone. CNCF Cloud Native Defenition v1.0, CNCF, 2018-11-28 (https://github.com/cncf/toc/blob/master/DEFINITION.md) What is “Cloud Native”
9. Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for everyone. CNCF Cloud Native Defenition v1.0, CNCF, 2018-11-28 (https://github.com/cncf/toc/blob/master/DEFINITION.md) • 3$  • ,% • * • (.0  • 15   
   42 "+ !) -  '/(& Open Scalable # What is “Cloud Native”
10. Docker / Container VM7… • %- 4= • 8/ *; • !# 6 • 05 )<' "& • Immutable Infrastructure ,1  • >2($ • :3+  • 9. 
11. • 3*Docker2/ • # $ # •  $ # / $ $ # • "$ # $ • #-.15 • 6&+!$ # • $ • "$# # • $,')%2/ Container Orchestration Engine
    (4/0
12. • XE Docker*" SO • 1 $ ,2.1 • 2.1 / 2"2.1 • 02.1)!2" • 1 $ KLRZ • ^>F /(&2.1 • 2'!%. • 02#%-11 • !2J?D; SO • QT9  + • B34 • SO • 8[M  • :N W67 
    _U IA \ H@] 54 = <G 8V Container Orchestration Engine CY OP
13. Kubernetes >/ Docker 9+$ Kubernetes >/ " 9+$ • "!"<: CRI !"#1 • docker • cri-o • containerd OCI !")1 • runC • gVisor • Kata Containers • Nabla Containers • Firecracker
    63"!" -2 %=. ' "8;*&    ( 
    40,75
14. Kubernetes Docker Kubernetes • D CRI D • docker • cri-o • containerd OCI D • runC • gVisor • Kata Containers • Nabla Containers • Firecracker
    D R
15. Concepts of Kubernetes(9?) Design Principles, Kubernetes Community, 2017-11-04 (https://github.com/kubernetes/community/blob/master/c ontributors/design-proposals/architecture/principles.md) 02 03 01 Self Healing :3$" #)&5< 6Node :3%#) -(/ Automation & Immutable Infrastructure Immutable Infrastructure*> #)' 6Load Balancer+
    !
     Declarative Code & API 4,1.Manifests =0&28 API;7 Infrastructure as Code
16. Key points of Kubernetes • KubernetesGoogle*+)(/  ' %#!$Borg"OSS,  → !
    $#  
    "  • 20, 
    !&1-YAML.  → Infrastructure EngineerOps   Server-side Engineer Dev
17. • Observability • Traffic Shifting • Circuit Break • Fault Injection • Rate Limit • Retry • mTLS Envoy App a Envoy App b Envoy App c *$
    & #+ "%) ! 
      ' (% Service Mesh with Istio
18. CyberAgent 
     Docker / Kubernetes
19. 7 K 7 T - 10 2 V a % 7 e  - A78 6 b
    
20. E AG K E
21. - - -    
    
     
    
        ! • : - - - a • - - - a • - :- - - a I K PEBCG I PL cAKE AG Japan Container Days v18.04 Keynote, MasayaAoyama, 2019-02-15, (http://bit.ly/jkd-keynote-ake-amsy810)
22. Google Kubernetes Engine Managed Kubernetes Serivce Azure Kubernetes Service 2014-11
    2017-02
    2017-11
    Amazon Elastic Container Service for Kubernetes
23. • LM A ig - - - • ac - • S K W L G W • I B l S K - • • be • - I - - - C B
24. - K 50 K 21 3 5 K
25. C 3 2 C 6 2 C 1 3 2 C
26. -W  
     2 -a 4E -W - 3 E E - 3 -W 3 -W 3 
    .es - t1 -a E -W 3 3 4 -W
27. . 2 2 1 .2 1
28. Kubernetes I SLO M PN S A1A9< ) 31:: (  A 5 35<A9:5 51 <:C ) 31:: 5 35 (  A 5 35<A9:5 51 <:C ) 31:: -1 5 135 (  A 5 35<A9:5 51 <:C ) 31:: : A5 ( 
     A 5 35<A9:5 0A1 A :1A5<3C A1A5:5 1< 3 5 :1 :5 (  A 5 35<A9:5 A A1: A A1: 3 <A19<5 < 5 5 < 5 https://github.com/kubernetes/community/blob/master/sig-scalability/slos/slos.md, Kubernetes, 2019-02-15 Kubernetes scalability and performance SLIs/SLOs https://kubernetes.io/docs/setup/cluster-large/, Kubernetes, 2019-02-15 Building Large Cluster - Kubernetes virtual and aggregated resources and Custom Resource Definitions R
29. a t 3 5VM t u I a t e 4 o 4 c r d f d s 5 1 2 12 . d 4 C n
30. A P C Pi R AB R k R i m A R , AB N G e
31. NodePool E GI KI Kubernetes 1.12 n1-standard-8 Kubernetes 1.13 n1-highmem-4 Kubernetes 1.12 n1-highcpu-32 Kubernetes 1.12 n1-ultramem-160 Kubernetes 1.13 n1-standard-8 + 2 GPU
32. NodePool
    
33. NodePool
    
34. Horizontal Pod Autoscaler   
     
35. "! # 
    
    
36.   $# "!%  
    
    
37.     
     
    
38. GKE   
     
    
39.   
       C2 )B / 4 E C TP S 1 (G K ! D 3 2 ) 3 2 ) 3 2 ) 2 ) 2 ) 2 )B A4 ) ./- E C ( 2 ) ?
40. Cloud Native CI/CD with Kubernetes / Docker
41. D D D D D CI CD D Kubernetes D Developer GitOps, Weaveworks, 2018-11-28 (https://www.weave.works/technologies/gitops/) GitOps - Overview
42. 
      CD Kubernetes   GitOps - by Pull Requests
43. (staging) CI CD Staging D Developer staging branch master branch (production) Production GitOps - Staging
44. (staging) CI CD D Developer staging branch master branch (production) Staging Production GitOps - Production
45. (staging) S CI CD Developer staging branch master branch (production) Staging T E Production T E S !$%& #  "  $ R E T ""%&  !%& GitOps
46. Conclusion
47. 8 g ( a nl 2 t od N EK Ers 2 Vt 4 7 18 68 C c 2 1/0C he Tbu 2 4 1 8A id vm KE
48. + + + KubeCon + CloudNativeCon Barcelona 2019 Call for Proposals (CFP) Is Open, CNCF, 2018-11-28 (https://www.cncf.io/blog/2018/11/16/kubecon-barcelona-2019-call-for-proposals-cfp-is-open/)
49. CNCF Community Presentation, CNCF, 2018 (https://github.com/cncf/presentations)
50.     
     Kubernetes  "  !   Kubernetes is also framework for distributed system. CustomResourceDefinition & Controller (Operator) To be continue…
51. Kubernetes +*.  $!()  "& %', #-  
52. 3 I H uNs P /D 6 IBK c a io sy P f s 0 C 1 DD 3 I H W3 I HS M 3 I H & 72 uNs C ID 9 CD H uNs 0BH K L 4 uNs / B 8I uNs /D HI uNs 5 I I uNs uNs N snN s h l o y f fl y s N snr N tk ye & hs c N uNsu ghd v o y /2 /0 fl N s 8 KB 5 HA 3 I H dNkyl 3 I H b
53. @amsy810      Call for Proposal 
    https://bit.ly/cndt2019cfp 
    : Japan Container Days  Co-chair
54. Thank you for your attention follow me on Twitter: @amsy810