Upgrade to Pro — share decks privately, control downloads, hide ads and more …

쿠버네티스 펀하고 쿨하고 섹시하게 만들기

kakao
December 09, 2022

쿠버네티스 펀하고 쿨하고 섹시하게 만들기

#Kubernetes #ClusterAPI #Kubernetesasaservice

카카오 엔터프라이즈에서는 cluster api와 다양한 plugin을 이용하여, 개발자들에게 용도에 맞는 k8s를 제공하고 있습니다. 그리고 이 환경을 기반으로 모든 클라우드 배포 환경을 k8s를 이용하여 구현할 수 있었습니다.

카카오 엔터프라이즈에서 사용중인 선언적인 kubernetes as a service 개발 방법 공유!, 이것이 선언적인 방법이니까! 음!

발표자 : gonzales.son
카카오엔터프라이즈에서 퍼블릭 클라우드 서비스를 개발하고 있는 벅과 곤잘레스 입니다. 카카오아이클라우드 최고에요~!

buck.hur
카카오엔터프라이즈에서 퍼블릭 클라우드 서비스를 개발하고 있는 벅과 곤잘레스 입니다. 카카오아이클라우드 최고에요~!

kakao

December 09, 2022
Tweet

More Decks by kakao

Other Decks in Programming

Transcript

  1. Copyright 2022. Kakao Corp. All rights reserved. Redistribution or public

    display is not permitted without written permission from Kakao. 쿠버네티스 펀하고, 쿨하고, 섹시하게 손준호 gonzales.son, 허진 buck.hur 카카오엔터프라이즈, 멀티버스팀, 코어파트 if(kakao)2022
  2. 왜 이런 구조로 구성되었나? Master plan K8S 프로비저닝 서비스 KIC

    관리형 k8s 서비스 KIC 리전 프로비저닝/ 인스톨 패키징 KIC 리전 인스톨러
  3. 설명하기에 앞서 - Term Term Description KIC kakao i cloud의

    약자 KIC Console kakao i cloud 리소스 생성 및 확인이 가능한 console dashboard Overcloud KIC Ҋёীѱ ௿ۄ਋٘ ܻࣗझܳ ઁҕೞӝ ਤೠ H/W, S/W੄ ૘೤. IaaS ࢲ࠺झܳ ਤೠ ױة ֎౟ਕ௼ Fabric ҳઑܳ ыח׮. ௿ۄ਋٘ী ੄೧ ٜ݅য૓ ܻࣗझо ઓ੤ೞח ৔৉. IKE ࢲ࠺झ۽ vpc ӝ߈੄ k8s ௿۞झఠ ೐۽࠺੷׬ Undercloud Overcloud ҙܻܳ ਤೠ Managementী ೙ਃೠ H/W, S/W੄ ૘೤. Overcloud৬ ҳ࠙غח ױة ֎౟ਕ௼ Fabric ҳઑܳ ыח׮. ௿ۄ਋٘ܳ ٜ݅ӝ ਤೠ ܻࣗझо ઓ੤ೞח ৔৉. Underground ࢲ࠺झ۽ k8s ௿۞झఠ ೐۽࠺੷׬
  4. Cluster API의 이해 선언적 방식의 cluster lifecycle management Controller patern

    / Reconcile In applications of robotics and automation, a control loop is a non - terminating loop that regulates the state of the system. In Kubernetes, a controller is a control loop that watches the shared state of the cluster through the apiserver and makes changes attempting to move the current state towards the desired state. ୹୊: https://kubernetes.io/docs/concepts/architecture/controller/
  5. Cluster API의 이해 선언적 방식의 cluster lifecycle management Custom Resource

    De fi nition The CustomResourceDe fi nition API resource allows you to de fi ne custom resources. De fi ning a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource. The name of a CRD object must be a valid DNS subdomain name. ୹୊: https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
  6. ୹୊: https://cluster-api.sigs.k8s.io/user/concepts.html Cluster API의 이해 - contollers/CRs Cluster Kubeadm ControlPlane

    InfrastructureCluster kubeadm con fi gtemplate Infrastructure machinetemplate Machine Deployment Infrastructure machinetemplate master worker
  7. ୹୊:https://cluster-api.sigs.k8s.io/developer/architecture/controllers.html Cluster API의 이해 - contollers/CRs Cluster Kubeadm ControlPlane InfrastructureCluster

    kubeadm con fi gtemplate Infrastructure machinetemplate Machine Deployment Infrastructure machinetemplate worker Machine InfrastructureMachine machineset KubeadmCon fi g Machine InfrastructureMachine KubeadmCon fi g Reference Management Core controller Controlplane controller master Bootstrap controller infrastructure controller Spec.clusterName Spec.controlPlaneRef Spec.InfrastructureRef providers
  8. ୹୊:https://cluster-api.sigs.k8s.io/developer/architecture/controllers.html Cluster API의 이해 - contollers/CRs Cluster InfrastructureCluster Infrastructure machinetemplate

    Machine InfrastructureMachine KubeadmCon fi g Reference Management master Spec.clusterName Spec.controlPlaneRef Spec.InfrastructureRef kubeadm con fi gtemplate Machine Deployment Infrastructure machinetemplate worker machineset Machine InfrastructureMachine KubeadmCon fi g Kubeadm ControlPlane Core controller Controlplane controller Bootstrap controller infrastructure controller reconcile()
  9. Kubeadm ControlPlane ୹୊:https://cluster-api.sigs.k8s.io/developer/architecture/controllers.html Cluster API의 이해 - contollers/CRs Cluster InfrastructureCluster

    Infrastructure machinetemplate Machine InfrastructureMachine KubeadmCon fi g Reference Management master Spec.clusterName Spec.controlPlaneRef Spec.InfrastructureRef kubeadm con fi gtemplate Machine Deployment Infrastructure machinetemplate worker machineset Machine InfrastructureMachine KubeadmCon fi g Core controller Controlplane controller Bootstrap controller infrastructure controller reconcile() reconcile()
  10. ୹୊:https://cluster-api.sigs.k8s.io/developer/architecture/controllers.html Cluster API의 이해 - contollers/CRs worker Reference Management Core

    controller master Controlplane controller Bootstrap controller infrastructure controller reconcile() reconcile() Cluster Kubeadm ControlPlane InfrastructureCluster kubeadm con fi gtemplate Infrastructure machinetemplate Machine Deployment Infrastructure machinetemplate Machine InfrastructureMachine machineset KubeadmCon fi g Machine InfrastructureMachine KubeadmCon fi g Spec.clusterName Spec.controlPlaneRef Spec.InfrastructureRef
  11. ୹୊:https://cluster-api.sigs.k8s.io/developer/architecture/controllers.html Cluster API의 이해 - contollers/CRs worker Reference Management Core

    controller master Controlplane controller Bootstrap controller infrastructure controller reconcile() reconcile() Cluster Kubeadm ControlPlane InfrastructureCluster kubeadm con fi gtemplate Infrastructure machinetemplate Machine Deployment Infrastructure machinetemplate Machine InfrastructureMachine machineset KubeadmCon fi g Machine InfrastructureMachine KubeadmCon fi g Spec.clusterName Spec.controlPlaneRef Spec.InfrastructureRef reconcile() reconcile()
  12. ୹୊:https://cluster-api.sigs.k8s.io/developer/architecture/controllers.html Cluster API의 이해 - contollers/CRs worker Reference Management Core

    controller master Controlplane controller Bootstrap controller infrastructure controller reconcile() Cluster Kubeadm ControlPlane InfrastructureCluster kubeadm con fi gtemplate Infrastructure machinetemplate Machine Deployment Infrastructure machinetemplate Machine InfrastructureMachine machineset KubeadmCon fi g Machine InfrastructureMachine KubeadmCon fi g Spec.clusterName
  13. ୹୊:https://cluster-api.sigs.k8s.io/developer/architecture/controllers.html Cluster API의 이해 - contollers/CRs worker Reference Management Core

    controller master Controlplane controller Bootstrap controller infrastructure controller reconcile() Cluster Kubeadm ControlPlane InfrastructureCluster kubeadm con fi gtemplate Infrastructure machinetemplate Machine Deployment Infrastructure machinetemplate Machine InfrastructureMachine machineset KubeadmCon fi g Machine InfrastructureMachine KubeadmCon fi g Spec.clusterName reconcile() reconcile()
  14. ୹୊:https://cluster-api.sigs.k8s.io/developer/crd-relationships.html Cluster API의 이해 - contollers/CRs master worker Reference Management

    Infra provision bootstrap / init / join cluster provision Kubeadm ControlPlane InfrastructureCluster kubeadm con fi gtemplate Infrastructure machinetemplate Machine Deployment Infrastructure machinetemplate Machine InfrastructureMachine machineset KubeadmCon fi g Machine InfrastructureMachine KubeadmCon fi g Cluster
  15. provider openstack IaaS resource control 다양한 infrastructure provider 지원 provider

    metal3 IDC resource control Cluster API의 이해 - Infrastructure provider
  16. 어떻게 서비스화 할까? Kubernetes Cluster API Provider Openstack Provider Metal3

    OpenAPI Rest API CRUD/Auth Cluster provision engine Cluster control Infra Provider Infra resource control
  17. 서비스 Diagram - Underground Dashboard Ring0 Cluster API Provider Metal3

    Undercloud CSP Admin / SVC Developer CMDB Shared Rest API Provisioning request create CR access baremetal Network metadata
  18. 서비스 Diagram - IKE Cluster API Provider Openstack Undercloud Cloud

    User KIC Console Shared request create CR access IKE Mgmt IKE Rest API Overcloud VPC VPC VPC Provisioning kind: Service apiVersion: v1 metadata: ... spec: selector: app: web type: LoadBalancer
  19. 서비스 Diagram - IKE Cloud User Shared k8s api access

    IKE Mgmt VPC VPC VPC Provisioning Control plan VPC service access Undercloud Overcloud
  20. 성숙한 k8s 만들기 Upgrade Undercloud Cloud User Shared IKE Mgmt

    Overcloud 2. Upgrade MachineDeployments Control plan VPC upgrade request 1. Upgrade Control Plane User VPC IKE Rest API IKE Controller CAPI CAPO Console
  21. 앞으로의 계획 - Underground Metadata sync operator 개발 Ring0 Cluster

    API Provider Metal3 CMDB Network metadata Sync operator
  22. 앞으로의 계획 - IKE IKE GPU cluster IKE Mgmt IKE

    Rest API VPC Provisioning Cluster API Provider Openstack VPC VPC GPU VM GPU VM GPU VM
  23. 앞으로의 계획 trivy kube - hunter sigstore vaccine ୹୊ :

    https://blog.sigstore.dev/kubernetes-signals-massive-adoption-of-sigstore-for-protecting-open-source-ecosystem-73a6757da73 Supplychain security강화
  24. Q&A