Beyond Korea with Mixture Cloud

#Global #Hybrid #Kubernetes #Container

카카오에서는 kubernetes 클러스터를 자동화된 방법으로 생성하고 관리할 수 있는 DKOS 라는 클라우드 플랫폼을 자체 개발하여 6년 째 운영해오고 있으며 카카오톡, 카카오택시, 다음, 멜론 등 카카오의 거의 모든 서비스에 적용되어 안정적으로 운영하고 있습니다.

이제 카카오는 Beyond Korea 라는 비전 아래 한국이라는 시작점을 넘어 해외 시장이라는 새로운 땅을 빠르게 개척하고 있습니다. 이러한 기조에 발 맞춰 DKOS도 국내 데이터센터 기반 클라우드의 한계를 뛰어넘어 글로벌 비지니스 지원을 위한 글로벌 클라우드 플랫폼으로의 진화를 하고 있습니다. 자체 구축한 Private Cloud와 Public Cloud와의 Hybrid Cloud 구축을 통해 해외 데이터센터의 컴퓨팅 자원을 제공하고, 더 나아가 이러한 다양한 이기종 클라우드를 하나의 클라우드처럼 구분없이 사용할 수 있는 Mixture Cloud 로의 항해를 시작하였습니다.

본 세션에서는 DKOS가 글로벌 비지니스를 지원하기 위한 Mixture Cloud 로 발전하는 과정에서 고민했던 내용과 직면했던 어려움, 이를 어떻게 해결해나갔는지에 대해서 공유해 보고자 합니다.

발표자 : dennis.hong
카카오에서 클라우드 플랫폼을 개발하고 있는 데니스입니다.

hailey.sol
카카오에서 클라우드 플랫폼을 개발하고 있는 헤일리입니다.

Transcript

1. ഘࢳਊ Dennis.Hong, ӣ׮ࣛ Hailey.Sol
   ஠஠য়
   Copyright 2022. Kakao Corp. All rights reserved. Redistribution or public display is not permitted without written permission from Kakao.
   Beyond Korea with Mixture Cloud
   if(kakao)2022
   Ӗ۽ߥ झாੌ ޸झ୊ ௿ۄ਋٘ ҳ୷ ࢎ۹
   

   View Slide

2. DKOS ࣗѐ
   Mixture Cloud ҳ୷ ߓ҃
   Mixture Cloud ҳ୷ਸ ਤೠ ా೤ ਃࣗ
   - ੋૐ/ӂೠ
   - ֎౟ਕ௼
   - ੋఠಕ੉झ
   Mixture Cloud ജ҃ীࢲ੄ DevOps সޖ ߸ച
   

   View Slide

3. DKOS
   Datacenter of Kakao Operating System
   DKOS ࣗѐ
   

   View Slide

4. ࢎղ ҅੿ ۽Ӓੋ
   Kubernetes cluster ࢤࢿ
   ࢤࢿػ cluster ҙܻ
   (Node ૐࢸ, ߑച߷ न୒ ١)
   ௢ߡ֎౭झ ௿۞झఠܳ ѐߊ੗о ૒੽ ࢤࢿ, ҙܻ
   DKOS : Kubernetes as a Service
   

   View Slide

5. ੹ࢎ ௿ۄ਋٘ ੹ജ ৮ܐ, ੉ઁח Ӗ۽ߥ ഛ੢
   োب ஶప੉ց ӏݽ ઱ਃ ੸ਊ ࢲ࠺झ
   2018 1݅ ѐ ׮਺ ݫੌ, ׏झ
   2019 7݅ ѐ ஠஠য়ఖद, ݮۿ ୶ୌ दझమ
   2020 35݅ ѐ ஠஠য়స ߂ ੹ࢎ ࢲ࠺झ ௿ۄ਋٘ ੹ജ द੘
   2021 100݅ ѐ ੹ࢎ ࢲ࠺झ ௿ۄ਋٘ ੹ജ
   2022 110݅ ѐ+ Ӗ۽ߥ झாੌ ௿ۄ਋٘ ഛ੢
   

   View Slide

6. ੹ࢎ ௿ۄ਋٘ ੹ജ ৮ܐ, ੉ઁח Ӗ۽ߥ ഛ੢
   1M+
   Containers
   100k+
   Servers
   7k+
   Clusters
   

   View Slide

7. ੹ࢎ ௿ۄ਋٘ ੹ജ ৮ܐ, ੉ઁח Ӗ۽ߥ ഛ੢
   ߊ಴ ઱ઁ : Surviving From Endless Issues Coming From 7K+ Kubernetes Clusters
   

   View Slide

8. DKOS ࣗѐ
   Mixture Cloud ҳ୷ ߓ҃
   Mixture Cloud ҳ୷ਸ ਤೠ ా೤ ਃࣗ
   - ੋૐ/ӂೠ
   - ֎౟ਕ௼
   - ੋఠಕ੉झ
   Mixture Cloud ജ҃ীࢲ੄ DevOps সޖ ߸ച
   

   View Slide

9. Beyond Korea
   Mixture Cloud ҳ୷ ߓ҃
   

   View Slide

10. Hybrid Cloud
    Mixture Cloud ҳ୷ ߓ҃
    Public Cloud
    (ex. AWS, GCP, Azure …)
    Private Cloud
    (Kakao Cloud)
    

    View Slide

11. Cloud Native
    Kubernetes Native
    Mixture Cloud ҳ୷ ߓ҃
    গ೒ܻா੉࣌ ੉धࢿ, ഐജࢿ ࠁ੢
    ௿ۄ਋٘ ઁҕ੗ ઙࣘࢿ ઁѢ
    

    View Slide

12. Mixture Cloud ҳ୷ ߓ҃
    Cloud Native
    Kubernetes Native
    GCP
    AWS
    Azure
    Kakao
    

    View Slide

13. Cloud Native
    Kubernetes Native
    Mixture Cloud ҳ୷ ߓ҃
    GCP
    AWS
    Azure
    Kakao
    

    View Slide

14. Mixture Cloud ҳ୷ ߓ҃
    ೞ੉࠳ܻ٘ ௿ۄ਋٘੄ ইए਍ ੼
    1. ҅੿/ӂೠ/੿଼ ҙܻ য۰਑
    2. ௿ۄ਋٘߹ ೟ण ࠺ਊ
    3. ௿ۄ਋٘߹ ҙܻ ࠗ׸
    

    View Slide

15. Mixture Cloud ҳ୷ ߓ҃
    GCP
    AWS
    Azure
    Kakao
    Mixture
    Cloud
    User
    Mixture Cloud
    Kakao
    

    View Slide

16. DKOS ࣗѐ
    Mixture Cloud ҳ୷ ߓ҃
    Mixture Cloud ҳ୷ਸ ਤೠ ా೤ ਃࣗ
    - ੋૐ/ӂೠ
    - ֎౟ਕ௼
    - ੋఠಕ੉झ
    Mixture Cloud ജ҃ীࢲ੄ DevOps সޖ ߸ച
    

    View Slide

17. ੋૐ/ӂೠ
    ֎౟ਕ௼
    ੋఠಕ੉झ
    Mixture Cloud ҳ୷ਸ ਤೠ ా೤ ਃࣗ
    

    View Slide

18. ੋૐ/ӂೠ
    Mixture Cloud ীࢲ੄ ੋૐ/ӂೠ
    

    View Slide

19. Public Cloud
    (ex. AWS, GCP, Azure …)
    user
    Mixture DKOS
    ੋૐ/ӂೠ ҙ۲ ӝמ
    ?
    ?
    Manager
    ?
    

    View Slide

20. Public Cloud KCP
    Public Cloud ӝמ ୭؀ೠ ഝਊ ૒੽ ѐߊೠ ӝמ ୭؀ೠ ഝਊ
    VS
    য٣ীࢲ ੋૐ/ӂೠਸ ҙܻೡө?
    

    View Slide

21. KCP(KaKao Cloud Portal) Services
    

    View Slide

22. KCP SSO KCP IAM
    KCP Single Sign On KCP Identity Access Management
    

    View Slide

23. KCP SSO
    - KaKao Cloudܳ ਤೠ Single Sign On
    - OAuth 2.0ਸ ҳഅೠ ੋૐ ࢲ࠺झ
    - KCP ੋૐ ష௾ਸ ࢎਊೞৈ ݽٚ API ੋૐ оמ
    

    View Slide

24. KCP IAM
    - KaKao Cloudܳ ਤೠ ӂೠ ҙܻ ࢲ࠺झ
    - п ࢲ࠺झ੄ ӂೠ ҙܻܳ ೠҔীࢲ оמೞѱ ೧ષ
    - KCP ੋૐ ష௾ਸ ӝ߈ਵ۽ ӂೠ ҙܻ ߂ ੋо ઁҕ
    project01
    project02
    project07
    project06
    project05
    project04
    project03
    

    View Slide

25. Public Cloud ੋૐ/ӂೠ KCP ੋૐ/ӂೠ
    ӝמ ݏ୺ച ࠛоמ оמ
    ഛ੢ࢿ ծ਺ ֫਺
    ѐߊ ࠙۝ ੸਷ ಞ ݆਷ ಞ
    ೟ण ࠺ਊ ֫਺ ࠁా
    

    View Slide

26. Public Cloud ੋૐ/ӂೠ KCP ੋૐ/ӂೠ
    ӝמ ݏ୺ച ࠛоמ оמ
    ഛ੢ࢿ ծ਺ ֫਺
    ѐߊ ࠙۝ ੸਷ ಞ ݆਷ ಞ
    ೟ण ࠺ਊ ֫਺ ࠁా
    

    View Slide

27. Public Cloud ੋૐ/ӂೠ KCP ੋૐ/ӂೠ
    ӝמ ݏ୺ച ࠛоמ оמ
    ഛ੢ࢿ ծ਺ ֫਺
    ѐߊ ࠙۝ ੸਷ ಞ ݆਷ ಞ
    ೟ण ࠺ਊ ֫਺ ࠁా
    

    View Slide

28. Public Cloud ੋૐ/ӂೠ KCP ੋૐ/ӂೠ
    ӝמ ݏ୺ച ࠛоמ оמ
    ഛ੢ࢿ ծ਺ ֫਺
    ѐߊ ࠙۝ ੸਷ ಞ ݆਷ ಞ
    ೟ण ࠺ਊ ֫਺ ࠁా
    

    View Slide

29. Public Cloud ੋૐ/ӂೠ KCP ੋૐ/ӂೠ
    ӝמ ݏ୺ച ࠛоמ оמ
    ഛ੢ࢿ ծ਺ ֫਺
    ѐߊ ࠙۝ ੸਷ ಞ ݆਷ ಞ
    ೟ण ࠺ਊ ֫਺ ࠁా
    

    View Slide

30. Public Cloud ੋૐ/ӂೠ KCP ੋૐ/ӂೠ
    ӝמ ݏ୺ച ࠛоמ оמ
    ഛ੢ࢿ ծ਺ ֫਺
    ѐߊ ࠙۝ ੸਷ ಞ ݆਷ ಞ
    ೟ण ࠺ਊ ֫਺ ࠁా
    

    View Slide

31. Public Cloud
    (AWS, GCP, Azure …)
    ETC…
    KCP Service
    Mixture Cloud
    

    View Slide

32. Public Cloud
    (AWS, GCP, Azure …)
    ETC…
    KCP Service
    Mixture Cloud
    

    View Slide

33. Public Cloud ੋૐ/ӂೠ KCP ੋૐ/ӂೠ
    ӝמ ݏ୺ച ࠛоמ оמ
    ഛ੢ࢿ ծ਺ ֫਺
    ѐߊ ࠙۝ ੸਷ ಞ ݆਷ ಞ
    ೟ण ࠺ਊ ֫਺ ࠁా
    

    View Slide

34. HOW?
    “Ӓېࢲ যڌѱ ੋૐ/ӂೠ ҙܻܳ ೠ׮ח Ѥоਃ?”
    

    View Slide

35. Public Cloud
    (ex. AWS, GCP, Azure …)
    Mixture IAM
    KCP SSO
    user
    KCP IAM
    Mixture DKOS
    Manager
    

    View Slide

36. Public Cloud
    (ex. AWS, GCP, Azure …)
    Mixture IAM
    KCP SSO
    user
    Mixture DKOS
    Manager
    KCP IAM
    

    View Slide

37. Public Cloud
    (ex. AWS, GCP, Azure …)
    Mixture IAM
    user
    KCP IAM
    Mixture DKOS
    KCP SSO
    Manager
    KCP ష௾
    

    View Slide

38. Manager
    Public Cloud
    (ex. AWS, GCP, Azure …)
    Mixture IAM
    KCP SSO
    user
    KCP IAM
    Mixture DKOS
    KCP ష௾
    

    View Slide

39. Public Cloud
    (ex. AWS, GCP, Azure …)
    KCP SSO
    user
    KCP IAM
    Mixture DKOS Mixture IAM
    Manager
    KCP ష௾
    

    View Slide

40. Public Cloud
    (ex. AWS, GCP, Azure …)
    Mixture IAM
    KCP SSO
    user
    Mixture DKOS KCP IAM
    OR
    Manager
    KCP ష௾
    

    View Slide

41. Public Cloud
    (ex. AWS, GCP, Azure …)
    KCP SSO
    user
    KCP IAM
    Mixture DKOS Mixture IAM
    Manager
    Public Cloud ష௾
    

    View Slide

42. Public Cloud
    (ex. AWS, GCP, Azure …)
    KCP SSO
    user
    KCP IAM
    Mixture DKOS Mixture IAM
    Manager
    Public Cloud ష௾
    

    View Slide

43. Public Cloud
    (ex. AWS, GCP, Azure …)
    KCP SSO
    user
    KCP IAM
    Mixture DKOS Mixture IAM
    Manager
    Public Cloud ష௾
    

    View Slide

44. DKOS ࣗѐ
    Mixture Cloud ҳ୷ ߓ҃
    Mixture Cloud ҳ୷ਸ ਤೠ ా೤ ਃࣗ
    - ੋૐ/ӂೠ
    - ֎౟ਕ௼
    - ੋఠಕ੉झ
    Mixture Cloud ജ҃ীࢲ੄ DevOps সޖ ߸ച
    

    View Slide

45. Mixture Cloud ҳ୷ : ֎౟ਕ௼
    

    View Slide

46. DKOS ࣗѐ
    Mixture Cloud ҳ୷ ߓ҃
    Mixture Cloud ҳ୷ਸ ਤೠ ా೤ ਃࣗ
    - ੋૐ/ӂೠ
    - ֎౟ਕ௼
    - ੋఠಕ੉झ
    Mixture Cloud ജ҃ীࢲ੄ DevOps সޖ ߸ച
    

    View Slide

47. Mixture Cloud ҳ୷ : ੋఠಕ੉झ
    Kakao
    Provider
    AWS
    Provider
    Legacy
    Provider
    Kakao
    Cloud Portal
    Mixture Cloud
    API
    Deployer
    

    View Slide

48. Mixture Cloud ҳ୷ : ੋఠಕ੉झ
    Kakao
    Provider
    AWS
    Provider
    Legacy
    Provider
    Kakao
    Cloud Portal
    Mixture Cloud
    API
    Deployer
    

    View Slide

49. Mixture Cloud ҳ୷ : ੋఠಕ੉झ
    Kakao
    Provider
    AWS
    Provider
    Legacy
    Provider
    Kakao
    Cloud Portal
    Mixture Cloud
    API
    Deployer
    

    View Slide

50. Mixture Cloud ҳ୷ : ੋఠಕ੉झ
    ࢎղ ҅੿ ۽Ӓੋ
    ௿۞झఠ ࢤࢿद
    AWS us-west-1 ܻ੹ ࢶఖ DKOS ௿۞झఠ ࢤࢿ ৮ܐ
    

    View Slide

51. Mixture Cloud ҳ୷ : ੋఠಕ੉झ
    पઁ۽ח AWS ী ࢤࢿغযઉ ੓ח EKS ௿۞झఠ
    

    View Slide

52. Mixture Cloud ҳ୷ : ੋఠಕ੉झ
    ௿۞झఠ ֢٘ Ӓܛ ࢤࢿ, ֢٘ ୶о DKOS ীࢲ ഛੋغח ֢٘ ୶о Ѿҗ
    

    View Slide

53. Mixture Cloud ҳ୷ : ੋఠಕ੉झ
    पઁ۽ח AWS ী ࢤࢿغযઉ ੓ח Node Groupҗ Node
    

    View Slide

54. Mixture Cloud ҳ୷ : ੋఠಕ੉झ
    पઁ۽ח AWS ী ࢤࢿغযઉ ੓ח EC2 VM
    

    View Slide

55. DKOS ࣗѐ
    Mixture Cloud ҳ୷ ߓ҃
    Mixture Cloud ҳ୷ਸ ਤೠ ా೤ ਃࣗ
    - ੋૐ/ӂೠ
    - ֎౟ਕ௼
    - ੋఠಕ੉झ
    Mixture Cloud ജ҃ীࢲ੄ DevOps সޖ ߸ച
    

    View Slide

56. Mixture Cloud о হח ജ҃ীࢲ੄ DevOps
    ѐߊ੗
    ੋ೐ۄ ਃ୒
    DevOps
    ׸׼੗
    "௢ߡ֎౭झ ௿۞झఠ ઱ࣁਃ~!"
    

    View Slide

57. Mixture Cloud о হח ജ҃ীࢲ੄ DevOps
    ѐߊ੗ DevOps
    ׸׼੗
    AWS ҅੿
    GCP ҅੿
    Azure ҅੿
    ੋ೐ۄ ਃ୒
    "௢ߡ֎౭झ ௿۞झఠ ઱ࣁਃ~!"
    

    View Slide

58. Mixture Cloud о হח ജ҃ীࢲ੄ DevOps
    GCP
    AWS
    Azure
    ѐߊ੗
    ௢ߡ֎౭झ
    ੋૐ ੿ࠁ
    DevOps
    ׸׼੗
    AWS ҅੿
    GCP ҅੿
    Azure ҅੿
    AWS UI
    GCP UI
    Azure UI
    ੋ೐ۄ ਃ୒
    "௢ߡ֎౭झ ௿۞झఠ ઱ࣁਃ~!"
    

    View Slide

59. Mixture Cloud о হח ജ҃ীࢲ੄ DevOps
    Inhouse
    IDC
    GCP
    AWS
    Azure
    ѐߊ੗ DevOps
    ׸׼੗
    AWS ҅੿
    GCP ҅੿
    Azure ҅੿
    AWS UI
    ੋ೐ۄ ׸׼੗ Inhouse System UI
    GCP UI
    Azure UI
    ੋ೐ۄ ਃ୒
    ੋ೐ۄ ਃ୒
    "௢ߡ֎౭झ ௿۞झఠ ઱ࣁਃ~!"
    ௢ߡ֎౭झ
    ੋૐ ੿ࠁ
    

    View Slide

60. Mixture Cloud о হח ജ҃ীࢲ੄ DevOps
    Inhouse
    IDC
    GCP
    AWS
    Azure
    ѐߊ੗
    AWS ҅੿
    GCP ҅੿
    Azure ҅੿
    AWS UI
    ੋ೐ۄ ׸׼੗ Inhouse System UI
    GCP UI
    Azure UI
    ֎౟ਕ௼ ߂ ߑച߷
    ੿଼
    ֎౟ਕ௼
    /ߑച߷ ੿଼
    ֎౟ਕ௼
    /ߑച߷ ੿଼
    ֎౟ਕ௼
    /ߑച߷ ੿଼
    ௢ߡ֎౭झ
    ੋૐ ੿ࠁ
    

    View Slide

61. Mixture Cloud ജ҃ীࢲ੄ DevOps
    Kakao
    IDC
    GCP
    AWS
    Azure
    ѐߊ੗
    Mixture Cloud (DKOS)
    ࢎղ ҅੿
    

    View Slide

62. Mixture Cloud ജ҃ীࢲ੄ DevOps
    Kargo : SaaS ഋ Kubernetes Native CI/CD
    ઱ਃ ӝמ
    - Progressive Delivery (Blue/Green, Canary ߓನ)
    - CI/CD ౵੉೐ۄੋ ҙܻ
    - ߓನ੹ ࢸ੿ ࠺Ү Ѩష
    - ߓನ ൤झషܻ ҙܻ ߂ ܀ߔ
    - ߓನ थੋ
    *Kargo = Kakao + Cargo ship(ചޛࢶ)
    

    View Slide

63. Mixture Cloud ജ҃ীࢲ੄ DevOps
    ѐߊ੗
    Mixture Cloud (Kargo)
    ࢎղ ҅੿
    Kakao
    IDC
    GCP
    AWS
    Azure
    

    View Slide

64. DKOS ࣗѐ
    Mixture Cloud ҳ୷ ߓ҃
    Mixture Cloud ҳ୷ਸ ਤೠ ా೤ ਃࣗ
    - ੋૐ/ӂೠ
    - ֎౟ਕ௼
    - ੋఠಕ੉झ
    Mixture Cloud ജ҃ীࢲ੄ DevOps সޖ ߸ച
    

    View Slide