Microservices on Kubernetes

Transcript

1. ©2018 Wantedly, Inc. Microservices on Kubernetes Convention, Enforcement, and Migration

   Kubernetes & Cloud Native Meetup 25.Oct.2018 - Shimpei Otsubo @potsbo

2. ©2018 Wantedly, Inc. 4IJNQFJ
   0UTVCP
   !QPUTCP Infrastructure Team, Wantedly, Inc. 2018 New

   Grads Kubernetes Dev Tools CI / CD AuthN / Z Productivity Dvorak

3. ©2018 Wantedly, Inc. All Services on 70 various sizes of

   microservices More than 2 per engineer

4. ©2018 Wantedly, Inc. Docker for 4 yrs. We have been

   early adaptors Kubernetes for 2.5 yrs. We have own best practices To manage this # of microservices

5. ©2018 Wantedly, Inc. Enforcement Conventions Migration Summary

6. ©2018 Wantedly, Inc. Enforcement Conventions Migration Summary

7. ©2018 Wantedly, Inc. Convention Bootstrap fast, maintain only logic one

   namespace for repo company-wide convention each env (production, qa, sandbox) has a Kubernetes cluster same third-party services across all services commit hash as image tag automatic deploy after PR merged for most apps configuration build/deploy monitoring etc…

8. ©2018 Wantedly, Inc. The Latest k8s ops flow Fully managed

   set image check pull deploy push push hook build build

9. ©2018 Wantedly, Inc. commit hash as docker image tag GitHub

   commit status shows docker image is ready Almost any commit is deployable push push hook build build Fully managed The Latest k8s ops flow

10. ©2018 Wantedly, Inc. `kube prod deploy master` to deploy Wrapper

    to invoke `kubectl set image` Use same image in every purpose prod, qa, and dev set image check pull deploy Fully managed The Latest k8s ops flow Automatic deploy to production

11. ©2018 Wantedly, Inc. Conventions make development productive •commit hash as

    tag •one namespace for one repo

12. ©2018 Wantedly, Inc. Enforcement Conventions Migration Summary

13. ©2018 Wantedly, Inc. Enforcement Library to install everything you need

    to build an app servicex • Log format and collection • Error collection • Performance monitoring • Distributed Tracing • Client Library for other services Following the rule has to be the easiest way Available in Golang, Ruby, Python Ideally less than an hour to deploy to the production a bootstrap document to launch a new app

14. ©2018 Wantedly, Inc. Enforcement everything with kube command Following the

    rule has to be the easiest way In-house build/deploy tool Same tools in CI and laptops Automation is transparent to developers kube build kube prod deploy master to build and push a docker image to deploy to production to generate manifest files auto generated manifests kube generate autoscale

15. ©2018 Wantedly, Inc. language: bash env: - // encrypted credentials

    here install: - bash <(curl -sL https://get.wantedlyapp.com/kube) script: - kube build deploy: - skip_cleanup: true provider: script script: kube prod deploy $TRAVIS_COMMIT on: branch: master Every command works in local laptops

16. ©2018 Wantedly, Inc. language: bash env: - // encrypted credentials

    here install: - bash <(curl -sL https://get.wantedlyapp.com/kube) script: - kube build deploy: - skip_cleanup: true provider: script script: kube prod deploy $TRAVIS_COMMIT on: branch: master Every command works in local laptops Install kube

17. ©2018 Wantedly, Inc. language: bash env: - // encrypted credentials

    here install: - bash <(curl -sL https://get.wantedlyapp.com/kube) script: - kube build deploy: - skip_cleanup: true provider: script script: kube prod deploy $TRAVIS_COMMIT on: branch: master Every command works in local laptops Install kube build and push a docker image

18. ©2018 Wantedly, Inc. language: bash env: - // encrypted credentials

    here install: - bash <(curl -sL https://get.wantedlyapp.com/kube) script: - kube build deploy: - skip_cleanup: true provider: script script: kube prod deploy $TRAVIS_COMMIT on: branch: master deploy the revision Install kube build and push a docker image Every command works in local laptops

19. ©2018 Wantedly, Inc. Conventions make development productive Enforcement to follow

    the conventions •commit hash as tag •servicex a library to install everything •kube in CI and laptops •one namespace for one repo

20. ©2018 Wantedly, Inc. Enforcement Conventions Migration Summary

21. ©2018 Wantedly, Inc. Migration We migrated the biggest legacy app

    to k8s Dockerfile from scratch Faster Build Hack assets:precompile QA/Dev first Communication with the backend team Drop fast-rollback feature Drop automatic deploy As test for both Infra team and the backend team

22. ©2018 Wantedly, Inc. build Legacy Build / Deploy Flow Diff

    tag Env Var Modern EC2 Travis Env Commit hash in images k8s secrets Deploy blue / green rolling Fully AWS Fully Managed Infra Modern has the best practice Strategy manual automatic

23. ©2018 Wantedly, Inc. %FQMPZ
    TBNF
    WFSTJPO
    UP
    UXP
    FOWT 0ME
    "-# /FX
    "-# JOUFSOBM DNS

24. ©2018 Wantedly, Inc. JOUFSOBM DNS 4PNF
    PG
    SFRVFTUT
    HP
    UP
    LT ;FSP
    TFDPOE
    TXJUDI
    QFSDFOUBHF
    PG
    USBGGJD
    UIBU
    HPFT
    UP
    LT &BTZ
    UP
    BCPSU 0ME
    "-# /FX
    "-#

25. ©2018 Wantedly, Inc. "MM
    USBGGJD
    UP
    LT JOUFSOBM DNS 0ME
    "-# /FX
    "-#

26. ©2018 Wantedly, Inc. "CPSU
    EVF
    UP
    FYQFDUFE
    FSSPS JOUFSOBM DNS *
    DBVTFE
    BO
    JODJEFOU
    EVSJOH
    UIJT
    PQFSBUJPO
    CVU
    HSFBU
    UP
    IBWF
    B
    SPMMCBDL
    QBUI 0ME
    "-# /FX
    "-#

27. ©2018 Wantedly, Inc. 5SZ
    BHBJO JOUFSOBM DNS 0ME
    "-# /FX
    "-#

28. ©2018 Wantedly, Inc. 4XJUDI
    %/4 JOUFSOBM DNS 0ME
    "-# /FX
    "-#

29. ©2018 Wantedly, Inc. JOUFSOBM DNS %POF 0ME
    "-# /FX
    "-#

30. ©2018 Wantedly, Inc. DNS %POF /FX
    "-#

31. ©2018 Wantedly, Inc. Conventions make development productive Summary Migration is

    worth performing Enforcement to follow the conventions •same third-party services •commit hash as tag •automatic deploy •servicex a library to install everything •kube in CI and laptops •one namespace for one repo •temporary exception is ok •but, have to follow the rule sooner or later

32. ©2018 Wantedly, Inc. We’re Hiring .JDSPTFSWJDFT
    .POEBZ HJUIVCDPNXBOUFEMZNJDSPTFSWJDFT@NPOEBZ