on-premises-cloud-and-docker

Hatena Engineer Seminar #3 @ Tokyo

Transcript

1. ΦϯϓϨϛεɺΫϥ΢υ
   ͦͯ͠%PDLFS
   Hatena Engineer Seminar #3
   id:y_uuki
   

   View Slide

2. @y_uuk1
   id:y_uuki
   ӡ༻
   

   View Slide

3. Agenda
   ͸ͯͳͷΠϯϑϥࣄ৘
   Docker ΛͲͷΑ͏ʹଊ͍͑ͯΔ͔
   ͸ͯͳͷWebΦϖϨʔγϣϯΤϯδχΞ
   

   View Slide

4. ͸ͯͳͷ
   Πϯϑϥࣄ৘
   

   View Slide

5. %BUB$FOUFS "84
   ࣗ࡞
   αʔό
   9FO
   ϕϯμ
   αʔό
   &$
   4
   $MPVE'SPOU
   3PVUF
   -74LFFQBMJWFE
   (MBJDJFS
   71$
   &-# )"1SPYZ
   .PHJMF'4
   "LBNBJ
   

   View Slide

6. DC ͱ AWS
   • DCͰ Xen ࢖ͬͯͨΓ͢Δͱ AWS ͷ಺෦ΛͳΜͱͳ͘༧૝
   Ͱ͖ͨΓ͢Δ
   • ͳΜ͔ϩʔυόϥϯαͷεϧʔϓοτͰͳ͍ → ϋΠύʔ
   όΠβଆͰ nf_conntrack ᷓΕٙ࿭
   • ΠϯελϯελΠϓ͝ͱʹ state ਺ͷ্ݶҧ͏ٙ࿭
   • Domain 0ಉډΠϯελϯε
   • ΞΫςΟϒɾελϯόΠ2୆ͱ΋ಉډͰ࢒೦
   • Ϧιʔεͷ৯͍߹͍ (CPU %steal)
   

   View Slide

7. %FW 0QT
   $IFG
   $BQJTUSBOP
   /BHJPT
   .BDLFSFM
   .JEMMFXBSF
   "QQMJDBUJPO
   +FOLJOT
   04
   

   View Slide

8. Dev ͱ Ops
   • αʔόͬΆ͍͜ͱʢσϓϩΠΑΓԼʣ͸͍͍ͩͨ Ops
   • Ξϥʔτ͸ Dev ΋ड͚ͱΔ
   • Dev ͱ Ops ͕ڠྗͯ͠ো֐ରԠ
   • DevνʔϜͷmeetingʹ΋ఆظతʹࢀՃ
   • Ops ͸ GHE ͷ issue ΋ΈΔ (ઃఆม͑Δͱ͖͸PR)
   • ༵ۚϦϦʔε͸͠ͳ͍
   • ஈ֊తͳϦϦʔε
   

   View Slide

9. ՝୊
   

   View Slide

10. DC؀ڥͱAWS؀ڥͷڞ௨Խ
    • αʔόߏ੒؅ཧ Chef ✓
    • ωοτϫʔΫ(VPC + VPN) ✓
    • αʔό؅ཧ/ϞχλϦϯάπʔϧ (Mackerel) ✓
    • ؂ࢹ(Nagios) ✓
    • ϑΣΠϧΦʔό ✘
    • Chef ద༻લͷOSΠϝʔδ(Packer) ✘
    

    View Slide

11. AWSͷݻ༗ͷػೳ͕࢖͍ͮΒ͍
    AWSϩοΫΠϯ
    AWSαʔϏεʹґଘͨ͠࢓૊ΈΛ࡞ΔͱDC
    ଆʹྲྀ༻ͮ͠Β͍
    

    View Slide

12. Dev ͱ Ops ͷίϛϡχέʔγϣϯ
    • Dev ͔Β Ops ʹϗετߏஙΛґཔ
    • ຊ൪ʹ Redis ͕ͳͯ͘ࣄނ
    • Dev ͔Β Ops ʹ Jenkins ϗετʹඞཁͳύοέʔ
    δΛΠϯετʔϧґཔ
    • ޓ͍ʹ໘౗
    • ΄͍͠ਓͱ༻ҙ͢Δਓ͕ҟͳΔͷ͕໰୊
    • ͦ΋ͦ΋ Dev ͕ϗετߏங͠΍͍͢؀ڥ͕Α͍
    

    View Slide

13. Docker
    

    View Slide

14. It is Not Docker
    • VMͱ͔ੜLXC Έ͍ͨʹϩάΠϯͯ͠ੜ׆͢Δ
    ΋ͷͰ͸ͳ͍
    • Linuxίϯςφ ≠ Docker
    • Docker ͸ಛผɺେن໛؀ڥ޲͚ͱ͍͏Θ͚Ͱ
    ͸ͳ͍
    

    View Slide

15. It is Docker
    • Linux ͷϢʔβϥϯυ؀ڥΛؙ͝ͱΠϝʔδԽ
    • ΞϓϦέʔγϣϯ͝ͱʹίϯςφԽ
    • ls ίϚϯυ͚ͩͷ Docker ίϯςφͱ͔
    • Dockerίϯςφͷ࣮ମ͸ͨͩͷOSͷϓϩηε
    • Χʔωϧ಺෦Ͱִ཭͞ΕͯΔ
    

    View Slide

16. όʔδϣϯͷҧ͏ MySQL Λ3୆
    ಉ͡ϗετͰಈ͔ͤ·͔͢ʁ
    

    View Slide

17. user, datadir, port, socket,
    pidfile, install_dir, my.cnf
    !
    ͳͲͷϦιʔεΛ3ͭͣͭ࡞Δ
    UNIX ʹ͸఻౷తʹϦιʔε෼཭
    ػೳ͕ͳ͍
    

    View Slide

18. $ docker run -d mysql:5.5
    $ docker run -d mysql:5.6
    $ docker run -d mysql:5.7
    It is Docker
    

    View Slide

19. Chef ͱ͔࢖ͬͯ΋OSͷϢʔβϥϯυ
    γεςϜ͕ෳࡶ͗ͯ͢؅ཧͰ͖ͳ͍
    !
    ·ͱ΋ʹ؅ཧ͢ΔͷΛ͖͋ΒΊͯ
    ΞϓϦέʔγϣϯ͝ͱʹ
    ϢʔβϥϯυΛ༻ҙ͠΍͍͢࢓૊ΈΛ
    ఏڙͨ͠΋ͷ͕ Docker
    

    View Slide

20. DockerͰͳʹ͕Ͱ͖Δ͔
    • DC ؀ڥ ͱ Ϋϥ΢υ؀ڥͷڞ௨Խ
    • DCͱΫϥ΢υͰಉ͡ Docker image ͕ಈ͘
    • αʔϏεͷ੝ਰʹซͤͯɺDC <=> Ϋϥ΢υ
    • Dev ͱ Ops ͷίϛϡχέʔγϣϯ
    • Dev ΤϯδχΞ͕ඞཁͳΞϓϦέʔγϣϯͷ
    Dockerfile Λॻ͍ͯɺOps ΤϯδχΞ͕ϨϏϡʔͯ͠
    ͦͷ··σϓϩΠ
    • ΄͍͠ਓ͕४උ͠΍͍͢
    

    View Slide

21. ͦΕDockerͰͰ͖ΔΑ
    

    View Slide

22. Docker͕ؔΘΔྖҬ͸ଟ͍
    • ΞϓϦέʔγϣϯσϓϩΠ
    • ϩʔΧϧ؀ڥ͔Β CI Λܦ༝ͯ͠ຊ൪·Ͱ
    • ߏ੒؅ཧπʔϧʢChef, Puppet…)
    • Docker ࣌୅ͷαʔόϞχλϦϯά
    • Docker ࣌୅ͷϩάऩू
    • LinuxΧʔωϧ
    • LinuxίϯςφɺAUFS ͳͲ
    

    View Slide

23. ΞϓϦέʔγϣϯσϓϩΠ
    • શͯͷ؀ڥͰಉ͡ Docker image Λ࢖Θͳ͍ͱ
    ϝϦοτ͕ബ͍
    • ϩʔΧϧ؀ڥɺ։ൃ؀ڥɺstaging؀ڥɺCI؀
    ڥɺຊ൪؀ڥ
    • ෦෼తʹಋೖ͢Δͱ Docker ؀ڥͱ௨ৗͷ؀
    ڥΛ྆ํϝϯς͠ͳ͍ͱ͍͚ͳ͘ͳΔ
    • ։ൃνʔϜͷΤϯδχΞશһ+σβΠφʔ΋
    Docker؀ڥͰ։ൃ͢Δ͜ͱʹͳΔ
    

    View Slide

24. @ Hatena
    

    View Slide

25. ߹॓ͰσϓϩΠ؅ཧπʔϧ࣮૷
    ࣮ࡍʹ͸ෳࡶ࣮͗ͯ͢༻͸೉͔ͬͨ͠
    *NNVUBCMF*OGSBTUSVDUVSFʹඞཁͳཁૉ͸٧·ͬͯͨ
    

    View Slide

26. SQN EFCύοέʔδΛ࡞Δ؀ڥΛ࡞Δͷ͕໘౗ͩͬͨ
    

    View Slide

27. DBUVTSCJOSSEUPPM
    CJOTI
    FYFDEPDLFSSVOSNNBDLFSFMSSEUPPM!
    RRDtool ☓ Docker
    Πϯετʔϧ͕໘౗ͳίϚϯυΛ%PDLFSԽ
    $POUBJOFS$PNNBOE1BUUFSO
    

    View Slide

28. Graphite ☓ Docker
    • Graphite ͷΑ͏ͳ؀ڥߏங͕໘౗ͳ΋ͷ͸
    Docker ͰϩʔΧϧ؀ڥͱCI؀ڥΛߏங
    • https://registry.hub.docker.com/u/mackerel/graphite/
    EPDLFSSVOEOBNFHSBQIJUFWUNQMPHWBSMPHHSBQIJUFW
    UNQXIJTQFSWBSMJCHSBQIJUFTUPSBHFXIJTQFSQQ
    QNBDLFSFMHSBQIJUF
    

    View Slide

29. Docker ΞΠσΞ
    • ϒϥϯν͝ͱʹαʔϏεηοτΛ࡞Δ࢓૊Έ
    • ϚΠΫϩαʔϏεͷ֤ίϯϙʔωϯτΛDockerԽ
    • ؀ڥߏங͕؆୯ʹͳΔ
    • ςετͰଞαʔϏεΛୟ͘ͱ͖ʹϞοΫԽ͍ͯ͠
    ͨͱ͜ΖΛຊ෺Λ͚ͨͨͨΓ͢Δ
    • Docker cron
    • cron ͕ಈ͔ͳ͍໰୊
    • खݩͰ࣮ߦ͠΍͍͢Α͏ʹ
    

    View Slide

30. Docker ࣗମΛ
    ࣗલͰӡ༻͢Δ or ͠ͳ͍
    

    View Slide

31. Docker ͷӡ༻
    • Docker ࣗମͷτϥϒϧγϡʔςΟϯά
    • ಥવDNSҾ͚ͳ͘ͳΔ
    • ΰϛ૟আ (଺ཹ͢Δίϯςφ)
    • ϚϧνϗετσϓϩΠ
    • ΦʔέετϨʔγϣϯπʔϧԿΛ࢖͏͔
    • Kubernetes, fleet, Consul, …
    • DNSͰσΟεΧόϦͨ͘͠ͳ͍
    • Docker ίϯςφͷωοτϫʔΫߏ੒
    • ϗετଆͱϑϥοτ or NAPT
    • ໰୊ௐࠪͷํ๏ͷཱ֬
    • ssh ͰϩάΠϯͱ͍͏ੈքͰ͸ͳ͍ (docker exec /bin/bash)
    

    View Slide

32. Ϋϥ΢υαʔϏε
    • AWS EC2 Container Service
    • Google Container Engine
    !
    • Dockerͷӡ༻Λશ෦೚ͤΒΕΔ
    • AWS <-> Google ΋Մ
    

    View Slide

33. ઐ༻αʔό
    • શͯͷαʔϏεΛΫϥ΢υʹͷͤΔͱߴ͍
    • ωοτϫʔΫ&ϋʔυ΢ΣΞ؅ཧΛΦϑϩʔυͭ͠
    ্͕ͭ҆Γ
    • Xen ͰԾ૝Խͤͣʹɺ෺ཧϋʔυͷ্ʹίϯςφ
    • Docker ͳΒ Ϋϥ΢υ 㱻 ΦϯϓϨϛεͷҠಈ͕΍
    Γ΍͍͢
    • ৽αʔϏε͸શ෦Ϋϥ΢υ -> ྲྀߦΒͳ͔ͬͨΒ
    ઐ༻αʔό؀ڥ΁ୀආ ͷΑ͏ͳઓུ͕ͱΓ΍͍͢
    

    View Slide

34. ͦΕͰ΋·ͩෆ҆
    • ࠓ೔ॻ͍ͨ Dockerfile ͸ 1 ೥ޙಈ͔ͳ͍Մೳੑ
    (DockerfileࣗମͰόʔδϣϯݻఆͱ͔Ͱ͖ΔΘ
    ͚Ͱ͸ͳ͍)
    • Docker image Λޙੜେࣄʹ͓࣋ͬͯ͘ඞཁ
    ͕͋Δ
    • ࠓ೔࡞ͬͨ Docker image ͕1೥ޙͷDocker
    runtimeͰಈ͔͘Ͳ͏͔
    

    View Slide

35. Docker ͸ύϥμΠϜΛม͑Δ
    

    View Slide

36. ͔ͩΒͦ͜ӡ༻ج൫ͱ։ൃج൫Λ
    શͯDockerલఏͰߟ͑ͳ͍ͱ
    Ձ஋͕Ͱͳ͍
    

    View Slide

37. View Slide

38. ͸ͯͳͷ
    WebΦϖϨʔγϣϯ
    ΤϯδχΞ
    

    View Slide

39. View Slide

40. ΍ͬͯΔ͜ͱ͕ҧ͏
    

    View Slide

41. y_uuki ͷ৔߹
    

    View Slide

42. ɾΞϧόΠτ࣌୅: ࣾ಺Mackerel։ൃ(Perl) (YAPC Asia)
    ɾೖࣾ: EC2Ͱ HAProxy νϡʔχϯά
    http://yuuki.hatenablog.com/entry/2014/03/20/085600
    ɾXen, LVS(keepalived), Chef, Nagios Ұ௨Γ
    ɾ߹॓: Docker + Mesos σϓϩΠ؅ཧπʔϧ࡞Δ
    http://yuuki.hatenablog.com/entry/2013/12/22/174813
    !
    ɾ৽αʔϏεͷߏஙӡ༻ʢMackerelʣ
    ɾJVM, PostgreSQL, Graphite(࣌ܥྻDB)
    ɾmackerel-agent ύοέʔδఏڙ
    http://yuuki.hatenablog.com/entry/docker-package-ci
    !
    ɾJVM Operation Casual http://yuuki.hatenablog.com/entry/2014/04/08/074507
    ɾGraphiteνϡʔχϯά http://yuuki.hatenablog.com/entry/monitoringcasual6
    ɾMackerel OSS ·ΘΓϝϯςφϯε
    ɾISUCON4 ຊઓग़৔ http://yuuki.hatenablog.com/entry/dockerized-isucon
    ɾMackerel CLI tool github.com/y-uuki/gomkr
    ɾ߹॓: GoͰδϣϒΩϡʔγεςϜ
    

    View Slide

43. WebΞϓϦέʔγϣϯ։ൃ
    ϋʔυ΢ΣΞௐୡͱ͔ωοτϫʔΫઃܭ
    ্
    Լ
    OSΧʔωϧ
    ϛυϧ΢ΣΞ: MySQL, PostgreSQL, Redis, memcached,
    Nginx, Apache, Elasticsearch, Solr, Varnish, Squid
    Jenkins(CI)
    Capistrano(σϓϩΠ)
    Nagios
    Chef
    

    View Slide

44. ࣗ෼ͷϨΠϠʔ: N
    N → N - 1
    N + 1 → N
    !
    ͕Ͱ͖Δ؀ڥ
    

    View Slide

45. ਺ઍ୆ਓ
    

    View Slide

46. ਺ઍ୆ਓ
    ʜ ৽αʔϏε ৽αʔϏε
    ৽αʔϏε
    

    View Slide

47. 5ਓʂʁ
    

    View Slide

48. CRITICAL
    

    View Slide

49. We are Hiring!
    IUUQIBUFOBDPSQKQSFDSVJUDBSFFSPQFSBUJPOFOHJOFFS
    

    View Slide