無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of autonomous and infinite scalable Jenkins with AWS

672ee2598bea695496f11022a9a9c4c8?s=47 moznion
August 18, 2018

無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of autonomous and infinite scalable Jenkins with AWS

Kyotoなんか #4の資料です

672ee2598bea695496f11022a9a9c4c8?s=128

moznion

August 18, 2018
Tweet

Transcript

  1. ແݶʹεέʔϧ͢Δ্ʹ
 ࣗ཯తͳJenkinsʹݟΔເ
 ~AWSἫ~ KyotoͳΜ͔ #4 @moznion

  2. @moznion Software engineer Hatena Intern 2013

  3. Jenkins

  4. Jenkins - CI (Continuous Integration) Platform - Workflow - ศརͳcron

  5. Why Jenkins?

  6. Why Jenkins? - ΋ͱ΋ͱ͸ (Ұ෦͸ࠓ΋) CircleCIΛ࢖͍ͬͯͨ - Job਺͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……

    - ΋ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ੔͍͑ͨ -
  7. Why Jenkins? - ΋ͱ΋ͱ͸ (Ұ෦͸ࠓ΋) CircleCIΛ࢖͍ͬͯͨ - Job਺͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……

    - ΋ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ੔͍͑ͨ - ΋ͱ΋ͱJenkinsΛεέʔϧͤ͞Δํ๏Λ஌͍ͬͯͨ
  8. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊

  9. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊ - ฒྻ࣮ߦJob਺ͷ૿Ճʹ൐͏ੑೳྼԽ - ߜΔͱbuild଴ͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ໰୊ - Master

    InstanceࢮͶͳ͍໰୊ - ͓ئ͍͠·͢೴ͷڪා (™ @kyanny)
  10. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊ - ฒྻ࣮ߦJob਺ͷ૿Ճʹ൐͏ੑೳྼԽ - ߜΔͱbuild଴ͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ໰୊ - Master

    InstanceࢮͶͳ͍໰୊ - ͓ئ͍͠·͢೴ͷڪා (™ @kyanny)
  11. ฒྻjob਺্͛ΔͷϚδَ໳ʂ

  12. ฒྻjob਺্͛ΔͷϚδَ໳ SlaveΛ଍͢ͷ͸؆୯ʂʂʂʂ

  13. ฒྻjob਺্͛ΔͷϚδَ໳ - slave଍͢ͷ͸؆୯ͳΜ͚ͩͲ…… - ୯७ʹ؅ཧίετ͕૿͑Δͧʂ - ʮ͜ͷslaveͩͱͳ͔ͥςετ͕ίέ·͢ʯ

  14. ฒྻjob਺্͛ΔͷϚδَ໳ - slave଍͢ͷ͸؆୯ͳΜ͚ͩͲ…… - ΦʔτεέʔϦϯάΛͪ·ͪ·΍͍ͬͯ͘ඞཁ͕͋Γ…… - ΠϯελϯεΛ༡͹͓ͤͯ࣌ؒ͘͸ແ͍ - CI͸Α΄ͲϓϩδΣΫτཱ͕ͯࠐΜͰͳ͍ݶΓ৸ͯΔ͕࣌ؒ ࢧ഑త

    - ׂͱઃఆ͕೉͍͠ (ίπ͕͍Δ)
  15. ͏͏……

  16. զʑ͸΄ͬͱ͍ͯ΋ແݶʹϏϧυ͕εέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……

  17. ͦΜͳ΋ͷ͸…… ͋Δʂʂʂ

  18. ͦΜͳ΋ͷ͸…… ͋Δʂʂʂ

  19. AWS CodeBuild

  20. AWS CodeBuild - ෼୯Ґͷ࣌ؒି͠ - ೚ҙͷίϯςφΛ࣮ߦͰ͖Δ - ͦͷίϯςφ্ͰϏϧυ΍Βςετ΍Β͢Δ - (LambdaΑΓ΋͕͔͔࣌ؒΔλεΫͷ࣮ߦ؀ڥʹ΋……)

  21. AWS CodeBuild - Jenkins Plugin͕͋Δ (AWSۘ੡) - https://github.com/awslabs/aws-codebuild-jenkins-plugin - JenkinsͰड͚ͨτϦΨͰCodeBuildΛൃՐͰ͖Δ

  22. https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/

  23. https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/

  24. https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/ ࣮࣭ແݶʂʂʂʂ

  25. ΋ͨΒ͞ΕͨԸܙ - slave nodeͷϝϯς஌Βͣ - ΨϯΨϯฒྻϏϧυ͢Δͧʂʂʂ - ࣌ؒି͠ (= ࢖ͬͨ෼͚ͩ)

    ͷྉۚઃఆͳͷͰ
 ࣗ෼ͰΦʔτεέʔϧॲཧΛ͠ͳͯ͘Α͍
  26. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊ - ฒྻ࣮ߦJob਺ͷ૿Ճʹ൐͏ੑೳྼԽ - ߜΔͱbuild଴ͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ໰୊ - Master

    InstanceࢮͶͳ͍໰୊ - ͓ئ͍͠·͢೴ͷڪා (™ @kyanny)
  27. Master Instance͕ਆ֨Խ͢Δ

  28. Master Instanceਆ֨Խ - Jenkinsͷmaster data͸masterϚγϯͷ*ϩʔΧϧετϨʔδ*ʹ
 อଘ͞ΕΔ - Πϯελϯε͕ࢮΜͩΒऴΘΓ - ͔͠͠ࢮ͸ආ͚ΒΕͳ͍

    - ͔ͩΒόοΫΞοϓΛऔͬͨΓ͢Δ - e.g. https://papix.hatenablog.com/entry/2016/02/09/101918 - όοΫΞοϓ͔ΒͷϦετΞ͸࣮ࡍ໘౗ - ָ͍͠࢓ࣄͰ΋ͳ͍…… - μ΢ϯλΠϜग़Δ͠……
  29. Master Instanceਆ֨Խ - ݁Ռͱͯ͠ຖ೔ଉࡂΛفΔ೔ʑ…… - ӡѱ͘Πϯελϯε͕ࢮΜͩΒσΠϦʔόοΫΞοϓ ͔ΒϦετΞ - σΠϦʔόοΫΞοϓͱࢮ͵ؒࡍͱͷࠩ෼ʹ͍ͭͯ͸ ఘΊΔ͔͠ແ͍……

  30. ͏͏……

  31. զʑ͸ৎ෉Ͱεέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……

  32. ͦΜͳ΋ͷ͸…… ͋Δʂʂʂ

  33. ͦΜͳ΋ͷ͸…… ͋Δʂʂʂ

  34. Amazon Elastic File System

  35. AWS Elastic File System (EFS) - AmazonͷnfsΈ͍ͨͳ΍ͭ - Instanceʹmountͯ͠࢖͏ -

    EBS + nfsతͳ…… - ϘϦϡʔϜαΠζ͕ࣗಈͰ֦ுɾॖখ͢Δ - Disk full஌Βͣͷੜ׆
  36. AWS Elastic File System (EFS) - $JENKINS_HOMEΛEFSϘϦϡʔϜ্ʹ͢Δ - Πϯελϯε͕ࢮΜͰ΋৽͍͠Πϯελϯεʹ
 ͦͷϘϦϡʔϜΛϚ΢ϯτ͢Δ

    - => Πϯελϯε͸લੈͷهԱΛҾ͖ܧ͗෮׆ʂʂʂ
  37. mount Working...

  38. ❌mount Instance down...

  39. ❌ New instance coming up...

  40. ❌ mount Works fine!!

  41. AWS Elastic File System (EFS) - EBSͰ΋ྑ͍ͷͰ͸ʁ - ͸͍ʂɹEBSͷ΄͏͕͍҆͠ʂʂ -

    ͨͩEFS͸ϘϦϡʔϜࣗಈ֦ு͕خ͍͠ - (͏͔ͬΓdisk fullʹͳΒͳ͍) - ͋ͱEFSͩͱmulti masterΈ͍ͨͳ͜ͱ΋Ͱ͖Δ (ޙड़)
  42. Jenkins Multi Master - ଟ෼Ͱ͖Δ (௕࣌ؒӡ༻ͯ͠ͳ͍) - ͔͠͠CodeBuildͱڞଘ͍ͯ͠Δ؀ڥͰMulti Masterʹ͢Δ
 ϝϦοτ΄΅ͳ͘ͳ͍Ͱ͔͢ʁ

    - σϓϩΠ࣌ʹҰॠ2୆masterΛཱ͓͍ͯͯͯɼݹ͍ํΛ
 ࡴ͢Έ͍ͨͳ௿μ΢ϯλΠϜσϓϩΠ޲͚ͷ࢖͍ํ͸Մೳ
  43. ΋ͨΒ͞ΕͨԸܙ - ΧδϡΞϧʹΠϯελϯε͕ࢮΜͰ΋҆৺ʂ - ͱ͸͍͑όοΫΞοϓ͸औͬͯΔ - ݁Ռͱͯ͠ΠϯελϯεͷೖΕସָ͕͑ʹͳͬͨ - ετϨʔδ΋ࣗಈ֦ு͢ΔͷͰDisk full஌Βͣ

  44. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊ - ฒྻ࣮ߦJob਺ͷ૿Ճʹ൐͏ੑೳྼԽ - ߜΔͱbuild଴ͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ໰୊ - Master

    InstanceࢮͶͳ͍໰୊ - ͓ئ͍͠·͢೴ͷڪා (™ @kyanny)
  45. ࣾձ໰୊: Jenkins͓͡͞ΜͷͳΓखෆ଍

  46. Jenkinsͷӡ༻Λ୭͔ʹԡ͠෇͚ΔͷΛ΍ΊΔ - ୭Ͱ΋ӡ༻Ͱ͖ΔΑ͏ʹ͢Δ - શਓຽJenkins͓͡͞Μ/͓͹͞ΜԽ - σϓϩΠ΍ϝϯς͸ۃྗলྗԽ͢Δ

  47. Jenkinsͷӡ༻Λ୭͔ʹԡ͠෇͚ΔͷΛ΍ΊΔ - Amazon Elastic Beanstalk + Docker - Dockerfile +

    Dockerrun.aws.jsonʹΑΔߏ੒ͷίʔυԽ - .ebextensionsʹΑΔࣗಈԽ - EFS mountͳͲ - https://github.com/moznion/jenkins_eb_efs_codebuild_example
  48. - Elastic Beanstalkͷmanaged updateͰࣗಈతʹύον͕౰ͨΔ - ͍ͭͰʹΠϯελϯεͷೖΕସ͑΋ߦΘΕΔ - EFS؀ڥԼͳͷͰΠϯελϯεೖΕସ͑΋ָʑࣗಈԽ - جຊతʹӡ༻͸์ஔͯ͠·͢

    - EFSͷϘϦϡʔϜࣗಈ֦ுͷύϫʔͰ΋͋Δ ΋ͨΒ͞ΕͨԸܙ
  49. ·ͱΊ

  50. ·ͱΊ - JenkinsͷslaveϊʔυΛAWS CodeBuildʹҕৡ͢Δ͜ͱͰ࣮࣭ ແݶʹฒྻϏϧυ਺͕εέʔϧ͢ΔΑ͏ʹͳΓ·ͨ͠ - JenkinsͷετϨʔδΛEFSʹ͢Δ͜ͱͰϚελʔσʔλͷ ϩόετԽΛߦ͍·ͨ͠ - Elastic

    BeanstalkΛ༻͍ͯJenkinsΛӡ༻͢Δ͜ͱͰӡ༻ͷ௿ ίετԽΛ͠·ͨ͠ - Α͔ͬͨͰ͢Ͷ
  51. Q?

  52. Wait...

  53. ͜ͷJenkinsʹ͸໰୊͕͋Δ - EFS͸஗͍ʂʂʂʂʂ - Jenkins͕git clone͢ΔͱϚδ͔͔࣌ؒΓ·͢ - git cloneΛճආ͢Δํ๏͸͍·ͷͱ͜Ζແ͍…… -

    shallow cloneΛ༗ޮʹ࢖͏ - sparse checkoutΛ༗ޮʹ࢖͏ - ͳͲͷ޻෉͕ඞཁ - ͦ΋ͦ΋CodeBuild͕git clone͢ΔΜͩ͠jenkins͸͠ͳͯ͘
 ྑ͍Μ͚ͩͲ…… - ͋Δ͍͸EBSΛ࢖͏……
  54. ຊ౰ʹJenkins͸ඞཁʁ - ͭ͡͸AWS CodeBuild͸࠷ۙΊͬͪΌͰ͖͕Αͯ͘ɼ
 ୯ମͰ΋े෼࢖͑Δ - GitHubͷhookड͚ΕΔ͠ - Commit statusΛม͑Δػೳ·Ͱ͍͍ͭͯΔ

    - Ϗϧυཤྺ΋ҰཡͰ͖Δ͠ - CloudWatch LogsʹϏϧυͷϩά΋ग़Δ - JenkinsෆཁͰ͸ʁʁʁʁɹ͜ΕͰे෼Ͱ͸ʁʁʁʁ - ͸͍
  55. ຊ౰ʹJenkins͸ඞཁʁ - ࣗ༝ͳWebhookΛड͚Δඞཁ͕͋Δͱ͔…… - ͋ͱ௨஌ͱ͔Ͷʂ - ͋Δ͍͸΋ͬͱଞʹϏϧυύΠϓϥΠϯΛܨ ͛Δඞཁ͕͋Δͱ͔…… - ͦ͏͍͏ͱ͖͸΂ΜΓͰ͢

    - (͋ͱ͸·͋Ұཡੑ͕Α͍) (ॾઆ͋Γ·͢)
  56. Q?