無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of autonomous and infinite scalable Jenkins with AWS

Transcript

1. ແݶʹεέʔϧ͢Δ্ʹ
 ࣗ཯తͳJenkinsʹݟΔເ
 ~AWSἫ~ KyotoͳΜ͔ #4 @moznion

2. @moznion Software engineer Hatena Intern 2013

3. Jenkins

4. Jenkins - CI (Continuous Integration) Platform - Workflow - ศརͳcron

5. Why Jenkins?

6. Why Jenkins? - ΋ͱ΋ͱ͸ (Ұ෦͸ࠓ΋) CircleCIΛ࢖͍ͬͯͨ - Job਺͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……

   - ΋ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ੔͍͑ͨ -

7. Why Jenkins? - ΋ͱ΋ͱ͸ (Ұ෦͸ࠓ΋) CircleCIΛ࢖͍ͬͯͨ - Job਺͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……

   - ΋ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ੔͍͑ͨ - ΋ͱ΋ͱJenkinsΛεέʔϧͤ͞Δํ๏Λ஌͍ͬͯͨ

8. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊

9. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊ - ฒྻ࣮ߦJob਺ͷ૿Ճʹ൐͏ੑೳྼԽ - ߜΔͱbuild଴ͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ໰୊ - Master

   InstanceࢮͶͳ͍໰୊ - ͓ئ͍͠·͢೴ͷڪා (™ @kyanny)

10. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊ - ฒྻ࣮ߦJob਺ͷ૿Ճʹ൐͏ੑೳྼԽ - ߜΔͱbuild଴ͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ໰୊ - Master

    InstanceࢮͶͳ͍໰୊ - ͓ئ͍͠·͢೴ͷڪා (™ @kyanny)

11. ฒྻjob਺্͛ΔͷϚδَ໳ʂ

12. ฒྻjob਺্͛ΔͷϚδَ໳ SlaveΛ଍͢ͷ͸؆୯ʂʂʂʂ

13. ฒྻjob਺্͛ΔͷϚδَ໳ - slave଍͢ͷ͸؆୯ͳΜ͚ͩͲ…… - ୯७ʹ؅ཧίετ͕૿͑Δͧʂ - ʮ͜ͷslaveͩͱͳ͔ͥςετ͕ίέ·͢ʯ

14. ฒྻjob਺্͛ΔͷϚδَ໳ - slave଍͢ͷ͸؆୯ͳΜ͚ͩͲ…… - ΦʔτεέʔϦϯάΛͪ·ͪ·΍͍ͬͯ͘ඞཁ͕͋Γ…… - ΠϯελϯεΛ༡͹͓ͤͯ࣌ؒ͘͸ແ͍ - CI͸Α΄ͲϓϩδΣΫτཱ͕ͯࠐΜͰͳ͍ݶΓ৸ͯΔ͕࣌ؒ ࢧ഑త

    - ׂͱઃఆ͕೉͍͠ (ίπ͕͍Δ)

15. ͏͏……

16. զʑ͸΄ͬͱ͍ͯ΋ແݶʹϏϧυ͕εέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……

17. ͦΜͳ΋ͷ͸…… ͋Δʂʂʂ

18. ͦΜͳ΋ͷ͸…… ͋Δʂʂʂ

19. AWS CodeBuild

20. AWS CodeBuild - ෼୯Ґͷ࣌ؒି͠ - ೚ҙͷίϯςφΛ࣮ߦͰ͖Δ - ͦͷίϯςφ্ͰϏϧυ΍Βςετ΍Β͢Δ - (LambdaΑΓ΋͕͔͔࣌ؒΔλεΫͷ࣮ߦ؀ڥʹ΋……)

21. AWS CodeBuild - Jenkins Plugin͕͋Δ (AWSۘ੡) - https://github.com/awslabs/aws-codebuild-jenkins-plugin - JenkinsͰड͚ͨτϦΨͰCodeBuildΛൃՐͰ͖Δ

22. https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/

23. https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/

24. https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/ ࣮࣭ແݶʂʂʂʂ

25. ΋ͨΒ͞ΕͨԸܙ - slave nodeͷϝϯς஌Βͣ - ΨϯΨϯฒྻϏϧυ͢Δͧʂʂʂ - ࣌ؒି͠ (= ࢖ͬͨ෼͚ͩ)

    ͷྉۚઃఆͳͷͰ
 ࣗ෼ͰΦʔτεέʔϧॲཧΛ͠ͳͯ͘Α͍

26. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊ - ฒྻ࣮ߦJob਺ͷ૿Ճʹ൐͏ੑೳྼԽ - ߜΔͱbuild଴ͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ໰୊ - Master

    InstanceࢮͶͳ͍໰୊ - ͓ئ͍͠·͢೴ͷڪා (™ @kyanny)

27. Master Instance͕ਆ֨Խ͢Δ

28. Master Instanceਆ֨Խ - Jenkinsͷmaster data͸masterϚγϯͷ*ϩʔΧϧετϨʔδ*ʹ
 อଘ͞ΕΔ - Πϯελϯε͕ࢮΜͩΒऴΘΓ - ͔͠͠ࢮ͸ආ͚ΒΕͳ͍

    - ͔ͩΒόοΫΞοϓΛऔͬͨΓ͢Δ - e.g. https://papix.hatenablog.com/entry/2016/02/09/101918 - όοΫΞοϓ͔ΒͷϦετΞ͸࣮ࡍ໘౗ - ָ͍͠࢓ࣄͰ΋ͳ͍…… - μ΢ϯλΠϜग़Δ͠……

29. Master Instanceਆ֨Խ - ݁Ռͱͯ͠ຖ೔ଉࡂΛفΔ೔ʑ…… - ӡѱ͘Πϯελϯε͕ࢮΜͩΒσΠϦʔόοΫΞοϓ ͔ΒϦετΞ - σΠϦʔόοΫΞοϓͱࢮ͵ؒࡍͱͷࠩ෼ʹ͍ͭͯ͸ ఘΊΔ͔͠ແ͍……

30. ͏͏……

31. զʑ͸ৎ෉Ͱεέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……

32. ͦΜͳ΋ͷ͸…… ͋Δʂʂʂ

33. ͦΜͳ΋ͷ͸…… ͋Δʂʂʂ

34. Amazon Elastic File System

35. AWS Elastic File System (EFS) - AmazonͷnfsΈ͍ͨͳ΍ͭ - Instanceʹmountͯ͠࢖͏ -

    EBS + nfsతͳ…… - ϘϦϡʔϜαΠζ͕ࣗಈͰ֦ுɾॖখ͢Δ - Disk full஌Βͣͷੜ׆

36. AWS Elastic File System (EFS) - $JENKINS_HOMEΛEFSϘϦϡʔϜ্ʹ͢Δ - Πϯελϯε͕ࢮΜͰ΋৽͍͠Πϯελϯεʹ
 ͦͷϘϦϡʔϜΛϚ΢ϯτ͢Δ

    - => Πϯελϯε͸લੈͷهԱΛҾ͖ܧ͗෮׆ʂʂʂ

37. mount Working...

38. ❌mount Instance down...

39. ❌ New instance coming up...

40. ❌ mount Works fine!!

41. AWS Elastic File System (EFS) - EBSͰ΋ྑ͍ͷͰ͸ʁ - ͸͍ʂɹEBSͷ΄͏͕͍҆͠ʂʂ -

    ͨͩEFS͸ϘϦϡʔϜࣗಈ֦ு͕خ͍͠ - (͏͔ͬΓdisk fullʹͳΒͳ͍) - ͋ͱEFSͩͱmulti masterΈ͍ͨͳ͜ͱ΋Ͱ͖Δ (ޙड़)

42. Jenkins Multi Master - ଟ෼Ͱ͖Δ (௕࣌ؒӡ༻ͯ͠ͳ͍) - ͔͠͠CodeBuildͱڞଘ͍ͯ͠Δ؀ڥͰMulti Masterʹ͢Δ
 ϝϦοτ΄΅ͳ͘ͳ͍Ͱ͔͢ʁ

    - σϓϩΠ࣌ʹҰॠ2୆masterΛཱ͓͍ͯͯͯɼݹ͍ํΛ
 ࡴ͢Έ͍ͨͳ௿μ΢ϯλΠϜσϓϩΠ޲͚ͷ࢖͍ํ͸Մೳ

43. ΋ͨΒ͞ΕͨԸܙ - ΧδϡΞϧʹΠϯελϯε͕ࢮΜͰ΋҆৺ʂ - ͱ͸͍͑όοΫΞοϓ͸औͬͯΔ - ݁Ռͱͯ͠ΠϯελϯεͷೖΕସָ͕͑ʹͳͬͨ - ετϨʔδ΋ࣗಈ֦ு͢ΔͷͰDisk full஌Βͣ

44. Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢໰୊ - ฒྻ࣮ߦJob਺ͷ૿Ճʹ൐͏ੑೳྼԽ - ߜΔͱbuild଴ͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ໰୊ - Master

    InstanceࢮͶͳ͍໰୊ - ͓ئ͍͠·͢೴ͷڪා (™ @kyanny)

45. ࣾձ໰୊: Jenkins͓͡͞ΜͷͳΓखෆ଍

46. Jenkinsͷӡ༻Λ୭͔ʹԡ͠෇͚ΔͷΛ΍ΊΔ - ୭Ͱ΋ӡ༻Ͱ͖ΔΑ͏ʹ͢Δ - શਓຽJenkins͓͡͞Μ/͓͹͞ΜԽ - σϓϩΠ΍ϝϯς͸ۃྗলྗԽ͢Δ

47. Jenkinsͷӡ༻Λ୭͔ʹԡ͠෇͚ΔͷΛ΍ΊΔ - Amazon Elastic Beanstalk + Docker - Dockerfile +

    Dockerrun.aws.jsonʹΑΔߏ੒ͷίʔυԽ - .ebextensionsʹΑΔࣗಈԽ - EFS mountͳͲ - https://github.com/moznion/jenkins_eb_efs_codebuild_example

48. - Elastic Beanstalkͷmanaged updateͰࣗಈతʹύον͕౰ͨΔ - ͍ͭͰʹΠϯελϯεͷೖΕସ͑΋ߦΘΕΔ - EFS؀ڥԼͳͷͰΠϯελϯεೖΕସ͑΋ָʑࣗಈԽ - جຊతʹӡ༻͸์ஔͯ͠·͢

    - EFSͷϘϦϡʔϜࣗಈ֦ுͷύϫʔͰ΋͋Δ ΋ͨΒ͞ΕͨԸܙ

49. ·ͱΊ

50. ·ͱΊ - JenkinsͷslaveϊʔυΛAWS CodeBuildʹҕৡ͢Δ͜ͱͰ࣮࣭ ແݶʹฒྻϏϧυ਺͕εέʔϧ͢ΔΑ͏ʹͳΓ·ͨ͠ - JenkinsͷετϨʔδΛEFSʹ͢Δ͜ͱͰϚελʔσʔλͷ ϩόετԽΛߦ͍·ͨ͠ - Elastic

    BeanstalkΛ༻͍ͯJenkinsΛӡ༻͢Δ͜ͱͰӡ༻ͷ௿ ίετԽΛ͠·ͨ͠ - Α͔ͬͨͰ͢Ͷ

51. Q?

52. Wait...

53. ͜ͷJenkinsʹ͸໰୊͕͋Δ - EFS͸஗͍ʂʂʂʂʂ - Jenkins͕git clone͢ΔͱϚδ͔͔࣌ؒΓ·͢ - git cloneΛճආ͢Δํ๏͸͍·ͷͱ͜Ζແ͍…… -

    shallow cloneΛ༗ޮʹ࢖͏ - sparse checkoutΛ༗ޮʹ࢖͏ - ͳͲͷ޻෉͕ඞཁ - ͦ΋ͦ΋CodeBuild͕git clone͢ΔΜͩ͠jenkins͸͠ͳͯ͘
 ྑ͍Μ͚ͩͲ…… - ͋Δ͍͸EBSΛ࢖͏……

54. ຊ౰ʹJenkins͸ඞཁʁ - ͭ͡͸AWS CodeBuild͸࠷ۙΊͬͪΌͰ͖͕Αͯ͘ɼ
 ୯ମͰ΋े෼࢖͑Δ - GitHubͷhookड͚ΕΔ͠ - Commit statusΛม͑Δػೳ·Ͱ͍͍ͭͯΔ

    - Ϗϧυཤྺ΋ҰཡͰ͖Δ͠ - CloudWatch LogsʹϏϧυͷϩά΋ग़Δ - JenkinsෆཁͰ͸ʁʁʁʁɹ͜ΕͰे෼Ͱ͸ʁʁʁʁ - ͸͍

55. ຊ౰ʹJenkins͸ඞཁʁ - ࣗ༝ͳWebhookΛड͚Δඞཁ͕͋Δͱ͔…… - ͋ͱ௨஌ͱ͔Ͷʂ - ͋Δ͍͸΋ͬͱଞʹϏϧυύΠϓϥΠϯΛܨ ͛Δඞཁ͕͋Δͱ͔…… - ͦ͏͍͏ͱ͖͸΂ΜΓͰ͢

    - (͋ͱ͸·͋Ұཡੑ͕Α͍) (ॾઆ͋Γ·͢)

56. Q?