Distributed Tracing with OpenCensus at Wantedly, Inc.

OpenCensus meetup vol.1
https://opencensus.connpass.com/event/123885/

Transcript

1. ©2018 Wantedly, Inc.
   Distributed Tracing with OpenCensus
   at Wantedly, Inc.
   OpenCensus meetup vol.1
   03.Apr.2019 - @munisystem
   

   View Slide

2. ©2018 Wantedly, Inc.
   0QFO$FOTVTͷಋೖഎܠ
   8BOUFEMZ *ODʹ͓͚Δ෼ࢄτϨʔγϯά
   ݱঢ়ͷ՝୊
   ·ͱΊ
   Agenda
   

   View Slide

3. ©2018 Wantedly, Inc.
   !NVOJTZTUFN:VJDIJ4BJUP
   *OGSBTUSVDUVSF5FBNBU8BOUFEMZ *OD
   

   View Slide

4. ©2018 Wantedly, Inc.
   0QFO$FOTVTͷಋೖഎܠ
   

   View Slide

5. ©2018 Wantedly, Inc.
   8BOUFEMZʹ͓͚Δ.JDSPTFSWJDFTͷݱঢ়
   w 1PMZHMPU
   w 3VCZͱ(P͕ϝΠϯ
   w αʔϏεͷυϝΠϯʹΑͬͯ1ZUIPO΍/PEFɺ3VTU΋࠾༻
   w .VMUJ$MPVE
   w "84 ,VCFSOFUFTPO&$ 3%4 &MBTUJDBDIF ʜ
   
   w ($1 #JH2VFSZ $MPVE%BUBTUPSF $MPVE1VC4VC ʜ
   
   

   View Slide

6. ©2018 Wantedly, Inc.
   ͳͥ.JDSPTFSWJDFTΛ࠾༻ͨ͠ͷ͔
   ೥ͷ8BOUFEMZ1FPQMFͷ։ൃ͔Β.JDSPTFSWJDFTΛ࠾༻
   w ౰࣌͸ϞϊϦγοΫͳ3BJMTΞϓϦέʔγϣϯ͕਺ݸଘࡏ
   w αʔϏεͷಛੑ্ɺ3BJMTͰ͸࣮ݱ͠ʹ͍͘ػೳΛ࣮ݱ͢Δඞཁ͕͋ͬͨ
   w ฒྻԽɺػցֶश
   w ߴ଎ͳ։ൃͱϦϦʔε΁ͷཁٻ
   w ϦϦʔεαΠΫϧͷ෼཭
   

   View Slide

7. ©2018 Wantedly, Inc.
   ύϑΥʔϚϯεܭଌͷ೉͠͞
   w 8BOUFEMZͰ͸"1.ͷج൫ͱͯ͠/FX3FMJDΛ࠾༻͍ͯͨ͠
   

   View Slide

8. ©2018 Wantedly, Inc.
   ύϑΥʔϚϯεܭଌͷ೉͠͞
   w 8BOUFEMZͰ͸"1.ͷج൫ͱͯ͠/FX3FMJDΛ࠾༻͍ͯͨ͠
   w SVOUJNFϨϕϧͰͷϝτϦΫεͷऩू
   w -BUFODZ 4UBDLUSBDF ($IFBQTJ[F ʜ
   w .JDSPTFSWJDFTͰ͸αʔϏεؒͷσʔλͷ΍ΓऔΓ͕ωοτϫʔΫӽ͠ʹߦΘΕΔ
   w ϦΫΤετઌͰͲΜͳॲཧ͕ߦΘΕ͍ͯΔͷ͔Λ൑அͰ͖ͳ͍
   

   View Slide

9. ©2018 Wantedly, Inc.
   ύϑΥʔϚϯεܭଌͷ೉͠͞
   Receive A /api/products/1
   timeline
   DB Access
   Request B /api/reviews/1
   #Ͱͳʹ͕ߦΘΕ͍ͯΔͷ͔͸"ͷ4UBDL5SBDF্͔Β͸൑அͰ͖ͳ͍
   

   View Slide

10. ©2018 Wantedly, Inc.
    ύϑΥʔϚϯεܭଌͷ೉͠͞
    w 8BOUFEMZͰ͸"1.ͷج൫ͱͯ͠/FX3FMJDΛ࠾༻͍ͯͨ͠
    w SVOUJNFϨϕϧͰͷϝτϦΫεͷऩू
    w -BUFODZ 4UBDLUSBDF ($IFBQTJ[F ʜ
    w .JDSPTFSWJDFTͰ͸αʔϏεؒͷσʔλͷ΍ΓऔΓ͕ωοτϫʔΫӽ͠ʹߦΘΕΔ
    w ϦΫΤετઌͰͲΜͳॲཧ͕ߦΘΕ͍ͯΔͷ͔Λ൑அͰ͖ͳ͍
    ‎෼ࢄτϨʔγϯάͷಋೖ
    

    View Slide

11. ©2018 Wantedly, Inc.
    ෼ࢄτϨʔγϯά
    

    View Slide

12. ©2018 Wantedly, Inc.
    ෼ࢄτϨʔγϯάಋೖͷΞϓϩʔν
    ͬ͘͟Γಋೖํ๏͸௨Γʹ෼ྨͰ͖Δ
    w ϕϯμʔ͕ఏڙ͢Δ-JCSBSZͷར༻ͱ࣮૷ʢྫ"NB[PO93BZ
    
    w 0QFO$FOTVT΍0QFO5SBDJOHʹ୅ද͢Δڞ௨Խ࣮૷
    

    View Slide

13. ©2018 Wantedly, Inc.
    ٕज़બ୒Λ͢Δͱ͖ʹߟ͑ͨ͜ͱ
    w όοΫΤϯυͷมߋ༰қੑͷ୲อ
    w ܗ੒͞ΕͨΤίγεςϜͷར༻͕Մೳ
    w 1PMZHMPUͳ؀ڥͰ͋Δ͜ͱΛҙࣝ͢Δͱɺ

    গ͠Ͱ΋ΤίγεςϜʹ৐ΕΔܗʹ͍ͨ͠
    ‎ڞ௨ԽϑϨʔϜϫʔΫͷ࠾༻
    

    View Slide

14. ©2018 Wantedly, Inc.
    Կނ0QFO$FOTVTΛ࠾༻ͨ͠ͷ͔
    w ͲͪΒ΋࣮૷ΛϕϯμʔΛ෼཭͢Δͱ͍͏໨త͸Ұॹ
    w ෼ࢄτϨʔγϯάҎ֎ͷར༻Ձ஋Λߟ͑ͨ
    w 0QFO5SBDJOH͸෼ࢄτϨʔγϯά͚ͩͰ͋Δͷʹର͠ɺ

    0QFO$FOTVT͸0CTFSWBCJMJUZશൠʹϑΥʔΧε͍ͯ͠Δ
    w ΞϓϦέʔγϣϯͷϝτϦΫεͷऩूΛݟਾ͑ͨ
    w ΤίγεςϜͷ੒௕ͷݟࠐΈ
    w (PPHMF͸4UBDLESJWFSͰ0QFO$FOTVTͷར༻Λਪ঑͍ͯͨ͠
    

    View Slide

15. ©2018 Wantedly, Inc.
    8BOUFEMZ *ODʹ͓͚Δ෼ࢄτϨʔγϯά
    

    View Slide

16. ©2018 Wantedly, Inc.
    ෼ࢄτϨʔγϯάͷόοΫΤϯυ
    w όοΫΤϯυʹ͸%BUBEPH"1.ͱ4UBDLESJWFS5SBDFΛ࠾༻
    w %BUBEPH"1.͕ϝΠϯ
    w ΋ͱ΋ͱ%BUBEPHΛར༻͍ͯͨ͠ͱ͍͏എܠ
    w %BUBEPH"1.Ͱ͸ಛఆͷ؀ڥ͚ͩ͢΂ͯͷ5SBDFΛ࢒͢ࣄ͕Ͱ͖ͳ͍
    w ͜ΕΛ୲อ͢ΔͨΊʹ4UBDLESJWFS5SBDFͱซ༻͍ͯ͠Δ
    w কདྷతʹ͸όοΫΤϯυΛ౷Ұͤ͞Δ
    

    View Slide

17. ©2018 Wantedly, Inc.
    4IBSFE-JCSBSZʹΑΔڞ௨Խ
    w ࣾ಺ʹ͸ϚΠΫϩαʔϏεʹඞཁͳϩδοΫΛ֤αʔϏεʹఏڙ͢ΔͨΊͷ

    ڞ௨Խ࣮૷͕ଘࡏ͍ͯͨ͠
    w 5JNFPVU3FUSZ"VUIFOUJDBUPO-PHHJOH
    w ෼ࢄτϨʔγϯάͷ࣮૷΋͜Εʹ಺แ͍ͤͯ͞Δ
    w 3VCZͱ(PͷΈͰఏڙ
    w 1ZUIPO΍ͦͷଞͷݴޠ͸ϦΫΤετͷ຤୺Ͱར༻͞ΕΔέʔε͕ଟ͍
    w ߋʹ%#΁ͷSFBEXSJUF΋΄΅ແ͍
    w VQTUSFBN΁ͷϦΫΤετͷ4QBO͚ͩͰ͋Δఔ౓൑அ͕Մೳ
    

    View Slide

18. ©2018 Wantedly, Inc.
    ར༻͍ͯ͠Δ-JCSBSZ
    w (Pʹؔͯ͠͸ެࣜͷ΋ͷΛར༻
    w 3VCZ͸GPSL IUUQTHJUIVCDPNXBOUFEMZPQFODFOTVTSVCZ
    Λར༻
    w ౰࣌UPQMFWFMͷ4QBO͕ࢠ4QBOʹ఻ൖ͞Εͳ͍ͱ͍͏க໋తͳόά͕͋ͬͨ
    w 13Λૹ͕ͬͨɺϝϯςφෆࡏͰϦϦʔε͞Εͳ͍࠷ۙϚʔδ͞Εͨ
    w UPQMFWFM4QBOʹ"UUSJCVUFΛࠩ͠ࠐΉͨΊͷIPPLΛࣗ࡞͍ͯͯ͠ɺ

    ͦΕ͕·ͩຊମʹ13Λग़͍ͯ͠ͳ͍
    w %BUBEPHͷ&YQPSUFSʹؔͯ͠͸ࣗ࡞
    w IUUQTHJUIVCDPNNVOJTZTUFNPQFODFOTVTEBUBEPH
    

    View Slide

19. ©2018 Wantedly, Inc.
    ϝλσʔλͷૠೖ
    w ݕࡧͷͨΊʹϝλσʔλ
    w ,VCFSOFUFTNFUBEBUB
    w 3FRVFTU*%
    w 6TFS*%
    w %BUBEPH"1.ͷ࠷దԽͷͨΊͷϝλσʔλ
    

    View Slide

20. ©2018 Wantedly, Inc.
    3FRVFTU*%ͷ఻ൖ
    w 3FRVFTU*%͸ผαʔϏεʹ఻ൖ͍ͯ͠Δ
    w "DDFTT-PHHJOH
    w &SSPSUSBDLFS )POFZCBEHFS
    
    

    View Slide

21. ©2018 Wantedly, Inc.
    )551)FBEFS1SPQBHBUJPO
    w 5SBDF$POUFYUQSPQBHBUJPOGPSNBUΛ࠾༻
    w 0QFO$FOTVTͷ࢓༷Ͱ5SBDF$POUFYUͷ࣮૷ؚ͕·Ε͍ͯΔ
    w PQFODFOTVTSVCZ͸#͕·࣮ͩ૷͞Ε͍ͯͳ͍
    

    View Slide

22. ©2018 Wantedly, Inc.
    4BNQMJOH
    w ։ൃ؀ڥ͸͢΂ͯͷϦΫΤετͷॲཧΛτϨʔεͰ͖ΔΑ͏ʹɺ

    αϯϓϦϯά͸ߦ͍ͬͯͳ͍
    w 1SPEVDUJPO؀ڥ͸αʔϏε΍ΤϯυϙΠϯτʹؔΘΒͣҰ཯ˋͰઃఆ
    w ϦΫΤετ਺΍4QBOͷ਺͕ଟ͍ͨΊ
    w ΞϓϦέʔγϣϯଆͷQFSGPSNBODFJTTVF
    w όοΫΤϯυͷQSJDJOH
    

    View Slide

23. ©2018 Wantedly, Inc.
    ($1Ͱͷར༻
    w 4%,ଆͰ0QFO$FOTVT͕ରԠ͞Ε͍ͯΔ
    w ಛʹͳʹ΋ͤͣFDPTZTUFNʹ৐ΕΔ
    

    View Slide

24. ©2018 Wantedly, Inc.
    ݱঢ়ͷ՝୊
    

    View Slide

25. ©2018 Wantedly, Inc.
    όοΫΤϯυݻ༗ͷ࠷దԽͷ࣮ݱ
    w ෇༩͞Ε͍ͯΔ"UUSJCVUFʹΑͬͯόοΫΤϯυͷৼΔ෣͍͕มΘΔ
    w %BUBEPH"1.Ͱ͸TQBOUZQFͷ஋ͰαʔϏεͷΧςΰϥΠζ͕มΘΔ
    w UPQMFWFMͷ4QBOͰղܾ͢Δ΋ͷͳΒ·͍͍ͩ
    w SFEJTDMJFOUʹಛఆͷ"UUSJCVUFΛࠩ͠ࠐΉʹ͸Ͳ͏͢Δ
    w SFEJTDMJFOUଆͰIPPLΛ༻ҙ͢Δ
    w NPOLFZQBUDIΛ͋ͯΔ
    

    View Slide

26. ©2018 Wantedly, Inc.
    ΞϓϦέʔγϣϯίʔυͱϕϯμʔͷ׬શͳ෼཭
    w &YQPSUFS͸ίʔυʹґଘ
    w ϥΠϒϥϦͷΞοϓσʔτͷ೉͠͞
    w 1PMZHMPUͱ&YQPSUFS࣮૷
    w ओཁͳݴޠ͚࣮ͩ૷͢Δͷ΋ݱ࣮తͰ͸ͳ͍
    w &YQPSUFSͷࣗ࡞ͱ͍͏બ୒ࢶ
    w ϕϯμʔଆͷػೳ΁ͷ௥ैͷίετ
    

    View Slide

27. ©2018 Wantedly, Inc.
    0QFO$FOTVT4FSWJDF
    https://github.com/census-instrumentation/opencensus-service/blob/master/images/opencensus-service-deployment-models.png
    

    View Slide

28. ©2018 Wantedly, Inc.
    0QFO$FOTVT4FSWJDF
    w ϕϯμʔͱΞϓϦέʔγϣϯͷ׬શͳ෼཭ͷ࣮ݱ
    w "HFOUʜ5SBDF΍4UBUTͷऩूɺ$PMMFDUPS΁ͷू໿
    w $PMMFDUPSʜ೚ҙͷόοΫΤϯυʹू໿ͨ͠5SBDF΍4UBUTΛૹ৴
    w ར఺
    w TBNQMJOHTUSBUFHZͷมߋ΍ϝλσʔλͷ෇༩Λ$PMMFDUPSଆͰ࣮ݱ
    w ϕϯμʔ͸(Pͷ࣮૷͚ͩ༻ҙ͢Ε͹ྑ͍
    

    View Slide

29. ©2018 Wantedly, Inc.
    ·ͱΊ
    w 0QFO$FOTVTΛ෼ࢄτϨʔγϯάͰར༻͢ΔϝϦοτ
    w ϕϯμʔͱίʔυͷ෼཭ͷ࣮ݱ
    w ϩά΍ϝτϦΫεͷऩू΁ͷ֦ுੑ
    w ΤίγεςϜͷେ͖͞
    w 8BOUFEMZ *ODʹ͓͚Δ෼ࢄτϨʔγϯά
    w 4IBSFE-JCSBSZʹΑΔ࣮૷ͷڞ௨Խ
    w ݕࡧͷͨΊͷϝλσʔλͷૹ৴
    

    View Slide