Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Distributed Tracing with OpenCensus at Wantedly, Inc.

Distributed Tracing with OpenCensus at Wantedly, Inc.

Yuichi Saito

April 03, 2019
Tweet

More Decks by Yuichi Saito

Other Decks in Technology

Transcript

  1. ©2018 Wantedly, Inc.
    Distributed Tracing with OpenCensus
    at Wantedly, Inc.
    OpenCensus meetup vol.1
    03.Apr.2019 - @munisystem

    View Slide

  2. ©2018 Wantedly, Inc.
    0QFO$FOTVTͷಋೖഎܠ
    8BOUFEMZ *ODʹ͓͚Δ෼ࢄτϨʔγϯά
    ݱঢ়ͷ՝୊
    ·ͱΊ
    Agenda

    View Slide

  3. ©2018 Wantedly, Inc.
    !NVOJTZTUFN:VJDIJ4BJUP
    *OGSBTUSVDUVSF5FBNBU8BOUFEMZ *OD

    View Slide

  4. ©2018 Wantedly, Inc.
    0QFO$FOTVTͷಋೖഎܠ

    View Slide

  5. ©2018 Wantedly, Inc.
    8BOUFEMZʹ͓͚Δ.JDSPTFSWJDFTͷݱঢ়
    w 1PMZHMPU
    w 3VCZͱ(P͕ϝΠϯ
    w αʔϏεͷυϝΠϯʹΑͬͯ1ZUIPO΍/PEFɺ3VTU΋࠾༻
    w .VMUJ$MPVE
    w "84 ,VCFSOFUFTPO&$ 3%4 &MBTUJDBDIF ʜ

    w ($1 #JH2VFSZ $MPVE%BUBTUPSF $MPVE1VC4VC ʜ

    View Slide

  6. ©2018 Wantedly, Inc.
    ͳͥ.JDSPTFSWJDFTΛ࠾༻ͨ͠ͷ͔
    ೥ͷ8BOUFEMZ1FPQMFͷ։ൃ͔Β.JDSPTFSWJDFTΛ࠾༻
    w ౰࣌͸ϞϊϦγοΫͳ3BJMTΞϓϦέʔγϣϯ͕਺ݸଘࡏ
    w αʔϏεͷಛੑ্ɺ3BJMTͰ͸࣮ݱ͠ʹ͍͘ػೳΛ࣮ݱ͢Δඞཁ͕͋ͬͨ
    w ฒྻԽɺػցֶश
    w ߴ଎ͳ։ൃͱϦϦʔε΁ͷཁٻ
    w ϦϦʔεαΠΫϧͷ෼཭

    View Slide

  7. ©2018 Wantedly, Inc.
    ύϑΥʔϚϯεܭଌͷ೉͠͞
    w 8BOUFEMZͰ͸"1.ͷج൫ͱͯ͠/FX3FMJDΛ࠾༻͍ͯͨ͠

    View Slide

  8. ©2018 Wantedly, Inc.
    ύϑΥʔϚϯεܭଌͷ೉͠͞
    w 8BOUFEMZͰ͸"1.ͷج൫ͱͯ͠/FX3FMJDΛ࠾༻͍ͯͨ͠
    w SVOUJNFϨϕϧͰͷϝτϦΫεͷऩू
    w -BUFODZ 4UBDLUSBDF ($IFBQTJ[F ʜ
    w .JDSPTFSWJDFTͰ͸αʔϏεؒͷσʔλͷ΍ΓऔΓ͕ωοτϫʔΫӽ͠ʹߦΘΕΔ
    w ϦΫΤετઌͰͲΜͳॲཧ͕ߦΘΕ͍ͯΔͷ͔Λ൑அͰ͖ͳ͍

    View Slide

  9. ©2018 Wantedly, Inc.
    ύϑΥʔϚϯεܭଌͷ೉͠͞
    Receive A /api/products/1
    timeline
    DB Access
    Request B /api/reviews/1
    #Ͱͳʹ͕ߦΘΕ͍ͯΔͷ͔͸"ͷ4UBDL5SBDF্͔Β͸൑அͰ͖ͳ͍

    View Slide

  10. ©2018 Wantedly, Inc.
    ύϑΥʔϚϯεܭଌͷ೉͠͞
    w 8BOUFEMZͰ͸"1.ͷج൫ͱͯ͠/FX3FMJDΛ࠾༻͍ͯͨ͠
    w SVOUJNFϨϕϧͰͷϝτϦΫεͷऩू
    w -BUFODZ 4UBDLUSBDF ($IFBQTJ[F ʜ
    w .JDSPTFSWJDFTͰ͸αʔϏεؒͷσʔλͷ΍ΓऔΓ͕ωοτϫʔΫӽ͠ʹߦΘΕΔ
    w ϦΫΤετઌͰͲΜͳॲཧ͕ߦΘΕ͍ͯΔͷ͔Λ൑அͰ͖ͳ͍
    ‎෼ࢄτϨʔγϯάͷಋೖ

    View Slide

  11. ©2018 Wantedly, Inc.
    ෼ࢄτϨʔγϯά

    View Slide

  12. ©2018 Wantedly, Inc.
    ෼ࢄτϨʔγϯάಋೖͷΞϓϩʔν
    ͬ͘͟Γಋೖํ๏͸௨Γʹ෼ྨͰ͖Δ
    w ϕϯμʔ͕ఏڙ͢Δ-JCSBSZͷར༻ͱ࣮૷ʢྫ"NB[PO93BZ

    w 0QFO$FOTVT΍0QFO5SBDJOHʹ୅ද͢Δڞ௨Խ࣮૷

    View Slide

  13. ©2018 Wantedly, Inc.
    ٕज़બ୒Λ͢Δͱ͖ʹߟ͑ͨ͜ͱ
    w όοΫΤϯυͷมߋ༰қੑͷ୲อ
    w ܗ੒͞ΕͨΤίγεςϜͷར༻͕Մೳ
    w 1PMZHMPUͳ؀ڥͰ͋Δ͜ͱΛҙࣝ͢Δͱɺ

    গ͠Ͱ΋ΤίγεςϜʹ৐ΕΔܗʹ͍ͨ͠
    ‎ڞ௨ԽϑϨʔϜϫʔΫͷ࠾༻

    View Slide

  14. ©2018 Wantedly, Inc.
    Կނ0QFO$FOTVTΛ࠾༻ͨ͠ͷ͔
    w ͲͪΒ΋࣮૷ΛϕϯμʔΛ෼཭͢Δͱ͍͏໨త͸Ұॹ
    w ෼ࢄτϨʔγϯάҎ֎ͷར༻Ձ஋Λߟ͑ͨ
    w 0QFO5SBDJOH͸෼ࢄτϨʔγϯά͚ͩͰ͋Δͷʹର͠ɺ

    0QFO$FOTVT͸0CTFSWBCJMJUZશൠʹϑΥʔΧε͍ͯ͠Δ
    w ΞϓϦέʔγϣϯͷϝτϦΫεͷऩूΛݟਾ͑ͨ
    w ΤίγεςϜͷ੒௕ͷݟࠐΈ
    w (PPHMF͸4UBDLESJWFSͰ0QFO$FOTVTͷར༻Λਪ঑͍ͯͨ͠

    View Slide

  15. ©2018 Wantedly, Inc.
    8BOUFEMZ *ODʹ͓͚Δ෼ࢄτϨʔγϯά

    View Slide

  16. ©2018 Wantedly, Inc.
    ෼ࢄτϨʔγϯάͷόοΫΤϯυ
    w όοΫΤϯυʹ͸%BUBEPH"1.ͱ4UBDLESJWFS5SBDFΛ࠾༻
    w %BUBEPH"1.͕ϝΠϯ
    w ΋ͱ΋ͱ%BUBEPHΛར༻͍ͯͨ͠ͱ͍͏എܠ
    w %BUBEPH"1.Ͱ͸ಛఆͷ؀ڥ͚ͩ͢΂ͯͷ5SBDFΛ࢒͢ࣄ͕Ͱ͖ͳ͍
    w ͜ΕΛ୲อ͢ΔͨΊʹ4UBDLESJWFS5SBDFͱซ༻͍ͯ͠Δ
    w কདྷతʹ͸όοΫΤϯυΛ౷Ұͤ͞Δ

    View Slide

  17. ©2018 Wantedly, Inc.
    4IBSFE-JCSBSZʹΑΔڞ௨Խ
    w ࣾ಺ʹ͸ϚΠΫϩαʔϏεʹඞཁͳϩδοΫΛ֤αʔϏεʹఏڙ͢ΔͨΊͷ

    ڞ௨Խ࣮૷͕ଘࡏ͍ͯͨ͠
    w 5JNFPVU3FUSZ"VUIFOUJDBUPO-PHHJOH
    w ෼ࢄτϨʔγϯάͷ࣮૷΋͜Εʹ಺แ͍ͤͯ͞Δ
    w 3VCZͱ(PͷΈͰఏڙ
    w 1ZUIPO΍ͦͷଞͷݴޠ͸ϦΫΤετͷ຤୺Ͱར༻͞ΕΔέʔε͕ଟ͍
    w ߋʹ%#΁ͷSFBEXSJUF΋΄΅ແ͍
    w VQTUSFBN΁ͷϦΫΤετͷ4QBO͚ͩͰ͋Δఔ౓൑அ͕Մೳ

    View Slide

  18. ©2018 Wantedly, Inc.
    ར༻͍ͯ͠Δ-JCSBSZ
    w (Pʹؔͯ͠͸ެࣜͷ΋ͷΛར༻
    w 3VCZ͸GPSL IUUQTHJUIVCDPNXBOUFEMZPQFODFOTVTSVCZ
    Λར༻
    w ౰࣌UPQMFWFMͷ4QBO͕ࢠ4QBOʹ఻ൖ͞Εͳ͍ͱ͍͏க໋తͳόά͕͋ͬͨ
    w 13Λૹ͕ͬͨɺϝϯςφෆࡏͰϦϦʔε͞Εͳ͍࠷ۙϚʔδ͞Εͨ
    w UPQMFWFM4QBOʹ"UUSJCVUFΛࠩ͠ࠐΉͨΊͷIPPLΛࣗ࡞͍ͯͯ͠ɺ

    ͦΕ͕·ͩຊମʹ13Λग़͍ͯ͠ͳ͍
    w %BUBEPHͷ&YQPSUFSʹؔͯ͠͸ࣗ࡞
    w IUUQTHJUIVCDPNNVOJTZTUFNPQFODFOTVTEBUBEPH

    View Slide

  19. ©2018 Wantedly, Inc.
    ϝλσʔλͷૠೖ
    w ݕࡧͷͨΊʹϝλσʔλ
    w ,VCFSOFUFTNFUBEBUB
    w 3FRVFTU*%
    w 6TFS*%
    w %BUBEPH"1.ͷ࠷దԽͷͨΊͷϝλσʔλ

    View Slide

  20. ©2018 Wantedly, Inc.
    3FRVFTU*%ͷ఻ൖ
    w 3FRVFTU*%͸ผαʔϏεʹ఻ൖ͍ͯ͠Δ
    w "DDFTT-PHHJOH
    w &SSPSUSBDLFS )POFZCBEHFS

    View Slide

  21. ©2018 Wantedly, Inc.
    )551)FBEFS1SPQBHBUJPO
    w 5SBDF$POUFYUQSPQBHBUJPOGPSNBUΛ࠾༻
    w 0QFO$FOTVTͷ࢓༷Ͱ5SBDF$POUFYUͷ࣮૷ؚ͕·Ε͍ͯΔ
    w PQFODFOTVTSVCZ͸#͕·࣮ͩ૷͞Ε͍ͯͳ͍

    View Slide

  22. ©2018 Wantedly, Inc.
    4BNQMJOH
    w ։ൃ؀ڥ͸͢΂ͯͷϦΫΤετͷॲཧΛτϨʔεͰ͖ΔΑ͏ʹɺ

    αϯϓϦϯά͸ߦ͍ͬͯͳ͍
    w 1SPEVDUJPO؀ڥ͸αʔϏε΍ΤϯυϙΠϯτʹؔΘΒͣҰ཯ˋͰઃఆ
    w ϦΫΤετ਺΍4QBOͷ਺͕ଟ͍ͨΊ
    w ΞϓϦέʔγϣϯଆͷQFSGPSNBODFJTTVF
    w όοΫΤϯυͷQSJDJOH

    View Slide

  23. ©2018 Wantedly, Inc.
    ($1Ͱͷར༻
    w 4%,ଆͰ0QFO$FOTVT͕ରԠ͞Ε͍ͯΔ
    w ಛʹͳʹ΋ͤͣFDPTZTUFNʹ৐ΕΔ

    View Slide

  24. ©2018 Wantedly, Inc.
    ݱঢ়ͷ՝୊

    View Slide

  25. ©2018 Wantedly, Inc.
    όοΫΤϯυݻ༗ͷ࠷దԽͷ࣮ݱ
    w ෇༩͞Ε͍ͯΔ"UUSJCVUFʹΑͬͯόοΫΤϯυͷৼΔ෣͍͕มΘΔ
    w %BUBEPH"1.Ͱ͸TQBOUZQFͷ஋ͰαʔϏεͷΧςΰϥΠζ͕มΘΔ
    w UPQMFWFMͷ4QBOͰղܾ͢Δ΋ͷͳΒ·͍͍ͩ
    w SFEJTDMJFOUʹಛఆͷ"UUSJCVUFΛࠩ͠ࠐΉʹ͸Ͳ͏͢Δ
    w SFEJTDMJFOUଆͰIPPLΛ༻ҙ͢Δ
    w NPOLFZQBUDIΛ͋ͯΔ

    View Slide

  26. ©2018 Wantedly, Inc.
    ΞϓϦέʔγϣϯίʔυͱϕϯμʔͷ׬શͳ෼཭
    w &YQPSUFS͸ίʔυʹґଘ
    w ϥΠϒϥϦͷΞοϓσʔτͷ೉͠͞
    w 1PMZHMPUͱ&YQPSUFS࣮૷
    w ओཁͳݴޠ͚࣮ͩ૷͢Δͷ΋ݱ࣮తͰ͸ͳ͍
    w &YQPSUFSͷࣗ࡞ͱ͍͏બ୒ࢶ
    w ϕϯμʔଆͷػೳ΁ͷ௥ैͷίετ

    View Slide

  27. ©2018 Wantedly, Inc.
    0QFO$FOTVT4FSWJDF
    https://github.com/census-instrumentation/opencensus-service/blob/master/images/opencensus-service-deployment-models.png

    View Slide

  28. ©2018 Wantedly, Inc.
    0QFO$FOTVT4FSWJDF
    w ϕϯμʔͱΞϓϦέʔγϣϯͷ׬શͳ෼཭ͷ࣮ݱ
    w "HFOUʜ5SBDF΍4UBUTͷऩूɺ$PMMFDUPS΁ͷू໿
    w $PMMFDUPSʜ೚ҙͷόοΫΤϯυʹू໿ͨ͠5SBDF΍4UBUTΛૹ৴
    w ར఺
    w TBNQMJOHTUSBUFHZͷมߋ΍ϝλσʔλͷ෇༩Λ$PMMFDUPSଆͰ࣮ݱ
    w ϕϯμʔ͸(Pͷ࣮૷͚ͩ༻ҙ͢Ε͹ྑ͍

    View Slide

  29. ©2018 Wantedly, Inc.
    ·ͱΊ
    w 0QFO$FOTVTΛ෼ࢄτϨʔγϯάͰར༻͢ΔϝϦοτ
    w ϕϯμʔͱίʔυͷ෼཭ͷ࣮ݱ
    w ϩά΍ϝτϦΫεͷऩू΁ͷ֦ுੑ
    w ΤίγεςϜͷେ͖͞
    w 8BOUFEMZ *ODʹ͓͚Δ෼ࢄτϨʔγϯά
    w 4IBSFE-JCSBSZʹΑΔ࣮૷ͷڞ௨Խ
    w ݕࡧͷͨΊͷϝλσʔλͷૹ৴

    View Slide