라스트 마일에서 데이터센터까지: Network Error Logging과 eBPF를 통한 네트워크 성능 모니터링

#Network Performance Monitoring #eBPF #Network Error Logging

라스트마일 네트워크와 데이터센터 네트워크 성능을 모니터링하기 위해 각각 Network Error Logging과 eBPF를 도입한 과정과 이를 통해 네트워크 성능을 개선시킨 경험을 공유합니다.

발표자 : dal.gona
카카오에서 네트워크 운영 및 관련 서비스 개발을 담당하고 있습니다.

Transcript

2. 
   ۄझ౟
   ݃ੌ
   ֎౟ਕ௼
   ࢿמ
   ݽפఠ݂ 
   ؘ੉ఠࣃఠ
   ֎౟ਕ௼
   ࢿמ
   ݽפఠ݂
3. ۄझ౟
   ݃ੌ
   ֎౟ਕ௼
   ࢿמ
   ݽפఠ݂
4. ө׮۽਍
   ۄझ౟
   ݃ੌ
   ֎౟ਕ௼
   ޙઁٜ  ౠ੿
   Ҵо
   ాनࢎ
   ױ݈ӝ
   ࢎਊ੗੄
   ਢ
   ࢎ੉౟
   ੽ࣘ
   ࠛ۝
   ޙઁ  ৻ࠗ
   $%/
   স୓੄
   ֎౟ਕ௼
   ޙઁ  %/4
   য়ܨ
   ֎౟ਕ௼
   ۄ਋౴
   য়ܨ
   ١
   ࢲߡীࢲ
   х૑ೞӝ
   য۰਍
   ޙઁٜ j
5. /FUXPSL
   &SSPS
   -PHHJOH
   /&-  ਢ
   ࠳ۄ਋੷ীࢲ
   पઁ
   ࢎਊ੗о
   ѻ਷
   ֎౟ਕ௼
   য়ܨܳ
   ӝ۾೧઱ח
   ӝࣿ  8$
   8PSLJOH
   %SBGU  $ISPNJVN
   ӝ߈
   ࠳ۄ਋੷ী
   ҳഅؽ 
   $SFBHFS
   %
   
   $MFMMBOE
   *
   
   0DUPCFS
    
   /FUXPSL
   &SSPS
   -PHHJOH
   80SH
   IUUQT XXXXPSH53OFUXPSL - FSSPS - MPHHJOH
6. /&-
   ز੘
   ߑध
7. /&-
   ز੘
   ߑध
8. /&-
   ز੘
   ߑध
9. /&-
   ز੘
   ߑध
10. /&-
    ز੘
    ߑध
11. /&-
    ܻನ౟
    ഋध  3FRVFTU
    63-  3FRVFTU
    .FUIPE  4FSWFS
    *1  )551
    7FSTJPO  3FTQPOTF
    4UBUVT
    $PEF  1IBTF
    %/4
    
    $POOFDUJPO
    
    "QQMJDBUJPO  &MBQTFE
    5JNF
12. /&-
    ܻನ౟
    ഋध  5ZQF  EOTOBNF@OPU@SFTPMWFE  UDQSFGVTFE  UDQBEESFTT@VOSFBDIBCMF  UMTWFSTJPO@PS@DJQIFS@NJTNBUDI  IUUQFSSPS  j  4PVSDF
    *1  6TFS - "HFOU
13. /&-
    ഝਊ
    ࢎ۹ ࢎ۹
    ੌࠄ
    ౠ੿
    ాनࢎ
    ࢎਊ੗੄
    ஠஠য়ݎ
    ੽ࣘ
    ࠛо
    ੉ग
    ഛੋ
14. /&-
    ഝਊ
    ࢎ۹ ࢎ۹
    ۄझ౟݃ੌ
    ֎౟ਕ௼
    ࢿמ
    ஏ੿ 
    )551
    ߡ੹
    ߹
    )551
    ਽׹
    दр
    ࠺Ү
15. /&-
    ഝਊ
    ࢎ۹ ࢎ۹
    ۄझ౟݃ੌ
    ֎౟ਕ௼
    ࢿמ
    ஏ੿ 
    ࢎਊ੗
    ੽ࣘ
    Ҵо
    ߹
    )551
    ਽׹
    दр
    ࠺Ү
16. ؘ੉ఠࣃఠ
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
17. -BUFODZ 5ISPVHIQVU ֎౟ਕ௼
    ࢿמ
18. 0VU
    PG
    0SEFS -PTT )JHI
    355 ֎౟ਕ௼
    ࢿמ
19. ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
    ߑߨ OTUBU TT
    J UDQEVNQ GUSBDF F#1' ݽפఠ݂
    ࠗೞ ೞ ೞ ࢚ ೞ ೞ ੿ࠁ۝ ೞ ઺ ઺ ઺ ࢚ ױ੼ 'MPX
    ߹
    ా҅ח
    ঌ
    ࣻ
    হ਺ ࡅܰѱ
    ࢎۄ૑ח
    ࣗ௄੄
    ੿ࠁח
    ഛੋೞӝ
    য۰਑ ޙઁܳ
    ੗ز੸ਵ۽
    
 ߊѼೞӝ
    য۰਑ ೐۽Ӓې߁
    ೠ҅
20. F#1'
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂ 5$1-JGF 5$13FUSBOT 4LC%SPQ ౟ۨ੉य
    ೣࣻ JOFU@TPDL@TFU@TUBUF UDQ@SFUSBOTNJU@TLC LGSFF@TLC ഛੋ
    оמೠ
    ੿ࠁ 'MPX
    ߹
    MPTT
    PVU
    PG
    PSEFS
    
 355
    ੤੹࣠ػ
    ಁఉ ഐझ౟ীࢲ
    ٘ۉػ
    ಁఉҗ
    Ӓ
    ਗੋ ઱੄੼ ૑ࣘ
    ழ֏࣌ਸ
    ࢎਊೞח
    ҃਋
    
 ౟ۨ੉यغ૑
    ঋ਺ 5SBDFQPJOU
    ࢎਊ
    द
    UBJM
    MPTT
    QSPCF
    ੉߮౟ী
    ઱੄ 
    (SFHH
    #
    
    /PWFNCFS
     
    -JOVY
    #DD#1'
    5DQMJGF
    #SFOEBOHSFHH$PN
    IUUQT XXXCSFOEBOHSFHHDPNCMPHMJOVY - CDD - UDQMJGFIUNM 
 
    (SFHH
    #
    
    4FQUFNCFS
     
    -JOVY
    'USBDF
    5$1
    3FUSBOTNJU
    5SBDJOH
    #SFOEBOHSFHH$PN
    IUUQT XXXCSFOEBOHSFHHDPNCMPHMJOVY - GUSBDF - UDQ - SFUSBOTNJU - USBDJOHIUNM
21. 5$1-JGF USBDFQPJOU : TPDL : JOFU@TPDL@TFU@TUBUF  ࣗ௄੄
    ࢚కо
    ߸҃ؼ
    ٸ
    ഐ୹
    ৘
    5$1@5*.&@8"*5
    ->
    5$1@$-04&  BSHT  TUSVDU
    TPDL
    TL  JOU
    PMETUBUF  *OU
    OFXTUBUF
22. 5$1-JGF #1'
    )FMQFS
    'VODUJPO
    CQG@TLD@UP@UDQ@TPDL  TPDL
    ನੋఠܳ
    UDQ@TPDL
    ನੋఠ۽
    ߸ജ  UDQ@TPDL
    ઱ਃ
    ೙٘  V
    TSUU@VT  V
    MPTU  V
    UPUBM@SFUSBOT  V
    SDW@PPPQBDL  V
    SFPSE@TFFO 

23. 5$13FUSBOT USBDFQPJOU : UDQ : UDQ@SFUSBOTNJU@TLC  4MPX
    TUBSU
    SFUSBOT
    GBTU
    SFUSBOT
    UBJM
    MPTT
    QSPCF
    ߊࢤ
    द
    ഐ୹  5BJM
    MPTT
    QSPCFח
    ಁఉ
    ਬप੉
    হח
    ࢚ടীࢲب
    ੗઱
    ߊࢤೞ޲۽
    ઱੄  BSHT  TUSVDU
    TPDL
    TL  TUSVDU
    TL@CVGG
    TLC
24. 5$13FUSBOT LQSPCF : UDQ@SFUSBOTNJU@TLC  4MPX
    TUBSU
    SFUSBOT
    GBTU
    SFUSBOT
    ߊࢤ
    द
    ഐ୹  BSHT  TUSVDU
    TPDL
    TL  TUSVDU
    TL@CVGG
    TLC  *OU
    TFHT
25. 4LC%SPQ USBDFQPJOU : TLC : LGSFF@TLC 
    ಁఉਸ
    ߡܾ
    ٸ
    ഐ୹  BSHT  TUSVDU
    TL@CVGG
    TLC  WPJE
    MPDBUJPO  FOVN
    TLC@ESPQ@SFBTPO
    SFBTPO
26. 4LC%SPQ FOVN
    TLC@ESPQ@SFBTPO  /0@40$,&5  /&5'*-5&3@%301  40$,&5@3$7#6''  2%*4$@%301  $16@#"$,-0(  '6--@3*/(  /05@41&$*'*&%  j
27. F#1'
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
    ഝਊ
    ࢎ۹ ࢎ۹
    য೒ܻா੉࣌
    ਽׹
    ૑ো੉
    ֎౟ਕ௼
    ޙઁ۽
    ੋೠ
    Ѫੋ૑
    ഛੋೡ
    ٸ  5$1-JGF
    5$13FUSBOT۽
    MPTT
    PGP
    IJHI
    355
    ١
    ֎౟ਕ௼
    ޙઁ
    ഛੋ  ಁఉ
    ਬप੉
    ੓׮ݶ
    4LC%SPQਵ۽
    ಁఉ੉
    ഐझ౟ীࢲ
    ٘ۉػ
    Ѥ
    ইצ૑
    ഛੋ
28. F#1'
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
    ഝਊ
    ࢎ۹
29. F#1'
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
    ഝਊ
    ࢎ۹ ࢎ۹
    5$1-JGFܳ
    ా೧
    ؘ੉ఠࣃఠ
    ղ
    PVU
    PG
    PSEFS
    ಁఉ
    х૑  *ࢎ
    /*$੄
    ౠ੿
    ӝמ
    ৔ೱ
30. ࢎ۹
    5$1-JGFܳ
    ా೧
    ؘ੉ఠࣃఠ
    ղ
    PVU
    PG
    PSEFS
    ಁఉ
    х૑  *ࢎ
    /*$੄
    ౠ੿
    ӝמ
    ৔ೱ F#1'
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
    ഝਊ
    ࢎ۹
31. ࢎ۹
    5$1-JGFܳ
    ా೧
    ؘ੉ఠࣃఠ
    ղ
    PVU
    PG
    PSEFS
    ಁఉ
    х૑  *ࢎ
    /*$੄
    ౠ੿
    ӝמ
    ৔ೱ F#1'
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
    ഝਊ
    ࢎ۹
32. ࢎ۹
    5$1-JGFܳ
    ా೧
    ؘ੉ఠࣃఠ
    ղ
    PVU
    PG
    PSEFS
    ಁఉ
    х૑  *ࢎ
    /*$੄
    ౠ੿
    ӝמ
    ৔ೱ F#1'
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
    ഝਊ
    ࢎ۹
33. ࢎ۹
    5$1-JGFܳ
    ా೧
    ؘ੉ఠࣃఠ
    ղ
    PVU
    PG
    PSEFS
    ಁఉ
    х૑  *ࢎ
    /*$੄
    ౠ੿
    ӝמ
    ৔ೱ F#1'
    ֎౟ਕ௼
    ࢿמ
    ݽפఠ݂
    ഝਊ
    ࢎ۹
34. ߣ৻
    F#1'
    ֎౟ਕ௼
    ࢿמ
    ౚ׬  ֿܻझীࢲ
    JOJU@SUPח
     NT
    NJO@SUPח
    NT  ੉ઁ
    F#1'ܳ
    ా೧
    ݽف
    ઑ੺
    оמ  #1'@40$,@014@5*.&065@*/*5ਵ۽
    JOJU@SUP
    ઑ੺  CQG@TFUTPDLPQU
    5$1@#1'@350@.*/ਵ۽
    NJO@SUP
    ઑ੺  #1'@40$,@014@5*.&065@*/*5ח
    KJG fi FT
    5$1@#1'@350@.*/਷
    VT
    ױਤ੐ਸ
    ઱੄
35. ߣ৻
    F#1'
    ֎౟ਕ௼
    ࢿמ
    ౚ׬ .BY
    )551
    
 3FTQPOTF
    5JNF 
    -PTT 
    -PTT 
    -PTT 
    -PTT *OJU
    350
     NT .JO
    350
    NT 
 EFGBVMU NT NT  NT  NT *OJU
    350
    NT .JO
    350
    NT NT NT NT NT
36. 2"
37. ଵҊ
    ޙ೴  $SFBHFS
    %
    
    $MFMMBOE
    *
    
    0DUPCFS
     
    /FUXPSL
    &SSPS
    -PHHJOH
    80SH
    IUUQT XXXXPSH53OFUXPSL - FSSPS - MPHHJOH  (SFHH
    #
    
    /PWFNCFS
     
    -JOVY
    #DD#1'
    5DQMJGF
    #SFOEBOHSFHH$PN
    IUUQT XXXCSFOEBOHSFHHDPNCMPHMJOVY - CDD - UDQMJGFIUNM  (SFHH
    #
    
    4FQUFNCFS
     
    -JOVY
    'USBDF
    5$1
    3FUSBOTNJU
    5SBDJOH
    #SFOEBOHSFHH$PN
    IUUQT XXXCSFOEBOHSFHHDPNCMPHMJOVY - GUSBDF - UDQ - SFUSBOTNJU - USBDJOHIUNM
38. &0%