自作ロードバランサ開発 / JANOG40 SP6LB

Transcript

1. ࣗ࡞ϩʔυόϥϯα։ൃ :PIFJ ,BOFNBSV -*/&
   $PSQPSBUJPO ZPIFJLBOFNBSV BU
   MJOFDPSQDPN 
   !ZVOB[VOP

2. ͜Μͳιϑτ΢ΣΞ-#Λ։ൃ͍ͯ͠·͢ • -*/&ͷαʔϏεʹඞཁͳػೳΛඋ͍͑ͯΔ • ී௨ͷY
   -JOVYαʔό্Ͱಈ࡞͢Δ • ߴ͍εέʔϥϏϦςΟ͕͋Δ 2017-07-28 JANOG 40

   Meeting 2 ৄ͍͠എܠ͸ +"/0(
   ൃදࢿྉ 
   ొஃϨϙʔτʹͯ [1] LINE    (JANOG39 ) https://www.janog.gr.jp/meeting/janog39/program/line.html [2] JANOG39
   LINE    https://engineering.linecorp.com/ja/blog/detail/135

3. ϩʔυόϥϯαͷΞʔΩςΫνϟ͍Ζ͍Ζ  $FOUSBMJ[FE
   -PBE
   #BMBODFST
   .JDSPTPGU
   "OBOUB
   'BDFCPPL
   TIJW (PPHMF
   .BHMFW
     %FDFOUSBMJ[FE
   -PBE
   #BMBODFST ($1
   *OUFSOBM
   -#

   
   $MPVEGMBSF 1P1T
    2017-07-28 JANOG 40 Meeting Ωʔϫʔυ
   / 
   
   .VMUJ5JFS MS Ananta FB shiv Google Maglev [1] Microsoft Ananta https://doi.org/10.1145/2534169.2486026 [2] Facebook Shiv: https://www.usenix.org/conference/srecon15/program/presentation/shuff [3] Google Maglev https://research.google.com/pubs/pub44824.html 3

4.  ߏ੒͔Β /   .VMUJUJFSߏ੒΁ 2017-07-28 JANOG 40 Meeting

   4 
   
    /
   
   
   
   .VMUJUJFS Client L4 LB APP : In service : Not in service L4 LB APP L7 LB L3 Switch Client L3DSR TCP/HTTP(S) termination L3DSR ECMP ࠓճ͝঺հ͢Δͷ͸ ͜ͷ෦෼

5. ஈ໨  #(1 &$.1 1FSGMPX
   &$.1
   ʹΑΓɺ--#΁͓͓Αͦۉ౳ʹτϥϑΟοΫΛ෼ࢄ 2017-07-28 JANOG 40 Meeting 5

   Equal Cost Multi Path (ECMP) VIP: 192.0.2.1/32 L4LB-1 Hash(5-tuple) -> L4LB-[1-3] L3 Switch VIP: 192.0.2.1/32 L4LB-2 VIP: 192.0.2.1/32 L4LB-3 eBGP peer Flow X Flow Y

6. ஈ໨  4UBUFMFTT --# $POTJTUFOU
   )BTIͰ෼ࢄઌ--#Λܾఆ͠ɺ-%43Ͱύέοτసૹ 2017-07-28 JANOG 40 Meeting 6

   L3 Switch L4LB-1 L7LB-1 L7LB-2 L4LB-2 L7LB-3 L7LB-4 ConsistentHash(Hash(5-tuple)) -> L7LB-[1-4] Flow X Flow Y

7. --#  F9QSFTT %BUB 1BUI 9%1 Λ׆༻ -JOVY
   LFSOFM಺Ͱಈ࡞͢ΔɺF#1'Λ༻͍ͨߴ଎ύέοτॲཧج൫ 2017-07-28 JANOG

   40 Meeting eBPF Map VIP-Backend table Statistics table C eBPF C code (Data Plane) Clang/LLVM eBPF Bytecode Python Management Plane eBPF JIT eBPF VM API VIP->Backend lookup Statistics Kernel User eBPF Verifier 7 BGP daemon XDP Program From L3 Switch To L7LB Kernel NIC Advertise VIP Compiler  -PPLVQ
   7*1  )BTI UVQMF  .PEJGZ
   *1
   IFBEFS Maintain map

8. --#  ϊʔυ୯ମύϑΥʔϚϯε 2017-07-28 JANOG 40 Meeting 8 )BSEXBSF
   
   Y
   *OUFM
   9FPO
   &
   W
   
   DPSFT

   
   
   (
   %%3
   
   .FMMBOPY
   $POOFDU9
   -Y
   &/
   (C& 4PGUXBSF
   -JOVY
   ,FSOFM
   SD EDCG 
   #1'
   +*5
   FOBCMFE &OWJSPONFOU 19.1 15.0 10.8 5.8 2.9 2.0 0% 20% 40% 60% 80% 100% 0 4 8 12 16 20 64 128 256 512 1024 1518 Link Utilization [%] Throughput [Mpps] Frame size [byte] Throughput [Mpps] Link Utilization (/25GbE) [%] 25GbE wire-rate

9. ࣮ࡍͷͱ͜ΖͲΜͳঢ়گ ౰ॳͷૂ͍͸͓͓ΉͶୡ੒Ͱ͖ͦ͏ खݩͷίʔυͱಉҰͷ΋ͷ͕࣮ωοτϫʔΫͰͦͷ··ಈ࡞͢Δ ςετ͕͠΍͍͢ɾಈ࡞͕ݟ͑΍͍͢ / ߏ੒͕໰୊ʹͳΔέʔε΋͋Δ .VMUJQBUI
   5$1
   *1
   'SBHNFOU
   Ұ෦ͷಛघҊ݅
   

   2017-07-28 JANOG 40 Meeting 9

10. ࠓޙͷల๬ ຊ֨తʹϓϩμΫγϣϯ؀ڥ΁ͷ౤ೖΛ։࢝ ػೳ௥Ճ΍ߏ੒ͷΞοϓσʔτ %P4 
    %%P4
    ๷ޚ εςʔτϑϧηογϣϯΩϟογϡ -36
    DBDIF
    ͷซ༻ #(1
    VOOVNCFSFE
    3'$
    

    %$
    OFUXPSLͷ-Խ ϋʔυ΢ΣΞدΓͷΞϓϩʔνΛௐࠪɾݕ౼ 9%1
    F#1'
    IBSEXBSF
    PGGMPBE
    1SPHSBNNBCMF
    "4*$
    1
     2017-07-28 JANOG 40 Meeting 10