Upgrade to Pro — share decks privately, control downloads, hide ads and more …

自作ロードバランサ開発 / JANOG40 SP6LB

自作ロードバランサ開発 / JANOG40 SP6LB

JANOG40 ショートプログラム 「自作ロードバランサ開発」
https://www.janog.gr.jp/meeting/janog40/program/sp6lb

7389db41aa4054d74d533abefd698a94?s=128

yunazuno

July 28, 2017
Tweet

Transcript

  1. ࣗ࡞ϩʔυόϥϯα։ൃ :PIFJ ,BOFNBSV -*/&$PSQPSBUJPO ZPIFJLBOFNBSV BUMJOFDPSQDPN !ZVOB[VOP

  2. ͜Μͳιϑτ΢ΣΞ-#Λ։ൃ͍ͯ͠·͢ • -*/&ͷαʔϏεʹඞཁͳػೳΛඋ͍͑ͯΔ • ී௨ͷY-JOVYαʔό্Ͱಈ࡞͢Δ • ߴ͍εέʔϥϏϦςΟ͕͋Δ 2017-07-28 JANOG 40

    Meeting 2 ৄ͍͠എܠ͸ +"/0(ൃදࢿྉ ొஃϨϙʔτʹͯ [1] LINE    (JANOG39 ) https://www.janog.gr.jp/meeting/janog39/program/line.html [2] JANOG39 LINE    https://engineering.linecorp.com/ja/blog/detail/135
  3. ϩʔυόϥϯαͷΞʔΩςΫνϟ͍Ζ͍Ζ  $FOUSBMJ[FE-PBE#BMBODFST .JDSPTPGU"OBOUB 'BDFCPPLTIJW (PPHMF.BHMFW   %FDFOUSBMJ[FE-PBE#BMBODFST ($1*OUFSOBM-#

    $MPVEGMBSF 1P1T  2017-07-28 JANOG 40 Meeting Ωʔϫʔυ/ .VMUJ5JFS MS Ananta FB shiv Google Maglev [1] Microsoft Ananta https://doi.org/10.1145/2534169.2486026 [2] Facebook Shiv: https://www.usenix.org/conference/srecon15/program/presentation/shuff [3] Google Maglev https://research.google.com/pubs/pub44824.html 3
  4.  ߏ੒͔Β /   .VMUJUJFSߏ੒΁ 2017-07-28 JANOG 40 Meeting

    4   / .VMUJUJFS Client L4 LB APP : In service : Not in service L4 LB APP L7 LB L3 Switch Client L3DSR TCP/HTTP(S) termination L3DSR ECMP ࠓճ͝঺հ͢Δͷ͸ ͜ͷ෦෼
  5. ஈ໨  #(1 &$.1 1FSGMPX&$.1ʹΑΓɺ--#΁͓͓Αͦۉ౳ʹτϥϑΟοΫΛ෼ࢄ 2017-07-28 JANOG 40 Meeting 5

    Equal Cost Multi Path (ECMP) VIP: 192.0.2.1/32 L4LB-1 Hash(5-tuple) -> L4LB-[1-3] L3 Switch VIP: 192.0.2.1/32 L4LB-2 VIP: 192.0.2.1/32 L4LB-3 eBGP peer Flow X Flow Y
  6. ஈ໨  4UBUFMFTT --# $POTJTUFOU)BTIͰ෼ࢄઌ--#Λܾఆ͠ɺ-%43Ͱύέοτసૹ 2017-07-28 JANOG 40 Meeting 6

    L3 Switch L4LB-1 L7LB-1 L7LB-2 L4LB-2 L7LB-3 L7LB-4 ConsistentHash(Hash(5-tuple)) -> L7LB-[1-4] Flow X Flow Y
  7. --#  F9QSFTT %BUB 1BUI 9%1 Λ׆༻ -JOVYLFSOFM಺Ͱಈ࡞͢ΔɺF#1'Λ༻͍ͨߴ଎ύέοτॲཧج൫ 2017-07-28 JANOG

    40 Meeting eBPF Map VIP-Backend table Statistics table C eBPF C code (Data Plane) Clang/LLVM eBPF Bytecode Python Management Plane eBPF JIT eBPF VM API VIP->Backend lookup Statistics Kernel User eBPF Verifier 7 BGP daemon XDP Program From L3 Switch To L7LB Kernel NIC Advertise VIP Compiler  -PPLVQ7*1  )BTI UVQMF  .PEJGZ*1IFBEFS Maintain map
  8. --#  ϊʔυ୯ମύϑΥʔϚϯε 2017-07-28 JANOG 40 Meeting 8 )BSEXBSFY*OUFM9FPO&W DPSFT

    (%%3.FMMBOPY$POOFDU9-Y&/(C& 4PGUXBSF-JOVY,FSOFMSD EDCG #1'+*5FOBCMFE &OWJSPONFOU 19.1 15.0 10.8 5.8 2.9 2.0 0% 20% 40% 60% 80% 100% 0 4 8 12 16 20 64 128 256 512 1024 1518 Link Utilization [%] Throughput [Mpps] Frame size [byte] Throughput [Mpps] Link Utilization (/25GbE) [%] 25GbE wire-rate
  9. ࣮ࡍͷͱ͜ΖͲΜͳঢ়گ ౰ॳͷૂ͍͸͓͓ΉͶୡ੒Ͱ͖ͦ͏ खݩͷίʔυͱಉҰͷ΋ͷ͕࣮ωοτϫʔΫͰͦͷ··ಈ࡞͢Δ ςετ͕͠΍͍͢ɾಈ࡞͕ݟ͑΍͍͢ / ߏ੒͕໰୊ʹͳΔέʔε΋͋Δ .VMUJQBUI5$1 *1'SBHNFOU Ұ෦ͷಛघҊ݅ 

    2017-07-28 JANOG 40 Meeting 9
  10. ࠓޙͷల๬ ຊ֨తʹϓϩμΫγϣϯ؀ڥ΁ͷ౤ೖΛ։࢝ ػೳ௥Ճ΍ߏ੒ͷΞοϓσʔτ %P4 %%P4๷ޚ εςʔτϑϧηογϣϯΩϟογϡ -36DBDIF ͷซ༻ #(1VOOVNCFSFE 3'$

    %$OFUXPSLͷ-Խ ϋʔυ΢ΣΞدΓͷΞϓϩʔνΛௐࠪɾݕ౼ 9%1 F#1' IBSEXBSFPGGMPBE 1SPHSBNNBCMF"4*$ 1  2017-07-28 JANOG 40 Meeting 10