ネットワーク機能のベンチマーク自動化 / How to benchmark network functions in LINE

by LINE Developers

Slide 1

Slide 1 text

ωοτϫʔΫػೳͷϕϯνϚʔΫࣗಈԽ ాޱ ༤ن :VLJ5BHVDIJ )PXUPCFODINBSLOFUXPSLGVODUJPOTJO-*/& -*/&%FWFMPQFS.FFUVQ

Slide 2

Slide 2 text

:VLJ5BHVDIJ 7FSEBࣨωοτϫʔΫ։ൃνʔϜ ʙࡢ೥݄̏ɿֶ෦म࢜ʢ/'7ͷੑೳ޲্ʣ ࡢ೥݄ɿ -*/&ʹ৽ଔೖࣾ ۀ຿ɿϓϥΠϕʔτΫϥ΢υʹؔ࿈͢Δ %1MBOFͷੑೳվળ -PBE #BMBODFSͷ$1MBOFվળ %$ /8ΦʔέετϨʔλʔ։ൃ

Slide 3

Slide 3 text

ࠓ೔ͷ಺༰ "HFOEB -#ͷ։ൃϑϩʔʹϕϯνϚʔΫΛ૊ΈࠐΉ Ͳ͏΍ͬͯϕϯνϚʔΫΛࣗಈԽ͍ͯ͠Δʁࢫຯ͸ʁ νʔϜڞ༗ςετϕου঺հ ݶΓ͋ΔϦιʔεΛͲ͏΍ͬͯڞ༗͍ͯ͠Δ͔ʁ ੑೳ໰୊ͷ੾Γ෼͚ʹ΋ϕϯνϚʔΫ͕໾ʹཱͭ 43Wͷੑೳ໰୊ͷݪҼΛಛఆͯ͠ղܾͨ͠ࣄྫ

Slide 4

Slide 4 text

1. LB

Slide 5

Slide 5 text

-*/&ͷιϑτ΢ΣΞ-# -#ͷৄࡉ͸-*/&%&7&-01&3%":ͷൃදࢿྉ<>Λࢀর <> IUUQTMJOFEFWEBZMJOFDPSQDPNKQTFTTJPOT' • ී௨ͷY-JOVYαʔό্Ͱಈ࡞͢Δ • σʔλϓϨʔϯ͸9%1ϕʔεͰϑϧεΫϥον࣮૷ • ϝοηʔδϯά΍ϝσΟΞɾ޿ࠂ഑৴ͳͲ෯޿͍αʔϏεͰར༻ Messaging LINEBLOG LINE Clova LB ߴ͍ύϑΥʔϚϯεͱ҆ఆੑͷҡ͕࣋ෆՄܽ

Slide 6

Slide 6 text

-#ͷςετঢ়گ #FGPSF • -#ੑೳࢼݧ͸ɺ-#։ൃऀ͕ຊ൪σϓϩΠલͳͲʹखಈͰ࣮ࢪ͍ͯͨ͠ • σʔλϓϨʔϯͷςετ͕։ൃϑϩʔʹ৐͍ͬͯͳ͍ ػೳςετ ʢҙਤͨ͠௨Γͷಈ࡞͔ʣ ੑೳςετ ʢҙਤͨ͠௨Γͷੑೳ͔ʣ ࣗಈԽ͕೉͍͠ طʹࣗಈԽͰ͖Δঢ়ଶ

Slide 7

Slide 7 text

खಈςετͷ໰୊ ςετͷ࠶ݱੑ աڈͷ݁Ռͱͷൺֱ • ։ൃ ςετ࣮ߦ ͷϧʔϓΛճ͢ͷ͕େม • Ͳ͏ͯ͠΋ςετͷճ਺͸ݮΔ • ։ൃ్தͷόʔδϣϯͷੑೳ͸෼͔Βͳ͍ • ઃఆϛε • ༻ҙͰ͖ΔϦιʔεʹΑΔɺ؀ڥͷࠩҟ ։ൃऀ΁ͷࢧԉෆ଍ • ͲͷϚγϯͰɺͲΜͳίϯϑΟάͰɺͲΜͳςετΛͯ͠Δʁ

Slide 8

Slide 8 text

ࣗಈԽͷΞϓϩʔν Ξϓϩʔν ୡ੒͍ͨ͜͠ͱ ࠶ݱՄೳͳϕϯνϚʔΫ աڈͷςετ݁Ռ ͱൺֱՄೳ ։ൃऀ΁ͷࢧԉ • ઃఆ͸એݴతʹ è "OTJCMFΛ࢖ͬͯߏ੒؅ཧ • ͳΔ΂͘ࡉ͔͘ઃఆΛࣗಈԽ • ઐ༻ͷςετϕουΛ੔උ • ຖճಉ͡ϚγϯͰϕϯνϚʔΫ͢ΔΑ͏ʹ • Ұൠతͳ$*γεςϜʹγʔϜϨεʹ࿈ܞ • աڈͷ݁ՌΛ֬ೝͰ͖Δ 1 d

Slide 9

Slide 9 text

࡞੒ͨࣗ͠ಈϕϯνϚʔΫγεςϜ Automated Testbed Developer Plot Figures (pandas/matplotlib) Tester LB Continuous feedback 1. Provisioning 2. Run benchmark 3. Collect results GitHub Storage Compare Save Ansible provisioning PR Drone

Slide 10

Slide 10 text

ϕϯνϚʔΫγφϦΦ • ੑೳ௿Լ͕ݒ೦͞ΕΔಛघͳύλʔϯ TRex Generator (software-base) new scenario (python) Tester 4:/'MPPEJOH࣌ͷੑೳ͸ Measurement Dropped? Decrease rate yes no Increase rate TRex Generator LB XJSFSBUF PVUQVU • ࠷େϨʔτ ʢύέϩεڐ༰ʣ • ;FSP1BDLFU-PTTςετ

Slide 11

Slide 11 text

-#ݻ༗ͷϕϯνϚʔΫͷ೉͠͞ • 7FSEBͷ-#Ͱ͸*1*1τϯωϦϯάΛར༻͓ͯ͠ΓɺඇରশτϥϑΟοΫΛੜ੒ • δΣωϨʔλʢ53FYʣ͸ૹ৴ϑϩʔͱड৴ϑϩʔΛผʑʹΧ΢ϯτ • ϑϩʔຖͷ౷ܭ৘ใͳͲ͕ར༻Ͱ͖ͳ͍ LB Traffic Generator (TRex) IP IP’ IP ҧ͏ϑϩʔͩ • ؔ܎ͷͳ͍"31΍--%1΋Χ΢ϯτ͞ΕΔΑ͏ͳ άϩʔόϧ౷ܭΛ୅ΘΓʹར༻ • ׬ᘳͳθϩύέοτϩεςετ͸Ͱ͖ͳ͍ • ύέοτड৴Ͱ͖͍ͯΕ͹ɺϩεແ ͠ͱΈͳ͢

Slide 12

Slide 12 text

࣮ࡍͷΠϝʔδ $*1JQFMJOF #FODINBSL-PHT Repo name Repo name Changed file Changed file

Slide 13

Slide 13 text

Slide 14

Slide 14 text

ઐ༻ςετϕουߏஙͷഎܠ • -#ʹݶΒͣɺଞʹ΋༷ʑͳωοτϫʔΫػೳͷධՁɾݕূ͕ඞཁ • ػثࡏݿͷ੍໿΋͋Γɺ͙͢ʹݕূ؀ڥΛ༻ҙ͢Δͷ͸ࠔ೉ • ϓϥΠϕʔτΫϥ΢υͷ෺ཧػث͕Ұ௨Γଗͬͨςετϕου؀ڥ͕͋Δͱศར • աڈͷ݁Ռͱൺֱ͠΍͍͢ϕϯνϚʔΫΛߦ͍͍ͨ

Slide 15

Slide 15 text

-*/&Πϯϑϥݕূʹదͨ͠ςετϕουઃܭ • -εΠονΛհͨ͠ϊʔυؒ઀ଓ • 73'΍1#3ͳͲͰτϥϑΟοΫΛॊೈʹ੍ޚͰ͖Δ • -#ͷΑ͏ʹΧϓηϧԽΛ൐͏τϥϑΟοΫ΋੍ޚ͕༰қ L3 Switch (Cumulus Linux) LB Traffic Generator Management Switch Provisioning via SSH ࠷େ(C&QPSU ʜ

Slide 16

Slide 16 text

ςετϕουͷαʔόΛνʔϜͰڞ༗͢Δ • ཁٻ • લʹ࢖ͬͨਓͷίϯϑΟά͸࢒ͨ͘͠ͳ͍ • LFSOFMόʔδϣϯ΋੾Γସ͑ͯݕূ͍ͨ͠ Management Server • 19&%)$1 • ,FSOFMJNBHF • /'4 • 04JOTUBMMFS ςεταʔό Diskless Boot • ղܾࡦσΟεΫϨεϒʔτ

Slide 17

Slide 17 text

ςετϕουͷ-εΠονΛνʔϜͰڞ༗͢Δ • ݒ೦ • ৽ͨʹ௥Ճͨ͠ϧʔτ͕ଞऀͷطଘͷઃఆʹӨڹΛ༩͑ɺϕϯνϚʔΫ͕ࣦ ഊ͢Δ͔΋͠Εͳ͍ • ղܾࡦར༻ऀ͸ඞͣ73'Λ࡞੒͢Δ L3 Switch Traffic Generator VRF LB VRF VRF Traffic Generator VRF A B

Slide 18

Slide 18 text

5FTUCFE.BOBHFS ୭͕ͲͷϚγϯΛར༻த͔ Ϛγϯͷ֬อ -εΠονͷઃఆ݁Ռ ʢ"OTJCMF-PHʣ ઃఆ൓өϘλϯ hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname hostname Server name Switch name Switch name Switch name Switch name Switch name

Slide 19

Slide 19 text

5FTUCFE.BOBHFSͷεΠονઃఆࣗಈԽ • ֘౰Ϛγϯ͕઀ଓ͞Ε͍ͯΔϙʔτ൪߸Λऔಘ͠ɺ*OUFSGBDFͱ3PVUJOHͷઃఆςϯ ϓϨʔτΛ(JU)VC্ʹࣗಈੜ੒͢Δ • FUDGSSGSSDPOGE66*% • FUDOFUXPSLJOUFSGBDFTE66*% Tester Tester DuT Testbed GitHub DPOGJHςϯϓϨʔτੜ੒ Provision L3 SW config Testbed Manager

Slide 20

Slide 20 text

Slide 21

Slide 21 text

ϓϥΠϕʔτΫϥ΢υͷϚϧνςφϯγʔ • -*/&Ͱ͸43WΛ࢖ͬͯϚϧνςφϯγʔΛ࣮ݱ • 'VMM-ͷ-*/&ͷωοτϫʔΫΛ׆͔ͤΔ • -JOVYΧʔωϧͷ%1MBOF࣮૷Λར༻ -*/&ͷ43Wಋೖͷৄࡉ͸-*/&%&7&-01&3%":΍+"/0(ͷൃදࢿྉ< >Λࢀর <>IUUQTXXXKBOPHHSKQNFFUJOHKBOPHQSPHSBNTSW <>IUUQTMJOFEFWEBZMJOFDPSQDPNKQTFTTJPOT& IP-Fabric (underlay) Tenant A Tenant B Tenant C ß *1Wύέοτసૹ ß αʔϏε͝ͱʹಠཱͨ͠ςφϯτ

Slide 22

Slide 22 text

-JOVYΧʔωϧͷ43Wੑೳ໰୊ • 43WͰߏஙͨ͠Ϛϧνςφϯτ؀ڥ͸ɺసૹੑೳ͕௿͍ • γϯάϧςφϯτ؀ڥͷ෼ͷ̍ఔ౓ʢ(CQTʣͷసૹੑೳ • Ͳ͏ʹ͔ͯ͠ɺੑೳΛ޲্Ͱ͖ͳ͍͔ʁ • /*$ϕϯμʔ͔Β৽ͨͳ5$1ηάϝϯςʔγϣϯΦϑϩʔυػೳ͕௥Ճ <>IUUQTXXXKBOPHHSKQNFFUJOHKBOPHQSPHSBNTSWYEQ 540 1VSF- ඇ43W Host VM/APP /*$Ͱ43Wύέοτ Ληάϝϯςʔγϣϯ Φʔόϔου࡟ݮ͕ظ଴

Slide 23

Slide 23 text

ࣗಈϕϯνϚʔΫͰੑೳ֬ೝ • ౰ॳͷݕূ؀ڥͰ͸ɺ540ͷޮՌ͕֬ೝͰ͖ͳ͍ • ֤*'ͷΦϑϩʔυઃఆΛ༷ʑͳύλʔϯΛࣗಈͰςετ SRv6 encap SRv6 decap Traffic Generator (iperf) 43WϊʔυΛϒϥοΫϘοΫεతʹςετͯ͠΋ݪҼ͕Θ͔Βͳ͍ Ansible ςετ ύϥϝʔλมߋ

Slide 24

Slide 24 text

)ZQFSWJTPSͷ43Wੑೳݕূ HV VM VRF tap veth veth p0 br End.DX4 T.encaps iperf ϘτϧωοΫʹͳΒͳ͍Α͏ ߴ଎ͳ711Λ43Wର޲ʹ Traffic Generator (iperf) VPP End.DX4 T.encaps ड৴ੑೳ ૹ৴ੑೳ • ϘτϧωοΫΛ୳͢ 540͕ޮ͍͍ͯΕ͹͜͜͸଎͍

Slide 25

Slide 25 text

࣮ࡍͷσʔλ 4 Gbps 4 Gbps 20 Gbps ड৴ੑೳ 540PGG ૹ৴ੑೳ ड৴ੑೳ 540PO ࣮͸ૹ৴ଆ͸540͕ޮ͍͓ͯΓ ੑೳ͕͔ͳΓ޲্ ड৴ଆ͸540͕ಇ͍͍ͯͳͦ͞͏ 43W OPO43W 43W` 43W OPO43W 43W` 43W OPO43W 43W` 711

Slide 26

Slide 26 text

ύέοτΛτϨʔεͯ͠ΈΔ HV VM VRF tap veth veth p0 br End.DX4 T.encaps iperf ෳ਺ͷ5$1ηάϝϯτ͕ ू໿͞Εͯड৴ -(0(30 Traffic Generator (iperf) VPP End.DX4 T.encaps ηάϝϯςʔγϣϯ ຊ౰͸͜͜·Ͱ540Ͱ ू໿͞Ε͍ͯͯ΄͍͠ • Ͳ͜Ͱηάϝϯςʔγϣϯ͞Ε͍ͯΔ͔Λ୳͢ Per-Packet ड৴ύέοτ

Slide 27

Slide 27 text

࠷ޙ͸Χʔωϧ಺෦ͷௐࠪ • ҙਤ͠ͳ͍ηάϝϯςʔγϣϯͷݪҼΛ͔֬ΊΔ • ΧʔωϧωοτϫʔΫௐࠪʹJQGUSBDF<> • ύέοτ͕-JOVYΧʔωϧͷͲͷॲཧGVODUJPOͰॲཧ͞Ε͔ͨΛτϨʔε • TLCͷΦϑϩʔυϑϥά΋֬ೝͰ͖Δ <>IUUQTHJUIVCDPN:VUBSP)BZBLBXBJQGUSBDF %FDBQ࣌ʹ࡟আ͞ΕΔ͸ͣͷ ϑϥά͕࢒͍ͬͯΔ è 4,#@(40@*19*1 VRF doesn’t support this!

Slide 28

Slide 28 text

-JOVY,FSOFM΁ͷ$POUSJCVUJPO ղܾࡦ • 43Wͷ%FDBQॲཧޙʹɺ4,#@(40@*19*1Λ࡟আ͢ΔॲཧΛ௥Ճ <>IUUQTHJUIVCDPNUPSWBMETMJOVYDPNNJUFCBFBFEFFD EBFDBFD 7.·Ͱू໿͞Εͨ··సૹ͞ΕΔ͜ͱΛ֬ೝ è,FSOFMʹQBUDIΛૹͬͯϚʔδࡁΈʂ Remove!

Slide 29

Slide 29 text

ੑೳվળ • ड৴ੑೳ͸(CQTΛ௒͑ɺγϯάϧςφϯτ؀ڥʹ͍ۙੑೳ·Ͱվળ 12 Gbps 20 Gbps 43W OPO43W 43W` 43W OPO43W 43W` )7ड৴ੑೳ )7ૹ৴ੑೳ

Slide 30

Slide 30 text

·ͱΊ -#ͷ։ൃϑϩʔʹϕϯνϚʔΫΛ૊ΈࠐΉ ੑೳΛ։ൃऀʹϑΟʔυόοΫɺ༧ظͤ͵ੑೳ௿Լʹؾ͚ͮΔ νʔϜڞ༗ςετϕου঺հ -εΠονͷίϯϑΟάΛࣗಈԽͯ͠ಉ࣌ʹςετΛ࣮ߦ ੑೳ໰୊ͷ੾Γ෼͚ʹ΋ϕϯνϚʔΫ͕໾ʹཱͭ ࣗಈϕϯνϚʔΫͰूΊͨσʔλ͔Βɺ43WͷΧʔωϧόάΛൃݟ

Slide 31

Slide 31 text

THANK YOU