XDPの最新動向とLINEにおける取り組み事例 / Recent updates on XDP

Transcript

1. 9%1ͷ࠷৽ಈ޲ͱ -*/&ʹ͓͚ΔऔΓ૊Έࣄྫ :PIFJ ,BOFNBSV -*/&
   $PSQPSBUJPO ZPIFJLBOFNBSV BU
   MJOFDPSQDPN 0QFO
   /FUXPSLJOH
   $POGFSFODF
   +BQBO
   
   r 
2. ࣗݾ঺հ ؙۚ ༸ฏ 
   ZVOB[VOP ωοτϫʔΫӡ༻  d  σʔληϯλωοτϫʔΫͷӡ༻ɺϐΞϦϯάௐ੔ ΦϑΟεແઢ-"/ͷߏஙɾӡ༻ ϓϥΠϕʔτΫϥ΢υ l7FSEBz
   ͷωοτϫʔΫؔ࿈։ൃ 
   d -PBE
   #BMBODFS
   BT
   B
   4FSWJDF
   -#BB4
   ͷઃܭɾ։ൃɾߏஙɾӡ༻ Open Networking Conference Japan 2019 2019-11-01 2
3. "HFOEB • 9%1ͷ֓ཁͱϢʔεέʔε • ࠷ۙͷ9%1ʹ·ͭΘΔΞοϓσʔτ • -*/&ʹ͓͚Δ9%1ར༻ͷྺ࢙ͱݱঢ় Open Networking Conference Japan 2019 2019-11-01 3
4. 9%1ͷ֓ཁͱϢʔεέʔε Open Networking Conference Japan 2019 2019-11-01 4
5. 9%1 F9QSFTT %BUB 1BUI -JOVY
   LFSOFMͷ಺Ͱಈ࡞͢Δɺߴ଎ύέοτॲཧج൫ lGBTUQBUIz /*$υϥΠόͷϨϕϧͰύέοτॲཧΛద༻ ,FSOFM
   
   ೥݄
   ͕࠷ॳͷϦϦʔε F#1'Λ༻͍ͨߴ଎͔ͭݎ࿚ͳج൫ Χʔωϧ಺ͰͷόΠτίʔυ࣮ߦ؀ڥ F#1' WFSJGJFSΛ༻͍࣮ͨߦલͷ੩తݕূʹΑΓ҆શੑΛ୲อ .BQʹΑΔঢ়ଶ؅ཧ͓ΑͼϢʔβۭؒͱͷ࿈ܞ +*5ʹΑΔߴ଎ॲཧ Open Networking Conference Japan 2019 2019-11-01 5
6. 9%1ͱF#1'ͰͰ͖Δ͜ͱ Open Networking Conference Japan 2019 Kernel User Device Driver Kernel NIC Device Driver XDP skb Network Interface Network Interface IP TCP / UDP / ICMP Socket BPF maps AF_XDP App App App Transmit Redirect Redirect Pass Drop Parse Lookup Modify Push/Pop Hdr Verdict 2019-11-01 App 6
7. 9%1ͳΒͰ͸ͷಛ௃ -JOVYʹ࣮૷͞Εͨଟ͘ͷػೳ΍࢓૊ΈΛ׆༻Մೳ Χʔωϧͷ'*#ΛϓϩάϥϜ಺͔ΒࢀরՄೳ ௨ৗͷωοτϫʔΫελοΫ΁ॲཧΛҠৡ Ϣʔβۭؒͷϓϩηεͱͷ࿈ܞ͕༰қ ΧʔωϧϞδϡʔϧͱൺֱͯ͠։ൃ͕༰қ ύέοτॲཧઐ༻ͷ$16ׂΓ౰͕ͯෆཁ ࣮ߦதͷϓϩάϥϜΛಈతʹࠩ͠ସ͑Մೳ 2019-11-01 Open Networking Conference Japan 2019 7
8. 9%1ͷϢʔεέʔε -PBE
   #BMBODFS
   4FDVSJUZ෼໺Ͱͷద༻ྫ͕ଟ͍ 'BDFCPPL
   ,BUSBO
   -*/&
   'BCSJD-# $MPVEGMBSF
   -%SPQ
   4VSJDBUB
   $JMJVN
   $BMJDP ࠷ۙͷద༻ྫ "'@9%1΋ؚΉ /"5 %FDBQ 1BDLFU TBNQMFS ! NJYJ <> 43W 1SPYZ z&OE"$z ! *OUFSPQ 5PLZP  <> 0QFO W4XJUDI <> %1%, "'@9%1 1.% <> Open Networking Conference Japan 2019 2019-11-01 8 [1]   
      https://bcu30.jp/2019/talk/mabuchi-toshiya/ [2] edenden/end.ac https://github.com/edenden/end.ac [3] Open vSwitch with AF_XDP http://docs.openvswitch.org/en/latest/intro/install/afxdp/ [4] AF_XDP Poll Mode Driver https://doc.dpdk.org/guides/nics/af_xdp.html
9. ࠷ۙͷ9%1ʹ·ͭΘΔΞοϓσʔτ πʔϧνΣΠϯ΍֤छαϙʔτͷ֦ॆ F#1'ͷ֦ு "'@9%1 2019-11-01 Open Networking Conference Japan 2019 9
10. πʔϧνΣΠϯ΍࢖༻؀ڥͷ֦ॆ ϥΠϒϥϦ΍πʔϧ͕੔උ͞ΕɺओཁύοέʔδϚωʔδϟͰఏڙ MJCCQG Ϣʔβۭ͔ؒΒF#1'Λѻ͏ͨΊͷϥΠϒϥϦ CQGUPPM
    Ξλον͞Ε͍ͯΔϓϩάϥϜ΍Ϛοϓͷμϯϓ͕Մೳͳπʔϧ ֤छσΟετϦϏϡʔγϣϯͰσϑΥϧτͰ࢖༻Մೳʹ 'FEPSB
    6CVOUVͰ͸"'@9%1ؚΊ͙͢ʹ࢖༻Մೳ 3)&-Ͱ΋9%1͕࢖༻Մೳʹ UFDIOPMPHZ QSFWJFX 2019-11-01 Open Networking Conference Japan 2019 10
11. YEQUVUPSJBM ໢ཏతͳνϡʔτϦΞϧ 9%1։ൃʹର͢Δ೉қ౓Λܰݮ “Kernel selftests/bpf and samples/bpf: bad starting point”[1] 2019-11-01 Open Networking Conference Japan 2019 11 https://github.com/xdp-project/xdp-tutorial [1] https://linuxplumbersconf.org/event/4/contributions/460/attachments/244/426/xdp-distro-view.pdf
12. F#1'ͷ֦ு #PVOEFE
    -PPQͷαϙʔτ<> ैདྷɺF#1'Ͱ͸MPPQΛ࢖༻Ͱ͖ͳ͔ͬͨ l3FTUSJDUFE
    $z F#1' WFSJGJFSͷڧԽʹΑΓɺ༗ݶ௕ͷϧʔϓ͸࢖༻Մೳʹ lάϩʔόϧม਺zͷαϙʔτ<> ैདྷɺάϩʔόϧͳઃఆ৘ใͷѻ͍ʹ೉͕͋ͬͨ ํ๏
    EFGJOF ͯ͠࠶ίϯύΠϧ ํ๏ ௕͞ͷBSSBZ NBQΛ࡞੒࣮ͯ͠ߦ࣌ʹ౎౓MPPLVQ ಺෦తʹ͸௕͞ͷNBQͱͯ͠ѻΘΕΔ͕ɺMPPLVQແ͠ͰࢀরͰ͖ΔΑ͏ʹ ಁաతʹల։͞ΕΔ 2019-11-01 Open Networking Conference Japan 2019 12 [1] Bounded loops in BPF for the 5.3 kernel https://lwn.net/Articles/794934/ [2] BPF support for global data https://lore.kernel.org/bpf/20190409210910.32048-1-daniel@iogearbox.net/
13. F#1'ͷ֦ு DPOU ϓϩάϥϜඇґଘͷDIBJOJOH
    8*1 <> ैདྷΑΓɺUBJM
    DBMMͰϓϩάϥϜͷDIBJOJOH͸Մೳͩͬͨ ݺͼग़͠ݩϓϩάϥϜͰ໌ࣔతʹUBJM DBMMΛ࢖༻ͤͣͱ΋DIBJOJOHͰ͖ΔΑ͏ʹ ྫ -#ϓϩάϥϜͷલʹɺYEQEVNQΛࠩ͠ࠐΜͰύέοτΩϟϓνϟ 2019-11-01 Open Networking Conference Japan 2019 13 [1] xdp: Support multiple programs on a single interface through chain calls https://lore.kernel.org/netdev/157046883502.2092443.146052429591277809.stgit@alrua-x1/
14. App UMEM "'@9%1 9%1Λ༻͍ͯɺύέοτΛϢʔβۭؒʹ౉͢࢓૊Έ F#1'ʹ੍ݶ͞Εͳ͍ɺॊೈͰߴ଎ͳΞϓϦέʔγϣϯΛ࣮ݱ 2019-11-01 Open Networking Conference Japan 2019 14 Device Driver XDP BPF map (XSKMAP) NIC Network Stack Queue Queue Parse Lookup Modify Verdict Queue Filter Kernel NIC AF_XDP socket (XSK) User Kernel Tx Completion Fill Redirect Rx Lookup
15. "'@9%1  /*$ͱͷؔ܎ ϋʔυ΢ΣΞΩϡʔͱ94,͸ ର৅Ωϡʔʹύέοτ͕ೖΔΑ͏ߟྀ͕ඞཁ ํ๏ ΩϡʔΛݸʹ͢Δ ํ๏ OUVQMF GJMUFSΛઃఆ͢Δ ํ๏ Ωϡʔͷ਺͚ͩ94,Λ࡞੒͢Δ Ωϡʔͷ؅ཧػೳ͸ผ్ݕ౼த<> 2019-11-01 Open Networking Conference Japan 2019 15 NIC Queue Queue Queue Filter Kernel NIC [1] Making Networking Queues a First Class Citizen in the Kernel https://linuxplumbersconf.org/event/4/contributions/462/
16. "'@9%1  9%1ϓϩάϥϜͰͷॲཧ 94,."1ΛݟͯϦμΠϨΫτઌΛܾఆ ͜ͷϓϩάϥϜ͸ݱ࣌఺Ͱ͸ࣗ෼Ͱ༻ҙ লུ͢Δ࢓૊Έ͕ݕ౼த<> ͲͷύέοτΛϦμΠϨΫτ͢Δ͔ɺ ϓϩάϥϜͰهड़Մೳ ׬શͳΧʔωϧόΠύεͰ͸ͳ͍ 2019-11-01 Open Networking Conference Japan 2019 16 Device Driver XDP BPF map (XSKMAP) Network Stack Parse Lookup Modify Verdict Redirect Lookup [1] Enable direct receive on AF_XDP sockets https://lore.kernel.org/bpf/1570515415-45593-1-git-send-email-sridhar.samudrala@intel.com/
17. App UMEM "'@9%1  ϢʔβۭؒͰͷॲཧ ϝϞϦ؅ཧ͸ΞϓϦέʔγϣϯଆͷ੹೚ ड৴༻όοϑΝ͸'JMM
    3JOHΛ௨ͯ͠Χʔωϧʹ ద੾ʹڙڅ͢Δඞཁ͕͋Δ "'@9%1 TPDLFUࣗମ͸੍ޚ༻νϟϯωϧ ύέοτͷૹड৴͸3JOHܦ༝ ;FSPDPQZ΋ར༻Մೳ /*$͔Β6.&.ʹ%." 2019-11-01 Open Networking Conference Japan 2019 17 AF_XDP socket (XSK) User Kernel Tx Completion Fill Rx
18. -*/&ʹ͓͚Δ9%1ར༻ͷྺ࢙ͱݱঢ় 2019-11-01 Open Networking Conference Japan 2019 18
19. ར༻ྫ -BZFS ϩʔυόϥϯα Open Networking Conference Japan 2019 L4LB (Software) Real Server L7LB (Software) L3 Switch (Hardware) Client TCP/HTTP(S) Proxy IPIP encap ECMP [2] #*%($%:4 ;>?
    /&'-0!26 @  https://www.slideshare.net/linecorp/ss-116879618 [3] LINE0*-+-$%*,19=!)"". 85 "%3 0*-7<  https://www.slideshare.net/linecorp/lines-infrastructure-platform-how-it-scales-massive-services-and-maintains-low-operational-cost 2019-11-01 19
20. ར༻ྫ 43W /FUXPSL /PEF 2019-11-01 Open Networking Conference Japan 2019 20 Push/Pop SRH [1] LINE Data Center Networking with SRv6 https://speakerdeck.com/line_developers/line-data-center-networking-with-srv6 [2]  
    
     
    
      https://engineering.linecorp.com/ja/blog/intern2019-report-infra/
21. ͜Ε·Ͱͷঢ়گ ೥݄ࠒ͔Β࣮؀ڥͰ࢖༻த 9%1σʔλϓϨʔϯ༝དྷͷো֐͸ݱ࣌఺Ͱ݅ Ή͠Ζ9%1Ҏ֎ͷ෦෼ ಛʹϔϧενΣοΫ َ͕໳ ໌֬ͳ$%1MBOF෼཭ʹΑΓɺൺֱతͳେ͖ͳมߋ΋εϜʔζʹ ։ൃ؀ڥ΍ӡ༻؀ڥͷݟ௚͠͸ਵ࣮࣌ࢪ ؀ڥߏ੒ɾσϓϩΠํࣜͷมߋ $POUJOVPVT
    *OUFHSBUJPO
    $*
    ͷಋೖ ϞχλϦϯάͷ֦ॆ 2019-11-01 Open Networking Conference Japan 2019 21
22. ࣗಈԽ͞Εͨςετ࣮ߦ 2019-11-01 Open Networking Conference Japan 2019 22 linelinux ver. X on VM linelinux ver. Y on VM linelinux ver. X on Baremetal L4LB XDP Program L4LB XDP Program L4LB XDP Program Performance Test Behavior Test (BPF_PROG_TEST_RUN) Issue PR Kernel build system (“linelinux”) GitHub Enterprise Drone CI Run tests
23. ϞχλϦϯά 1SPNFUIFVTΛ࢖༻ ઐ༻ͷFYQPSUFSΛ࣮૷͠ɺҎԼͷΑ͏ͳϝτϦΫεΛऩू #1' NBQʹॻ͔Εͨ౷ܭ৘ใ /*$ͷ9%1 RVFVFؔ࿈ϝτϦΫε FUIUPPM 4ܦ༝ ਖ਼ৗੑ൑ఆͱΞϥʔτ͸֎ܗ؂ࢹΛॏࢹ ϊʔυ୆਺͕ଟ͍ͨΊɺ֤छϝτϦΫεͰࡉ͔͘ΞϥʔτΛઃఆ͢Δͷ͕ࠔ೉ 2019-11-01 Open Networking Conference Japan 2019 23
24. 2019-11-01 Open Networking Conference Japan 2019 24
25. ࠷ۙͷࠔΓ͝ͱ #1'
    NBQͷಡΈॻ͖଎౓͕஗͍ ΤϯτϦͣͭಡΈॻ͖͢Δඞཁ͕͋Δ ΞϓϦέʔγϣϯىಈ࣌ʹઃఆྲྀ͠ࠐΈ଴͕ͪൃੜ --#Ͱ࢖༻͢Δ-36
    -FBTU
    3FDFOUMZ
    6TFE
    UBCMFͷσόοά͕ਏ͍ ΤϯτϦ͕ଟ͍ͷͰμϯϓ͕େม ΤϯτϦΛಡΈࠐΉͱ-36
    UBCMFͷڍಈʹӨڹΛ༩͑ͯ͠·͏ +VNCP
    GSBNFαϙʔτ ݱঢ়͸,#·Ͱ<> 2019-11-01 Open Networking Conference Japan 2019 25 [1] XDP multi buffer design https://github.com/xdp-project/xdp-project/blob/master/areas/core/xdp-multi-buffer01-design.org
26. ·ͱΊ 9%1͸ϦϦʔεҎޙண࣮ʹ։ൃ͕ਐΊΒΕ͍ͯΔ جຊࢥ૝͸มΘΒͣɺͦͷपลͷվળ΋ଟ͍ ҰํͰ"'@9%1ͷΑ͏ͳେ͖ͳมԽ΋͋Δ ػೳͷෆ଍΋·ͩ·ͩ͋Δ͕ɺվળʹ޲͚׆ൃͳٞ࿦͕ߦͳΘΕ͍ͯΔ ద༻ྫ΋ଟ͘ɺ௕ظؒ҆ఆՔಈ͍ͯ͠Δ ։ൃ΍ӡ༻ͷ͠΍͢͞΋౰ॳ͔Βང͔ʹվળ͞Ε͍ͯΔ Open Networking Conference Japan 2019 2019-11-01 26
27. <13> -*/& %&7&-01&3 %":  ݄೔ ! άϥϯυχοίʔ౦ژ ୆৔ 2019-11-01 Open Networking Conference Japan 2019 27 https://linedevday.linecorp.com/jp/2019/