NSDI2016 Technical Sessions Distributed Systems Session Overview TANAKA Daisuke@PFN

CONTENTS • Consensus in a Box: Inexpensive Coordination in Hardware (ETH Zürich) • StreamScope: Continuous Reliable Distributed Processing of Big Data Streams (Microsoft, Microsoft Research) • Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks (Facebook) • The Design and Implementation of the Warp Transactional Filesystem (Cornell University) • BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores
 (University of California, Berkeley) ঺հ͢Δ࿦จ https://www.usenix.org/conference/nsdi16/technical-sessions

ਐΊํ 1. ͲΜͳ՝୊Λղܾ͠Α͏ͱ͍ͯ͠Δͷ͔ 2. Ͳ͏΍ͬͯղܾͨ͠ͷ͔ (ΞϧΰϦζϜ) 3. ݕূ݁Ռ΍Ԡ༻ΞϓϦέʔγϣϯʹ͍ͭͯ

CONTENTS • Consensus in a Box: Inexpensive Coordination in Hardware • StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks • The Design and Implementation of the Warp Transactional Filesystem • BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores ঺հ͢Δ࿦จ

ͲΜͳ՝୊Λղܾ͠Α͏ͱ͍ͯ͠Δͷ͔ Consensus in a Box: Inexpensive Coordination in Hardware • γεςϜ߹ҙ͸ίετ (࣮૷ͷෳࡶ͞ɾॲཧ࣌ؒ) ͕େ͖͍ • γεςϜ߹ҙͱ͸جຊతʹҎԼͷ4ͭΛຬͨ͢ඞཁ͕͋Δ • Termination (Liveness) / Validity / Integrity / Agreement • PAXOS, RAFT ͳͲ • ݫີͳworkload͕ඞཁͱ͞Ε͍ͯΔγεςϜͰ͸߹ҙ͕ඞਢͩ ͕ɺύϑΥʔϚϯε΍εέʔϥϏϦςΟͷ੍໿ʹͳΔ͜ͱ͕ଟ͍

Ͳ͏΍ͬͯղܾͨ͠ͷ͔ • Zookeeperͷ atomic broadcast (ZAB) Λ FPGA ʹ࣮ͯ૷ͨ͠ Consensus in a Box: Inexpensive Coordination in Hardware

Zookeeper’s atomic broadcast (ZAB) Consensus in a Box: Inexpensive Coordination in Hardware

TCP/IP hardware Consensus in a Box: Inexpensive Coordination in Hardware

ݕূ݁Ռ Consensus in a Box: Inexpensive Coordination in Hardware

CONTENTS • Consensus in a Box: Inexpensive Coordination in Hardware • StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks • The Design and Implementation of the Warp Transactional Filesystem • BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores ঺հ͢Δ࿦จ

ͲΜͳ՝୊Λղܾ͠Α͏ͱ͍ͯ͠Δͷ͔ StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • ϏοάσʔλͰετϦʔϜॲཧ͕΍Γ͍ͨ • ෳࡶੑɺεέʔϥϏϦςΟɺfault tolerance (଱ނোੑ) ͕ٻΊΒ ΕΔ • ετϦʔϜॲཧͷσʔλϑϩʔͷ෮چ • ೖྗΠϕϯτͷ࠶ૹɺεςʔτ (ঢ়ଶ) ͷ෮چ
 ˠΠϕϯτͷϩετ͋Δ͍͸ॏෳΠϕϯτΛ๷͍͗ͨ

StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • σʔλϑϩʔΛ DAG ͱͯ͠දݱ͠ɺϊʔυͱΤοδͷґଘΛ rStream / rVertex ͱͯ͠ந৅Խ • recoveryͷ࣮ݱ • SCOPE ͱ͍͏ Parallel Map Reduce 
 ࣮૷ͷ֦ு Ͳ͏΍ͬͯղܾͨ͠ͷ͔ http://www.vldb.org/pvldb/1/1454166.pdf

StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • STREAM SCOPE (StreamS) ͷ࣮ߦ Ͳ͏΍ͬͯղܾͨ͠ͷ͔

StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • STREAM SCOPE (StreamS) ͷ࣮ߦ Ͳ͏΍ͬͯղܾͨ͠ͷ͔ rStream rVertex

StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • rStream • vertex (ΠϕϯτΛॲཧ͢Δϊʔυͷ૯শ) ΁ͷґଘΛ෼཭ɾந৅ Խͨ͠ඇಉظίϛϡχέʔγϣϯνϟϯωϧ • seq Λ࣋ͭɻಉ͡ seq Λ࣋ͭ event ͷॻ͖ࠐΈ͕੒ޭ͢Δ·Ͱ ಡΈࠐΈ͸ऴΘΒͳ͍ɻ • φΠʔϒʹ࣮૷͢ΔͱಉظϞσϧʹͳΔͷͰ GC ϞσϧΛ࠾༻ Ͳ͏΍ͬͯղܾͨ͠ͷ͔

StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • rVertex • vertex Ͱͷܭࢉʹରͯ͠γϯϓϧͳεφοϓγϣοτΛऔΔɻ • εςʔτͷ restart ٴͼ failure recovery Λ࣮૷ Ͳ͏΍ͬͯղܾͨ͠ͷ͔

StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • νΣοΫϙΠϯτͷִؒ΍ӬଓԽͷλΠϛϯάʹΑͬͯෳ਺ͷނো ෮چϞσϧΛ࠷খ͢Δ͜ͱ͕Ͱ͖Δ • strict model / relaxed model … • Ͳ͜Ͱނোͨ͠ͷ͔σόοά͢Δͷ͕؆୯ • σϓϩΠٴͼ (rStream / rVertex) Ҏ֎ͷ࣮૷͸طଘࢿ࢈ (ओʹ SCOPE) Λྲྀ༻͢Δ͜ͱ͕Ͱ͖Δ ݕূ

CONTENTS • Consensus in a Box: Inexpensive Coordination in Hardware • StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks • The Design and Implementation of the Warp Transactional Filesystem • BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores ঺հ͢Δ࿦จ

ͲΜͳ՝୊Λղܾ͠Α͏ͱ͍ͯ͠Δͷ͔ • େྔͷHTTPϦΫΤετΛͲͷΑ͏ʹࡹ͖ɺΩϟογϡΛ֤Ϋϥελ ຖʹͲ͏΍ͬͯ෼ࢄͤ͞Δ͔ɻޮ཰తͳ෼ࢄΛߦ͍͍ͨ • balanced / adaptive / stable / fast decision Λຬ͍ͨͨ͠ Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks

• ιʔγϟϧάϥϑΛ੩తʹ഑ஔͰ͖Δ “object” ͱ ಈతʹ഑ஔ͢Δ “group” ʹ໌ࣔతʹ෼཭ • Facebook ͷ TAO (ιʔγϟϧάϥϑ޲͚෼ࢄDB) Λ࢖༻ • static assignment • ࣅͨάϥϑͷ object ΛूΊΔɻσʔλΞΫηεύλʔϯΛάϥϑ ͱͯ͠දݱ͠ɺ෼ׂɻσʔλ͕େ͖͍ͱ஗͍ɻ Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks Ͳ͏΍ͬͯղܾͨ͠ͷ͔

• dynamic assignment • มԽ͢Δ৘ใʹରͯ͠ಈతʹόϥϯγϯάɻݕূͰ͸ bipartite graph partitioning Λ࢖༻ Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks Ͳ͏΍ͬͯղܾͨ͠ͷ͔

Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks ݕূ

Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks ݕূ

CONTENTS • Consensus in a Box: Inexpensive Coordination in Hardware • StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks • The Design and Implementation of the Warp Transactional Filesystem • BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores ঺հ͢Δ࿦จ

ͲΜͳ՝୊Λղܾ͠Α͏ͱ͍ͯ͠Δͷ͔ • ͜Ε·Ͱͷ෼ࢄϑΝΠϧγεςϜ • ෆे෼ͳอূ / ੍໿ͷଟ͍ΠϯλʔϑΣʔε / εέʔϧ͠ͳ͍ • ෼ࢄϑΝΠϧγεςϜͷϓϩτίϧͷ֦ுɻུͯ͠ WTF • PAXOS API + new zero-copy API The Design and Implementation of the Warp Transactional Filesystem

• file slicing API The Design and Implementation of the Warp Transactional Filesystem Ͳ͏΍ͬͯղܾͨ͠ͷ͔

• file slicing API • metadata strage ͕ slice ͷϙΠϯλΛ࣋ͭ • file offset ͢Δɻoverwrite ͨ͠৔߹͸ compaction ͕૸Δɻmeta data compaction • ࢖Θͳ͍෦෼͸ GC ͞ΕΔ • fragmentation ͕ى͜ΔͷͰɺܧଓతʹ locality-aware slice placement Λ࢖༻ The Design and Implementation of the Warp Transactional Filesystem Ͳ͏΍ͬͯղܾͨ͠ͷ͔

• Map Reduce Sort ࣌ͷ store ʹ࢖༻ˠ࣮ߦ͕࣌ؒ 70min ͔Β 15min • Videoฤूˠ࣌ܥྻιʔτ͕ૣ͘ͳͬͨ • ͦͷଞ2ͭ঺հ The Design and Implementation of the Warp Transactional Filesystem Ԡ༻ΞϓϦέʔγϣϯ

CONTENTS • Consensus in a Box: Inexpensive Coordination in Hardware • StreamScope: Continuous Reliable Distributed Processing of Big Data Streams • Social Hash: An Assignment Framework for Optimizing Distributed Systems Operations on Social Networks • The Design and Implementation of the Warp Transactional Filesystem • BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores ঺հ͢Δ࿦จ

ͲΜͳ՝୊Λղܾ͠Α͏ͱ͍ͯ͠Δͷ͔ • σʔλετΞʹஔ͍ͯ͸ɺϥϯμϜΞΫηεͱݕࡧͷ2͕ͭجຊతͳ ૢ࡞ • NoSQL͸େ͖ͳσʔλ΁ͷରԠ͕·ͣઌͰɺͦͷ࣍ʹ store Λ͍͔ ʹૣ͘͢Δ͔ͱ͍͏Ξϓϩʔν • େ͖ͳσʔλʹରԠ͢ΔͨΊʹѹॖΛར༻͍ͯ͠Δ͕ɺεϧʔϓο τͱͷτϨʔυΦϑ͕ଘࡏ͢Δ BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores

ͲΜͳ՝୊Λղܾ͠Α͏ͱ͍ͯ͠Δͷ͔ BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores

• Layered Sampled Array (LSA) ͱ͍͏৽͍͠σʔλߏ଄ΛఏҊ͠ɺτ ϨʔυΦϑۂઢΛಈతʹม͑ΒΕΔΑ͏ʹͨ͠ BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores Ͳ͏΍ͬͯղܾͨ͠ͷ͔

• Succinct store BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores Ͳ͏΍ͬͯղܾͨ͠ͷ͔ https://www.usenix.org/conference/nsdi15/technical-sessions/presentation/agarwal

• Succinct Λෳ਺ͷ sampling rate Ͱ store Ͱ͖ΔΑ͏ʹ֦ு BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores Ͳ͏΍ͬͯղܾͨ͠ͷ͔

• γϟʔσΟϯά࣌ʹͲ͏΍ͬͯ sampling rate Λௐઅ͢Δ͔ʁ
 ˠઌߦݚڀ: Back-pressure style scheduling BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores Ͳ͏΍ͬͯղܾͨ͠ͷ͔ http://dl.acm.org/citation.cfm?id=1285032

• FacebookͷΫϥελͰ͸ 90% ͕ transient failure • 1ͭͷϨϓϦΧ͕ނোͨ࣌͠ͷϩʔυϦΧόϦ͕3ഒߴ଎Խ BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores ධՁ

• [slide backup]