Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DynamoDBとBloomFilterのムフフなカンケイ

Avatar for co-sche co-sche
November 11, 2016
Technology
0
1.5k

 DynamoDBとBloomFilterのムフフなカンケイ

メディアのための広告配信システム、A.J.A.でのDynamoDBとBloomFilterの活用事例
Avatar for co-sche

co-sche

November 11, 2016
Tweet

More Decks by co-sche

See All by co-sche
PackerとAnsibleでコンパクトに始めるInfrastructure as Code
Avatar for co-sche co_sche
0
260

Other Decks in Technology

See All in Technology
モバイル界のMCPを考える
Avatar for naoto33 naoto33
0
450
OPENLOGI Company Profile for engineer
Avatar for OPENLOGI hr01
1
34k
Delta airlines®️ USA Contact Numbers: Complete 2025 Support Guide
Avatar for ape1422 airtravelguide
0
340
高速なプロダクト開発を実現、創業期から掲げるエンタープライズアーキテクチャ
Avatar for かわうそ kawauso
2
8.6k
American airlines ®️ USA Contact Numbers: Complete 2025 Support Guide
Avatar for markhor64456 airhelpsupport
0
370
開発生産性を測る前にやるべきこと - 組織改善の実践 / Before Measuring Dev Productivity
Avatar for 株式会社カオナビ kaonavi
8
2.5k
KubeCon + CloudNativeCon Japan 2025 Recap
Avatar for Ren ren510dev
1
370
SmartNewsにおける 1000+ノード規模 K8s基盤 でのコスト最適化 – Spot・Gravitonの大規模導入への挑戦
Avatar for ishikawa ryu vsanna2
0
120
2025-07-06 QGIS初級ハンズオン「はじめてのQGIS」
Avatar for kou_kita kou_kita
0
160
Claude Code に プロジェクト管理やらせたみた
Avatar for 佐藤圭吾 unson
6
3.3k
ネットワーク保護はどう変わるのか？re:Inforce 2025最新アップデート解説
Avatar for Shun Tokuyama tokushun
0
190
United airlines®️ USA Contact Numbers: Complete 2025 Support Guide
Avatar for heron31238 unitedflyhelp
0
210

Featured

See All Featured
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.3k
Building an army of robots
Avatar for Kyle Neath kneath
306
45k
Scaling GitHub
Avatar for Zach Holman holman
459
140k
KATA
Avatar for Megan Lloyd mclloyd
30
14k
Why Our Code Smells
Avatar for Brandon Keepers bkeepers
PRO
336
57k
Practical Orchestrator
Avatar for Shlomi Noach shlominoach
189
11k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
Avatar for panda_program panda_program
107
19k
A better future with KSS
Avatar for Kyle Neath kneath
238
17k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
Avatar for Sam Stephenson sstephenson
161
15k
10 Git Anti Patterns You Should be Aware of
Avatar for Lemi Orhan Ergin lemiorhan
PRO
656
60k
Fantastic passwords and where to find them - at NoRuKo
Avatar for Phil Nash philnash
51
3.3k
How to Ace a Technical Interview
Avatar for Jacob Kaplan-Moss jacobian
277
23k

Transcript

  1. DynamoDBͱBloomFilterͷ ϜϑϑͳΧϯέΠ

  2. ࣗݾ঺հ ฏদ ߛయ • A.J.A. ϓϩάϥϚʔɾΞʔΩςΫτ • നนͷ֗ ૔ෑग़਎ •

    Ϊλʔ͍ͬͺ͍࣋ͬͯΔ • ͓ڕ৯΂ΒΕͳ͍ • https://github.com/co-sche
  3. None

  4. લճɺBloomFilterͷಈ࡞ݪ ཧͳͲʹ͍ͭͯ࿩Λ͠·ͨ͠ɻ IUUQXXXTMJEFTIBSFOFUDPTDIFTT

  5. A.J.A.ͰͲ͏࢖ΘΕ͍ͯΔͷ͔Λ஌ Γ͍ͨͱ͍͏੠্͕͕ͬͨͷͰɺࠓ ճ͸ͦͷลΛத৺ʹ͓࿩͠͠·͢ɻ

  6. ΋͘͡ • DynamoDB • DynamoDBͷ׆༻ࣄྫ • DynamoDBͷ໰୊఺ • BloomFilter •

    DynamoDB with BloomFilter

  7. DynamoDB

  8. DynamoDB • AWSͷϑϧϚωʔδυͳ NoSQL DB • ΩϟύγςΟϢχοτͱ͍͏୯ҐͰRead/WriteΛݟ ੵ΋ͬͯઃఆ͢Δ͚ͩͰɺμ΢ϯλΠϜແ͠Ͱεέʔ ϧ •

    ઃఆͨ͠ΩϟύҎ্ͷΞΫηεʹ͸جຊ଱͑ΒΕͳ͍ • ઃఆͨ͠Ωϟύʹରͯ͠՝ۚ

  9. DynamoDB׆༻ࣄྫ A.J.A.ͷ৔߹

  10. ϒϥ΢βΫοΩʔͷ୅ସͱͯ͠ಋೖ

  11. ֨ೲͯ͠Δ৘ใ • ͲΜͳ಺༰ͷ޿ࠂͳͷ͔ • ͲΜͳ৔ॴʹग़ͯΔͷ͔ • ͲΜͳબग़ͷ͞ΕํΛͨ͠ͷ͔

  12. WriteͷλΠϛϯά • ϝσΟΞαΠτͰͷ޿ࠂΫϦοΫ࣌

  13. ReadͷλΠϛϯά • ޿ࠂओαΠτʹຒΊࠐ·ΕͨϏʔίϯλά ϑοΫ • PV΍CVͷϩάʹు͍ͯɺ͔࣍Βͷ޿ࠂબ ग़ʹ׆͔͢

  14. ͜ͷέʔεͰͷ໰୊఺ • Read͠Α͏ͱ߲ͨ͠໨͕ݟ͔ͭͬͯ΋ݟ͔ͭ Βͳͯ͘΋ΩϟύγςΟϢχοτ͸ফඅ͞Ε Δ

  15. A.J.A. Ձ֨.c̋m͞Μ͔ΒϏʔίϯ ड৴͔ͨ͠Β͜ͷID͕ͯ͞͠΍ɻ

  16. DynamoDB ͕͚ͨ͞͠Ͳɺ ͦΜͳ΋Μ͋Β΁ΜͰɻ

  17. A.J.A. ͞Α͏͔ɻ ͢·Μͳɻ

  18. DynamoDB ͓͏ɻ ͋ɺۚ͸΋Ζͯ͘Ͱɻ

  19. ͜ͷέʔεͰͷ໰୊఺ • ReadͷλΠϛϯά͸޿ࠂओαΠτʹຒΊࠐ·ΕͨϏʔίϯ ϑοΫ • Ձ֨.c̋mͷ঎඼ৄࡉϖʔδʹຒΊࠐ·ΕͨΒʁ • YAH̋̋!γϣοϐϯάͷߪೖ׬ྃϖʔδʹషΒΕͨΒʁ • ͜ΕΒͷϏʔίϯͷ΄ͱΜͲ͸ɺA.J.A.޿ࠂΛclickͯ͠ભҠ͠

    ͨΘ͚Ͱ͸ͳ͍ແؔ܎ͳϏʔίϯ • ͭ·ΓɺReadͰݟ͔ͭΒͳ͍͜ͱ͕΄ͱΜͲ

  20. ͜ͷέʔεͰͷ໰୊఺ • ΄ͱΜͲ͕ϛεώοτ͢ΔେྔͷRead • ↑ͷ๫෩͕ಥવਧ͖ߥΕ࢝ΊΔڪා • ͜ΕΒʹରͯ͠ɺΩϟύγςΟΛେྔʹݟੵ ΋Βͳ͚Ε͹ͳΒͳ͍ෆໟײ

  21. A.J.A. σʔλͷ༗ແ͚ͩͰ΋ ࣄલʹ෼͔Ε͹͍͍ͷʹ…

  22. BloomFilter

  23. BloomFilter • ཁૉΛ༩͑ɺޙʹ໰͍߹ΘͤΔͱʮΈͨ͜ͱ ͋ΔΘʔʯ͘Β͍ʹ͍֮͑ͯͯ͘ΕΔσʔλ ߏ଄ • ཁૉΛ༩͑ͯͳ͍ͷʹ஌͔ͬͨͿΔ͜ͱ͕͋ Δ • ༩͑ͨͷʹ֮͑ͯͳ͍ͱ͍͏͜ͱ͸ɺઈରͳ

    ͍

  24. ཁૉΛ༩͑ͯͳ͍ͷʹ஌ͬͨ ͔ͿΔ͜ͱ͕͋Δ ِཅੑ͕͋Δ

  25. ༩͑ͨͷʹ֮͑ͯͳ͍ͱ͍͏ ͜ͱ͸ɺઈରͳ͍ ِӄੑ͸ͳ͍

  26. BloomFilter • ِཅੑΛڐ༰͢Δ͜ͱͰɺੌ·͘͡লεϖʔε Ͱཁૉͷ༗ແΛهԱ͢Δ͜ͱ͕Ͱ͖Δ • Ұཁૉ͋ͨΓ਺bits

  27. DynamoDB with BloomFilter • DynamoDBʹΫΤϦ͢ΔલʹBloomFilterʹฉ ͘͜ͱͰɺϋζϨReadΛ΄΅Χοτ • ِཅੑ͸ൃੜ͢Δ͕ɺͦΕʹΑͬͯى͜Δͷ ͸ͨͩͷϋζϨReadͳͷͰ໰୊ͳ͠

  28. DynamoDB with BloomFilter • A.J.A.ͷ৔߹ • PVϏʔίϯ਺ 1000/s • DynamoDB΁ͷΫΤϦ਺

    50/s • ໿ 1 / 20 • ແؔ܎ͳPVϏʔίϯ਺͕ٸಅͯ͠΋໰୊ͳ͠

  29. ·ͱΊ • A.J.AͰ͸ɺBloomFilterΛDynamoDBͷલஈ ͷωΨςΟϒΩϟογϡͱͯ͠ར༻͍ͯ͠·͢ɻ

  30. ͓·͚ • ͜ͷྨͷਖ਼֬ੑΛͪΐͼͬͱ٘ਜ਼ʹͯ͠ɺ࣌ؒ తɾۭؒతίετΛେ෯ʹઅ໿͢Δσʔλߏ଄ ͸ɺཚ୒σʔλߏ଄ͱݺ͹Ε͍ͯ·͢ɻ • BloomFilterҎ֎ʹ͸ɺHyperLogLog΍MinHash ͱ͍͏ͷ΋͋Γ·͢ɻ • ͦΕͧΕ໘ന͕͋͞ΔͷͰɺௐ΂ͯΈͯԼ͍͞

    Ͷʂ

  31. Have a nice ཚ୒ʂ

  32. ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ