Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DynamoDBとBloomFilterのムフフなカンケイ

Avatar for co-sche co-sche
November 11, 2016
Technology
1.5k
0

 DynamoDBとBloomFilterのムフフなカンケイ

メディアのための広告配信システム、A.J.A.でのDynamoDBとBloomFilterの活用事例

Avatar for co-sche

co-sche

November 11, 2016

More Decks by co-sche

See All by co-sche
PackerとAnsibleでコンパクトに始めるInfrastructure as Code
Avatar for co-sche co_sche
0
270

Other Decks in Technology

See All in Technology
BigQuery の Cross-cloud Lakehouse への歩み
Avatar for Tomonori Hayashi / ぴーはや phaya72
2
540
JJUG CCC 2026 Spring　AI時代の開発こそ標準化を武器に！ ― 方式・プロセス・プラットフォームの標準化
Avatar for WatanabeShun s27watanabe
2
690
最低限これだけ押さえれ大丈夫_Claude Enterprise/Team企業展開ガバナンス入門
Avatar for t-kikuchi tkikuchi
1
730
TROCCOで始めるクラウドコストを民主化するためのFinOps
Avatar for tk3fftk tk3fftk
3
560
Dynamic Workersについて
Avatar for Yusuke Wada yusukebe
2
580
Claude Codeを組織で使いこなす— サーバサイドAIエージェント運用の実践知
Avatar for PERSOL CAREER Dev | techtekt techtekt
PRO
0
190
APIテストとは？
Avatar for 草薙昭彦 nagix
0
180
Terraformモジュールは、なぜ「魔境」化するのか
Avatar for hayama hayama17
1
180
「コーディング」しない人のための Claude Code 入門　ChatGPT の次の一歩 — 業務に組み込む 育成・共有・自動化
Avatar for ふくだ（fukuda ryu） rfdnxbro
2
1.1k
インフラが苦手でも大丈夫！ 紙芝居 Kubernetes -WWGT 10周年編-
Avatar for Aoi Takahashi aoi1
1
340
[モダンアプリ勉強会]今更聞けないGit/GitHub入門
Avatar for つくぼし tsukuboshi
0
220
AIガバナンス実践 - 生成AIコネクタのデータ漏洩リスクと実務対策
Avatar for 西岡 賢一郎 (Kenichiro Nishioka) knishioka
0
170

Featured

See All Featured
Effective software design: The role of men in debugging patriarchy in IT @ Voxxed Days AMS
Avatar for Kenny Baas-Schwegler baasie
0
380
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
Avatar for Andy Polaine apolaine
1
340
How to build an LLM SEO readiness audit: a practical framework
Avatar for Nick Samuel nmsamuel
1
770
No one is an island. Learnings from fostering a developers community.
Avatar for Antonio thoeni
21
3.7k
The Director’s Chair: Orchestrating AI for Truly Effective Learning
Avatar for Mike Taylor tmiket
1
180
Organizational Design Perspectives: An Ontology of Organizational Design Elements
Avatar for Dr. Kim W Petersen kimpetersen
PRO
1
720
Bioeconomy Workshop: Dr. Julius Ecuru, Opportunities for a Bioeconomy in West Africa
Avatar for AKADEMIYA2063 akademiya2063
PRO
1
130
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
49
3.5k
The Limits of Empathy - UXLibs8
Avatar for Cassini Nazir cassininazir
1
350
AI Search: Implications for SEO and How to Move Forward - #ShenzhenSEOConference
Avatar for Aleyda Solis aleyda
1
1.3k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
Avatar for Addy Osmani addyosmani
10
1.2k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
231
23k

Transcript

  1. DynamoDBͱBloomFilterͷ ϜϑϑͳΧϯέΠ

  2. ࣗݾ঺հ ฏদ ߛయ • A.J.A. ϓϩάϥϚʔɾΞʔΩςΫτ • നนͷ֗ ૔ෑग़਎ •

    Ϊλʔ͍ͬͺ͍࣋ͬͯΔ • ͓ڕ৯΂ΒΕͳ͍ • https://github.com/co-sche
  3. None

  4. લճɺBloomFilterͷಈ࡞ݪ ཧͳͲʹ͍ͭͯ࿩Λ͠·ͨ͠ɻ IUUQXXXTMJEFTIBSFOFUDPTDIFTT

  5. A.J.A.ͰͲ͏࢖ΘΕ͍ͯΔͷ͔Λ஌ Γ͍ͨͱ͍͏੠্͕͕ͬͨͷͰɺࠓ ճ͸ͦͷลΛத৺ʹ͓࿩͠͠·͢ɻ

  6. ΋͘͡ • DynamoDB • DynamoDBͷ׆༻ࣄྫ • DynamoDBͷ໰୊఺ • BloomFilter •

    DynamoDB with BloomFilter

  7. DynamoDB

  8. DynamoDB • AWSͷϑϧϚωʔδυͳ NoSQL DB • ΩϟύγςΟϢχοτͱ͍͏୯ҐͰRead/WriteΛݟ ੵ΋ͬͯઃఆ͢Δ͚ͩͰɺμ΢ϯλΠϜແ͠Ͱεέʔ ϧ •

    ઃఆͨ͠ΩϟύҎ্ͷΞΫηεʹ͸جຊ଱͑ΒΕͳ͍ • ઃఆͨ͠Ωϟύʹରͯ͠՝ۚ

  9. DynamoDB׆༻ࣄྫ A.J.A.ͷ৔߹

  10. ϒϥ΢βΫοΩʔͷ୅ସͱͯ͠ಋೖ

  11. ֨ೲͯ͠Δ৘ใ • ͲΜͳ಺༰ͷ޿ࠂͳͷ͔ • ͲΜͳ৔ॴʹग़ͯΔͷ͔ • ͲΜͳબग़ͷ͞ΕํΛͨ͠ͷ͔

  12. WriteͷλΠϛϯά • ϝσΟΞαΠτͰͷ޿ࠂΫϦοΫ࣌

  13. ReadͷλΠϛϯά • ޿ࠂओαΠτʹຒΊࠐ·ΕͨϏʔίϯλά ϑοΫ • PV΍CVͷϩάʹు͍ͯɺ͔࣍Βͷ޿ࠂબ ग़ʹ׆͔͢

  14. ͜ͷέʔεͰͷ໰୊఺ • Read͠Α͏ͱ߲ͨ͠໨͕ݟ͔ͭͬͯ΋ݟ͔ͭ Βͳͯ͘΋ΩϟύγςΟϢχοτ͸ফඅ͞Ε Δ

  15. A.J.A. Ձ֨.c̋m͞Μ͔ΒϏʔίϯ ड৴͔ͨ͠Β͜ͷID͕ͯ͞͠΍ɻ

  16. DynamoDB ͕͚ͨ͞͠Ͳɺ ͦΜͳ΋Μ͋Β΁ΜͰɻ

  17. A.J.A. ͞Α͏͔ɻ ͢·Μͳɻ

  18. DynamoDB ͓͏ɻ ͋ɺۚ͸΋Ζͯ͘Ͱɻ

  19. ͜ͷέʔεͰͷ໰୊఺ • ReadͷλΠϛϯά͸޿ࠂओαΠτʹຒΊࠐ·ΕͨϏʔίϯ ϑοΫ • Ձ֨.c̋mͷ঎඼ৄࡉϖʔδʹຒΊࠐ·ΕͨΒʁ • YAH̋̋!γϣοϐϯάͷߪೖ׬ྃϖʔδʹషΒΕͨΒʁ • ͜ΕΒͷϏʔίϯͷ΄ͱΜͲ͸ɺA.J.A.޿ࠂΛclickͯ͠ભҠ͠

    ͨΘ͚Ͱ͸ͳ͍ແؔ܎ͳϏʔίϯ • ͭ·ΓɺReadͰݟ͔ͭΒͳ͍͜ͱ͕΄ͱΜͲ

  20. ͜ͷέʔεͰͷ໰୊఺ • ΄ͱΜͲ͕ϛεώοτ͢ΔେྔͷRead • ↑ͷ๫෩͕ಥવਧ͖ߥΕ࢝ΊΔڪා • ͜ΕΒʹରͯ͠ɺΩϟύγςΟΛେྔʹݟੵ ΋Βͳ͚Ε͹ͳΒͳ͍ෆໟײ

  21. A.J.A. σʔλͷ༗ແ͚ͩͰ΋ ࣄલʹ෼͔Ε͹͍͍ͷʹ…

  22. BloomFilter

  23. BloomFilter • ཁૉΛ༩͑ɺޙʹ໰͍߹ΘͤΔͱʮΈͨ͜ͱ ͋ΔΘʔʯ͘Β͍ʹ͍֮͑ͯͯ͘ΕΔσʔλ ߏ଄ • ཁૉΛ༩͑ͯͳ͍ͷʹ஌͔ͬͨͿΔ͜ͱ͕͋ Δ • ༩͑ͨͷʹ֮͑ͯͳ͍ͱ͍͏͜ͱ͸ɺઈରͳ

    ͍

  24. ཁૉΛ༩͑ͯͳ͍ͷʹ஌ͬͨ ͔ͿΔ͜ͱ͕͋Δ ِཅੑ͕͋Δ

  25. ༩͑ͨͷʹ֮͑ͯͳ͍ͱ͍͏ ͜ͱ͸ɺઈରͳ͍ ِӄੑ͸ͳ͍

  26. BloomFilter • ِཅੑΛڐ༰͢Δ͜ͱͰɺੌ·͘͡লεϖʔε Ͱཁૉͷ༗ແΛهԱ͢Δ͜ͱ͕Ͱ͖Δ • Ұཁૉ͋ͨΓ਺bits

  27. DynamoDB with BloomFilter • DynamoDBʹΫΤϦ͢ΔલʹBloomFilterʹฉ ͘͜ͱͰɺϋζϨReadΛ΄΅Χοτ • ِཅੑ͸ൃੜ͢Δ͕ɺͦΕʹΑͬͯى͜Δͷ ͸ͨͩͷϋζϨReadͳͷͰ໰୊ͳ͠

  28. DynamoDB with BloomFilter • A.J.A.ͷ৔߹ • PVϏʔίϯ਺ 1000/s • DynamoDB΁ͷΫΤϦ਺

    50/s • ໿ 1 / 20 • ແؔ܎ͳPVϏʔίϯ਺͕ٸಅͯ͠΋໰୊ͳ͠

  29. ·ͱΊ • A.J.AͰ͸ɺBloomFilterΛDynamoDBͷલஈ ͷωΨςΟϒΩϟογϡͱͯ͠ར༻͍ͯ͠·͢ɻ

  30. ͓·͚ • ͜ͷྨͷਖ਼֬ੑΛͪΐͼͬͱ٘ਜ਼ʹͯ͠ɺ࣌ؒ తɾۭؒతίετΛେ෯ʹઅ໿͢Δσʔλߏ଄ ͸ɺཚ୒σʔλߏ଄ͱݺ͹Ε͍ͯ·͢ɻ • BloomFilterҎ֎ʹ͸ɺHyperLogLog΍MinHash ͱ͍͏ͷ΋͋Γ·͢ɻ • ͦΕͧΕ໘ന͕͋͞ΔͷͰɺௐ΂ͯΈͯԼ͍͞

    Ͷʂ

  31. Have a nice ཚ୒ʂ

  32. ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ