Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DynamoDBとBloomFilterのムフフなカンケイ

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for co-sche co-sche
November 11, 2016
Technology
0
1.5k

 DynamoDBとBloomFilterのムフフなカンケイ

メディアのための広告配信システム、A.J.A.でのDynamoDBとBloomFilterの活用事例

Avatar for co-sche

co-sche

November 11, 2016
Tweet

More Decks by co-sche

See All by co-sche
PackerとAnsibleでコンパクトに始めるInfrastructure as Code
Avatar for co-sche co_sche
0
260

Other Decks in Technology

See All in Technology
Context Engineeringの取り組み
Avatar for nutslove nutslove
0
340
~Everything as Codeを諦めない~ 後からCDK
Avatar for mu7889yoon / Yuta Nakamura mu7889yoon
3
340
10Xにおける品質保証活動の全体像と改善 #no_more_wait_for_test
Avatar for nihonbuson nihonbuson
PRO
2
240
Introduction to Sansan for Engineers / エンジニア向け会社紹介
Avatar for Sansan, Inc. sansan33
PRO
6
68k
Frontier Agents (Kiro autonomous agent / AWS Security Agent / AWS DevOps Agent) の紹介
Avatar for msysh msysh
3
170
CDKで始めるTypeScript開発のススメ
Avatar for つくぼし tsukuboshi
1
390
プロダクト成長を支える開発基盤とスケールに伴う課題
Avatar for yuu26 yuu26
4
1.3k
CDK対応したAWS DevOps Agentを試そう_20260201
Avatar for モブエンジニア（Masaki Okuda） masakiokuda
1
260
ブロックテーマでサイトをリニューアルした話 / 2026-01-31 Kansai WordPress Meetup
Avatar for Toro_Unit (Hiroshi Urabe) torounit
0
460
変化するコーディングエージェントとの現実的な付き合い方 〜Cursor安定択説と、ツールに依存しない「資産」〜
Avatar for empitsu empitsu
4
1.4k
Webhook best practices for rock solid and resilient deployments
Avatar for Guillaume Laforge glaforge
1
290
Data Hubグループ 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
2.7k

Featured

See All Featured
Stewardship and Sustainability of Urban and Community Forests
Avatar for Eric Wiseman pwiseman
0
110
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
25
1.7k
Docker and Python
Avatar for Tania Allard trallard
47
3.7k
Agile Leadership in an Agile Organization
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
80
10 Git Anti Patterns You Should be Aware of
Avatar for Lemi Orhan Ergin lemiorhan
PRO
659
61k
SEO for Brand Visibility & Recognition
Avatar for Aleyda Solis aleyda
0
4.2k
The Curse of the Amulet
Avatar for Matthew Lei leimatthew05
1
8.5k
Thoughts on Productivity
Avatar for Jon Yablonski jonyablonski
74
5k
[RailsConf 2023] Rails as a piece of cake
Avatar for Vladimir Dementyev palkan
59
6.3k
Large-scale JavaScript Application Architecture
Avatar for Addy Osmani addyosmani
515
110k
Leveraging LLMs for student feedback in introductory data science courses - posit::conf(2025)
Avatar for Mine Cetinkaya-Rundel minecr
0
140
Unsuck your backbone
Avatar for Amy Palamountain ammeep
671
58k

Transcript

  1. DynamoDBͱBloomFilterͷ ϜϑϑͳΧϯέΠ

  2. ࣗݾ঺հ ฏদ ߛయ • A.J.A. ϓϩάϥϚʔɾΞʔΩςΫτ • നนͷ֗ ૔ෑग़਎ •

    Ϊλʔ͍ͬͺ͍࣋ͬͯΔ • ͓ڕ৯΂ΒΕͳ͍ • https://github.com/co-sche
  3. None

  4. લճɺBloomFilterͷಈ࡞ݪ ཧͳͲʹ͍ͭͯ࿩Λ͠·ͨ͠ɻ IUUQXXXTMJEFTIBSFOFUDPTDIFTT

  5. A.J.A.ͰͲ͏࢖ΘΕ͍ͯΔͷ͔Λ஌ Γ͍ͨͱ͍͏੠্͕͕ͬͨͷͰɺࠓ ճ͸ͦͷลΛத৺ʹ͓࿩͠͠·͢ɻ

  6. ΋͘͡ • DynamoDB • DynamoDBͷ׆༻ࣄྫ • DynamoDBͷ໰୊఺ • BloomFilter •

    DynamoDB with BloomFilter

  7. DynamoDB

  8. DynamoDB • AWSͷϑϧϚωʔδυͳ NoSQL DB • ΩϟύγςΟϢχοτͱ͍͏୯ҐͰRead/WriteΛݟ ੵ΋ͬͯઃఆ͢Δ͚ͩͰɺμ΢ϯλΠϜແ͠Ͱεέʔ ϧ •

    ઃఆͨ͠ΩϟύҎ্ͷΞΫηεʹ͸جຊ଱͑ΒΕͳ͍ • ઃఆͨ͠Ωϟύʹରͯ͠՝ۚ

  9. DynamoDB׆༻ࣄྫ A.J.A.ͷ৔߹

  10. ϒϥ΢βΫοΩʔͷ୅ସͱͯ͠ಋೖ

  11. ֨ೲͯ͠Δ৘ใ • ͲΜͳ಺༰ͷ޿ࠂͳͷ͔ • ͲΜͳ৔ॴʹग़ͯΔͷ͔ • ͲΜͳબग़ͷ͞ΕํΛͨ͠ͷ͔

  12. WriteͷλΠϛϯά • ϝσΟΞαΠτͰͷ޿ࠂΫϦοΫ࣌

  13. ReadͷλΠϛϯά • ޿ࠂओαΠτʹຒΊࠐ·ΕͨϏʔίϯλά ϑοΫ • PV΍CVͷϩάʹు͍ͯɺ͔࣍Βͷ޿ࠂબ ग़ʹ׆͔͢

  14. ͜ͷέʔεͰͷ໰୊఺ • Read͠Α͏ͱ߲ͨ͠໨͕ݟ͔ͭͬͯ΋ݟ͔ͭ Βͳͯ͘΋ΩϟύγςΟϢχοτ͸ফඅ͞Ε Δ

  15. A.J.A. Ձ֨.c̋m͞Μ͔ΒϏʔίϯ ड৴͔ͨ͠Β͜ͷID͕ͯ͞͠΍ɻ

  16. DynamoDB ͕͚ͨ͞͠Ͳɺ ͦΜͳ΋Μ͋Β΁ΜͰɻ

  17. A.J.A. ͞Α͏͔ɻ ͢·Μͳɻ

  18. DynamoDB ͓͏ɻ ͋ɺۚ͸΋Ζͯ͘Ͱɻ

  19. ͜ͷέʔεͰͷ໰୊఺ • ReadͷλΠϛϯά͸޿ࠂओαΠτʹຒΊࠐ·ΕͨϏʔίϯ ϑοΫ • Ձ֨.c̋mͷ঎඼ৄࡉϖʔδʹຒΊࠐ·ΕͨΒʁ • YAH̋̋!γϣοϐϯάͷߪೖ׬ྃϖʔδʹషΒΕͨΒʁ • ͜ΕΒͷϏʔίϯͷ΄ͱΜͲ͸ɺA.J.A.޿ࠂΛclickͯ͠ભҠ͠

    ͨΘ͚Ͱ͸ͳ͍ແؔ܎ͳϏʔίϯ • ͭ·ΓɺReadͰݟ͔ͭΒͳ͍͜ͱ͕΄ͱΜͲ

  20. ͜ͷέʔεͰͷ໰୊఺ • ΄ͱΜͲ͕ϛεώοτ͢ΔେྔͷRead • ↑ͷ๫෩͕ಥવਧ͖ߥΕ࢝ΊΔڪා • ͜ΕΒʹରͯ͠ɺΩϟύγςΟΛେྔʹݟੵ ΋Βͳ͚Ε͹ͳΒͳ͍ෆໟײ

  21. A.J.A. σʔλͷ༗ແ͚ͩͰ΋ ࣄલʹ෼͔Ε͹͍͍ͷʹ…

  22. BloomFilter

  23. BloomFilter • ཁૉΛ༩͑ɺޙʹ໰͍߹ΘͤΔͱʮΈͨ͜ͱ ͋ΔΘʔʯ͘Β͍ʹ͍֮͑ͯͯ͘ΕΔσʔλ ߏ଄ • ཁૉΛ༩͑ͯͳ͍ͷʹ஌͔ͬͨͿΔ͜ͱ͕͋ Δ • ༩͑ͨͷʹ֮͑ͯͳ͍ͱ͍͏͜ͱ͸ɺઈରͳ

    ͍

  24. ཁૉΛ༩͑ͯͳ͍ͷʹ஌ͬͨ ͔ͿΔ͜ͱ͕͋Δ ِཅੑ͕͋Δ

  25. ༩͑ͨͷʹ֮͑ͯͳ͍ͱ͍͏ ͜ͱ͸ɺઈରͳ͍ ِӄੑ͸ͳ͍

  26. BloomFilter • ِཅੑΛڐ༰͢Δ͜ͱͰɺੌ·͘͡লεϖʔε Ͱཁૉͷ༗ແΛهԱ͢Δ͜ͱ͕Ͱ͖Δ • Ұཁૉ͋ͨΓ਺bits

  27. DynamoDB with BloomFilter • DynamoDBʹΫΤϦ͢ΔલʹBloomFilterʹฉ ͘͜ͱͰɺϋζϨReadΛ΄΅Χοτ • ِཅੑ͸ൃੜ͢Δ͕ɺͦΕʹΑͬͯى͜Δͷ ͸ͨͩͷϋζϨReadͳͷͰ໰୊ͳ͠

  28. DynamoDB with BloomFilter • A.J.A.ͷ৔߹ • PVϏʔίϯ਺ 1000/s • DynamoDB΁ͷΫΤϦ਺

    50/s • ໿ 1 / 20 • ແؔ܎ͳPVϏʔίϯ਺͕ٸಅͯ͠΋໰୊ͳ͠

  29. ·ͱΊ • A.J.AͰ͸ɺBloomFilterΛDynamoDBͷલஈ ͷωΨςΟϒΩϟογϡͱͯ͠ར༻͍ͯ͠·͢ɻ

  30. ͓·͚ • ͜ͷྨͷਖ਼֬ੑΛͪΐͼͬͱ٘ਜ਼ʹͯ͠ɺ࣌ؒ తɾۭؒతίετΛେ෯ʹઅ໿͢Δσʔλߏ଄ ͸ɺཚ୒σʔλߏ଄ͱݺ͹Ε͍ͯ·͢ɻ • BloomFilterҎ֎ʹ͸ɺHyperLogLog΍MinHash ͱ͍͏ͷ΋͋Γ·͢ɻ • ͦΕͧΕ໘ന͕͋͞ΔͷͰɺௐ΂ͯΈͯԼ͍͞

    Ͷʂ

  31. Have a nice ཚ୒ʂ

  32. ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ