Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DynamoDBとBloomFilterのムフフなカンケイ

Avatar for co-sche co-sche
November 11, 2016
Technology
1.5k
0

 DynamoDBとBloomFilterのムフフなカンケイ

メディアのための広告配信システム、A.J.A.でのDynamoDBとBloomFilterの活用事例

Avatar for co-sche

co-sche

November 11, 2016

More Decks by co-sche

See All by co-sche
PackerとAnsibleでコンパクトに始めるInfrastructure as Code
Avatar for co-sche co_sche
0
270

Other Decks in Technology

See All in Technology
猫付きpingコマンドを自作
Avatar for uyuki uyuki234
0
250
Bits AI を制するものは Datadog を制す / The player that controls  Bits AI, controls Datadog
Avatar for 株式会社カミナシ kaminashi
0
110
LLM Internals: 언어 모델의 계보와 알고리즘 진화 (2023~2026)
Avatar for Jeongkyu Shin inureyes
PRO
0
290
生成 AI の基礎 〜 サンプル実装で学ぶ基本原理
Avatar for Etsuji Nakai enakai00
7
4.4k
トヨタ⽣産⽅式(TPS)⼊⾨
Avatar for Recruit recruitengineers
PRO
3
920
【CEDEC2026】『Relink』を拡張せよ - 『GRANBLUE FANTASY: Relink - Endless Ragnarok』の開発速度と品質を守るCI運用
Avatar for Cygames, Inc. cygames
PRO
0
190
『三匹の子ぶた』から学ぶネットワークセキュリティの昔と今 / Network Security: Then and Now Through the Lens of The Three Little Pigs
Avatar for NTT docomo Business nttcom
1
6.2k
Contract One Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
19k
ボトムアップ文化が強い組織で セキュリティをどう根付かせていくかの現在進行形の話 / Making Security Stick in a Bottom-Up Organization
Avatar for Takashi Yamaguchi yamaguchitk333
0
240
LLM・AIエージェントシステムベストプラクティス
Avatar for shibuiwilliam shibuiwilliam
6
1.4k
FDEの心得
Avatar for Noriaki Oji noriakioji
5
6.1k
事業価値と Engineering 2026年度版
Avatar for Recruit recruitengineers
PRO
50
24k

Featured

See All Featured
Easily Structure & Communicate Ideas using Wireframe
Avatar for Afnizar Nur Ghifari afnizarnur
194
17k
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.3k
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
Avatar for Akash Hashmi akashhashmi
0
420
Deep Space Network (abreviated)
Avatar for Tony Rice tonyrice
0
260
Building the Perfect Custom Keyboard
Avatar for Naoto Takai takai
2
840
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
53k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
3k
Raft: Consensus for Rubyists
Avatar for Patrick Van Stee vanstee
141
7.6k
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
2.3k
Practical Orchestrator
Avatar for Shlomi Noach shlominoach
191
12k
So, you think you're a good person
Avatar for Per Axbom axbom
PRO
2
2.1k
[RailsConf 2023] Rails as a piece of cake
Avatar for Vladimir Dementyev palkan
59
6.9k

Transcript

  1. DynamoDBͱBloomFilterͷ ϜϑϑͳΧϯέΠ

  2. ࣗݾ঺հ ฏদ ߛయ • A.J.A. ϓϩάϥϚʔɾΞʔΩςΫτ • നนͷ֗ ૔ෑग़਎ •

    Ϊλʔ͍ͬͺ͍࣋ͬͯΔ • ͓ڕ৯΂ΒΕͳ͍ • https://github.com/co-sche
  3. None

  4. લճɺBloomFilterͷಈ࡞ݪ ཧͳͲʹ͍ͭͯ࿩Λ͠·ͨ͠ɻ IUUQXXXTMJEFTIBSFOFUDPTDIFTT

  5. A.J.A.ͰͲ͏࢖ΘΕ͍ͯΔͷ͔Λ஌ Γ͍ͨͱ͍͏੠্͕͕ͬͨͷͰɺࠓ ճ͸ͦͷลΛத৺ʹ͓࿩͠͠·͢ɻ

  6. ΋͘͡ • DynamoDB • DynamoDBͷ׆༻ࣄྫ • DynamoDBͷ໰୊఺ • BloomFilter •

    DynamoDB with BloomFilter

  7. DynamoDB

  8. DynamoDB • AWSͷϑϧϚωʔδυͳ NoSQL DB • ΩϟύγςΟϢχοτͱ͍͏୯ҐͰRead/WriteΛݟ ੵ΋ͬͯઃఆ͢Δ͚ͩͰɺμ΢ϯλΠϜແ͠Ͱεέʔ ϧ •

    ઃఆͨ͠ΩϟύҎ্ͷΞΫηεʹ͸جຊ଱͑ΒΕͳ͍ • ઃఆͨ͠Ωϟύʹରͯ͠՝ۚ

  9. DynamoDB׆༻ࣄྫ A.J.A.ͷ৔߹

  10. ϒϥ΢βΫοΩʔͷ୅ସͱͯ͠ಋೖ

  11. ֨ೲͯ͠Δ৘ใ • ͲΜͳ಺༰ͷ޿ࠂͳͷ͔ • ͲΜͳ৔ॴʹग़ͯΔͷ͔ • ͲΜͳબग़ͷ͞ΕํΛͨ͠ͷ͔

  12. WriteͷλΠϛϯά • ϝσΟΞαΠτͰͷ޿ࠂΫϦοΫ࣌

  13. ReadͷλΠϛϯά • ޿ࠂओαΠτʹຒΊࠐ·ΕͨϏʔίϯλά ϑοΫ • PV΍CVͷϩάʹు͍ͯɺ͔࣍Βͷ޿ࠂબ ग़ʹ׆͔͢

  14. ͜ͷέʔεͰͷ໰୊఺ • Read͠Α͏ͱ߲ͨ͠໨͕ݟ͔ͭͬͯ΋ݟ͔ͭ Βͳͯ͘΋ΩϟύγςΟϢχοτ͸ফඅ͞Ε Δ

  15. A.J.A. Ձ֨.c̋m͞Μ͔ΒϏʔίϯ ड৴͔ͨ͠Β͜ͷID͕ͯ͞͠΍ɻ

  16. DynamoDB ͕͚ͨ͞͠Ͳɺ ͦΜͳ΋Μ͋Β΁ΜͰɻ

  17. A.J.A. ͞Α͏͔ɻ ͢·Μͳɻ

  18. DynamoDB ͓͏ɻ ͋ɺۚ͸΋Ζͯ͘Ͱɻ

  19. ͜ͷέʔεͰͷ໰୊఺ • ReadͷλΠϛϯά͸޿ࠂओαΠτʹຒΊࠐ·ΕͨϏʔίϯ ϑοΫ • Ձ֨.c̋mͷ঎඼ৄࡉϖʔδʹຒΊࠐ·ΕͨΒʁ • YAH̋̋!γϣοϐϯάͷߪೖ׬ྃϖʔδʹషΒΕͨΒʁ • ͜ΕΒͷϏʔίϯͷ΄ͱΜͲ͸ɺA.J.A.޿ࠂΛclickͯ͠ભҠ͠

    ͨΘ͚Ͱ͸ͳ͍ແؔ܎ͳϏʔίϯ • ͭ·ΓɺReadͰݟ͔ͭΒͳ͍͜ͱ͕΄ͱΜͲ

  20. ͜ͷέʔεͰͷ໰୊఺ • ΄ͱΜͲ͕ϛεώοτ͢ΔେྔͷRead • ↑ͷ๫෩͕ಥવਧ͖ߥΕ࢝ΊΔڪා • ͜ΕΒʹରͯ͠ɺΩϟύγςΟΛେྔʹݟੵ ΋Βͳ͚Ε͹ͳΒͳ͍ෆໟײ

  21. A.J.A. σʔλͷ༗ແ͚ͩͰ΋ ࣄલʹ෼͔Ε͹͍͍ͷʹ…

  22. BloomFilter

  23. BloomFilter • ཁૉΛ༩͑ɺޙʹ໰͍߹ΘͤΔͱʮΈͨ͜ͱ ͋ΔΘʔʯ͘Β͍ʹ͍֮͑ͯͯ͘ΕΔσʔλ ߏ଄ • ཁૉΛ༩͑ͯͳ͍ͷʹ஌͔ͬͨͿΔ͜ͱ͕͋ Δ • ༩͑ͨͷʹ֮͑ͯͳ͍ͱ͍͏͜ͱ͸ɺઈରͳ

    ͍

  24. ཁૉΛ༩͑ͯͳ͍ͷʹ஌ͬͨ ͔ͿΔ͜ͱ͕͋Δ ِཅੑ͕͋Δ

  25. ༩͑ͨͷʹ֮͑ͯͳ͍ͱ͍͏ ͜ͱ͸ɺઈରͳ͍ ِӄੑ͸ͳ͍

  26. BloomFilter • ِཅੑΛڐ༰͢Δ͜ͱͰɺੌ·͘͡লεϖʔε Ͱཁૉͷ༗ແΛهԱ͢Δ͜ͱ͕Ͱ͖Δ • Ұཁૉ͋ͨΓ਺bits

  27. DynamoDB with BloomFilter • DynamoDBʹΫΤϦ͢ΔલʹBloomFilterʹฉ ͘͜ͱͰɺϋζϨReadΛ΄΅Χοτ • ِཅੑ͸ൃੜ͢Δ͕ɺͦΕʹΑͬͯى͜Δͷ ͸ͨͩͷϋζϨReadͳͷͰ໰୊ͳ͠

  28. DynamoDB with BloomFilter • A.J.A.ͷ৔߹ • PVϏʔίϯ਺ 1000/s • DynamoDB΁ͷΫΤϦ਺

    50/s • ໿ 1 / 20 • ແؔ܎ͳPVϏʔίϯ਺͕ٸಅͯ͠΋໰୊ͳ͠

  29. ·ͱΊ • A.J.AͰ͸ɺBloomFilterΛDynamoDBͷલஈ ͷωΨςΟϒΩϟογϡͱͯ͠ར༻͍ͯ͠·͢ɻ

  30. ͓·͚ • ͜ͷྨͷਖ਼֬ੑΛͪΐͼͬͱ٘ਜ਼ʹͯ͠ɺ࣌ؒ తɾۭؒతίετΛେ෯ʹઅ໿͢Δσʔλߏ଄ ͸ɺཚ୒σʔλߏ଄ͱݺ͹Ε͍ͯ·͢ɻ • BloomFilterҎ֎ʹ͸ɺHyperLogLog΍MinHash ͱ͍͏ͷ΋͋Γ·͢ɻ • ͦΕͧΕ໘ന͕͋͞ΔͷͰɺௐ΂ͯΈͯԼ͍͞

    Ͷʂ

  31. Have a nice ཚ୒ʂ

  32. ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ