Upgrade to Pro — share decks privately, control downloads, hide ads and more …

big data

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for ngarneau ngarneau
March 09, 2012
Programming
5
400

big data

big data keynote at the opencode quebec, introducing cassandra, hadoop and pig.

Avatar for ngarneau

ngarneau

March 09, 2012
Tweet

More Decks by ngarneau

See All by ngarneau
Introduction au machine learning avec Scitkit-learn
Avatar for ngarneau ngarneau
0
46
Mocks, stubs & seams
Avatar for ngarneau ngarneau
0
110

Other Decks in Programming

See All in Programming
AIに仕事を丸投げしたら、本当に楽になれるのか
Avatar for ディップ株式会社 dip_tech
PRO
0
160
AIコーディングの理想と現実 2026 | AI Coding: Expectations vs. Reality 2026
Avatar for Tomohisa Takaoka tomohisa
0
630
AIによる開発の民主化を支える コンテキスト管理のこれまでとこれから
Avatar for Mulyu mulyu
3
2.1k
個人開発は儲からない - それでも開発開始1ヶ月で300万円売り上げた方法
Avatar for Taishi Yamasaki taishiyade
0
120
RAGでハマりがちな"Excelの罠"を、データの構造化で突破する
Avatar for harumi harumiweb
1
280
並行開発のためのコードレビュー
Avatar for Miyuki miyukiw
2
2k
今、アーキテクトとして 品質保証にどう関わるか
Avatar for Nealle nealle
0
180
要求定義・仕様記述・設計・検証の手引き - 理論から学ぶ明確で統一された成果物定義
Avatar for Kuniwak orgachem
PRO
1
450
grapheme_strrev関数が採択されました（あと雑感）
Avatar for てきめん tekimen youkidearitai
PRO
1
160
AI主導でFastAPIのWebサービスを作るときに 人間が構造化すべき境界線
Avatar for Jun okazaki okajun35
0
310
The Ralph Wiggum Loop: First Principles of Autonomous Development
Avatar for Ryuichiro Semba sembayui
0
3.7k
米国のサイバーセキュリティタイムラインと見る Goの暗号パッケージの進化
Avatar for tom twinkle tomtwinkle
1
300

Featured

See All Featured
Building a Scalable Design System with Sketch
Avatar for Laura Van Doore lauravandoore
463
34k
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
49
3.3k
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
210
24k
Measuring Dark Social's Impact On Conversion and Attribution
Avatar for Stephen Akadiri stephenakadiri
1
140
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
130
Crafting Experiences
Avatar for Bethany Jepchumba bethany
1
66
From Legacy to Launchpad: Building Startup-Ready Communities
Avatar for Dug Song dugsong
0
160
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
2.7k
Information Architects: The Missing Link in Design Systems
Avatar for Michelle Chin soysaucechin
0
810
Neural Spatial Audio Processing for Sound Field Analysis and Control
Avatar for NII S. Koyama's Lab skoyamalab
0
190
Designing for humans not robots
Avatar for Tammie Lister tammielis
254
26k
SEO Brein meetup: CTRL+C is not how to scale international SEO
Avatar for Linda Hogenes lindahogenes
0
2.4k

Transcript

  1. big data

  2. cassandra - Facebook - 2007. - Apache - 2008. -

    Netflix, Digg, Twitter, Rackspace...

  3. cassandra - non-relationnal - schema-less - open-source - horizontally scalable

    - easy replication - large datasets

  4. cassandra - datacenters - «no single point of failure».

  5. cassandra data model - no joins (maybe joints, we don’t

    know as of version 1.0.9..) - denormalization

  6. cassandra data model - keyspace - column family - row

    key - super column - column / value

  7. cassandra data model application = { users = { ‘ngarneau’:

    { ‘first_name’: ‘nicolas’, ‘last_name’: ‘garneau’ } } }

  8. cassandra data model application = { users = { ‘ngarneau’:

    { ‘first_name’: ‘nicolas’, ‘last_name’: ‘garneau’ } } } keyspace

  9. cassandra data model application = { users = { ‘ngarneau’:

    { ‘first_name’: ‘nicolas’, ‘last_name’: ‘garneau’ } } } keyspace column family

  10. cassandra data model application = { users = { ‘ngarneau’:

    { ‘first_name’: ‘nicolas’, ‘last_name’: ‘garneau’ } } } keyspace column family row key

  11. cassandra data model application = { users = { ‘ngarneau’:

    { ‘first_name’: ‘nicolas’, ‘last_name’: ‘garneau’ } } } keyspace column family row key column

  12. cassandra data model application = { users = { ‘ngarneau’:

    { ‘first_name’: ‘nicolas’, ‘last_name’: ‘garneau’ } } } keyspace column family row key column value

  13. cassandra keep in mind memory disk memtable commit log

  14. cassandra keep in mind memory disk memtable commit log

  15. cassandra keep in mind memory disk memtable commit log

  16. cassandra keep in mind memory disk memtable commit log memtable

  17. cassandra keep in mind memory disk memtable commit log memtable

    memtable

  18. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables

  19. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables

  20. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables SSTables

  21. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables SSTables SSTables

  22. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables SSTables SSTables SSTables

  23. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables SSTables SSTables SSTables

  24. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables SSTables SSTables SSTables SSTables

  25. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables SSTables SSTables SSTables SSTables SSTables

  26. cassandra keep in mind memory disk memtable commit log memtable

    memtable memtables SSTables SSTables SSTables SSTables SSTables SSTables

  27. hadoop - Yahoo! - 2006. - Apache - 2008.

  28. hadoop - mapreduce - hadoop distributed filesystem

  29. hadoop mapreduce - map - reduce

  30. hadoop mapreduce

  31. hadoop mapreduce

  32. hadoop mapreduce

  33. hadoop HDFS

  34. hadoop HDFS data

  35. hadoop HDFS hadoop data

  36. hadoop HDFS hadoop data

  37. hadoop HDFS hadoop data hadoop

  38. hadoop HDFS hadoop data hadoop hadoop

  39. hadoop HDFS hadoop data hadoop hadoop data

  40. hadoop HDFS hadoop data hadoop hadoop data data

  41. hadoop HDFS hadoop data hadoop hadoop data data data

  42. hadoop HDFS hadoop data hadoop hadoop data data data

  43. hadoop HDFS hadoop data hadoop hadoop data data data

  44. hadoop HDFS hadoop data hadoop hadoop data data data cassandra

  45. hadoop HDFS hadoop data hadoop hadoop data data data cassandra

    cassandra

  46. hadoop HDFS hadoop data hadoop hadoop data data data cassandra

    cassandra cassandra

  47. hadoop keep in mind - business intelligence - machine learning

    - collective intelligence

  48. pig - Yahoo! - 2007. - Apache - 2008.

  49. pig - pigs eat anything. - pigs live anywhere. -

    pigs are domestic. - pigs fly.

  50. pig keep in mind

  51. let’s play! https://[email protected]/ngarneau/opencode.git

  52. let’s play! dataset Salons = { ’1’: { ‘id’: 1,

    ‘attendants’: 47, ‘name’: ‘Salon Laval’, ‘year’: 2010 } } Commandes = { ’1’: { ‘amount’: 799, ‘salon’: 1 } }

  53. let’s play! we want to know what is the correlation

    between the number of attendants and the total revenues by salon.