Small Data: Storage For The Rest Of Us

Small Data: Storage For The Rest Of Us

A talk I gave at PyWaw Summit 2015.

077e9a0cb34fa3eba2699240c9509717?s=128

Andrew Godwin

May 26, 2015
Tweet

Transcript

  1. 8.

    You don't need to scale from day one But always

    leave yourself scaling points
  2. 11.

    Explicit Schema ID int Name text Weight uint 1 2

    3 Alice Bob Charles 76 84 65 Implicit Schema { "id": 342, "name": "David", "weight": 44, }
  3. 12.

    Silent Failure { "id": 342, "name": "David", "weight": 74, }

    { "id": 342, "name": "Ellie", "weight": "85kg", } { "id": 342, "nom": "Frankie", "weight": 77, } { "id": 342, "name": "Frankie", "weight": -67, }
  4. 28.
  5. 31.

    SSDs Magnetic Tape Hard Drives Consumer Flash CDs/DVDs Long-life Flash

    Metal-Carbon DVDs 3-6 months 5-10 years 3-5 years 100+ years Approximate time to bit flip, unpowered at room temperature
  6. 32.

    Big Data isn't one thing It depends on type, size,

    complexity, throughput, latency...