Scaling by Cheating: Approximation, Sampling and Fault-friendliness for Scalable Big Learning

talk by Sean Owen @Myrrix at Data Science London @ds-ldn meetup

Transcript

1. Scaling by Cheating Approximation, Sampling and
2. TWO BIG PROBLEMS
3. I Today’s big is just tomorrow’s small. We’re expected to process arbitrarily large data sets by just adding computers. You can’t tell the boss that anything’s too big to handle these days. David, Sr. IT Manager “ ”
4. I Speed is king. People expect up-to-the- second results, and millisecond response times. No more overnight reporting jobs. My data grows 10x but my latency has to drop 10x. Shelly, CTO “ ”
5. TWO BIG SOLUTIONS
6. I Disk and CPU are cheap, on-demand. Frameworks to harness them, like Hadoop, are free and mature. We can easily bring to bear plenty of resources to process data quickly and cheaply. “Scooter”, White Lab “ ”
7. Cheating
8. I Kirk What would you say the odds are on our getting out of here? Spock Difficult to be precise, Captain. I should say approximately seven thousand eight hundred twenty four point seven to one. Kirk Difficult to be precise? Seven thousand eight hundred and twenty four to one? Spock Seven thousand eight hundred twenty four point seven to one. Kirk That's a pretty close approximation. Star Trek, “Errand of Mercy” http://www.redbubble.com/people/feelmeflow
9. WHEN TO CHEAT APPROXIMATE „ Only a few significant
11. APPROXIMATION
12. THE MEAN „  Huge number of data points: x1 … xN „  Independent, from one roughly-normal distribution „  “True” population mean µ? „  Best estimate is sample mean:
13. “CLOSE ENOUGH” MEAN „  Tight-enough estimate of µ? „  With high confidence (>p), µ = (1± ε) µsample „  Student’s t distribution with n-1 degrees of freedom
14. SAMPLING
16. WORD COUNT: TOY EXAMPLE „ Input are text documents „ Exactly how many times does
17. WORD COUNT: USEFUL EXAMPLE „ Input are text documents „ About how many times does
18. COMMONS CRAWL „  s3n://aws-publicdatasets/common-crawl/
 parse-output/segment/*/textData-*" „  Count words, Proper Nouns, etc. in 35GB subset of crawl „  github.com/srowen/commoncrawl" „  Amazon EMR
19. RAW RESULTS „  40 minutes „  40.1% Proper Nouns „  Most frequent words:
20. SAMPLE 10% OF DOCUMENTS „  21 minutes „  39.9% Proper Nouns „  Most frequent words:
21. STOP WHEN “CLOSE ENOUGH” „  CloseEnoughMean.java" „  Stop mapping documents
22. MORE SAMPLING
24. ITEM-ITEM SIMILARITY „  Input: user-item ratings, click count „  Compute item-item similarity „  Takes more than
25. PRUNING „  ItemSimilarityJob" „  --threshold
26. PRUNING EXPERIMENT „  Libimseti data set „  135K users x 165K items „  17M data points „  Rating on scale of 1-10 „  Compute item-item Pearson correlation „  Amazon EMR: 2 x m1.xlarge"
27. RESULTS NO PRUNING PRUNING „  0 threshold „  <10000 pairs per item „  <1000 prefs per user „  178 minutes „  20,400 MB output „  >0.3 threshold „  <10 pairs per item „  <100 prefs per user „  11 minutes „  2 MB output
28. FAULT-FRIENDLINESS
29. Not a Bank
30. FAULTS THREATEN… CONSISTENCY DURABILITY „  Give same answer in same state „  But answer might be approximate „  Many answers are “close” „  OK to give inconsistent answers ? „  Data should not disappear „  But deleting data often has
32. DESIGN FOR… FAST AVAILABILITY FAST 99% DURABILITY „  Multiple replicas „  Need not have a consistent view „  Clients have consistent view through smart load balancing „  Push data into durable store „  Buffer a little locally „  Tolerate loss of “a little”
33. RESOURCES „  Apache Mahout
34. ?