Paranormal Statistics: Computing what doesn't add up.

Non-parametric statistical analysis provides a rigorous approach to analyzing non-numeric, non-scalar data such as evaluations or 'buckets' of data. The approach is also designed to handle counts and intervals, thus avoiding several pitfalls of blindly applying a normal or t distribution to right-tailed data. This talk looks at the basics of non-parametric distributions and tests with examples of applicable data.

Transcript

1. Para-normal Statistics: Analyzing what doesn't add up. Steven Lembark Workhorse Computing lembark@wrkhors.com
2. Normality We expect data is normal. It's what we are trained for. Chi-Squared, F depend on it. It's the guts of ANOVA. Theory guarantees it, sort of.
3. What is "normal"? Normal data is: Parametric Real Symmetric Unimodal
4. Ab-normal data Not all data is parametric.
5. Ab-normal data Not all data is parametric: "Bold" + "Tide" / 2 ==
6. Ab-normal data Not all data is parametric: Nominal Data "Bold" + "Tide" / 2 == ?? "Bald" - "Harry" >= 0 ??
7. Ab-normal data Not all data is parametric: Ordinal Data "On a scale of 1 to 5 how would you rate..." Is the average really 3? Are differences between ranks unform?
8. Ab-normal data Not all data is parametric: Ordinal Data "On a scale of 1 to 5 how would you rate..." Is the average really 3? For different people?
9. Ab-normal data Not all data is unimodal, symmetric. Bi-modal data has higher sample variance. Positive data is skewed.
10. Ab-normal data Counts usually Binomial or Poission. Binomial: Coin flips. Poisson: Sample success/failure.
11. Power of Positive Thinking Binomial: Count of success from IID experiments. Mean = np Variance = npq
12. Power of Positive Thinking Poisson: Count of occurrances in sample size n. Mean = np Variance = np
13. Power of Positive Thinking Curves all positive. Right tailed. Binomial has highest power if sample data is binomial. Result: Smaller n for given Beta.
14. Kinda normal Approximations work...
15. Kinda normal Approximations work some of the time. Rule: npq > 5 for binomial approximation. Goal: Keep mean > 3σ so normal is all positive. Q: How good an approximation? A: It depends...
16. The middle way Binomial: n=20, p=0.5 Normal: µ 10, σ = 2.23 Decent approximation.
17. Off to one side Binomial: n=20, p=0.3 Normal: µ = 6, σ = 2.0 Drifting negative.
18. Life on the edge Binomial: n=20, p=0.1 Normal: µ = 2, σ = 1.3 Significant negative.
19. Neverneverland Binomial: n=20, p=0.0013 Normal: µ = 0.26 σ = 0.16 Heavily negative.
20. General rule: npq > 5 Small or large p is skewed. Six-sigma range should be positive. At that point n > 5 / pq. For p = 0.0013, n = 3582. Sample size around 4000?
21. When we assume we make... Assuming normal data leaves a less robust conclusion. Stronger, less robust: Sensitive to individual datasets. Not reproducable.
22. Non-parametric Statistics Origins in Psychology, Biology, Marketing. Analyze counts, ranks. Tests based on discrete distributions.
23. Common in Quality Frequency of failures. QC with No-Go guages. Variations between batch runs. Customer feedback.
24. Example: Safety study Q: Are departments equally "safe"? Q: Is a new configuration any "safer"? Compare sample populations.
25. What is "safe"? Fewer reported injurys? What is P( injury ) per operation?
26. What is "safe"? Fewer reported injurys? What is P( injury ) per operation? 0.5? 0.3?
27. What is "safe"? Fewer reported injurys? What is P( injury ) per operation? 0.5? 0.1? A whole lot less?
28. What is "safe"? Fewer reported injurys? What is P( injury ) per operation? 0.5? 0.1? A whole lot less? N(0.01, 0.01) is heavily negative.
29. Severe? Parametric measure of injurys?
30. Severe? Parametric ranking of injurys? ( Finger + Thumb ) / 2 == ?
31. Severe? Parametric ranking of injurys? ( Finger + Thumb ) / 2 == ? ( Hand + Eye ) == Arm ? ( Hand + Hand ) == 2 * Hand ?
32. Ordinal Data Ranked data, not scaled.
33. Ordinal Data Ranked data, not scaled. Hangnail < Finger Tip < Finger < Hand < Arm "Fuzzy Buckets" Have p( accident ) from history.
34. Kolomogrov-Smirnov Got tonic?
35. Kolomogrov-Smirnov Nope, not Vodka. Like F or ANOVA: Populations are "different".
36. K-S Test Compare cumulative data (blue) vs. Expeted (red). Measure is largest difference (arrow).
37. K-S for safety Rank the injurys on relative scale. Compare counts by bucket. Cumulative distribution: accomodates empty cells. minor mis-catagorization.
38. A good datum is hard to find, You always get the other kind. Apologies to Bessie Smith Sliding-scale questions: "How would you rate..." "How well did..." "How likely are you to..."
39. A good datum is hard to find, You always get the other kind. Apologies to Bessie Smith Reproducability: Variable skill. Variable methods. Variable data handling.
40. A good datum is hard to find, You always get the other kind. Apologies to Bessie Smith Big Data: Multiple sources. Multiple populations. Multiple data standards.
41. Repeatable Analysis Variety of NP tests for "messy" data. Handle protocol, sampling variations. Robust conclusions with real data.
42. Summary Non-parametric data: counts, nominal, ordinal data. Non-parametric analysis avoids NID assumptions. Robust analysis of real data. Even the para-normal.
43. Questions?
44. References: N-P http://www.uta.edu/faculty/sawasthi/Statistics/stnonpar.html http://www.ncbi.nlm.nih.gov/pmc/articles/PMC153434/ Nice writeups.
45. References: K-S http://itl.nist.gov/div898/handbook/eda/section3/eda35g.htm Exploratory data analysis is worth exploring. https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test As always... really good writeup of the test definition, math.
46. References: Robust Analysis https://en.wikipedia.org/wiki/Robust_statistics https://en.wikipedia.org/wiki/Robust_regression Decent introductions. Also look up "robust statistics" at nist.gov or "robust statistical analysis" at duckduckgo.
47. References: This talk http://slideshare.net/lembark Along with everything else I've done...