SOC 4015 & SOC 5050 - Week 06

Lecture slides for Lecture 06 of the Saint Louis University Course Quantitative Analysis: Applied Inferential Statistics. These slides cover the central limit theorem, confidence intervals, and more details on hypothesis testing.

Transcript

1. FOUNDATIONS FOR INFERENCE QUANTITATIVE ANALYSIS CHRISTOPHER PRENER, PH.D. FALL 2018 WEEK 06 LECTURE 06
2. AGENDA QUANTITATIVE ANALYSIS / WEEK 06 / LECTURE 06 1. Front Matter 2. Naming Things 3. Inferential Goals 4. Central Limit Theorem 5. Confidence Intervals 6. More on Hypothesis Testing 7. Back Matter
3. 1 FRONT 
 MATTER
4. Lab 05, PS-03, and Lecture Prep 07 are due before the next lecture. SOC 5050 only - annotated bibliographies for your final projects are due next week! 1. FRONT MATTER ANNOUNCEMENTS
5. NAMING THINGS 2
6. 2. NAMING THINGS A PROBLEMATIC FILE NAME Final project/Jane’s final project research log.md ▸ Spaces ▸ Punctuation ▸ Unnecessary words
7. 2. NAMING THINGS A BETTER FILE NAME finalProject/research_log.md ▸ No spaces; underscore used instead ▸ No punctuation ▸ Unnecessary words removed ▸ Shorter name means easier to type in R
8. 2. NAMING THINGS A PROBLEMATIC FILE NAME Final project/final draft one.md Final project/final draft two.md ▸ Ambiguous naming ▸ Computers cannot take advantage of numeric words
9. 2. NAMING THINGS A BETTER FILE NAME finalProject/draft01.md finalProject/draft02.md ▸ Naming simplified ▸ No spaces; camel case used instead ▸ Using 01 and 02 allows computer to present file names in proper order through Windows File Explorer or macOS Finder
10. 2. NAMING THINGS A BETTER FILE NAME finalProject/2018-01-04_draft.md finalProject/2018-01-05_draft.md ▸ Naming simplified ▸ No spaces; dashes and underscores used instead ▸ Using dates in year-month-day formatting allows computer to present file names in proper order through Windows File Explorer or macOS Finder
11. INFERENTIAL GOALS 3
12. 3. INFERENTIAL GOALS GENERALIZATION = 1 observation Universe or Population
13. 3. INFERENTIAL GOALS GENERALIZATION = 1 observation Sample
 Population Universe
14. 3. INFERENTIAL GOALS GENERALIZATION = 1 observation Sample
 Population Sample Universe
15. 3. INFERENTIAL GOALS GENERALIZATION = 1 observation Sample
 Population Sample Draw Inferences
 About Population Universe
16. CENTRAL LIMIT THEOREM 4
17. 4. CENTRAL LIMIT THEOREM DE MOIVRE–LAPLACE THEOREM ABRAHAM DE MOIRE PIERRE-SIMON LAPLACE
18. 4. CENTRAL LIMIT THEOREM SAMPLING DISTRIBUTIONS ABRAHAM DE MOIRE PIERRE-SIMON LAPLACE
19. A POPULATION > library(“testDriveR”) > autoData <- auto17 > nrow(autoData) [1] 1216 > summary(autoData$combFE) Min. 1st Qu. Median Mean 3rd Qu. Max. 11.00 19.00 23.00 23.27 26.00 58.00 > sd(autoData$combFE) [1] 5.83503 4. CENTRAL LIMIT THEOREM
20. A SAMPLE > library(“dplyr”) > sample1 <- dplyr::sample_n(autoData, 500) > > summary(sample1$combFE) Min. 1st Qu. Median Mean 3rd Qu. Max. 12.00 19.00 23.00 23.38 26.25 56.00 > sd(sample1$combFE) [1] 5.814742 4. CENTRAL LIMIT THEOREM
21. A SECOND SAMPLE > sample2 <- dplyr::sample_n(autoData, 500) > > summary(sample2$combFE) Min. 1st Qu. Median Mean 3rd Qu. Max. 12.0 19.0 23.0 23.3 26.0 58.0 > sd(sample2$combFE) [1] 6.263133 4. CENTRAL LIMIT THEOREM
22. ANOTHER 4,998 SAMPLES > summary(mpgSample$mean) Min. 1st Qu. Median Mean 3rd Qu. Max. 22.63 23.14 23.27 23.27 23.41 23.95 > sd(mpgSample$mean) [1] 0.2003956 4. CENTRAL LIMIT THEOREM
23. DISTRIBUTION OF SAMPLE MEANS 4. CENTRAL LIMIT THEOREM
24. 4. CENTRAL LIMIT THEOREM THE “MAGIC” OF THE CLT This will hold up regardless of the underlying distribution that the sample is drawn from (as long as it has sufficient size)… https://goo.gl/qYaZlx
25. DEFINITION ▸ Population: • Parameters of μ , • Sample size of n • Sample means of ▸ Distribution of : • Has mean of μ • Has a standard deviation of • Normal as n → ∞ 4. CENTRAL LIMIT THEOREM
26. COMPARING POPULATION TO SAMPLES > mean(autoData$combFE) [1] 23.27385 > sd(autoData$combFE) [1] 5.83503 > mean(mpgSample$mean) [1] 23.27471 > sd(mpgSample$mean) [1] 0.2003956 4. CENTRAL LIMIT THEOREM
27. ▸ = distribution of sample 
 means ▸ σ = standard deviation ▸ n = sample size “A means for assessing the reliability of a particular statistic by estimating the difference between the sample statistic and the population statistic.” 4. DESCRIBING DISTRIBUTIONS Let: STANDARD ERROR ¯ x
28. Population
29. Population
30. Population Samples of
31. Population Samples of
32. COMPARING MODELS “REAL” WORLD GOOD FIT POOR FIT
33. ▸ z = standardized value ▸ x = value of observation ▸ σ = standard deviation ▸ µ = mean “The value of an observation expressed in standardized units that can be compared to the normal distribution.” 4. CENTRAL LIMIT THEOREM Let: Z-SCORES
34. ▸ = distribution of sample 
 means ▸ z = standardized value ▸ x = value of observation ▸ σ = standard deviation ▸ µ = mean “Standardized score of sample mean.” 4. DESCRIBING DISTRIBUTIONS Let: Z-SCORES ¯ x
35. Z-SCORES ▸ Taking repeated samples of n=500 from this population, what proportion of these samples will have means ≥ 25? 4. DESCRIBING DISTRIBUTIONS
36. Z-SCORES ▸ Taking repeated samples of n=500 from this population, what proportion of these samples will have means ≥ 25? 4. DESCRIBING DISTRIBUTIONS > pnorm(6.6389, mean = 0, sd = 1, lower.tail = FALSE) [1] 1.580164e-11 ▸ The likelihood of obtaining a sample mean that is ≥ 25 from that population is very, very small.
37. ESTIMATING SAMPLE SIZES ▸ The CLT can be used to estimate sample sizes based on how close we want our sample to be to the population. This is one version of what we call power analyses.
 
 
 
 ▸ The Greek uppercase letter ∆ (“Delta”) is used to represent the amount of error we are willing to tolerate. ▸ We want our sample to be within ± ∆ of the population mean. 4. DESCRIBING DISTRIBUTIONS
38. ESTIMATING SAMPLE SIZES ▸ Given the population parameters we have been using in this case for miles per gallon, what sample size would we need to have sample mean that is within 3 miles per gallon of the population’s? 4. DESCRIBING DISTRIBUTIONS
39. ESTIMATING SAMPLE SIZES ▸ Given the population parameters we have been using in this case for miles per gallon, what sample size would we need to have sample mean that is within 3 miles per gallon of the population’s? 4. DESCRIBING DISTRIBUTIONS ▸ We need to have a sample size of at least 15 vehicles to have a sample mean within 3 miles per gallon of the population’s. ▸ To be within 2 miles per gallon, we need n=32. ▸ To be within 1 miles per gallon, we need n=127.
40. CONFIDENCE INTERVALS 5
41. 5. CONFIDENCE INTERVALS THE PREDICTIVE INTERVAL ▸ Related to the confidence interval. ▸ Can be used prior to sampling to estimate a value for both x and . ▸ Use z-scores from two-sided critical values.
42. z 1.96 2.58 3.29 0.05 0.01 0.001 % of scores inside 95% 99% 99.9% % of scores outside 5% 1% 0.1%
43. 5. CONFIDENCE INTERVALS THE PREDICTIVE INTERVAL ▸ Based on the predictive interval, a given value of x selected at random will fall between 10.035 and 32.559 95% percent of the time.
44. 5. CONFIDENCE INTERVALS THE PREDICTIVE INTERVAL ▸ Based on the predictive interval, a sample mean for n = 40 will fall between 19.515 and 23.079 95% percent of the time.
45. 5. CONFIDENCE INTERVALS THE CONFIDENCE INTERVAL ▸ Used after sampling to the amount of possible error between the given sample mean (for example) and the population sample mean. ▸ Like predictive intervals, use z-scores from two-sided critical values.
46. z 1.96 2.58 3.29 0.05 0.01 0.001 % of scores inside 95% 99% 99.9% % of scores outside 5% 1% 0.1%
47. 5. CONFIDENCE INTERVALS THE CONFIDENCE INTERVAL ▸ If we take a sample of size n = 40 from our population, the interval of the sample mean ± 1.782 has a 95% chance of covering µ.
48. 5. CONFIDENCE INTERVALS WIDTH OF CONFIDENCE INTERVALS Confidence Interval Formula Width 95% 99%
49. 5. CONFIDENCE INTERVALS WIDTH OF CONFIDENCE INTERVALS n 95% CI for μ Width 10 100 1000
50. MORE ON HYPOTHESIS TESTING 6
51. INFORMALLY, A P-VALUE IS THE PROBABILITY UNDER A SPECIFIED STATISTICAL MODEL THAT A STATISTICAL SUMMARY OF THE DATA WOULD BE EQUAL TO OR MORE EXTREME THAN ITS OBSERVED VALUE. American Statistical Association “Statement on p-values”
 (2016)
52. THE PROBABILITY OF GETTING RESULTS AT LEAST AS EXTREME AS THE ONES YOU OBSERVED, GIVEN THAT THE NULL HYPOTHESIS IS CORRECT Christie Aschwanden "Not Even Scientists Can Easily Explain P-values"
 (2015)
53. IMAGINE, HE SAID, THAT YOU HAVE A COIN THAT YOU SUSPECT IS WEIGHTED TOWARD HEADS. (YOUR NULL HYPOTHESIS IS THEN THAT THE COIN IS FAIR.) YOU FLIP IT 100 TIMES AND GET MORE HEADS THAN TAILS. THE P-VALUE WON’T TELL YOU WHETHER THE COIN IS FAIR, BUT IT WILL TELL YOU THE PROBABILITY THAT YOU’D GET AT LEAST AS MANY HEADS AS YOU DID IF THE COIN WAS FAIR. THAT’S IT — NOTHING MORE. Christie Aschwanden "Not Even Scientists Can Easily Explain P-values"
 (2015)
54. AES STATEMENT ON P-VALUES 1. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone. 2. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold. 3. Proper inference requires full reporting and transparency. 4. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result. 5. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis. 6. MORE ON HYPOTHESIS TESTING
55. 6. MORE ON HYPOTHESIS TESTING SHAPIRO-FRANCIA TEST 
 Assumptions: 1. Results are valid for 5 ≤ n ≤ 5000
56. 6. MORE ON HYPOTHESIS TESTING SHAPIRO-FRANCIA TEST Data are not markedly different from the normal distribution. H0 Data are markedly different from the normal distribution. HA
57. 6. MORE ON HYPOTHESIS TESTING SHAPIRO-FRANCIA TEST Data are not markedly different from the normal distribution. H0 Data are markedly different from the normal distribution. HA If the p value associated with the 
 test statistic is greater than .05…
58. 6. MORE ON HYPOTHESIS TESTING SHAPIRO-FRANCIA TEST Data are not markedly different from the normal distribution. H0 Data are markedly different from the normal distribution. HA If the p value associated with the 
 test statistic is less than .05…
59. SHAPIRO-FRANCIA TEST > library("ggplot2") > library("nortest") > > sf.test(mpg$cty) Shapiro-Francia normality test data: mpg$cty W = 0.95577, p-value = 5.293e-06 6. MORE ON HYPOTHESIS TESTING TEST STATISTIC
60. SHAPIRO-FRANCIA TEST > library("ggplot2") > library("nortest") > > sf.test(mpg$cty) Shapiro-Francia normality test data: mpg$cty W = 0.95577, p-value = 5.293e-06 6. MORE ON HYPOTHESIS TESTING P VALUE
61. 6. MORE ON HYPOTHESIS TESTING SHAPIRO-FRANCIA TEST Data are not markedly different from the normal distribution. H0 Data are markedly different from the normal distribution. HA Since the p value associated with the test statistic 
 is less than .05 (p < .001), we reject the null 
 hypothesis and take on the alternative hypothesis.
62. 6. MORE ON HYPOTHESIS TESTING SHAPIRO-FRANCIA TEST The results of the Shapiro-Francia test (W = 0.956,
 p < .001) suggest that the the city fuel efficiency 
 variable is not normally distributed. data: mpg$cty W = 0.95577, p-value = 5.293e-06 Report: ✓ The name of the test ✓ The test statistic (in this case, W) and the p value ✓ A plain English interpretation of the relationship identified
63. 6. MORE ON HYPOTHESIS TESTING SHAPIRO-FRANCIA TEST The results of the Shapiro-Francia test (W = 0.956,
 p = .049) suggest that the the city fuel efficiency 
 variable is not normally distributed. data: mpg$cty W = 0.95577, p-value = .0485235 Report: ✓ The name of the test ✓ The test statistic (in this case, W) and the p value ✓ A plain English interpretation of the relationship identified ✓ The exact p value if feasible
64. 6. MORE ON HYPOTHESIS TESTING SHAPIRO-FRANCIA TEST The results of the Shapiro-Francia test (W = 0.956,
 p = .423) suggest that the the city fuel efficiency 
 variable is normally distributed. data: mpg$cty W = 0.95577, p-value = .4234583 Report: ✓ The name of the test ✓ The test statistic (in this case, W) and the p value ✓ A plain English interpretation of the relationship identified ✓ The exact p value if feasible
65. PROBLEM: EVERYTHING WE HAVE DONE SO FAR ASSUMES WE KNOW THE POPULATION PARAMETERS OR DOES NOT
 REQUIRE THEM (AS IN THE SHAPIRO-FRANCIA TEST)
66. ▸ William Sealy Gosset was an English statistician who worked for the Guinness Brewery in Dublin, Ireland at the turn of the 20th century ▸ We’ve already discussed his impact on our use of degrees of freedom ▸ Gosset published under the pen name “Student” to avoid betraying Guinness trade secrets 6. MORE ON HYPOTHESIS TESTING GOSSET 1876-1937
67. ▸ William Sealy Gosset was an English statistician who worked for the Guinness Brewery in Dublin, Ireland at the turn of the 20th century ▸ The Student’s t distribution approximates normal once the degrees of freedom (n-1) is ≥ 30. 6. MORE ON HYPOTHESIS TESTING GOSSET 1876-1937
72. z 1.96 2.58 3.29 0.05 0.01 0.001 % of scores inside 95% 99% 99.9% % of scores outside 5% 1% 0.1%
73. 6. MORE ON HYPOTHESIS TESTING ERROR Sample Population μ = μ0 μ ≠ μ0 Not Reject yes Type II Reject Type I yes *The null hypothesis is that μ=μ0
74. 6. MORE ON HYPOTHESIS TESTING ERROR Sample Population μ = μ0 μ ≠ μ0 Not Reject yes Type II Reject Type I yes *The null hypothesis is that μ=μ0 FALSE POSITIVE
75. 6. MORE ON HYPOTHESIS TESTING ERROR Sample Population μ = μ0 μ ≠ μ0 Not Reject yes Type II Reject Type I yes *The null hypothesis is that μ=μ0 FALSE NEGATIVE
76. 6. MORE ON HYPOTHESIS TESTING ERROR Sample Population μ = μ0 μ ≠ μ0 Not Reject yes Type II Reject Type I yes *The null hypothesis is that μ=μ0 Pr(Type II)=β 1-β=power Pr(Type I)=
77. P-HACKING https://goo.gl/3oVKaP
78. 5. HYPOTHESIS TESTING P-HACKING
79. 7 BACK 
 MATTER
80. AGENDA REVIEW 7. BACK MATTER 2. Naming Things 3. Inferential Goals 4. Central Limit Theorem 5. Confidence Intervals 6. More on Hypothesis Testing
81. REMINDERS 7. BACK MATTER Lab 05, PS-03, and Lecture Prep 07 are due before the next lecture. SOC 5050 only - annotated bibliographies for your final projects are due next week!