Basic Steps in Working with Data

A section from chapter 5 of "The Data Journalism Handbook"

Transcript

1. Basic Steps in Working with Data “The Data Journalism Handbook”, chap. 5
2. 下⾯面哪個順序才是對的？ • 先把數據弄到⼿手 • 再看數據裡可以看出什麼端倪 • 先決定要探索什麼議題 • 再看這個議題需要些什麼數據
3. 我⾃自⼰己學⽤用 GA 的經驗 • 數據應⽤用盡有... • 要怎麼開始分析？
4. 我⾃自⼰己學⽤用 GA 的經驗 • Different business objectives • Drive product sales • Drive contact form submissions • Encourage engagement & awareness • Encourage frequent visitation • Provide information quickly • Different objective needs different metrics
5. 下⾯面哪個順序才是對的？ • 先把數據弄到⼿手 • 再看數據裡可以看出什麼端倪 • 先決定要探索什麼議題 • 再看這個議題需要些什麼數據
6. Step 1: Know the Questions You Want to Answer • Work BACKWARD i. list the data-evidenced statements you want to make ii. decide which variables and records to acquire
8. Step 2: Clean Messy Data • Quick way to look for messiness: create frequency tables of categorical variables • Excel • Filter • Pivot Tables • Demo? • Standardize into a shorter list of possibilities • Open Refine (formerly Google Refine) - http://openrefine.org/
9. Data dictionary • how the data file is formatted • explain the codes being used by particular variables
10. IDEALY, the data dictionary is all we need to decipher the data file.
11. In reality, people find ways to workaround design constraints (data dictionary) to fulfill their uncommon needs not preconceived at design time.
12. Example • A analysis by Miami Herald • “the varying rates of punishment that different judges were giving to drunk-driving people” • 1%~2% no punishment? • That’s against the state law (that required anyone convicted of drunk-driving be punished).
13. Example (cont.d) • court clerks use zeros in the prison-jail-fine variable to denote “community service”
14. Step 3: Resolve Undocumented Features in Data • always ask the source if there are any undocumented usage in the data • always examine the results of your analysis and see if they make sense
15. Recap: Basic Steps in Working w/ Data 1. Know the Questions You Want to Answer 2. Clean Messy Data 3. Resolve Undocumented Features