A case study of Visualizing Government Data

Transcript

1. ffunction inc. © FFUNCTION INC, 2011 A CASE STUDY OF VISUALIZING (NON-OPEN) GOVERNMENT DATA Sébastien Pierre, FFunction inc. @Data, Stories & Co., September 2012 www.ffctn.com
2. ffunction inc. © FFUNCTION INC, 2011 GOALS ➔ SHARE our experience with visualizing gov't data ➔ SHOW how we built an interactive dataviz ➔ TELL why open-data is important
3. ffunction inc. © FFUNCTION INC, 2011 INFOGRAPHIC : SE7EN SUMMITS
4. ffunction inc. © FFUNCTION INC, 2011 GOOGE DATAVIZ CHALLENGE 2010 (FINALIST)
5. ffunction inc. © FFUNCTION INC, 2011 NATIONAL GEOGRAPHIC SOCIETY'S PROJECTS
6. ffunction inc. © FFUNCTION INC, 2011 2008 Canadian Federal Travel & Hospitality Expenses 2010 US Federal Budget 2012 Call for tender Montréal & Québec FFUNCTION'S GOV-DATA VISUALIZATIONS
7. ffunction inc. © FFUNCTION INC, 2011 2008 Canadian Federal Travel & Hospitality Expenses
8. ffunction inc. © FFUNCTION INC, 2011 SOME THINGS HAVEN'T CHANGED SINCE 2008 ➔ SCRAPING DATA in the absence of open-data, journalists will often be in the same context, having to spend time to collect, explore and assess the quality of the data.
9. ffunction inc. © FFUNCTION INC, 2011 SOME THINGS HAVEN'T CHANGED SINCE 2008 ➔ FROM DATA TO STORY Each dataset is a discovery, getting a (compelling) story out of it is still a major challenge.
10. ffunction inc. © FFUNCTION INC, 2011 THE DATA
11. ffunction inc. © FFUNCTION INC, 2011 As the result of a federal government directive*, Travel and Hospitality Expenses have been published on the web in Canada since 2004 * Called “proactive disclosure”
12. ffunction inc. © FFUNCTION INC, 2011 http://www.tbs-sct.gc.ca/pd-dp/gr-rg/index-eng.asp
13. ffunction inc. © FFUNCTION INC, 2011 Data is (still) not directly accessible, and hosted on each specific ministry website, in a specific format.
14. ffunction inc. © FFUNCTION INC, 2011
15. ffunction inc. © FFUNCTION INC, 2011 NON-OPEN DATA ACCURACY PROBLEMS DATA MAY BE MISSING DATA NOT UP TO DATE
16. ffunction inc. © FFUNCTION INC, 2011 22Mb SQL file scraped by citizens (available on Github)
17. ffunction inc. © FFUNCTION INC, 2011 A DATASET WHICH TURNS OUT TO BE A BIT OPAQUE...
18. ffunction inc. © FFUNCTION INC, 2011 BUILDING A TOOL TO EXPLORE THE DATA
19. ffunction inc. © FFUNCTION INC, 2011 Basic analysis of the data
20. ffunction inc. © FFUNCTION INC, 2011 Thinking about how to represent the data
21. ffunction inc. © FFUNCTION INC, 2011 Thinking about the flow of interaction
22. ffunction inc. © FFUNCTION INC, 2011 Importing and visualizing the data
23. ffunction inc. © FFUNCTION INC, 2011 Mapping out the different types of expenses (travel, hospitality & guidelines)
24. ffunction inc. © FFUNCTION INC, 2011 Simplifying the representation (expenses vs guidelines, over guidelines is in red)
25. ffunction inc. © FFUNCTION INC, 2011 Changing the focus (under/over guidelines instead of total spending)
26. ffunction inc. © FFUNCTION INC, 2011 Adding guides to improve reading the information
27. ffunction inc. © FFUNCTION INC, 2011 Adding filtering to narrow down to subsets of the data
28. ffunction inc. © FFUNCTION INC, 2011 Trying alternative representations on the data
29. ffunction inc. © FFUNCTION INC, 2011 Trying even more alternative representations on the data
30. ffunction inc. © FFUNCTION INC, 2011 THE RESULT http://ffctn.com/a/expensevisualizer
31. ffunction inc. © FFUNCTION INC, 2011
32. ffunction inc. © FFUNCTION INC, 2011
33. ffunction inc. © FFUNCTION INC, 2011 I just found out the 5 top spending Federal depts, check it out at http://ur1.ca/a3spt
34. ffunction inc. © FFUNCTION INC, 2011
35. ffunction inc. © FFUNCTION INC, 2011
36. ffunction inc. © FFUNCTION INC, 2011 1 FINDINGS
37. ffunction inc. © FFUNCTION INC, 2011 TRENDS ONLY BECOME APPARENT WITH THE PROPER MODE OF REPRESENTATION Cumulative spending Monthly spending
38. ffunction inc. © FFUNCTION INC, 2011 PROBLEMS IN THE DATA QUALITY BECOME VISIBLE
39. ffunction inc. © FFUNCTION INC, 2011 Spending of ministers for all departments THINGS YOU WOULD EXPECT ARE NOT NECESSARILY THERE
40. ffunction inc. © FFUNCTION INC, 2011 DATA TO STORY: CHALLENGES ➔ NON-OPEN DATA – Missing or incomplete data: is the problem in the scraper or in the actual data? – At least you now have a tool to assess (and improve) the data quality
41. ffunction inc. © FFUNCTION INC, 2011 DATA TO STORY: CHALLENGES ➔ NOT WHAT I THOUGHT – You might expect something about the data, but the visualization might prove your wrong – You might have been looking for something specific but you cannot see it in the visualization See my “30 min of data visualization” workshop for more on this...
42. ffunction inc. © FFUNCTION INC, 2011 DATA TO STORY: CHALLENGES ➔ DID I TRY HARD ENOUGH? – There's no secret: you'll find something interesting if you explore your data enough. – If everything fails, you can at least get fun facts or controversial examples out of it.
43. ffunction inc. © FFUNCTION INC, 2011 HOSPITALITY EXPENSES SKYROCKET IN 2008 !!
44. ffunction inc. © FFUNCTION INC, 2011 2.5x As much INDUSTRY CANADA'S BIG SPENDER MINISTER DIRECTOR
45. ffunction inc. © FFUNCTION INC, 2011 WAR IS COSTING CANADA AN ARM AND A LEG! 3 MILLIONS! (over a period of five years)
46. ffunction inc. © FFUNCTION INC, 2011 WE NEED OPEN DATA! THIS IS NOT AN ACCEPTABLE PROACTIVE DISCLOSURE! THE BEST STORY IS:
47. ffunction inc. © FFUNCTION INC, 2011
48. ffunction inc. © FFUNCTION INC, 2011 Hacking Corruption Montréal 10-11 Novembre 2012 http://quebecouvert.org/events/hackonslacorruption/
49. ffunction inc. © FFUNCTION INC, 2011 THANK YOU! sebastien@ffctn.com / @ffunction WWW.FFCTN.COM