ffunctioninc.© FFUNCTION INC, 2011A CASE STUDY OFVISUALIZING (NON-OPEN)GOVERNMENT DATASébastien Pierre, FFunction inc.@Data, Stories & Co., September 2012www.ffctn.com
View Slide
ffunctioninc.© FFUNCTION INC, 2011GOALS➔SHAREour experience with visualizing gov't data➔SHOWhow we built an interactive dataviz➔TELLwhy open-data is important
ffunctioninc.© FFUNCTION INC, 2011INFOGRAPHIC : SE7EN SUMMITS
ffunctioninc.© FFUNCTION INC, 2011GOOGE DATAVIZ CHALLENGE 2010 (FINALIST)
ffunctioninc.© FFUNCTION INC, 2011NATIONAL GEOGRAPHIC SOCIETY'S PROJECTS
ffunctioninc.© FFUNCTION INC, 20112008Canadian FederalTravel & HospitalityExpenses2010USFederalBudget2012Call for tenderMontréal &QuébecFFUNCTION'S GOV-DATA VISUALIZATIONS
ffunctioninc.© FFUNCTION INC, 20112008Canadian FederalTravel & HospitalityExpenses
ffunctioninc.© FFUNCTION INC, 2011SOME THINGS HAVEN'T CHANGED SINCE 2008➔SCRAPING DATAin the absence of open-data, journalists willoften be in the same context, having to spendtime to collect, explore and assess the quality ofthe data.
ffunctioninc.© FFUNCTION INC, 2011SOME THINGS HAVEN'T CHANGED SINCE 2008➔FROM DATA TO STORYEach dataset is a discovery, getting a(compelling) story out of it is still a majorchallenge.
ffunctioninc.© FFUNCTION INC, 2011THE DATA
ffunctioninc.© FFUNCTION INC, 2011As the result of a federal government directive*,Travel and Hospitality Expenses have beenpublished on the web in Canada since 2004* Called “proactive disclosure”
ffunctioninc.© FFUNCTION INC, 2011http://www.tbs-sct.gc.ca/pd-dp/gr-rg/index-eng.asp
ffunctioninc.© FFUNCTION INC, 2011Data is (still) notdirectly accessible,and hosted on eachspecific ministrywebsite, in aspecific format.
ffunctioninc.© FFUNCTION INC, 2011
ffunctioninc.© FFUNCTION INC, 2011NON-OPEN DATAACCURACY PROBLEMSDATA MAY BE MISSINGDATA NOT UP TO DATE
ffunctioninc.© FFUNCTION INC, 201122Mb SQL filescraped by citizens(available on Github)
ffunctioninc.© FFUNCTION INC, 2011A DATASET WHICH TURNS OUT TO BE A BIT OPAQUE...
ffunctioninc.© FFUNCTION INC, 2011BUILDING A TOOL TO EXPLORE THE DATA
ffunctioninc.© FFUNCTION INC, 2011Basic analysis of the data
ffunctioninc.© FFUNCTION INC, 2011Thinking about how to represent the data
ffunctioninc.© FFUNCTION INC, 2011Thinking about the flow of interaction
ffunctioninc.© FFUNCTION INC, 2011Importing and visualizing the data
ffunctioninc.© FFUNCTION INC, 2011Mapping out the different types of expenses (travel, hospitality & guidelines)
ffunctioninc.© FFUNCTION INC, 2011Simplifying the representation (expenses vs guidelines, over guidelines is in red)
ffunctioninc.© FFUNCTION INC, 2011Changing the focus (under/over guidelines instead of total spending)
ffunctioninc.© FFUNCTION INC, 2011Adding guides to improve reading the information
ffunctioninc.© FFUNCTION INC, 2011Adding filtering to narrow down to subsets of the data
ffunctioninc.© FFUNCTION INC, 2011Trying alternative representations on the data
ffunctioninc.© FFUNCTION INC, 2011Trying even more alternative representations on the data
ffunctioninc.© FFUNCTION INC, 2011THE RESULThttp://ffctn.com/a/expensevisualizer
ffunctioninc.© FFUNCTION INC, 2011I just found out the 5 top spendingFederal depts, check it out athttp://ur1.ca/a3spt
ffunctioninc.© FFUNCTION INC, 20111FINDINGS
ffunctioninc.© FFUNCTION INC, 2011TRENDS ONLY BECOME APPARENTWITH THE PROPER MODE OF REPRESENTATIONCumulative spendingMonthly spending
ffunctioninc.© FFUNCTION INC, 2011PROBLEMS IN THE DATA QUALITYBECOME VISIBLE
ffunctioninc.© FFUNCTION INC, 2011Spending of ministers for all departmentsTHINGS YOU WOULD EXPECTARE NOT NECESSARILY THERE
ffunctioninc.© FFUNCTION INC, 2011DATA TO STORY: CHALLENGES➔NON-OPEN DATA– Missing or incomplete data: is the problem in thescraper or in the actual data?– At least you now have a tool to assess (and improve)the data quality
ffunctioninc.© FFUNCTION INC, 2011DATA TO STORY: CHALLENGES➔NOT WHAT I THOUGHT– You might expect something about the data,but the visualization might prove your wrong– You might have been looking for something specificbut you cannot see it in the visualizationSee my “30 min of data visualization”workshop for more on this...
ffunctioninc.© FFUNCTION INC, 2011DATA TO STORY: CHALLENGES➔DID I TRY HARD ENOUGH?– There's no secret: you'll find something interesting ifyou explore your data enough.– If everything fails, you can at least get fun facts orcontroversial examples out of it.
ffunctioninc.© FFUNCTION INC, 2011HOSPITALITY EXPENSES SKYROCKET IN 2008 !!
ffunctioninc.© FFUNCTION INC, 20112.5xAs muchINDUSTRY CANADA'S BIG SPENDERMINISTER DIRECTOR
ffunctioninc.© FFUNCTION INC, 2011WAR IS COSTING CANADA AN ARM AND A LEG!3 MILLIONS!(over a period of five years)
ffunctioninc.© FFUNCTION INC, 2011WE NEED OPEN DATA!THIS IS NOT AN ACCEPTABLE PROACTIVE DISCLOSURE!THE BEST STORY IS:
ffunctioninc.© FFUNCTION INC, 2011Hacking CorruptionMontréal10-11 Novembre 2012http://quebecouvert.org/events/hackonslacorruption/
ffunctioninc.© FFUNCTION INC, 2011THANKYOU![email protected] / @ffunctionWWW.FFCTN.COM
sebastien
whywaita
hassaku63
nsakki55
kawaguti
moritamasami
yoshikieguchi
soracom
yoshikikomoriya
takaking22
.png){kind=avatar}
jisungbin
matsujirushi
sakaguchi
keathley
maggiecrowley
dotmariusz
geoffreycrofte
jmmastey
malarkey
akosma
hursman
keavy
eitanlees
frogandcode
jennybc