Getting data & why it matters

Presented at Nepal Data Bootcamp with various live demos, including:

* Example project: Dollars for docs- http://projects.propublica.org/docdollars/

Catalogues:

* Nepal Bureau of Statistics - http://cbs.gov.np/
* World Bank Data Catalogue - http://data.worldbank.org/
* WB Country Factsheet - http://data.worldbank.org/country/nepal
* WB Country Map - http://maps.worldbank.org/sa/nepal
* UN Data search - http://data.un.org/Search.aspx?q=nepal
* Collaborative data hub - http://datahub.io/
* Open Nepal data portal - http://opendatanepal.org/

Search:

* Searching google with Operators: https://www.google.de/advanced_search?q=inurl:gov.np+filetype:pdf&hl=en&biw=1024&bih=670&noj=1

Everything is data:

* Nepal Stock index: http://www.myrepublica.com/portal/index.php?action=stocks
* Energy downtimes: http://www.myrepublica.com/portal/index.php?action=pages&page_id=8
* CA members: http://can.gov.np/en/ca_members/index
* Movies on Wikipedia: http://en.wikipedia.org/wiki/List_of_Nepalese_films

Extraction tools

* Google Chrome Scraper Extension - https://chrome.google.com/webstore/detail/scraper/mbigbapnjcgaffohmbkdlecaccepngjd?hl=en
* Google Spreadsheets ImportHTML: http://support.google.com/drive/bin/answer.py?hl=en&answer=155182
* ScraperWiki for little programs: https://scraperwiki.com/

Dealing with PDFs:

* Budget 2012/13 as a PDF: http://mof.gov.np/article-Content-bWlbjg==
* Tabula: http://tabula.nerdpower.org/
* Cometdocs: http://www.cometdocs.com/
* FineReader (Desktop application) - http://finereader.abbyy.com/

Transcript

1. Getting data and why it matters.
3. Data with a purpose is evidence (Not all evidence is data)
4. Data Question
6. Some data tells more stories than other
7. Data re-usability
9. Actions, not just trends
11. Checklist •Who made it? Why? When? •How did they make it? •What is missing? •Which other data relates to it?
12. Data sources •Catalogues •Search •Extraction •Observation
13. Data sources •Catalogues •Search •Extraction •Observation
14. http://opendatanepal.org/ http://data.worldbank.org/ http://data.un.org/ http://datahub.io/
15. Data sources •Catalogues •Search •Extraction •Observation
16. Search operators •inurl:gov.np •filetype:xls
17. Data sources •Catalogues •Search •Extraction •Observation
18. Look inside the Matr... web!
19. Extracting content •Web site “scraping” •PDF “scraping”
20. Data sources •Catalogues •Search •Extraction •Observation
22. Data sources •Catalogues •Search •Extraction •Observation