Brenda Moon: Exploring Science on Twitter with IPython Notebook and Python Pandas

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Brenda Moon:
Exploring Science on Twitter with IPython Notebook and Python Pandas
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2013 - Sunday, 08 Sep 2013 - Track 1
http://nz.pycon.org/

**Audience level**

Intermediate

**Description**

Exploring discussions of 'science' on Twitter by analysing 13.5 million tweets from 2011. I'll demonstrate how I've been using IPython notebook with the Pandas data analysis library to work with large data tables and time series data. You will see the strength of IPython Notebook for research in allowing you to keep notes interleaved with python code.

**Abstract**

I'm studying Science Communication and looking at how people use the word science on Twitter as a way of understanding what they might mean when they say 'science'. My initial dataset is 13.5 million tweets I collected during 2011. Since the start of 2012 I've been using Python for my data analysis.

I'll demonstrate how I've been using IPython notebook [1] with the Pandas [2] data analysis library to work with large data tables and time series data. The IPython notebook is a great tool for research, allowing notes about research to be kept interleaved with the python code. The ability to quickly see results and plot them using Matplotlib [3] encourages interactive exploration of the data.

[1] http://ipython.org/notebook.html
[2] http://pandas.pydata.org/
[3] http://matplotlib.org/

**YouTube**

http://www.youtube.com/watch?v=FzCk747yjCU

Transcript

1. Exploring Science on Twitter with IPython Notebook and Python Pandas Brenda Moon @brendam
2. IPython Notebook http://www.ipython.org/
3. $ workon kiwipycon2013 (kiwipycon2013)$
4. $ workon kiwipycon2013 (kiwipycon2013)$ ipython notebook --pylab inline
5. (kiwipycon2013)$ ipython notebook --pylab inline 2013-08-31 22:33:35.614 [NotebookApp] Using existing profile dir: '~/.ipython/profile_default' Using local MathJax from ~/.ipython/profile_default/static/ mathjax/MathJax.js Serving notebooks from local directory: /pyconNZ2013talk The IPython Notebook is running at: http:// 127.0.0.1:8888/ Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
11. http://nbviewer.ipython.org
14. IPython %magic functions run %magic to see available options
15. %timeit Time the execution of a cell or line
16. %timeit Time the execution of a cell or line
17. %pastebin Share code as Gist on GitHub %pastebin [-d "Custom description"] 1-7
18. %save / %load Save a cell or range of cells to .py Load a file into a cell
19. %run Run a python script.
20. Pandas “high-performance, easy-to-use data structures and data analysis tools” • series (array like) • dataFrame (table) strong support for time based indexing http://pandas.pydata.org/
21. Tweets keyword: ‘science’
46. Questions? Slides and notebook available on GitHub https://github.com/brendam/pyconnz2013talk @brendam