EuroPython 2013: Intro to PyLadies + Python

An introduction to what PyLadies is, and a quick overview of the dataviz tutorial from newcoder.io. Video here: http://www.youtube.com/watch?v=tVB_krH04-g

Transcript

1. Tuesday, July 2, 13
2. Lynn Root @roguelynn roguelynn.com lynn@lynnroot.com Tuesday, July 2, 13
3. Lynn Root Software Engineer at Red Hat PyLadies of San Francisco Python Software Foundation Board Member Tuesday, July 2, 13
4. Today’s Plan Part 1: Intro to PyLadies Part 2: Intro to Python with Data Visualization Part 3: PyLadies Cocktail! Tuesday, July 2, 13
5. International mentorship group Women + friends Python + Open Source community Supported by sponsors, donors, and the PSF Tuesday, July 2, 13
6. Developers or Aspiring Full time or hobby Just in — with Python Tuesday, July 2, 13
7. Workshops Development sprints Speaker series Hack nights Tuesday, July 2, 13
8. ⚑ San Francisco ⚑ Los Angeles ⚑ Wash, DC ⚑ Atlanta ⚑ Seattle ⚑ Portland ⚑ San Diego ⚑ NYC ⚑ Nashville ⚑ Boston ⚑ Austin Tuesday, July 2, 13
9. ⚑ Toronto ⚑ Montreal ⚑ Taiwan ⚑ Berlin ⚑ Brno ⚑ Vienna ⚑ Stockholm ⚑ Tunisia ⚑ India Tuesday, July 2, 13
10. What does it take to lead a PyLadies group? Tuesday, July 2, 13
11. My story! Tuesday, July 2, 13
12. Ulterior motive: find an interested PyLady to start a local PyLadies chapter Tuesday, July 2, 13
13. pyladies.com #pyladies on Freenode github.com/pyladies Questions? Interested folks? Tuesday, July 2, 13
14. Intro to Python with Data Visualization Adapted from newcoder.io/dataviz Tuesday, July 2, 13
15. Follow along rogue.ly/dataviz-slides rogue.ly/dataviz-pdf Tuesday, July 2, 13
16. Workshop Plan Introduction Setup your Machine Part 1: Parsing Data Part 2: Plotting on Google Maps Tuesday, July 2, 13
17. Introduction Project Goals Python Libraries Tuesday, July 2, 13
18. Project Parse data from a CSV file Plot on Google Maps Tuesday, July 2, 13
19. Goals Run & import a Python file Python’s data structures Make a simple maps Tuesday, July 2, 13
20. Python Libraries csv xml + Google Mapping Tuesday, July 2, 13
21. Setup your Machine Python git pip + virtualenv virtualenvwrapper (Mac/Linux) Text Editor Tuesday, July 2, 13
22. Part 0: Setup Setup for Data Visualization Tuesday, July 2, 13
23. Setup the Project 1. Make project directory 2. Clone my repository 3. Install dependencies Tuesday, July 2, 13
24. Questions? Tuesday, July 2, 13
25. Part 1: parse.py Parsing our sample data Tuesday, July 2, 13
26. Part 1: parse.py 1. Module Setup 2. Attacking the Parse Function 3. Using the Parse Function 4. Putting it into Action 5. Explore it further Tuesday, July 2, 13
27. Part 1.1: Module Setup 1. import 2. contants Tuesday, July 2, 13
28. Part 1.2: Attacking 1. Scaffolding 2. Docstrings 3. Comments 4. Code Tuesday, July 2, 13
29. Part 1.2.1: Scaffolding def parse(raw_file, delimiter): return parsed_data Tuesday, July 2, 13
30. Part 1.2.2: Doc Strings Documentation strings, or “docstrings”, are denoted with triple quotes: """This function returns x.""" Tuesday, July 2, 13
31. Part 1.2.2: Doc Strings def parse(raw_file, delimiter): """Parses a raw CSV file to a JSON-like object.""" return parsed_data Tuesday, July 2, 13
32. Part 1.2.3: Comments def parse(raw_file, delimiter): ... # Open CSV file # Read CSV file # Close CSV file # Build a data structure to return parsed_data return parsed_data Tuesday, July 2, 13
33. Part 1.2.4: Code def parse(raw_file, delimiter): ... # Open CSV file opened_file = open(raw_file) ... return parsed_data Tuesday, July 2, 13
34. Part 1.2.4: Code def parse(raw_file, delimiter): ... # Read CSV file csv_data = csv.reader(opened_file, delimiter=delimiter) ... return parsed_data Tuesday, July 2, 13
35. Part 1.2.4: Code def parse(raw_file, delimiter): ... # Setup an empty list parsed_data = [] ... return parsed_data Tuesday, July 2, 13
36. Part 1.2.4: Code def parse(raw_file, delimiter): ... # Skip over first line for headers fields = csv_data.next() ... return parsed_data Tuesday, July 2, 13
37. Part 1.2.4: Code def parse(raw_file, delimiter): ... # Iterate over each row, zip field -> value for row in csv_data: parsed_data.append(dict(zip(fields, row))) ... return parsed_data Tuesday, July 2, 13
38. Part 1.2.4: Code def parse(raw_file, delimiter): ... # Close the CSV file opened_file.close() return parsed_data Tuesday, July 2, 13
39. Part 1.3: Using parse() def main(): # Call parse() and give parameters new_data = parse(MY_FILE, ",") # Let’s see what the data looks like! print new_data Tuesday, July 2, 13
40. Part 1.3: Using parse() def main(): # Call parse() and give parameters new_data = parse(MY_FILE, ",") # Let’s see what the data looks like! print new_data if __name__ == "__main__": main() Tuesday, July 2, 13
41. Part 1.4: Action Follow along with me in my terminal Tuesday, July 2, 13
42. Questions so far? Tuesday, July 2, 13
43. Part 2: map.py Plotting our sample data on Google Maps Tuesday, July 2, 13
44. Part 2: map.py 1. Module Setup 2. Helper Functions 3. Create G-Map Tuesday, July 2, 13
45. Part 3.1: Module Setup MOAR import statements Tuesday, July 2, 13
46. Part 2.2.1: Create Doc def create_document(title, description=""): """Create Overall KML Document""" return doc Tuesday, July 2, 13
47. def create_document(title, description=""): ... # Initialize XML doc doc = xml.dom.minidom.Document() ... Part 2.2.1: Create Doc Tuesday, July 2, 13
48. def create_document(title, description=""): ... # Define as a KML-type XML doc kml = doc.createElement("kml") ... Part 2.2.1: Create Doc Tuesday, July 2, 13
49. def create_document(title, description=""): ... # Pull in common attributes kml.setAttrebutes("xmlns", "http://www.opengist.net/kml/2.2") doc.appendChild(kml) ... Part 2.2.1: Create Doc Tuesday, July 2, 13
50. def create_document(title, description=""): ... # Pull in common attributes document = doc.createElement("Document") kml.appendChild(document) docName = doc.createElement("title") document.appendChild(docName) ... Part 2.2.1: Create Doc Tuesday, July 2, 13
51. def create_document(title, description=""): ... # Pull in common attributes (con’t) docName_text = doc.createTextNode(title) docName.appendChild(docName_text) docDesc = doc.createElement("description") document.appendChild(docDesc) docDesc_text = doc.createTextNode(description) docDesc.appendChild(docDesc_text) ... Part 2.2.1: Create Doc Tuesday, July 2, 13
52. def create_document(title, description=""): ... return doc Part 2.2.1: Create Doc Tuesday, July 2, 13
53. Part 2.2.2: Create Place def create_placemark(address): """Generate KML Placemark for given addr""" return doc Tuesday, July 2, 13
54. def create_placemark(address): ... # Initialize XML doc doc = xml.dom.minidom.Document() ... Part 2.2.2: Create Place Tuesday, July 2, 13
55. def create_placemark(address): ... # Create elements for Placemark and add to doc pm = doc.createElement("Placemark") doc.appendChild(pm) name = doc.createElement("name") pm.appendChild(name) ... Part 2.2.2: Create Place Tuesday, July 2, 13
56. def create_placemark(address): ... # con’t name_text = doc.createTextNode("%(name)s", % address) name.appendChild(name_text) desc = doc.createElement("description") pm.appendChild(desc) ... Part 2.2.2: Create Place Tuesday, July 2, 13
57. def create_placemark(address): ... # con’t desc_text = doc.CreateTextNode("Date: %(date)s, %(description)s", address) desc.appendChild(desc_text) pt = doc.createElement("Point") pm.appendChild(pt) ... Part 2.2.2: Create Place Tuesday, July 2, 13
58. def create_placemark(address): ... # con’t coords = doc.createElement("coordinates") pt.appendChild(coords) coords_text = doc.createTextNode( "%(longitude)s, %(latitude)s" % address) coords.appendChild(coords_text) ... Part 2.2.2: Create Place Tuesday, July 2, 13
59. def create_placemark(address): ... return doc Part 2.2.2: Create Place Tuesday, July 2, 13
60. Part 2.3 Create G-Map! def create_gmap(data_file): """Create G-Map-readable doc""" Tuesday, July 2, 13
61. Part 2.3 Create G-Map! def create_gmap(data_file): ... # Create new KML doc kml_doc = create_document("Crime map", "Plots of Recent SF Crime") ... Tuesday, July 2, 13
62. Part 2.3 Create G-Map! def create_gmap(data_file): ... # grab specific DOM element (all one line) document = kml_doc.documentElement. getElementByTagName("Document")[0] ... Tuesday, July 2, 13
63. Part 2.3 Create G-Map! def create_gmap(data_file): ... # iterate over data to create KML doc for line in data_file: ... Tuesday, July 2, 13
64. Part 2.3 Create G-Map! def create_gmap(data_file): ... # loop continued for line in data_file: # Parse the data into a dict placemark_info = { "longitude": line["X"], "latitude": line["Y"], "name": line["Category"], "description": line["Descript"], "date": line["Date"] ... Tuesday, July 2, 13
65. Part 2.3 Create G-Map! def create_gmap(data_file): ... # loop continued for line in data_file: ... # Avoid null values for lat/long if placemark_info["longitude"] == "0": continue ... Tuesday, July 2, 13
66. Part 2.3 Create G-Map! def create_gmap(data_file): ... # loop continued for line in data_file: ... # parse line of data into KML-format placemark = create_placemark(placemark_info) ... Tuesday, July 2, 13
67. Part 2.3 Create G-Map! def create_gmap(data_file): ... # loop continued for line in data_file: ... # Adds the placemark to KML doc document.appendChild( placemark.documentElement) ... Tuesday, July 2, 13
68. Part 2.3 Create G-Map! def create_gmap(data_file): ... # write parsed KML data to file with open("file_sf.kml", "w") as f: f.write(kml_doc.toprettyxml( intent=" ", encoding="UTF-8")) Tuesday, July 2, 13
69. Part 2.3 Create G-Map! def main(): data = p.parse(p.my_file, ",") return create_gmap(data) if __name__ == "__main__": main() Tuesday, July 2, 13
70. Follow me on uploading to Google Maps Part 2.3 Create G-Map! Tuesday, July 2, 13
71. Congrats! @roguelynn | roguelynn.com lynn@lynnroot.com newcoder.io Tuesday, July 2, 13