EuroPython 2013: Intro to PyLadies + Python

Slide 1

Slide 1 text

Tuesday, July 2, 13

Slide 2

Slide 2 text

Lynn Root @roguelynn roguelynn.com [email protected] Tuesday, July 2, 13

Slide 3

Slide 3 text

Lynn Root Software Engineer at Red Hat PyLadies of San Francisco Python Software Foundation Board Member Tuesday, July 2, 13

Slide 4

Slide 4 text

Today’s Plan Part 1: Intro to PyLadies Part 2: Intro to Python with Data Visualization Part 3: PyLadies Cocktail! Tuesday, July 2, 13

Slide 5

Slide 5 text

International mentorship group Women + friends Python + Open Source community Supported by sponsors, donors, and the PSF Tuesday, July 2, 13

Slide 6

Slide 6 text

Developers or Aspiring Full time or hobby Just in — with Python Tuesday, July 2, 13

Slide 7

Slide 7 text

Workshops Development sprints Speaker series Hack nights Tuesday, July 2, 13

Slide 8

Slide 8 text

⚑ San Francisco ⚑ Los Angeles ⚑ Wash, DC ⚑ Atlanta ⚑ Seattle ⚑ Portland ⚑ San Diego ⚑ NYC ⚑ Nashville ⚑ Boston ⚑ Austin Tuesday, July 2, 13

Slide 9

Slide 9 text

⚑ Toronto ⚑ Montreal ⚑ Taiwan ⚑ Berlin ⚑ Brno ⚑ Vienna ⚑ Stockholm ⚑ Tunisia ⚑ India Tuesday, July 2, 13

Slide 10

Slide 10 text

What does it take to lead a PyLadies group? Tuesday, July 2, 13

Slide 11

Slide 11 text

My story! Tuesday, July 2, 13

Slide 12

Slide 12 text

Ulterior motive: find an interested PyLady to start a local PyLadies chapter Tuesday, July 2, 13

Slide 13

Slide 13 text

pyladies.com #pyladies on Freenode github.com/pyladies Questions? Interested folks? Tuesday, July 2, 13

Slide 14

Slide 14 text

Intro to Python with Data Visualization Adapted from newcoder.io/dataviz Tuesday, July 2, 13

Slide 15

Slide 15 text

Follow along rogue.ly/dataviz-slides rogue.ly/dataviz-pdf Tuesday, July 2, 13

Slide 16

Slide 16 text

Workshop Plan Introduction Setup your Machine Part 1: Parsing Data Part 2: Plotting on Google Maps Tuesday, July 2, 13

Slide 17

Slide 17 text

Introduction Project Goals Python Libraries Tuesday, July 2, 13

Slide 18

Slide 18 text

Project Parse data from a CSV file Plot on Google Maps Tuesday, July 2, 13

Slide 19

Slide 19 text

Goals Run & import a Python file Python’s data structures Make a simple maps Tuesday, July 2, 13

Slide 20

Slide 20 text

Python Libraries csv xml + Google Mapping Tuesday, July 2, 13

Slide 21

Slide 21 text

Setup your Machine Python git pip + virtualenv virtualenvwrapper (Mac/Linux) Text Editor Tuesday, July 2, 13

Slide 22

Slide 22 text

Part 0: Setup Setup for Data Visualization Tuesday, July 2, 13

Slide 23

Slide 23 text

Setup the Project 1. Make project directory 2. Clone my repository 3. Install dependencies Tuesday, July 2, 13

Slide 24

Slide 24 text

Questions? Tuesday, July 2, 13

Slide 25

Slide 25 text

Part 1: parse.py Parsing our sample data Tuesday, July 2, 13

Slide 26

Slide 26 text

Part 1: parse.py 1. Module Setup 2. Attacking the Parse Function 3. Using the Parse Function 4. Putting it into Action 5. Explore it further Tuesday, July 2, 13

Slide 27

Slide 27 text

Part 1.1: Module Setup 1. import 2. contants Tuesday, July 2, 13

Slide 28

Slide 28 text

Part 1.2: Attacking 1. Scaffolding 2. Docstrings 3. Comments 4. Code Tuesday, July 2, 13

Slide 29

Slide 29 text

Part 1.2.1: Scaffolding def parse(raw_file, delimiter): return parsed_data Tuesday, July 2, 13

Slide 30

Slide 30 text

Part 1.2.2: Doc Strings Documentation strings, or “docstrings”, are denoted with triple quotes: """This function returns x.""" Tuesday, July 2, 13

Slide 31

Slide 31 text

Part 1.2.2: Doc Strings def parse(raw_file, delimiter): """Parses a raw CSV file to a JSON-like object.""" return parsed_data Tuesday, July 2, 13

Slide 32

Slide 32 text

Part 1.2.3: Comments def parse(raw_file, delimiter): ... # Open CSV file # Read CSV file # Close CSV file # Build a data structure to return parsed_data return parsed_data Tuesday, July 2, 13

Slide 33

Slide 33 text

Part 1.2.4: Code def parse(raw_file, delimiter): ... # Open CSV file opened_file = open(raw_file) ... return parsed_data Tuesday, July 2, 13

Slide 34

Slide 34 text

Part 1.2.4: Code def parse(raw_file, delimiter): ... # Read CSV file csv_data = csv.reader(opened_file, delimiter=delimiter) ... return parsed_data Tuesday, July 2, 13

Slide 35

Slide 35 text

Part 1.2.4: Code def parse(raw_file, delimiter): ... # Setup an empty list parsed_data = [] ... return parsed_data Tuesday, July 2, 13

Slide 36

Slide 36 text

Part 1.2.4: Code def parse(raw_file, delimiter): ... # Skip over first line for headers fields = csv_data.next() ... return parsed_data Tuesday, July 2, 13

Slide 37

Slide 37 text

Part 1.2.4: Code def parse(raw_file, delimiter): ... # Iterate over each row, zip field -> value for row in csv_data: parsed_data.append(dict(zip(fields, row))) ... return parsed_data Tuesday, July 2, 13

Slide 38

Slide 38 text

Part 1.2.4: Code def parse(raw_file, delimiter): ... # Close the CSV file opened_file.close() return parsed_data Tuesday, July 2, 13

Slide 39

Slide 39 text

Part 1.3: Using parse() def main(): # Call parse() and give parameters new_data = parse(MY_FILE, ",") # Let’s see what the data looks like! print new_data Tuesday, July 2, 13

Slide 40

Slide 40 text

Part 1.3: Using parse() def main(): # Call parse() and give parameters new_data = parse(MY_FILE, ",") # Let’s see what the data looks like! print new_data if __name__ == "__main__": main() Tuesday, July 2, 13

Slide 41

Slide 41 text

Part 1.4: Action Follow along with me in my terminal Tuesday, July 2, 13

Slide 42

Slide 42 text

Questions so far? Tuesday, July 2, 13

Slide 43

Slide 43 text

Part 2: map.py Plotting our sample data on Google Maps Tuesday, July 2, 13

Slide 44

Slide 44 text

Part 2: map.py 1. Module Setup 2. Helper Functions 3. Create G-Map Tuesday, July 2, 13

Slide 45

Slide 45 text

Part 3.1: Module Setup MOAR import statements Tuesday, July 2, 13

Slide 46

Slide 46 text

Part 2.2.1: Create Doc def create_document(title, description=""): """Create Overall KML Document""" return doc Tuesday, July 2, 13

Slide 47

Slide 47 text

def create_document(title, description=""): ... # Initialize XML doc doc = xml.dom.minidom.Document() ... Part 2.2.1: Create Doc Tuesday, July 2, 13

Slide 48

Slide 48 text

def create_document(title, description=""): ... # Define as a KML-type XML doc kml = doc.createElement("kml") ... Part 2.2.1: Create Doc Tuesday, July 2, 13

Slide 49

Slide 49 text

def create_document(title, description=""): ... # Pull in common attributes kml.setAttrebutes("xmlns", "http://www.opengist.net/kml/2.2") doc.appendChild(kml) ... Part 2.2.1: Create Doc Tuesday, July 2, 13

Slide 50

Slide 50 text

def create_document(title, description=""): ... # Pull in common attributes document = doc.createElement("Document") kml.appendChild(document) docName = doc.createElement("title") document.appendChild(docName) ... Part 2.2.1: Create Doc Tuesday, July 2, 13

Slide 51

Slide 51 text

def create_document(title, description=""): ... # Pull in common attributes (con’t) docName_text = doc.createTextNode(title) docName.appendChild(docName_text) docDesc = doc.createElement("description") document.appendChild(docDesc) docDesc_text = doc.createTextNode(description) docDesc.appendChild(docDesc_text) ... Part 2.2.1: Create Doc Tuesday, July 2, 13

Slide 52

Slide 52 text

def create_document(title, description=""): ... return doc Part 2.2.1: Create Doc Tuesday, July 2, 13

Slide 53

Slide 53 text

Part 2.2.2: Create Place def create_placemark(address): """Generate KML Placemark for given addr""" return doc Tuesday, July 2, 13

Slide 54

Slide 54 text

def create_placemark(address): ... # Initialize XML doc doc = xml.dom.minidom.Document() ... Part 2.2.2: Create Place Tuesday, July 2, 13

Slide 55

Slide 55 text

def create_placemark(address): ... # Create elements for Placemark and add to doc pm = doc.createElement("Placemark") doc.appendChild(pm) name = doc.createElement("name") pm.appendChild(name) ... Part 2.2.2: Create Place Tuesday, July 2, 13

Slide 56

Slide 56 text

def create_placemark(address): ... # con’t name_text = doc.createTextNode("%(name)s", % address) name.appendChild(name_text) desc = doc.createElement("description") pm.appendChild(desc) ... Part 2.2.2: Create Place Tuesday, July 2, 13

Slide 57

Slide 57 text

def create_placemark(address): ... # con’t desc_text = doc.CreateTextNode("Date: %(date)s, %(description)s", address) desc.appendChild(desc_text) pt = doc.createElement("Point") pm.appendChild(pt) ... Part 2.2.2: Create Place Tuesday, July 2, 13

Slide 58

Slide 58 text

def create_placemark(address): ... # con’t coords = doc.createElement("coordinates") pt.appendChild(coords) coords_text = doc.createTextNode( "%(longitude)s, %(latitude)s" % address) coords.appendChild(coords_text) ... Part 2.2.2: Create Place Tuesday, July 2, 13

Slide 59

Slide 59 text

def create_placemark(address): ... return doc Part 2.2.2: Create Place Tuesday, July 2, 13

Slide 60

Slide 60 text

Part 2.3 Create G-Map! def create_gmap(data_file): """Create G-Map-readable doc""" Tuesday, July 2, 13

Slide 61

Slide 61 text

Part 2.3 Create G-Map! def create_gmap(data_file): ... # Create new KML doc kml_doc = create_document("Crime map", "Plots of Recent SF Crime") ... Tuesday, July 2, 13

Slide 62

Slide 62 text

Part 2.3 Create G-Map! def create_gmap(data_file): ... # grab specific DOM element (all one line) document = kml_doc.documentElement. getElementByTagName("Document")[0] ... Tuesday, July 2, 13

Slide 63

Slide 63 text

Part 2.3 Create G-Map! def create_gmap(data_file): ... # iterate over data to create KML doc for line in data_file: ... Tuesday, July 2, 13

Slide 64

Slide 64 text

Part 2.3 Create G-Map! def create_gmap(data_file): ... # loop continued for line in data_file: # Parse the data into a dict placemark_info = { "longitude": line["X"], "latitude": line["Y"], "name": line["Category"], "description": line["Descript"], "date": line["Date"] ... Tuesday, July 2, 13

Slide 65

Slide 65 text

Part 2.3 Create G-Map! def create_gmap(data_file): ... # loop continued for line in data_file: ... # Avoid null values for lat/long if placemark_info["longitude"] == "0": continue ... Tuesday, July 2, 13

Slide 66

Slide 66 text

Part 2.3 Create G-Map! def create_gmap(data_file): ... # loop continued for line in data_file: ... # parse line of data into KML-format placemark = create_placemark(placemark_info) ... Tuesday, July 2, 13

Slide 67

Slide 67 text

Part 2.3 Create G-Map! def create_gmap(data_file): ... # loop continued for line in data_file: ... # Adds the placemark to KML doc document.appendChild( placemark.documentElement) ... Tuesday, July 2, 13

Slide 68

Slide 68 text

Part 2.3 Create G-Map! def create_gmap(data_file): ... # write parsed KML data to file with open("file_sf.kml", "w") as f: f.write(kml_doc.toprettyxml( intent=" ", encoding="UTF-8")) Tuesday, July 2, 13

Slide 69

Slide 69 text

Part 2.3 Create G-Map! def main(): data = p.parse(p.my_file, ",") return create_gmap(data) if __name__ == "__main__": main() Tuesday, July 2, 13

Slide 70

Slide 70 text

Follow me on uploading to Google Maps Part 2.3 Create G-Map! Tuesday, July 2, 13

Slide 71

Slide 71 text

Congrats! @roguelynn | roguelynn.com [email protected] newcoder.io Tuesday, July 2, 13