Python Training: Introductory and pragmatic hands-on

Transcript

1. python training: introductory and pragmatic hands-on Thiago Ribeiro twitter.com/ribeiroit / github.com/ribeiroit
2. python hands-on reflection The only source of knowledge is experience. Einstein, Albert.
3. python hands-on before we start you are free to copy and share this material (you can teach someone using this) this is just the beginning you need much more practice than this to become a python jedi we only start the changing when we provoke the changes and this is a change provoker fuck off and let's go ahead
4. python hands-on agenda • intro • install • package manager and virtual environment • python 2.7 / 3+ • compatibility • types • operators • namespaces and scope • importing • strings • files • errors and exceptions • looping • packages • functions • lambda functions • classes • debugging / profiling / logging • unittest • python enhancement proposals (peps) • python frameworks • simple httpserver • django • bottle api • flask api
5. python hands-on intro • open source • indented code as part of syntax • interpreted and high-level • there are two main versions: 2.7 (ascii) and 3+ (utf-8) • does not have native concurrency • programming paradigms: functional / imperative / procedural / oop • created by guido van rossum in 1991 • automagic: dynamic types • easy like a piece of cake • how to learn: get the fundamentals / try it out / again / again / pick up an open source project / change it and pull request more: https://www.python.org/about/gettingstarted/ conventions on this doc: $ command line indicator # comment pho- an acronym prefix for python hands on lines of code
6. python hands-on install • already supported on linux/mac • if you need something newer: debian/ubuntu $ sudo apt-get install python3 centos/fedora $ sudo yum install python3 macos $ brew install python3 docker $ docker pull python:3.7 for god, don't try this on windows
7. python hands-on pip / virtualenv / pipenv • python has a native pkg manager: pip / pip3 • you should use pip / pip3 to install some library or dependency • python also supports environment virtualization to sandboxing deps: virtualenv / pipenv debian/ubuntu $ sudo apt-get install python-setuptools python-dev; sudo easy_install pip; sudo pip install –-upgrade virtualenv centos $ sudo yum install –y epel-release; sudo yum makecache; sudo yum install python-setuptools python-pip; sudo pip install –-upgrade virtualenv python $ pip3 install virtualenv --user creating a new environment: $ cd; mkdir –p python/training; cd python/training; virtualenv deps using on your environment: $ source deps/bin/activate check it out: $ whereis pip install a library: $ pip install flask important tip: pip is for python2 and pip3 is for python3 tip: take a look at conda (machine learning package manager)
8. python hands-on looking closer python 2.7 program $ vi pho-py27.py #!/usr/bin/env python27 print "damn it" $ chmod +x pho-py27.py $ ./pho-py27.py python 3+ program $ vi pho-py3.py #!/usr/bin/env python3 print("oh fuck") $ chmod +x pho-py3.py $ ./pho-py3.py $ vi pho-noshebang.py print("bang, bang, no shebang") $ python3 pho-noshebang.py hell yeah?
9. python hands-on being compatible • to write compatible code between versions 2.7 and 3, use future library: commands: $ cd python/training $ source deps/enable $ pip install future $ vi compatible.py #!/usr/bin/env python27 from __future__ import (absolute_import, division, print_function, unicode_literals) from builtins import * print("Writing down compatible code between python versions") (save) $ chmod +x pho-compatible.py $ ./pho-compatible.py more: http://python-future.org/
10. python hands-on first look on types $ vi pho-types.py #!/usr/bin/env python3 a = 2 b = "Hello world fella" c = 1.244444 d = True e = 1.0j f, g = (1, 2, "x"), [1, 2, 5] h = {'key': 'value', 'name': 'Thiago'} print("Type %s: %s" % (type(a), a)) print("Type %s: %s" % (type(b), b)) print("Type %s: %s" % (type(c), c)) print("Type %s: %s" % (type(d), d)) print("Type %s: %s" % (type(e), e)) print(h['key']) print("three initial chars from my name: %s" % h['name'][:3]) # interpolating types print(a * b) print("Size of b is: %d" % len(b)) print("removing spaces after printing. ", end="") print("everbody on the same line")
11. python hands-on operators $ vi pho-operators.py #!/usr/bin/env python3 a = 4 b = 2 c = [2,3,4] d = "Some stupid string" print("sum: %d. sub: %d. div: %.2f. mul: %d. mod: %d. exp: %d" % ((a+b),(a-b),(a/b),(a*b),(a%b),(a**b))) print("a>b: ", a>b) print("a<b: ", a<b) print("a==b: ", a==b) print("a!=b: ", a!=b) if b in c: print("Yeah! There is %d in %s" % (b, c)) else: print(":/") x = 10 if a in c else 5 print(x) print("Some" in d)
12. python hands-on namespaces and scope $ vi pho-nm.py #!/usr/bin/env python3 first_var = "Hello" print(dir()) def function(): second_var = "World" print(dir()) foo = function foo() print(dir()) def bar(): first_var = "Hola" print(first_var) bar() def first(): global first_var print(first_var) first_var = "Bye" def second(): print("second level") second() print(dir()) first() second()
13. python hands-on importing $ vi pho-imp.py #!/usr/bin/env python3 print("initial namespace =>", dir()) from math import * print("next namespace =>", dir()) $ vi pho-imp2.py #!/usr/bin/env python3 print("initial namespace =>", dir()) from math import sqrt, pow print("next namespace =>", dir()) $ vi pho-imp3.py #!/usr/bin/env python3 print("initial namespace =>", dir()) import math print("next namespace =>", dir())
14. python hands-on strings $ vi pho-strings.py #!/usr/bin/env python3 a = "String" b = 'Identical string' c = """ Another identical string """ l = ('1', "2", """3""") print(a.upper()) print(b.lower()) print("+".join(l)) print(a.replace("ing", "")) print("first: %s." % b[0]) print("last: %s." % b[-1]) print("last second: %s." % b[-2]) print("all after 3: %s." % b[3:]) print("all before 5: %s" % b[:5]) print(a + " and " + b) print(u'Isso é um unicode') print("String \"escaping\".") del a print(dir()) print(b[40])
15. python hands-on files $ vi pho-files.py #!/usr/bin/env python3 # # modes: # [r] = readonly. offset root level # [w] = write. overwritten if file already exists. # [a] = append mode. continues writing after the last char. creates if file does not exist # # [b] = use this option with the others above to open in binary mode. eg: # [wb] = write in binary mode # import os fp = open("app.log", mode = "w") print("name: ", fp.name) print("state is closed: ", fp.closed) print("opening mode: ", fp.mode) fp.close() fp1 = open("app2.log", mode = "a", encoding = "utf-8") fp1.write("just a text inside the file.") fp1.close() with open("app3.log", mode = "w+") as fp3: fp3.write("elegant way to open. auto close the file after this block") print("state is closed: ", fp3.closed) if os.path.isfile("app3.log"): os.remove("app3.log")
16. python hands-on errors and exceptions $ vi pho-errors.py #!/usr/bin/env python3 # an error interrupts the execution of the program. related to syntax print("This is a python program" $ vi pho-exception.py #!/usr/bin/env python3 a = "foo bar" try: print(a[40]) except IndexError: print("this is an index detected error") except SyntaxError: print("should it be a syntax error?") else: print("i'm dumb. could not detect the error") $ vi pho-raising.py #!/usr/bin/env python3 try: a = int(input("Please, type a number greater than zero: ")) if a <= 0: raise ValueError("i said a fucking positive") except ValueError as msg: print(msg) take a look: https://docs.python.org/3.6/library/exceptions.html
17. python hands-on looping $ vi pho-looping.py #!/usr/bin/env python3 fruits = ['orange', 'watermelon', 'papaya'] profile = {'name': 'Thiago', 'age': 34} for x in range(0,10): print("Couting numbers: %d") for item in fruits: print(item) for key, item in profile.items(): print("key: %s. item: %s" % (key, item)) count = 0 while count < 5: print(count) count += 1 for x in range(1,100,2): if x % 27 == 0: break
18. python hands-on packages $ mkdir –p sound/effects $ touch sound/__init__.py $ touch sound/effects/__init__.py $ vi sound/effects/echo.py #!/usr/bin/env python3 def single(): print("echo!") def double(): print("echo!" * 2) if __name__ == "__main__": single() $ vi sound-effects.py #!/usr/bin/env python3 import sound.effects.echo sound.effects.echo.single() $ vi sound-effects2.py #!/usr/bin/env python3 from sound.effects import echo echo.single() echo.double() Pay attention to packages collision, eg: mkdir math; touch math/__init__.py If you import this module, you are conflicting that with original math module. Take a look: https://docs.python.org/3/py-modindex.html
19. python hands-on functions $ vi pho-functions.py #!/usr/bin/env python3 def sum1(a, b): print("A: %d. B: %d" % (a,b)) return(a+b) def sum2(a=0, b=0): print("A: %d. B: %d" % (a,b)) return(a+b) def myfunc(arg1, *mytuple): print("I know this arg: %s" % str(arg1)) for x in mytuple: print("This is new for me: %s" % str(x)) def myfunc2(arg1, **mydict): print("I know this arg: %s" % str(arg1)) for key,value in mydict.items(): print("This is new key [%s] and value [%s]" % (key,value)) print(sum1(1,2)) print(sum2(2)) myfunc(10, 20, 30, 40, 50) print(myfunc2(10, **{'course': 'python', 'grade': 1.5})) # print(sum1(a=1)) Try to comment out the last line to check the results
20. python hands-on lambda functions $ vi pho-lbd.py #!/usr/bin/env python3 # import functools # just a lambda example f = lambda x, y: x+ y f(1,1) # map function with lambda celsius = [39.5, 33.2, 39.5] farenheit = map(lambda x: 9.0/5 * x + 32, celsius) print(farenheit) print(list(farenheit)) # filter function with lambda numbers = [1, 3, 4, 5, 6] just_even = filter(lambda x: x%2 == 0, numbers) # reducing function with lambda result = functools.reduce(lambda x,y: x+y, numbers) print(result) Guido's preferred method is List Comprehensions, he hates lambda and specifically reduce. Lambda's is for the Lisp(ers).
21. python hands-on classes $ vi pho-classes.py #!/usr/bin/env python3 # class Animal: """ This is a tipical class doc """ total = 0 def sound(self): print('generic sound') class Dog(Animal): def sound(self): print('au au') class Chihuahua(Dog): def sound(self): print('au') animal = Animal() animal.total = 10 print(animal.total) animal.sound() dog = Dog() dog.sound() print(Dog.__bases__) chihuahua = Chihuahua() chihuahua.skin() this is not an oop class, please read more in: http://bit.ly/2D5EXnd
22. python hands-on classes: constructor & super $ vi pho-classes.py #!/usr/bin/env python3 # class Person: def __init__(self, name, surname): self.name = name self.surname = surname person = Person(name='thiago', surname='ribeiro') print(person.name.title()) $ vi pho-classes2.py #!/usr/bin/env python3 # class Person: def __init__(self, name, surname): self.name = name self.surname = surname class Employee(Person): def __init__(self, code, age, *args, **kwargs): self.code = code super().__init__(*args, **kwargs) employee1 = Employee(code=123, name='thiago', surname='ribeiro', age=34) # kwargs are mixed employee2 = Employee(123, 34, 'thiago', 'ribeiro') #args has an order print(employee1.name.title()) print(employee2.name.title()) try to comment the super() function to figure out what happens
23. python hands-on debugging $ vi pho-debugging.py #!/usr/bin/env python3 # import pdb from datetime import date pdb.set_trace() class Person: def __init__(self, name, surname, birth): self.name = name self.surname = surname self.birth = date(int(birth[0:4]), int(birth[5:7]), int(birth[8:])) def age(self): age = date.today().year - self.birth.year fyp = date.today().month < self.birth.month and date.today().day < self.birth.day if fyp: age = age - 1 return age person = Person('thiago', 'ribeiro', '1983-02-09') print(person.age()) take a look to iterate with pdb: https://docs.python.org/3/library/pdb.html
24. python hands-on profiling $ vi pho-profiling.py #!/usr/bin/env python3 # import cProfile from datetime import date pr = cProfile.Profile() pr.enable() class Person: def __init__(self, name, surname, birth): self.name = name self.surname = surname self.birth = date(int(birth[0:4]), int(birth[5:7]), int(birth[8:])) def age(self): age = date.today().year - self.birth.year fyp = date.today().month < self.birth.month and date.today().day < self.birth.day if fyp: age = age - 1 return age people = [] for x in range(1,1001): person = Person('thiago', 'ribeiro', '1983-02-09') people.append(person) pr.disable() pr.print_stats() take a look at documentation: https://docs.python.org/3/library/profile.html
25. python hands-on logging $ vi person.py #!/usr/bin/env python3 # import logging from datetime import date logging.basicConfig(filename='myapp.log', level=logging.INFO, format='%(asctime)s %(message)s') class Person: def __init__(self, name, surname, birth): self.name = name self.surname = surname self.birth = date(int(birth[0:4]), int(birth[5:7]), int(birth[8:])) logging.debug('name: %s. surname: %s. birth: %s' % (self.name, self.surname, str(self.birth))) def age(self): age = date.today().year - self.birth.year fyp = date.today().month < self.birth.month and date.today().day < self.birth.day if fyp: age = age – 1 logging.info('age: %d' % age) return age if __name__ == "__main__": person = Person('thiago', 'ribeiro', '1983-05-09') person.age() take a look at documentation: https://docs.python.org/3/howto/logging.html
26. python hands-on unittest $ vi test-person.py #!/usr/bin/env python3 # import unittest from person import Person class PersonTest(unittest.TestCase): def setUp(self): self.person = Person(name='thiago', surname='ribeiro', birth='1983-05-09') def test_name_is_capitalized(self): self.assertEqual(self.person.name(), 'Thiago') def tearDown(self): print('end of testing') if __name__ == "__main__": unittest.main() take a look at documentation: https://docs.python.org/3/library/unittest.html
27. python hands-on python enhancements proposals also known as PEPs you should read starting at 1 until 8 start reading by pep8 – style guide for python code complete list of peps and other documents: https://www.python.org/dev/peps/ validating your code $ pip3 install pep8 $ pep8 <yourfile>.py
28. python hands-on python frameworks • simplehttpserver • python embedded web server (just for samples) • django • mvt and dry concepts (good for cruds) • bottle • micro web framework (thin, thin, thin) • flask • micro web framework based on werkzeug and jinja2 (many modules) • tornado • scalable and non-blocking web server application (async operations) • cyclone • implements tornado api as a twisted protocol (like tornado, but easier)
29. python hands-on simplehttpserver $ vi pho-myserver.py #!/usr/bin/env python3 # import http.server import socketserver PORT = 8000 Handler = http.server.SimpleHTTPRequestHandler with socketserver.TCPServer(("", PORT), Handler) as httpd: print("serving at port", PORT) httpd.serve_forever() take a look at documentation: https://docs.python.org/3/library/http.server.html
30. python hands-on django $ pip3 install django $ django-admin startproject myproject $ cd myproject $ python3 manage.py startapp myapp $ vi myproject/settings.py ... INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'myapp' ] $ python3 manage.py runserver & $ curl http://localhost:8000/ • django project demonstration take a look at documentation: https://docs.djangoproject.com/en/2.0/ and https://scotch.io/tutorials/build-your-first-python-and-django- application
31. python hands-on bottle $ pip3 install bottle $ vi mybottleapi.py from bottle import route, run, template @route('/hello/<name>') def index(name): return {'name': name} run(host='localhost', port=8080)
32. python hands-on flask $ pip3 install flask $ vi helloflask.py #!/usr/bin/env python3 from flask import Flask, jsonify app = Flask(__name__) @app.route('/') def index(): return jsonify({'message': "Hello, World!"}) if __name__ == '__main__': app.run(debug=True) $ chmod +x helloflask.py $ ./helloflask.py & $ curl http://localhost:5000/
33. python hands-on creating a flask api what you need to know rest apis are created over HTTP protocol once you create a new api you should consider some versioning like git you should maintain the http methods to run the actions you need let's build some api to handle our customers. consider the customers api structure method uri description GET /v1/customers list all customers GET /v1/customer/[customer_id] retry a specific customer POST /v1/customer add a new customer PUT /v1/customer/[customer_id] update an existent customer PATCH /v1/customer/[customer_id] same as above DELETE /v1/customer/[customer_id] remove an existent customer preparing $ mkdir customers; cd customers $ touch config.py customer.py setup.py models.py if you need some documentation to your api, take a look: https://swagger.io/
34. python hands-on creating a flask api $ vi config.py #!/usr/bin/env python3 import os class Config: SQLALCHEMY_TRACK_MODIFICATIONS = False DEBUG = False TESTING = False SQLALCHEMY_DATABASE_URI = 'sqlite:////tmp/customers.db' class ProductionConfig(Config): """ Loads the string connection from an environment variable. Where: export DATABASE_URI = 'mysql://user@localhost/customers' """ SQLALCHEMY_DATABASE_URI = os.environ['DATABASE_URI'] class DevelopmentConfig(Config): DEBUG = True class TestingConfig(Config): TESTING = True class LoadConfig: env = DevelopmentConfig() def __init__(self): if 'FLASK_ENV' in os.environ: if os.environ['FLASK_ENV'] == 'prod': self.env = ProductionConfig() if os.environ['FLASK_ENV'] == 'homolog': self.env = TestingConfig()
35. python hands-on creating a flask api $ vi models.py #!/usr/bin/env python3 from datetime import date from flask_sqlalchemy import SQLAlchemy db = SQLAlchemy() class Customer(db.Model): id = db.Column(db.Integer, primary_key=True) name = db.Column(db.String(30), nullable=False) surname = db.Column(db.String(50), nullable=False) birth = db.Column(db.Date, nullable=False) def __init__(self, name, surname, birth): self.name = name self.surname = surname self.birth = birth def __repr__(self): return '<Customer (%d) %s %s>' % (self.id, self.name, self.surname) def age(self): pass
36. python hands-on creating a flask api $ vi customer.py #!/usr/bin/env python3 from datetime import datetime from flask import Flask, jsonify, request, make_response, abort from flask_sqlalchemy import SQLAlchemy from config import LoadConfig from models import db, Customer cnf = LoadConfig() app = Flask(__name__) app.config.from_object(cnf.env) db.init_app(app) # routing @app.errorhandler(404) def not_found(error): return make_response(jsonify({'error': 'Not found'}), 404) @app.route('/', methods=['GET']) def index(): return jsonify({'message': "welcome to customers api"}) # # next slide to continue #
37. python hands-on creating a flask api @app.route('/customer', methods=['POST']) def add_customer(): if not request.json or ('name','surname','birth’) not in request.json.keys(): abort(400) customer = Customer( name=request.json['name'], surname=request.json['surname'], birth=datetime.strptime(request.json['birth'], '%Y-%m-%d').date()) db.session.add(customer) db.session.commit() return jsonify({'name': customer.name, 'surname': customer.surname, 'birth': customer.birth, 'id': customer.id}), 201 @app.route('/customer/<customer_id>', methods=['GET']) def get_customer(customer_id): customer = Customer.query.filter_by(id=customer_id).first() if customer is None: return jsonify({'message': "customer not found"}), 404 else: return jsonify({'name': customer.name, 'surname': customer.surname, 'age': customer.age()}) if __name__ == '__main__': app.run(debug=True)
38. python hands-on creating a flask api $ vi setup.py #!/usr/bin/env python3 from flask import Flask from config import LoadConfig from models import db cnf = LoadConfig() app = Flask(__name__) app.config.from_object(cnf.env) db.init_app(app) with app.app_context(): db.create_all() $ chmod +x setup.py $ ./setup.py $ vi requirements.txt flask flask-sqlalchemy $ pip3 install –r requirements.txt
39. python hands-on creating a flask api running the api $ chmod +x customer.py $ ./customer.py & handling the api $ curl –v http://localhost:5000/ $ curl –H "Content-Type: application/json" –X POST –d'{"name":"Thiago", "surname":"Ribeiro", "birth":"1983-05-09"}' \ http://localhost:5000/customer $ curl –v http://localhost:5000/customer/1 homework • implement the function to calculate the customer age • implement the method GET to list all customers • consider using pagination: https://blog.fossasia.org/paginated-apis-in-flask/ • implement the methods to update and delete customers