From Zero to kind of a hero: Getting your Python side project ready for deployment by Sewagodimo Matlapeng

One evening, my little sister asked me for help with her homework at 9pm. She had already asked her classmates on a WhatsApp group, but nobody was able to help as all her peers had the same limited resources available to them. This gave me the idea for Buza (Zulu for “ask”), a platform for high school learners to ask questions that could be answered by volunteer university students in their free time.

Buza is the first side project of mine that I’ve ever truly wanted to deploy. After months of coding and adding loads of features, I finally reached out to a senior developer to help me deploy my truck of features. The first thing she said was “A lot of this code will have to be replaced with production-ready code”. These are words you never want to hear.

University Computer Science equipped me with the ability to write code to solve problems, but in industry extra skills were required to build production-ready software. This talk will share the valuable lessons I learned from getting this Django web app production-ready. This will start with how to choose the right tools, framework, environment for your project. I will also cover how to set up testing and continuous integration for your project.

What I will cover (Zero to Hero):

No code is the best code
Framework: DJango
Environment: Pipenv
Static Tests: Flake8, Mypy (checks types), isort(sorts imports)
Test Driven Development
Automated testing: Tox and Travis
Code Coverage

Transcript

1. Zero to some kind of hero! Sewagodimo Matlapeng Junior developer and ½ marathon runner WARNING: CAKE ANALOGIES 1
2. Building Buza 2
3. Building Buza 3
4. Deployment ready code • No code is the best code • Choosing a framework • Environment: Pipenv • Static Tests: Flake8, Mypy, isort • Test Driven Development • Code Coverage • Automated testing: Tox and Travis • Deploy 4
5. Dev Savior Syndrome 5
6. “As a software developer you are your own worst enemy and the sooner you realise that the better”- Rich Skrenta 6
7. 7 “...but my app will change the world”
8. Choosing a framework
9. Size and complexity of a project Full stack frameworks allow you to build projects that are large and packed with features. Microservice frameworks are for single module applications. They provide more fine-grain control 9 Choosing a framework
10. The framework for perfectionists with deadlines 10 Django web framework
11. Python Package control VE: Different versions of software packages installed in your machine Pip: Install and Uninstall python packages 11 Pip or Pipenv
12. 1. A library for package installation 2. A library for creating VE 3. A library for managing VE 4. All the commands associated with the libraries 12 Issues with Pip
13. 1. Managing transitive dependencies 2. Requirements.txt file 3. Complex interdependencies 13 More Issues with Pip
14. Pip + virtualenv = Pipenv 14 Do the fusion dance
15. Pipenv 15 Dependency management: Pipfile [[source]] url = "https://pypi.org/simple" verify_ssl = true name = "pypi" [packages] buza-website = {editable = true, path = "."} [dev-packages] [requires] python_version = "3.6"
16. Pipenv 16 Dependency management: Pipfile.lock "flake8": { "hashes": [ "sha256:7253265f7abd8b313e3892944044a365e3f4ac3fcdcfb4298f", "sha256:c7841163e2b576d435799169b78703ad6ac1bbb0f199994fc05" ], "index": "pypi", "version": "==3.5.0" },
17. Pipenv 17 $ pipenv graph pytest-django==3.4.2 - pytest [required: >=3.6, installed: 3.7.2] - atomicwrites [required: >=1.0, installed: 1.2.0] - attrs [required: >=17.4.0, installed: 18.1.0] - more-itertools [required: >=4.0.0, installed: 4.3.0] - six [required: >=1.0.0,<2.0.0, installed: 1.11.0] - pluggy [required: >=0.7, installed: 0.7.1] - py [required: >=1.5.0, installed: 1.5.4] - setuptools [required: Any, installed: 40.2.0]
18. Pipenv 18 Environment management - $ pip install pipenv - $ pipenv install django - $ pipenv install --dev Creating a pipfile.lock - $ pipenv lock - $ pipenv install requirements.txt
19. Don’t rush into coding 19 Don’t start coding yet!
20. Continuously add complexity Focus on the core functionality 20 Question Model: question_title question_body asked_by Question Model: question_title question_body question_grade tags user subject Image voice_notes
21. More features More problems
22. Describe your coding style Me: 22
23. Static Tests Writing code that is readable, maintainable, reusable and up to standard
24. Code reviews
25. Flake8 Flake8 is a Python library that wraps PyFlakes, pycodestyle and pep8 ./bootcamp/settings.py:165:5: F403 'from local_settings import *' used; unable to detect undefined names ./bootcamp/urls.py:5:1: F401 'django.contrib.auth.views as auth_views' imported but unused ./bootcamp/urls.py:6:1: F401 'django.core.urlresolvers.reverse' imported but unused ./bootcamp/urls.py:16:80: E501 line too long (88 > 79 characters) ./bootcamp/members/forms.py:5:1: F401 'django.forms.extras.SelectDateWidget' imported but unused
26. Getting started # Flake8 file [flake8] exclude = .git,*migrations*, ve max-line-length = 88 Installing $ pipenv install flake8 Flake8
27. MyPy Statically typed python???
28. iSort Sorts your file imports $ pipenv install isort Getting started # sort multiple files $ isort views.py urls.py # show a diff before applying any change isort views.py --diff # just check for errors isort urls.py --check-only
29. Place your screenshot here Lets get into it 29 Ready. Set. C0DE.
30. Place your screenshot here ◇ If it can break, it should be tested ◇ Each test should test on function ◇ Always run tests before PUSHING and after PULLING 30 Test Driven Development
31. Hello Buza 31 Create a model # models.py class Question(TimestampedModel, models.Model): author = models.ForeignKey(User) title = _CharField() body = models.TextField()
32. TDD: Unit tests 32 Testing the model class TestQuestion(TestCase) -> None : def test_repr(self) -> None: user: models.User = models.User(username='tester') question: models.Question = models.Question( author=user, title='Example question?', ) assert '<Question: By tester: Example question?>' == repr(question)
33. TDD: Unit tests 33 Testing the list views class TestQuestionList(TestCase): def test_get__empty(self) -> None: """ An empty question view """ response = self.client.get(reverse('question-list')) assert HTTPStatus.OK == response.status_code self.assertTemplateUsed(response, 'buza/question_list.html') self.assertQuerysetEqual(response.context['question_list'], [])
34. Hello Buza 34 Create the test view # views.py from project import models from django.views import generic class QuestionList(generic.ListView): model = models.Question ordering = ['-created']
35. Hello Buza 35 Make a url that calls the view # urls.py from django.conf.urls import url from project.views import QuestionList urlpatterns = [ url(r'^admin/', admin.site.urls), path('questions/', QuestionList.as_view(), name='question-list'), ]
36. Hello Buza 36 A template that will be rendered # question_list.html <html> <head> <title>Home Page</title> </head> <body> {% block content %} <h1>Questions</h1> {% for question in question_list %} {{ question.title }} {{ question.body }} {% endblock %} </body> </html>
37. TDD: Unit tests 37 Testing the list views class TestQuestionList(TestCase): def test_get__empty(self) -> None: """ An empty question view """ response = self.client.get(reverse('question-list')) assert HTTPStatus.OK == response.status_code self.assertTemplateUsed(response, 'buza/question_list.html') self.assertQuerysetEqual(response.context['question_list'], [])
38. More tests... 38 class TestQuestionList(TestCase): def test_get(self) -> None: """ Test Question list view, with a question """ self.author = models.User.objects.create(username='author') self.question = models.Question.objects.create( author=self.author, title='this is my question', ) response = self.client.get(reverse('question-list')) assert HTTPStatus.OK == response.status_code self.assertTemplateUsed(response, 'buza/question_list.html') self.assertQuerysetEqual(response.context['question_list'], []) self.assertContains(response, ‘this is my question’,1)
39. TDD: Unit tests 39 Testing authentication def test_get__anonymous(self) -> None: response = self.client.get(reverse('question-list')) self.assertRedirects(response, '/auth/login/?next=/question-list/') def test_post__anonymous(self) -> None: response = self.client.post(reverse('question-list')) self.assertRedirects(response, '/auth/login/?next=/question-list/')
40. Place your screenshot here How much testing is enough testing? - Misses - Hits - Coverage Percentage 40 Test Coverage
41. Test Coverage 41
42. Continuous integration 42 using Travis….
43. Who is travis? 43 language: python cache: - pip install: - pip install -e . - pip install -r requirements-dev.txt - pip install coveralls script: - flake8 - python manage.py runscript template_lint - py.test after_success: - coveralls
44. Place your screenshot here It’s Time to deploy your application... 44 Takeoff
45. Docker or Heroku 45
46. 46 Docker or Heroku
47. Heroku 47
48. Docker 48
49. Docker or Heroku 49
50. Thanks! The best question is no question at all. https://github.com/buza-project/buza-website/ You can find me: ◇ IG: @matlapeng ◇ Twitter: @Sewagodimo_M 50