Scaling Your Team

Talk presented at PyCon South Africa, 2012.

Transcript

1. Friday, October 5, 12
2. Scaling Your Project Alex Gaynor PyCon South Africa, 2012 Friday, October 5, 12
3. Me • Software Engineer at rdio.com • Python Software Foundation member • Django software Foundation board member • Django, PyPy, and CPython core developer Friday, October 5, 12
4. What is scaling? Friday, October 5, 12
5. Scaling is not performance Friday, October 5, 12
6. Scalability is the ability of a project to handle more users/traffic/customers linearly with resources Friday, October 5, 12
7. A website is scalable if adding N more server lets it handle N times more traffic than adding one server Friday, October 5, 12
8. A software project is scalable if adding N more engineers lets you ship code N times faster than adding one more engineer Friday, October 5, 12
9. “9 women can’t have a baby in one month” Friday, October 5, 12
10. How to scale a project and a team? Friday, October 5, 12
11. “Bus factor” Friday, October 5, 12
12. One-click deployment Friday, October 5, 12
13. • fabric • Roll out any revision at any time • Rollback to a known working state at any time Friday, October 5, 12
14. • Seems to be a relatively new idea. • Python tools for this aren’t great yet. • Chef, puppet, salt One click infrastructure Friday, October 5, 12
15. New developer onboarding Friday, October 5, 12
16. $ git clone http://url.to/your/project $ mkvirtualenv your-project $ pip install -r requirements.txt $ supervisord start Friday, October 5, 12
17. Developing for the web 5 years ago • HTTP server • Database Friday, October 5, 12
18. Developing for the web now • HTTP server • Databases • Queue workers • Custom daemons Friday, October 5, 12
19. Enter supervisord • It supervises processes • A good idea in production • Potentially really useful in development Friday, October 5, 12
20. Automated Testing Friday, October 5, 12
21. How do we avoid breaking software? • Intuition about what can break • Manual testing • Automated testing Friday, October 5, 12
22. Automated testing changes how we write software Friday, October 5, 12
23. Code Review Friday, October 5, 12
24. • Rietveld, github, phabricator • Code review every single patch • Helps defeat the bus factor Friday, October 5, 12
25. Make new developers do code reviews Friday, October 5, 12
26. Embrace the community Friday, October 5, 12
27. Cost to implement yourself vs. Cost to learn something Friday, October 5, 12
28. Community standards mean your developers come pre-trained Friday, October 5, 12
29. Maintain high code quality Friday, October 5, 12
30. Code Quality • PEP 8 • Good naming • Commenting Friday, October 5, 12
31. Measure and Analyze Friday, October 5, 12
32. • Useful logging • Powerful error analysis (Sentry) • Measurements (mmstats, statsd) Friday, October 5, 12
33. • Automate everything • Embrace the community • Make getting started easy Recap Friday, October 5, 12
34. Thank you! Questions? https://speakerdeck.com/u/alex Friday, October 5, 12