Python for the Enterprise

Language Features and why it’s a great investment for developers and businesses

Presented at Great Wide Open 2014 in Atlanta, Georgia

Transcript

1. April 2, 2014 Python for the Enterprise Language Features and why it’s a great investment for developers and businesses ! ! ! Great Wide Open 2014 - Atlanta, GA
2. April 2, 2014 whoami Lynn Root ! Partner Engineer @ Spotify PyLadies of San Francisco Founder Board Member of the Python Software Foundation
3. Spotify
4. Python
5. Awesome
6. Awesome About Spotify & Python Why Python and How we use it Looking forward
7. Spotify
8. April 2, 2014 About Spotify • Available in 55 countries • Over 6 million subscribers • Over 24 million monthly active users • Over 20 million songs • Paid over $1 billion to rights holders to date
9. April 2, 2014 About Spotify • 4.5 TB of compressed data from users per day • 400 GB of data from services per day • 64 TB of data generated in Hadoop each day • 590 node Hadoop cluster • 7,500 Hadoop jobs per day
10. Python
11. April 2, 2014 About Python • 23 years old, latest releases: 2.7.6 and 3.4.0 • Dynamic type system & auto-memory management • Scripting, web dev, networking, scientific computing, database management, 2/3D animation, natural language processing, infosec, artificial intelligence (what else is there?)
12. April 2, 2014 About Python • Scope defined by whitespace • Readability preferred over minor optimization • Cross-platform • Multiple implementations: CPython, PyPy, Jython, IronPython
13. April 2, 2014 These are some of my favorite things • List & generator comprehensions • SimpleHTTPServer, csv, profile, timeit, pdb • IPython notebook • Scapy, requests, Flask, NetworkX, Twisted
14. April 2, 2014 Who uses Python? • Google • Red Hat • Dropbox • Rackspace • Twitter • Facebook • Raspberry Pi • NASA • CERN • ITA • Yahoo! • Walt Disney • IBM • Reddit • YouTube
15. Awesome
16. We chose Python because… • Standard Library & Third-party packages • Community • Fast where it counts
17. April 2, 2014 Libraries & Packages • Python’s stdlib is very comprehensive • MIME, HTTP, SMTP, FTP • Interprocess Communication & Networking • Data persistence, archiving, file I/O • Unit testing, Doc testing, Logging, and CLI parsing • PyPI - over 38,000 packages • pip install $packagename • Networking like gevent, Twisted • Web Frameworks like Django, Flask, Pyramid • Test Frameworks & Doc tools, like pytest and read the docs and sphinx
18. April 2, 2014 Community • Rich ecosystem of third-party packages • Documentation is king • “Zen” of Python widely adopted
19. April 2, 2014 Fast where it counts • Code reviews are fast • Writing less code • Startup time is instant • Quicker overall development cycle • We’re not CPU-bound, more I/O bound
20. April 2, 2014 Where Spotify uses Python • User system: account creation and login • Social system • Payments system • All analytics jobs • Daemon & request handler • Build scripts • Internal PyPI
21. April 2, 2014 What Spotify uses • Django, Pyramid, Flask • Gevent, Twisted, Celery • requests • SQLAlchemy • lxml • Kazoo
22. How we’ve given back • Luigi & Snakebite • dh-virtualenv • gevent patches • Community diversity
23. April 2, 2014 Issues Spotify has faced with Python • A lot of home-grown solutions • No good HTTP client - every library is missing something • memcache client doesn’t work well with our gevent setup • Debian packaging for everything.
24. April 2, 2014 Issues Python face • The Great Debate: Python 2 versus 3 • The GIL • Python on Windows
25. April 2, 2014 Looking Forward • Moving to Ubuntu - revisit Python 3 • Java’s popularity internally is growing since difficult to hire Python developers • Wanting to incorporate more Python into our desktop client
26. April 2, 2014 Thanks! Lynn Root lynn@spotify.com @roguelynn | rogue.ly