Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Python for the Enterprise

Python for the Enterprise

Language Features and why it’s a great investment for developers and businesses

Presented at Great Wide Open 2014 in Atlanta, Georgia

Lynn Root

April 03, 2014
Tweet

More Decks by Lynn Root

Other Decks in Programming

Transcript

  1. April 2, 2014 Python for the Enterprise Language Features and

    why it’s a great investment for developers and businesses ! ! ! Great Wide Open 2014 - Atlanta, GA
  2. April 2, 2014 whoami Lynn Root ! Partner Engineer @

    Spotify PyLadies of San Francisco Founder Board Member of the Python Software Foundation
  3. April 2, 2014 About Spotify • Available in 55 countries

    • Over 6 million subscribers • Over 24 million monthly active users • Over 20 million songs • Paid over $1 billion to rights holders to date
  4. April 2, 2014 About Spotify • 4.5 TB of compressed

    data from users per day • 400 GB of data from services per day • 64 TB of data generated in Hadoop each day • 590 node Hadoop cluster • 7,500 Hadoop jobs per day
  5. April 2, 2014 About Python • 23 years old, latest

    releases: 2.7.6 and 3.4.0 • Dynamic type system & auto-memory management • Scripting, web dev, networking, scientific computing, database management, 2/3D animation, natural language processing, infosec, artificial intelligence (what else is there?)
  6. April 2, 2014 About Python • Scope defined by whitespace

    • Readability preferred over minor optimization • Cross-platform • Multiple implementations: CPython, PyPy, Jython, IronPython
  7. April 2, 2014 These are some of my favorite things

    • List & generator comprehensions • SimpleHTTPServer, csv, profile, timeit, pdb • IPython notebook • Scapy, requests, Flask, NetworkX, Twisted
  8. April 2, 2014 Who uses Python? • Google • Red

    Hat • Dropbox • Rackspace • Twitter • Facebook • Raspberry Pi • NASA • CERN • ITA • Yahoo! • Walt Disney • IBM • Reddit • YouTube
  9. April 2, 2014 Libraries & Packages • Python’s stdlib is

    very comprehensive • MIME, HTTP, SMTP, FTP • Interprocess Communication & Networking • Data persistence, archiving, file I/O • Unit testing, Doc testing, Logging, and CLI parsing • PyPI - over 38,000 packages • pip install $packagename • Networking like gevent, Twisted • Web Frameworks like Django, Flask, Pyramid • Test Frameworks & Doc tools, like pytest and read the docs and sphinx
  10. April 2, 2014 Community • Rich ecosystem of third-party packages

    • Documentation is king • “Zen” of Python widely adopted
  11. April 2, 2014 Fast where it counts • Code reviews

    are fast • Writing less code • Startup time is instant • Quicker overall development cycle • We’re not CPU-bound, more I/O bound
  12. April 2, 2014 Where Spotify uses Python • User system:

    account creation and login • Social system • Payments system • All analytics jobs • Daemon & request handler • Build scripts • Internal PyPI
  13. April 2, 2014 What Spotify uses • Django, Pyramid, Flask

    • Gevent, Twisted, Celery • requests • SQLAlchemy • lxml • Kazoo
  14. How we’ve given back • Luigi & Snakebite • dh-virtualenv

    • gevent patches • Community diversity
  15. April 2, 2014 Issues Spotify has faced with Python •

    A lot of home-grown solutions • No good HTTP client - every library is missing something • memcache client doesn’t work well with our gevent setup • Debian packaging for everything.
  16. April 2, 2014 Issues Python face • The Great Debate:

    Python 2 versus 3 • The GIL • Python on Windows
  17. April 2, 2014 Looking Forward • Moving to Ubuntu -

    revisit Python 3 • Java’s popularity internally is growing since difficult to hire Python developers • Wanting to incorporate more Python into our desktop client