PyConZA 2013: "Application To Platform: how we used Python to scale Yola" by Michael Joseph

This is the story of how Yola engineers transformed a website building application into a robust, distributed platform (for hosting and building websites).

By:

» decomposing and augmenting our services layer into multiple Python-driven API services
» in a series of small low-risk changes (which i’ll discuss in more detail)

we organically built out the service layer.

This talk mostly concerns itself with the python API services layer, since i’m the Services Team Lead. There are a number of other interesting talks that I have yet to convince some Yola colleagues to prepare:

» yodeploy and yoconfigurator
» yola’s hosting platform
» release automation and continuous deployment

I shall attempt to:

» extract and present some patterns that worked for us
» discuss the organisational flexibility and engineering workflow that enabled us to refactor the system architecture, and
» share the medium term roadmap for our system.

Transcript

1. Application To Platform How we used Python to scale Yola 1 / 25
2. michaeljoseph @github @twitter 2 / 25
3. this talk is about 3 / 25
4. follow along at home https://github.com/michaeljoseph/application-to-platform 4 / 25
5. Other talks yodeploy and yoconfigurator yola’s hosting platform release automation and continuous deployment 5 / 25
6. overview before and after lessons deployment / rollout patterns roadmap 6 / 25
7. History 7 / 25
8. architecture goals Simplicity is preferable to complexity Existing solutions are preferable to entirely new ones Open Source solutions are preferable to proprietary ones Avoid single points of failure 8 / 25
9. architecture changelog SynthaSite v2 beta — November 2007: The SynthaSite v2 beta went live. SynthaSite Platform 2.0 — June 2008: gh-pages step in creating a more scalable and reliable platform, introducing load balancing and failover databases and filesystems to the Site Building cluster SynthaSite Platform 2.1 — July 2008: Adds a dedicated hosting file system and database server, with drbd failover. SynthaSite Platform 2.2 — September 2008: Adds horizontal scalability to the Site Builder database tier, allowing additional Site Builder database servers to be added to improve capacity. 9 / 25
10. architecture changelog SynthaSite Platform 2.3 — October 2008: Adds the domain service and the payment service, allowing for the gh-pages company revenue stream. SynthaSite Platform 2.4 — November 2008: Adds hosting web server load balancing, allowing horizontal scaling as well as high availability. SynthaSite Platform 2.5 — December 2008: Adds the ability to have multiple hosting file servers, allowing horizontal scaling of hosting storage and I/O. 10 / 25
11. architecture changelog Yola Platform 2.6 — May 2009: Adds the ability to have multiple Site Building file servers, allowing horizontal scaling of Site Building storage and I/O. Yola Platform 2.7 — May 2009: Adds the Resource Store, which facilitates scalable and highly-available storage of site resources. Yola Platform 2.8 — June 2009: Adds the Checkout Service, which facilitates the provision of purchase flows in a secure fashion (it, over SSL). Yola Platform 2.9 — August 2009: Adds Resource Store instances to the publishing environment. 11 / 25
12. Before And After 12 / 25
13. lessons release early and often keep the changes small and pointy lowers the cognitive overhead of review increases confidence in pushing out new code small surface area of introduced bugs to inspect bugs are shallow anyway because we have many eyeballs 13 / 25
14. standardise applications and installables applications installables 14 / 25
15. deployment / rollout patterns new services first switch client applications using standard demands based service clients cleanup works for hardware too 15 / 25
16. evolving services with service-clients 16 / 25
17. generated docs 17 / 25
18. engineering workflow hiring meritocracy BDFL pull requests eyeballs 18 / 25
19. yola open source https://github.com/yola https://github.com/yola/demands https://github.com/yola/property-caching https://github.com/yola/auth_tkt https://github.com/yola/hashcache 19 / 25
20. roadmap database succession feature flags modernise all the things 20 / 25
21. roadmap: Monitoring graphite munin sentry 21 / 25
22. roadmap: what was hard slow boring singularity imminent 22 / 25
23. summary Slideshow created using remark. 23 / 25
24. questions 24 / 25
25. 25 / 25