Drupal and DevOps in a Large Organisation

In September 2013, @jtopper spoke at DrupalCon Prague, covering some of the things we learned whilst introducing a DevOps capability into a large, established organisation. A frank discussion of the issues, it covers what we did badly as well as what we did well.

You can view a recording of the talk: http://www.youtube.com/embed/euRbvxdEeLU?

Transcript

1. DevOps Track · Jon Topper, The Scale Factory · 24 September 2013 DevOps & Drupal in a Large Organisation
2. DevOps
3. DevOps
4. The Four Pillars of DevOps Culture Automation Monitoring Sharing
5. DevOps Success Stories Launched 2010 Founded 2004 Launched 2005 Founded 2006 Launched 2008
6. DevOps Success Stories
7. Other Companies?
9. ITV ‣ UK’s largest commercial TV network ‣ Launched 1955 ‣ >4000 employees ‣ £2,196 million (revenue 2012)
10. Q4 2010 - status quo ‣ Internally-built .Net CMS ‣ Small number of Drupal 6 sites ‣ Releases every 6 weeks (multi-hour, overnight) ‣ Onerous change management process ‣ Multi-tier operations team ‣ Poor availability and performance ‣ Minimal monitoring
11. Q4 2010 - status quo
12. Context ‣ Web deliverables tied to TV broadcast dates ‣ Broadcast dates are immutable ‣ Last-minute changes ‣ Curveball projects from other departments ‣ Recovering from lay-offs ‣ Third parties responsible for service delivery
13. Q4 2010 - in-flight projects ‣ Multi-million GBP data centre refresh ‣ Business transformation program ‣ New online video player project ‣ New CMS project
14. New Linux Platform ‣ On the new data centre platform (VMWare / RedHat) ‣ Automated using Puppet ‣ Development environments with Vagrant ‣ Continuous Deployment pipeline with Jenkins ‣ New team members hired
15. Puppet ‣ Configuration Management tool ‣ “Infrastructure as Code” ‣ Convergent ‣ Runs client/server or masterless
16. Vagrant ‣ Tool for managing dev environments ‣ Supports Linux, Mac, Windows ‣ Integrates with CM tools (eg. puppet) ‣ Folder sharing ‣ As portable as your laptop
17. Jenkins ‣ Orchestration of build/test/deploy pipeline ‣ Dashboard for build status ‣ Role based access control ‣ Huge plugin ecosystem
18. Zabbix ‣ Distributed monitoring platform ‣ Fetches time series data ‣ Can alert on anomaly ‣ Multiple data sources ‣ Zabbix agent/sender ‣ SNMP poll/traps ‣ JMX
19. Scope ‣ Initially two Drupal sites ‣ Intended to be a generic CMS ‣ Grew to include other Linux properties ‣ Grew further to include the new Player platform ‣ Several teams: on-site, and off-site ‣ ~50 developers ‣ Teams owned by different business units
20. Wins ‣ Automated deployment from day one ‣ New “automate everything” mindset ‣ New team members bring outside experience ‣ Removed complexity from change management ‣ Increased operational visibility
21. Challenges ‣ Environment and project built simultaneously ‣ Tight timescales - immutable ‣ All developers on contract ‣ Not all developers are full-stack competent ‣ Rapid team expansion
22. Consequences ‣ Automated testing largely ignored ‣ Low quality deliverables ‣ Blamestorming ‣ High staff turnover ‣ More experts brought in ‣ Fragile platform goes live ‣ Reduced confidence in Drupal
23. The Four Pillars of DevOps Culture Automation Monitoring Sharing
24. Learning Success takes more than just good technology Cultural factors will make or break a project Cultural change is hard Culture comes from the top Cultural change under pressure is nearly impossible All change takes time
25. Learning
26. http://www.scalefactory.com/ jon@scalefactory.com @jtopper scalefactory Jon Topper
27. THANK YOU! WHAT DID YOU THINK? Locate this session at the DrupalCon Prague website: http://prague2013.drupal.org/schedule Click the “Take the survey” link