Sydney AWS Meetup - September 2015

Domain's talk at Sydney AWS Meetup - September 2015

Transcript

1. ’s Migrating from Datacentre to Public Cloud
2. Overview •Getting Started with AWS •The Big Migration of 2015 •Current State & Learnings
3. Introduction ➤ Getting Started with AWS The Big Migration of 2015 Current State & Learnings
5. Original State •~100 apps/systems to migrate •18 web servers (Web + Mobile API) •5 “offline processing” servers •2 huge DB servers •A handful of ElasticSearch servers already in AWS •A lot of dependencies on Fairfax shared services and other business units
6. New Goals •New micro-services architecture – API-first •Cloud-first •Continuous Delivery •Generic hosting stack
7. Cultural change •Buy-in from the business •Buy-in from developers •Buy-in from ops •Trust the developers •Move fast, roll forward
9. How we do it
10. Introduction Getting Started with AWS ➤ The Big Migration of 2015 Current State & Learnings
11. Ramping it up •Dedicated team of Devs + Ops + DBA •4 months timeline •Lift-&-shift BUT: • Automate ALL deployments • Auto-scale as much as possible • No sticky session • Shared session state with Redis in Elasticache • Refactor “anti-cloud” applications as much as possible • Consolidation • Decommission • Spring cleaning in autumn
12. Going Hybrid •DirectConnect between “on-prem” and AWS •Minor latency impact •Enables seamless migration: • Database can still be shared
13. Weighted DNS 100% 0%
14. Weighted DNS 90% 10%
15. Weighted DNS 75% 25%
16. Weighted DNS 50% 50%
17. Weighted DNS 25% 75%
18. Weighted DNS 0% 100%
19. Communication
20. Instrumentation
21. Monitoring and Alerting
22. Introduction Getting Started with AWS The Big Migration of 2015 ➤ Current State & Learnings
23. Current State •Domain 100% in AWS •~430 EC2 instances (~200 for production) •Vast majority in ASGs and auto-provisioned •Run by a DevOps team of four engineers, one DBA and one manager (and the devs)
24. Current State •World-class CI/CD pipelines •World-first Octopus 3.0 HA cluster (auto-scaled) •Devs can spin up their own micro-service stack within minutes
25. 0 20 40 60 80 100 120 140 160 180 200 Dec 2014 Jan 2014 Feb 2014 Mar 2014 Apr 2014 May 2014 Jun 2014 Jul 2014 Aug 2014 Sep 2014 Oct 2014 Nov 2014 Dec 2014 Jan 2015 Feb 2015 Mar 2015 Apr 2015 May 2015 Jun 2015 Jul 2015 Deployments Per Month
26. Learnings •Bring together good people •Management buy-in •Trust The Developers •Be bold: move fast and roll forward •Spend the time to get the foundation right
27. What we did wrong •Original network stack •t2.medium EC2 instances for production •Overconfidence towards the end
28. Some things we wish had existed •Lambda •Elastic File System •RDS for MS SQL with true HA •Proper managed Active Directory •S3 VPC endpoints
29. Jason Brown Jason.brown@domain.com.au @cloudyopspoet Find Out More Fabien Ruffin fruffin@domain.com.au @fabienruffin http://fabienruffin.com Domain Tech Blog: http://tech.domain.com.au GitHub: http://github.com/DomainGroupOSS Twitter: @InsideDomain