Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Migrating Edmunds.com to AWS (re:Invent 2013 DMG205)

John Martin
November 14, 2013

Migrating Edmunds.com to AWS (re:Invent 2013 DMG205)

Taking a stack composed of 30 web applications and their service dependencies to the cloud is no easy feat. Do you take the entirety of the stack or go the hybrid path? How transparent should the end result be to your technology teams? Does it look exactly the same in the cloud as it does in your data center? These are not rhetorical questions; they were very real for those tasked with the challenge of taking Edmunds.com to the AWS Cloud. This talk addresses these questions and many more, examining the challenges, successes, and lessons learned as the team took their first steps out of their own data centers.

Accompanying video is available on YouTube: http://youtu.be/itbNET2dc3c

John Martin

November 14, 2013
Tweet

Other Decks in Technology

Transcript

  1. © 2013 Amazon.com, Inc. and its affiliates. All rights reserved.

    May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. John Martin November 14, 2013 Edmunds.com on AWS
  2. • A move isn’t easy • Taking something your familiar

    with elsewhere • How does that work again? Moving Isn’t Easy
  3. Today’s Agenda • Technology Overview • The Business Case •

    The Approach • Challenges • What’s Next?
  4. John Martin @tekbuddha WHO AM I? • 15+ years in

    .com • 10+ years of Java • Old School Ops • New School Cultures
  5. • Founded in 1966 • First online in 1994 as

    a gopher • First website in 1996 The Company
  6. • 30+ web applications across 300+ hosts. • Java on

    Redhat Linux • Tomcat, Solr, Coherence, Mongo, ActiveMQ The Environment
  7. • Chef + Cloudstack/UCS • Perforce, Jenkins, Nexus, Selenium, JMeter

    • AppDynamics, Splunk, RTview, Zenoss The Environment
  8. • OSS + Homegrown Tooling • All artifacts flow through

    the pipeline • Release Cycles: 1 Month > 3 Weeks > 1 Week The Deployment Pipeline
  9. • How to move 30+ apps and 300+ servers? •

    Make it run like it already does • Avoid biting off too much at once The Approach
  10. Objectives • Minimize change / leverage existing toolchain • Manage

    cost • Provide initial design patterns for future builds
  11. Source: http://is.gd/YmewdR • A move isn’t easy. • Making the

    things we were familiar with work elsewhere wasn’t easy Challenges
  12. • Different structure, similar logic • Defining single hosts, not

    groups of services • No definitions of network resources CloudStack JSON
  13. • Define network and services, not hosts • Live by

    cf-validate • Automate creation / avoid manual editing CloudFormation JSON
  14. Source: http://is.gd/rayzL1 • Still a core dependency • No great

    options in EC2 without refactor • Move static content to S3 NFS
  15. Source: http://is.gd/rayzL1 • No physical load balancers • ELBs, HAproxy,

    and Chef were the key • Learn how Public/Private ELBs work with VPC Load Balancing
  16. Source: http://is.gd/rayzL1 • Worked but can leave behind a mess

    • Tooling must be prepared for ephermal nodes • Helped survive outages in US-EAST [auto-]scaling
  17. Source: http://is.gd/xKdI6E • Additional live-traffic tests • Deployment of internal

    services • Prepare for full move out of data center What Next?
  18. Source: http://is.gd/D8bVaC • Greater adoption of SOA principles • Full

    refactor of data pipelines (in and out) • Refactor for fragility of cloud environments Refactor
  19. • Work thus far completed by a small team •

    The Big Move will be all hands on deck • A two year goal to be fully cloud-based The Big Move
  20. Please give us your feedback on this presentation As a

    thank you, we will select prize winners daily for completed surveys! DMG205 Thank You