Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MMT Edge

MMT Edge

Deployments Redesigned!

655c302c8f4b144ceb12d1ebb4be0aa4?s=128

Paritosh

April 10, 2019
Tweet

Transcript

  1. MMT Edge Deployments Redesigned!

  2. About Me Paritosh Anand – Lead DevOps @ MMT •

    Python Programmer • Docker Propellent • Inclined towards – CI / CD, Monitoring, AWS
  3. Agenda 1. MakeMyTrip’s journey towards achieving 100% deployment automation. 2.

    Platform agnostic approach for Deployments. 3. Blue Green deployments. 4. Importance of CANARY & Staggered deployments 5. Tracking deployments & ensuring compliance. 6. Empowering DEV teams.
  4. Inspirations

  5. Features 1. Multi Eco-system support & pluggable. 2. Agentless (Python’s

    Fabric) 3. JIRA & CANARY Integrated 4. Horizontally Scalable 5. DC Redundant 6. Dockerized 7. Logs Storage @Elasticsearch 8. Fast & Graphical Reports
  6. Cloud-Based Data Center Containers Platform & Env. Agnostic

  7. Why CANARY? ü Safety Net ü Fail Fast ü Faster

    Feedback ü Max. Damage Control ü Secure Deployment Strategy
  8. Blue Green Deployments

  9. Tech Stack & Architecture •Python • Django (Edge-UI) • Fabric

    (Remote Exec) • Celery (Task Exec) • Boto (AWS Integration) •Rabbit MQ (Task Queue) •Elastic Search (Logs/Reports) •MySQL (Config DB) •D3 (Visualization)
  10. Integration approach with Jira • Change control management • Tagging

    deployment tickets with dev tickets • Better compliance management • Scheduled deployments • Auto deployment trigger once change is approved.
  11. Staggered or Zero Downtime Deployments • Approach – Datacenters •

    Graceful traffic drain by taking servers OOS from F5 Load Balancer • Health Check validations before enabling traffic. • Canary metric comparison is based on server level comparison • Batch creation for parallel deployment execution • Approach – AWS Cloud (Blue/Green Deployments) • Ensures graceful termination of instances via ASG • Create replica green ASG from blue ASG • Canary metric comparison is based on deployment version
  12. Fabric – Remote Command Execution • Fabric – designed to

    execute shell commands remotely • Fabric uses SSH protocol • Agentless – No need to run agent service on production nodes • Benefits • Ability to execute commands in parallel i.e. on multiple servers at once • Sanitization of commands before execution • Setup and tracking is easy for SYS Admins • Extending Fabric for property deployments.
  13. Env. Agnostic Projects • Environments • Logical definitions of environments

    like AWS, DC, AWS-ECS, STORM • Health checks attributes • Projects • Canary comparison is on or off • Notifications are on or off • Project + Env level configs • Details of servers • Jira ID for tracking
  14. Action lifecycle | Plans | Env. Agnostic • Implementation approach

    for Actions • Implementation of Actions for different environments are defined exclusively. • Highly customized deployment strategy can be developed for each environment. • Support of Hard Deploy, Rollback and Restarts • Plans • Deploy (Hard Deploy) • Restart (Hard Restart) • Rollback (Hard Rollback)
  15. MakeMyTrip Medium Blogs - https://medium.com/makemytrip-engineering MakeMyTrip ECHO Tech Meetups -

    http://technology.makemytrip.com/echo/2018/ Connect with us