Automating AWS

Transcript

1. Automating AWS

2. Part 1 Automating Infrastructure

3. Page Birds-eye view / Copyright ©2014 by Readify Pty Ltd

   3 Step 1: Deploy infrastructure Step 2: Bootstrap instances Step 3: Deploy software

4. Page The infrastructure / Copyright ©2014 by Readify Pty Ltd

   4

5. Page / Copyright ©2014 by Readify Pty Ltd 5 Deploy

   Infrastructure Bootstrap Instances Register Tentacle Deploy Software Fire Project Trigger Nothing up my sleeve…

6. Page / Copyright ©2014 by Readify Pty Ltd 6 Deploy

   Infrastructure Bootstrap Instances Register Tentacle Deploy Software Fire Project Trigger Nothing up my sleeve…

7. Page Step 1: Deploy infrastructure › Deploy.ps1 › Package and

   upload bootstrapping resources to S3 › Package and upload configuration variables to S3 › Do we need to deploy a new stack? › Yes - New-CFNStack › No - New-CFNChangeSet › CloudFormation resource creation based on template / Copyright ©2014 by Readify Pty Ltd 7

8. Page / Copyright ©2014 by Readify Pty Ltd 8 Deploy

   Infrastructure Bootstrap Instances Register Tentacle Deploy Software Fire Project Trigger Nothing up my sleeve…

9. Page Step 2: Bootstrap instances › EC2 userdata › cfn-init.exe

   › AWS::CloudFormation::Init › Powershell DSC › Octopus tentacle › tentacle.exe --register-with / Copyright ©2014 by Readify Pty Ltd 9

10. Page / Copyright ©2014 by Readify Pty Ltd 10 Deploy

    Infrastructure Bootstrap Instances Register Tentacle Deploy Software Fire Project Trigger Nothing up my sleeve…

11. Page Step 3: Deploy software › Prior to Octopus 3.4

    => DSC › Octopus 3.4 => Project Triggers › Software is deployed by Octopus via the Project Trigger › DSC polls a local health check endpoint › ASG waits to be signalled before completion › DSC => cfn-signal.exe › fin / Copyright ©2014 by Readify Pty Ltd 11

12. Page / Copyright ©2014 by Readify Pty Ltd 12 Deploy

    Infrastructure Bootstrap Instances Register Tentacle Deploy Software Fire Project Trigger Nothing up my sleeve…

13. Part 2 Immutable infrastructure

14. Page Updating CF stacks What updating stacks used to be

    like / Copyright ©2014 by Readify Pty Ltd 14 Are you feeling lucky?

15. Page / Copyright ©2014 by Readify Pty Ltd 15

16. Page CloudFormation Change Sets › Supply your updated template to

    a Change Set via New-CFNChangeSet › AWS diffs the updated template against the current stack › A detailed list of what changes and what doesn’t is provided › Details the dependency chain and how it is effected / Copyright ©2014 by Readify Pty Ltd 16

17. Page Immutability › Rev infrastructure package => refresh instances ›

    CI + CloudFormation can do this for us › Inject version into Launch configuration userdata › Triggers a replacement of the launch configuration › How do we stay HA while our infrastructure refreshes? / Copyright ©2014 by Readify Pty Ltd 17

18. Page AutoScaleGroup Update Policies › Allows an ASG to stay

    HA while undergoing a refresh › Keeps a number of instances in service while terminating stale and creating fresh instances › Works with cfn-signal.exe to ensure updates are successful › Rolls back on failure / Copyright ©2014 by Readify Pty Ltd 18

19. Part 3 Phoenix environments The really really cool part!

20. Page Deploying feature branches › The Dream: › Developer checks

    in code on a feature branch › The feature branch is built by our CI service of choice › Once complete, our CI service then invokes Octopus and triggers an isolated stack of hardware to be deployed for it › Once the hardware is ready, the feature branch’s payload is automatically deployed to it by Octopus › From check-in to our feature running on new, isolated compute. Ready to test and destroy when we are done / Copyright ©2014 by Readify Pty Ltd 20

21. Page Octopus 3.4 › Introduces new features that allow us

    to work with a tonne more flexibility in elastic environments › Multi-tenancy › Project Triggers / Auto Deploy Overrides › Machine Policies / Copyright ©2014 by Readify Pty Ltd 21

22. Page Setup › We have built a new feature ryan

    on an isolated feature branch! We push the code up to our repository › Our CI service notices this and creates a release for our feature branch package, with a semver version number 1.0.0-ryan0001 / Copyright ©2014 by Readify Pty Ltd 22

23. Page Octo 3.4 – Multi-tenancy › We created a tenant

    ryan0001for our feature branch –we are using the semver from the built package as a naming convention › Allows us to isolate and group sets of machines within an environment › Allows us to configure projects with tenant-specific variables › Allows us to control what software gets deployed to what sets of machines via triggers within an environment / Copyright ©2014 by Readify Pty Ltd 23

24. Page Octo 3.4 – Auto Deploy Overrides › We created

    an auto deploy override for our feature branch tenant ryan0001 for the feature branch package version 1.0.0-ryan0001 › Part of Project Triggers › Auto Deploy Overrides allow us to say ‘when a project trigger fires, and the machine that triggered it is for a given tenant, deploy an explicit version of the project to the machine, not the default’ / Copyright ©2014 by Readify Pty Ltd 24

25. Page Octo 3.4 – Machine Policies › When our instances

    register with Octopus via tentacle.exe, they indicate a machine policy › When we clean up our CloudFormation, how do we ensure Octopus is cleaned up as well? › Old way: AWS Lambda › New way: Octopus Machine Policies › Healthchecks and unavailable machines › Cleanup unavailable machines / Copyright ©2014 by Readify Pty Ltd 25

26. Page Living The Dream 26

27. Thank you Andrew Best | @_andrewb | andrew.best@readify.net