Preventing Patching Problems (3P) with AWS Systems Manager

Not everything is a serverless application! A lot of us run EC2 instances on AWS, and one of the things those instances need from time to time is Patching! How can we make patching easier? Can we automate patching of both Windows and Linux instances on AWS? Well of course we can! In this session we look into preventing patching problems with AWS Systems Manager, Patch Management!

Transcript

1. Prevent Patching Problems
   (3P) with AWS SSM
   Darko Meszaros
   Developer Advocate - AWS
   @darkosubotica
   ln/darko-mesaros
   twitch.tv/ruptwelve
   youtu.be/ruptwelve
   

   View Slide

2. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Agenda
   • Patch Management
   • AWS Systems Manager
   • Manage patching with AWS Systems Manager
   • Demo
   

   View Slide

3. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Patch Management
   

   View Slide

4. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Not everything is a serverless workload
   

   View Slide

5. © 2020, Amazon Web Services, Inc. or its Affiliates.
   What is Patching? And why should we care?
   

   View Slide

6. © 2020, Amazon Web Services, Inc. or its Affiliates.
   

   View Slide

7. © 2020, Amazon Web Services, Inc. or its Affiliates.
   

   View Slide

8. © 2020, Amazon Web Services, Inc. or its Affiliates.
   What is Patch management?
   

   View Slide

9. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Why is Patch Management important?
   

   View Slide

10. © 2020, Amazon Web Services, Inc. or its Affiliates.
    

    View Slide

11. © 2020, Amazon Web Services, Inc. or its Affiliates.
    $(whoami) Darko Mesaroš / Darko Meszaros /
    Дарко Месарош
    @darkosubotica
    ln/darko-mesaros
    twitch.tv/ruptwelve
    youtu.be/ruptwelve
    

    View Slide

12. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Jerry
    

    View Slide

13. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Web Application
    BI Tooling
    Super Secure FTP Servers
    patch
    patch
    patch
    Jerry
    

    View Slide

14. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Web Application
    BI Tooling
    Super Secure FTP Servers
    Jerry
    

    View Slide

15. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Web Application
    BI Tooling
    Super Secure FTP Servers
    scale
    out
    scale
    out
    Jerry
    
    

    View Slide

16. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Systems Manager
    

    View Slide

17. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Management & Governance
    Optimize
    Analyze and reduce cost, improve
    efficiency and security posture
    Act
    Take operational
    action on resources
    Audit
    Audit resource configurations,
    user access, and policy enforcement
    Monitor
    Monitor resources
    and applications
    

    View Slide

18. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Management & Governance
    Monitor resources and applications
    Optimize to reduce cost and improve security posture
    Manage resources and take operational action
    Audit user activity and resource configurations
    Amazon CloudWatch
    AWS Trusted Advisor
    AWS Cost and Usage Report
    AWS Cost Explorer
    AWS Systems Manager
    AWS CloudTrail
    AWS Config
    

    View Slide

19. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Any environment
    Operate any
    AWS or external
    resource centrally
    Open
    Agent is
    open-sourced
    on GitHub
    Multi-platform
    Windows and
    Linux support
    Automated
    Multi-account,
    multi-Region
    automation
    AWS Systems Manager
    Centrally manage cloud resources at any scale
    

    View Slide

20. © 2020, Amazon Web Services, Inc. or its Affiliates.
    How it works
    AWS Systems Manager
    Systems Manager helps you
    safely manage and operate
    your resources at scale
    Group resources
    Create groups of
    resources across
    different AWS services,
    such as applications or
    different layers of an
    application stack
    Visualize data
    View aggregated
    operational data by
    resource group
    Take Action
    Respond to insights and
    automate operational
    actions across resource
    groups
    

    View Slide

21. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Patch Manager
    

    View Slide

22. © 2020, Amazon Web Services, Inc. or its Affiliates.
    So, what is this Patch Manager?
    

    View Slide

23. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Jerry
    

    View Slide

24. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Web Application
    BI Tooling
    Super Secure FTP Servers
    Jerry
    Write patch
    baseline
    patch
    

    View Slide

25. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Web Application
    BI Tooling
    Super Secure FTP Servers
    Jerry
    patch
    

    View Slide

26. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Select and deploy operating system and software
    patches automatically across large groups of Amazon
    EC2 or on-premises instances.
    • Automate patching
    • Use patch baselines to set rules for auto approval
    • Create exceptions to approve or reject patches
    • Schedule maintenance windows
    • Scan for compliance
    Patch Manager
    

    View Slide

27. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Patch Baselines
    • Patch your fleet to different levels
    • Set those exceptions
    • Use a custom Patch source (for Linux)
    

    View Slide

28. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Applying Patches
    Linux
    • Systems Manager evaluates
    patch baseline rules and every
    approved and denied patch on
    each managed instance;
    repository locally configured
    • OS and application patch repo
    are the same
    Windows
    • Systems Manager evaluates patch
    baseline rules and the list of
    approved and rejected patches
    directory in the service. Single
    repository for all patches
    • Supports Microsoft applications,
    such as Microsoft Word 2011 and
    Microsoft Exchange 2016
    

    View Slide

29. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Patch Groups
    • Patch groups can help you avoid deploying patches to the wrong set of
    instances (e.g. Dev, Test, Prod)
    • Install patches individually or to a fleet of EC2 Instances
    • Patch Group is associated per Patch Baseline
    • Tag Based
    

    View Slide

30. © 2020, Amazon Web Services, Inc. or its Affiliates.
    How does all that work now?
    

    View Slide

31. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS-RunPatchBaseLine (Window and Linux)
    • Document that enables you to control patch approvals using patch baselines.
    • Has a Scan Mode!
    • This is what installs patches
    • Windows – Uses PowerShell
    • Linux – Uses Python
    • Runs in a Maintenance Window
    

    View Slide

32. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Maintenance Window
    • Define one or more recurring windows of time during which it is acceptable for
    disruptive actions to occur
    • Built-in integration with Run Command and Patch Manager
    • Helps improve availability and reliability of your workloads by automatically
    performing tasks in a well-defined window of time
    • Targets specified by Patch Group and any other EC2 tag
    • Scan and Install patches
    

    View Slide

33. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Errors with Patching
    • Concurrency – number or percentage to run command at the same time
    • Error Threshold – specify when to stop running command on instances after
    fails, either number or percentages
    

    View Slide

34. © 2020, Amazon Web Services, Inc. or its Affiliates.
    All this to make sure you are compliant…
    Compliant?!
    

    View Slide

35. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Jerry
    

    View Slide

36. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Web Application
    BI Tooling
    Super Secure FTP Servers
    Jerry
    patch
    

    View Slide

37. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Web Application
    BI Tooling
    Super Secure FTP Servers
    Jerry
    patch
    Mike
    

    View Slide

38. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Compliance
    • Scan your fleet of managed instances for patch compliance and configuration
    inconsistencies (e.g. Snapshot ID)
    • Supports aggregating data from multiple AWS accounts and Regions
    • By default, displays Systems Manager Patch Manager patching and Systems
    Manager State Manager associations
    • Can customize the service and create your own compliance types based on
    your IT or business requirements
    • You can also port data to Amazon Athena and Amazon QuickSight to generate
    fleet-wide reports
    

    View Slide

39. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Compliance with Patch Manager
    Corp Data Center
    Individual instances
    not grouped
    Patch Group=WebServers
    Patch Group=SQLCluster
    Default Patch Baseline
    for the OS
    Web Server
    Patch Baseline
    Patch Manager
    Maintenance
    Window
    Compliance Notifications!
    

    View Slide

40. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Web Application
    BI Tooling
    Super Secure FTP Servers
    Jerry
    patch
    Mike
    
    

    View Slide

41. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Hey! I wanna patch my on-prem servers too!
    

    View Slide

42. © 2020, Amazon Web Services, Inc. or its Affiliates.
    IAM Service Role
    • Servers and virtual machines in hybrid require an IAM role to communicate
    with Systems Manager
    • Role grants AssumeRole trust to the Systems Manager Service
    • One IAM service role required per account
    • Required Trust Policy
    

    View Slide

43. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Managed-Instance Activation
    • Servers and virtual machines require a managed-instance activation
    • Consists of Activation Code and Activation ID
    • Similar to Access Key
    • Code/ID combination used in SSM Agent installation
    • Grants managed-instances secure access to Systems Manager
    • Activations expire – no impact to existing managed-instances
    • Create new activation when it expires
    

    View Slide

44. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Patch Manager Supported Operating Systems
    Linux
    • Amazon Linux 2012.03 - 2018.03
    • Amazon Linux 2 2 - 2.0
    • CentOS 6.5 - 7.8, 8.0-8.1
    • Red Hat Enterprise Linux (RHEL) 6.5 - 8.2
    • Debian 8.x and 9.x
    • Oracle Linux 7.5-7.8
    • SUSE Linux Enterprise Server (SLES) 12.0 and later 12.x versions, 15.0 and 15.1
    • Ubuntu Server 14.04 LTS, 16.04 LTS, and 18.04 LTS
    Windows
    • Windows Server 2008 through Windows Server 2019, including R2 versions
    

    View Slide

45. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Let’s see that in action …
    

    View Slide

46. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Getting started
    

    View Slide

47. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Getting Started
    1. Create a Patch
    Baseline to define
    approved patches (Add
    Patch Group)
    3. Maintenance
    Window executes
    patching
    4. Audit results
    with Patch
    Compliance
    2. Create a Maintenance
    Window to schedule
    patching for a set of
    instances
    

    View Slide

48. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Resources
    Patch Manager Documentation: https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-patch-working.html
    Systems Manager Prerequisites: https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-prereqs.html
    Supported Operating Systems: https://docs.aws.amazon.com/systems-manager/latest/userguide/prereqs-operating-systems.html
    Installing and Configuring SSM Agent on Windows Instance: https://docs.aws.amazon.com/systems-
    manager/latest/userguide/sysman-install-ssm-win.html
    Installing and Configuring SSM Agent on EC2 Linux Instances: https://docs.aws.amazon.com/systems-
    manager/latest/userguide/sysman-install-ssm-agent.html
    Multi-Account Automation: https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-automation-
    multiple-accounts-and-regions.html
    Multi-Account Patching: https://aws.amazon.com/blogs/mt/centralized-multi-account-and-multi-region-patching-with-aws-
    systems-manager-automation/
    Regional Endpoints: https://docs.aws.amazon.com/general/latest/gr/rande.html#ssm_region
    Manually Install SSM Agent on EC2: https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-manual-agent-
    install.html
    

    View Slide

49. Thank you!
    © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Darko Meszaros
    Developer Advocate - AWS
    @darkosubotica
    ln/darko-mesaros
    twitch.tv/ruptwelve
    youtu.be/ruptwelve
    

    View Slide