$30 off During Our Annual Pro Sale. View Details »

Auditing Your Automation's Access... Using More Automation

Rosemary Wang
October 19, 2022

Auditing Your Automation's Access... Using More Automation

Originally presented at Datadog DASH 2022.

From CI/DI pipelines to container orchestrators to developer debugging tools, you spend years building the automation needed to scale your systems. As we grow our automation footprint over time, we grant more and more access to new systems. But how do you know if that automation is accessing the right systems at the right time? And more recently, we’ve encountered new incidents in which an unauthorized user can exploit automation, compromise credentials, and access systems with us completely unaware.

This talk will discuss ways to build observability into your automation in order to audit its access across different parts of the software development lifecycle. Whether clicking on various things in the console or uploading .tar.gz files, too much human access to the phases of the SDLC can be risky. We will look at automation within your CI/CD pipelines and services, and talk through use cases where the replacement of human access might benefit from automation.

We will also outline some techniques to maintain least-privilege access, mitigate blast radius of compromised credentials, and better observe interactions between services so you quickly identify unauthorized access.

You will learn about managing, aggregating, and configuring audit logging and metrics using:
- Service mesh for authorization
- Secrets management for credentials
- Secure access management for manual access to systems

By the end of this talk, you will be able to identify ways to better secure your automation and observe its access across a system.

Rosemary Wang

October 19, 2022
Tweet

More Decks by Rosemary Wang

Other Decks in Technology

Transcript

  1. …with More Automation


    DASH | October 2022
    Auditing Access
    in Automation

    View Slide

  2. Automation


    Cycle
    Recognize and
    remediate.
    Implement
    automation.
    Identify
    functional issue.

    View Slide

  3. We deploy and
    operate quickly,
    securely, and
    reliably…

    View Slide

  4. until we find some
    anomalous activity.

    View Slide

  5. ⚠ What?


    ⚠ Who?


    ⚠ Where?

    View Slide

  6. Observability of Automation


    datadoghq.com/case-studies/hashicorp/

    View Slide

  7. Developer Advocate


    HashiCorp

    she/her


    @joatmon08 | joatmon08.github.io
    Rosemary Wang

    View Slide

  8. Automation


    Cycle (with
    Security)
    Recognize and
    remediate.
    Fix automation.
    Identify security
    issue.

    View Slide

  9. How do you


    speed up the
    automation cycle
    with security?

    View Slide

  10. Time to Resolution
    Reduce time for these stages.
    Fix automation. Identify security issue. Recognize and remediate.

    View Slide

  11. What should you
    observe in
    automation for
    security?

    View Slide

  12. Automation requires
    access.

    View Slide

  13. Automation need lots of access.
    Example - Deployment Pipelines
    •Check out code


    •Build infrastructure


    •Deploy application


    •Test application


    •Check code quality
    Need access to…
    •Version control


    •Infrastructure provider


    •Platform/release repository


    •Data/other services


    •Quality assurance tool

    View Slide

  14. Why audit access?
    Observe service interactions
    Maintain least privilege
    Identify and mitigate blast radius
    Standardize event information

    View Slide

  15. Application Deployment
    Local Testing Integration Testing Production

    View Slide

  16. Local Testing


    Auditing Engineering Access

    View Slide

  17. Engineers need access to build.
    Engineers…
    •Test


    •Debug


    •Deploy
    Needs access to…
    •Production Applications


    •Databases


    •Managed Services


    •Infrastructure APIs


    •Platform APIs

    View Slide

  18. Track human interactions
    Audit logs from cloud providers
    Audit logs from managed
    services
    Audit logs from secure access
    management tools

    View Slide

  19. Database Automation.


    Audit local connection to database.

    View Slide

  20. View Slide

  21. Is this fine?


    Identify unauthorized access to target.

    View Slide

  22. View Slide

  23. Integration Testing


    Auditing Service Access

    View Slide

  24. Services need access to other services.
    Services…
    •Test


    •Debug


    •Deploy
    Need access to…
    •Production Applications


    •Databases


    •Managed Services


    •Infrastructure APIs


    •Platform APIs

    View Slide

  25. Track service interactions
    Access logs
    Traces
    Authentication requests
    Network flow logs

    View Slide

  26. Service Automation.


    Tracing requests between services using APM.

    View Slide

  27. View Slide

  28. Is this fine?


    Anomalous access between services.

    View Slide

  29. View Slide

  30. Secrets provide
    access.

    View Slide

  31. Database Access.


    Tracking authentication requests.

    View Slide

  32. View Slide

  33. Production


    Auditing Even More Automation

    View Slide

  34. Production access involves…
    Local
    Temporary
    operational
    user access
    Integration
    Dynamic
    service access

    View Slide

  35. Track production interactions
    Audit logs from secrets manager
    Audit logs from providers
    Audit logs from secure access
    management tools

    View Slide

  36. Event-Driven Access Control
    WEBHOOK
    DATADOG
    Declare
    incident.
    status: active
    BOUNDARY
    Authorizes
    engineers to
    open sessions.
    Create
    temporary role
    with access to
    production.

    View Slide

  37. View Slide

  38. View Slide

  39. Event-Driven Access Control
    WEBHOOK
    DATADOG
    Resolve
    incident.
    status: resolved
    BOUNDARY
    Revokes access
    and terminates
    sessions.
    Delete
    temporary role
    with access to
    production.

    View Slide

  40. Secrets provide
    access.

    View Slide

  41. Audit usage of
    secrets.

    View Slide

  42. Infrastructure
    Automation.


    Audit static secret use for automation.

    View Slide

  43. View Slide

  44. Service Automation.


    Issuing certificates.

    View Slide

  45. View Slide

  46. View Slide

  47. Is this fine?


    Identify root access to a static secret.

    View Slide

  48. View Slide

  49. Automation requires
    access.

    View Slide

  50. Other Use
    Cases
    Infrastructure as Code Controllers
    GitOps Managed Services

    View Slide

  51. Why audit access?
    Observe service interactions
    Maintain least privilege
    Identify and mitigate blast radius
    Standardize event information

    View Slide

  52. Datadog Cloud SIEM


    Define Custom Rules
    datadoghq.com/blog/hashicorp-vault-security-datadog

    View Slide

  53. Time to Resolution
    Reduce time for these stages.
    Fix automation. Identify security issue. Recognize and remediate.

    View Slide

  54. Learn more Datadog & HashiCorp
    datadoghq.com/blog/tag/hashicorp/
    Learn More
    Example at

    joatmon08/hashicorp-stack-demoapp

    View Slide

  55. Rosemary Wang


    @joatmon08


    joatmon08.github.io
    Thank you!

    View Slide