Event-driven Access Controls

In this presentation, I look at how to use HashiCorp Boundary and PagerDuty Incident Response tooling to provide privileged access to on-call engineers for the duration of an incident.

This version of the talk was given at a HashiConf Europe in June 2022.

---

Companion Recording: youtu.be/GtrEh6iQVmEM

Companion Code: github.com/hashicorp-dev-advocates

Transcript

1. View Slide

2. brought a laptop with SSH keys
   allowlisted hotel IP range
   disabled VPN !
   Security absolutely hated it.
   

   View Slide

3. Get access to a service,
   but only when needed
   and only as long as needed.
   

   View Slide

4. to a production database,
   what:
   where:
   Get TCP access
   when: from 13:04 until 13:15,
   why: to resolve incident #34.
   

   View Slide

5. What you are about to see
   is not a product announcement.
   

   View Slide

6. Gate
   Privileged Access Controller
   Transit Secrets Engine
   Response Automation
   

   View Slide

7. View Slide

8. View Slide

9. View Slide

10. ~ >_ boundary targets list -recursive -filter '"/item/name"
    matches "postgres"'
    Target information:
    ID: ttcp_YMNb5RXsWQ
    Scope ID: p_yXNBa22BOY
    Version: 2
    Type: tcp
    Name: postgres
    Description: Postgres for var.project_identifier
    Authorized Actions:
    no-op
    read
    update
    delete
    authorize-session
    

    View Slide

11. ~ >_ boundary connect postgres -target-id ttcp_YMNb5RXsWQ
    

    View Slide

12. :activity -- Server activity
    :locks -- Lock info
    :waits -- Waiting queires
    :dbsize -- Database Size
    :tablesize -- Tables Size
    :uptime -- Server uptime
    Development queries:
    :sp -- Current Search Path
    :clear -- Clear screen
    :ll -- List
    psql (14.3)
    Type "help" for help.
    (ksatirli@[postgres]:5432) [postgres] >
    

    View Slide

13. :activity -- Server activity
    :locks -- Lock info
    :waits -- Waiting queires
    :dbsize -- Database Size
    :tablesize -- Tables Size
    :uptime -- Server uptime
    Development queries:
    :sp -- Current Search Path
    :clear -- Clear screen
    :ll -- List
    psql (14.3)
    Type "help" for help.
    (ksatirli@[postgres]:5432) [postgres] >
    

    View Slide

14. \l "hashicraft"
    (ksatirli@[postgres]:5432) [postgres] >
    List of databases
    +------------+----------+----------+-------------------+
    | Name | Owner | Encoding | Access privileges |
    +------------+----------+----------+-------------------+
    | hashicraft | postgres | UTF8 | |
    +------------+----------+----------+-------------------+
    (1 row)
    (ksatirli@[postgres]:5432) [postgres] >
    

    View Slide

15. (ksatirli@[postgres]:5432) [postgres] > ALTER DATABASE
    hashicraft OWNER TO hashicraft;
    ALTER DATABASE
    Time: 0.238 ms
    (ksatirli@[postgres]:5432) [postgres] > GRANT ALL PRIVILEGES ON
    DATABASE hashicraft TO hashicraft;
    GRANT
    Time: 0.688 ms
    (ksatirli@[postgres]:5432) [postgres] >
    

    View Slide

16. \l "hashicraft"
    (ksatirli@[postgres]:5432) [postgres] >
    List of databases
    +------------+------------+----------+---------------------------+
    | Name | Owner | Encoding | Access privileges |
    +------------+------------+----------+---------------------------+
    | hashicraft | hashicraft | UTF8 | =Tc/hashicraft +|
    | | | | hashicraft=CTc/hashicraft |
    +------------+------------+----------+---------------------------+
    (1 row)
    (ksatirli@[postgres]:5432) [postgres] >
    

    View Slide

17. TERMINAL
    \l "hashicraft"
    (ksatirli@[postgres]:5432) [postgres] >
    List of databases
    +------------+------------+----------+---------------------------+
    | Name | Owner | Encoding | Access privileges |
    +------------+------------+----------+---------------------------+
    | hashicraft | hashicraft | UTF8 | =Tc/hashicraft +|
    | | | | hashicraft=CTc/hashicraft |
    +------------+------------+----------+---------------------------+
    (1 row)
    (ksatirli@[postgres]:5432) [postgres] >
    

    View Slide

18. TERMINAL
    \l "hashicraft"
    (ksatirli@[postgres]:5432) [postgres] >
    List of databases
    +------------+------------+----------+---------------------------+
    | Name | Owner | Encoding | Access privileges |
    +------------+------------+----------+---------------------------+
    | hashicraft | hashicraft | UTF8 | =Tc/hashicraft +|
    | | | | hashicraft=CTc/hashicraft |
    +------------+------------+----------+---------------------------+
    (1 row)
    (ksatirli@[postgres]:5432) [postgres] >
    

    View Slide

19. \l "hashicraft"
    (ksatirli@[postgres]:5432) [postgres] >
    List of databases
    +------------+------------+----------+---------------------------+
    | Name | Owner | Encoding | Access privileges |
    +------------+------------+----------+---------------------------+
    | hashicraft | hashicraft | UTF8 | =Tc/hashicraft +|
    | | | | hashicraft=CTc/hashicraft |
    +------------+------------+----------+---------------------------+
    (1 row)
    (ksatirli@[postgres]:5432) [postgres] >
    Connection closed by foreign host.
    

    View Slide

20. Gate
    Privileged Access Controller
    Transit Secrets Engine
    Codified Configuration
    App Deployment
    Response Automation
    

    View Slide

21. Run Observe
    Build
    Release
    Delivery
    

    View Slide

22. Run Observe Response
    Automation
    Incident
    Response
    Team
    Incident
    Triggered
    !
    ALERT
    

    View Slide

23. Run Observe Response
    Automation
    Incident
    Response
    Team
    !?
    Incident
    Acknowledged
    ACKNOWLEDGE
    

    View Slide

24. Run Observe Response
    Automation
    Response
    Team
    {}
    Process
    Webhook
    !?
    Incident
    Acknowledged
    Incident
    

    View Slide

25. Boundary
    Desktop
    Postgres
    Database
    Grant
    Access
    Boundary
    CLI
    > _ ___ _
    _____ ___
    _ ___ __ _
    ___ __ __
    Incident
    Resolved
    {}
    Process
    Webhook
    Incident
    

    View Slide

26. Response
    Automation
    {}
    Process
    Webhook
    Revoke
    Access
    Observe
    Incident
    Resolved
    Resolution
    Relax
    

    View Slide

27. resource "boundary_host_catalog" "main" {
    scope_id = var.scope_id
    type = "static"
    name = var.host_name
    }
    resource "boundary_host" "main" {
    name = var.host_name
    address = var.host_address
    host_catalog_id = boundary_host_catalog.main.id
    type = "static"
    }
    resource "boundary_host_set" "main" {
    host_catalog_id = boundary_host_catalog.main.id
    type = "static"
    Boundary
    Create Resources for
    Host, Catalog, and
    Targets per Service.
    

    View Slide

28. host_ids = [
    boundary_host.main.id
    ]
    }
    resource "boundary_target" "main" {
    name = var.host_name
    scope_id = var.scope_id
    type = "tcp"
    host_source_ids = [
    boundary_host_set.main.id
    ]
    Boundary
    Create Resources for
    Host, Catalog, and
    Targets per Service.
    

    View Slide

29. resource "pagerduty_webhook_subscription" "gate" {
    type = "webhook_subscription"
    active = true
    delivery_method {
    type = "http_delivery_method"
    url = true
    custom_headers = [{
    name = "X-Boundary-Project",
    value = boundary_project.main.id
    }, {
    name = "X-Boundary-Targets",
    value = boundary_target.main.id
    }]
    }
    ...
    Webhook
    Custom Headers
    provide data to map
    affected service to
    Boundary Scopes.
    

    View Slide

30. resource "pagerduty_webhook_subscription" "gate" {
    ...
    events = [
    "incident.acknowledged",
    "incident.resolved",
    ]
    filter {
    id = pagerduty_service.gate.id
    type = "service_reference"
    }
    }
    Webhook
    Custom Headers
    provide data to map
    affected service to
    Boundary Scopes.
    

    View Slide

31. to a production database,
    what:
    where:
    Get TCP access
    when: from 13:04 until 13:15.
    ,
    why: to resolve incident #34.
    

    View Slide

32. See this workflow in action in the
    Dev Lounge during HashiConf.
    Code will be progressively released via
    github.com/hashicorp-dev-advocates
    Resources
    

    View Slide

33. Sr. Developer Advocate at HashiCorp
    he / him
    @ksatirli
    Kerim
    Satirli
    

    View Slide

34. @ksatirli
    kerim@hashicorp.com
    Thank you
    

    View Slide