Event-driven Access Controls

In this presentation, I look at how to use HashiCorp Boundary and PagerDuty Incident Response tooling to provide privileged access to on-call engineers for the duration of an incident.

This version of the talk was given at a HashiConf Europe in June 2022.

---

Companion Recording: youtu.be/GtrEh6iQVmEM

Companion Code: github.com/hashicorp-dev-advocates

Transcript

2. brought a laptop with SSH keys allowlisted hotel IP range disabled VPN ! Security absolutely hated it.
3. Get access to a service, but only when needed and only as long as needed.
4. to a production database, what: where: Get TCP access when: from 13:04 until 13:15, why: to resolve incident #34.
5. What you are about to see is not a product announcement.
6. Gate Privileged Access Controller Transit Secrets Engine Response Automation
10. ~ >_ boundary targets list -recursive -filter '"/item/name" matches "postgres"' Target information: ID: ttcp_YMNb5RXsWQ Scope ID: p_yXNBa22BOY Version: 2 Type: tcp Name: postgres Description: Postgres for var.project_identifier Authorized Actions: no-op read update delete authorize-session
11. ~ >_ boundary connect postgres -target-id ttcp_YMNb5RXsWQ
12. :activity -- Server activity :locks -- Lock info :waits -- Waiting queires :dbsize -- Database Size :tablesize -- Tables Size :uptime -- Server uptime Development queries: :sp -- Current Search Path :clear -- Clear screen :ll -- List psql (14.3) Type "help" for help. (ksatirli@[postgres]:5432) [postgres] >
13. :activity -- Server activity :locks -- Lock info :waits -- Waiting queires :dbsize -- Database Size :tablesize -- Tables Size :uptime -- Server uptime Development queries: :sp -- Current Search Path :clear -- Clear screen :ll -- List psql (14.3) Type "help" for help. (ksatirli@[postgres]:5432) [postgres] >
14. \l "hashicraft" (ksatirli@[postgres]:5432) [postgres] > List of databases +------------+----------+----------+-------------------+ | Name | Owner | Encoding | Access privileges | +------------+----------+----------+-------------------+ | hashicraft | postgres | UTF8 | | +------------+----------+----------+-------------------+ (1 row) (ksatirli@[postgres]:5432) [postgres] >
15. (ksatirli@[postgres]:5432) [postgres] > ALTER DATABASE hashicraft OWNER TO hashicraft; ALTER DATABASE Time: 0.238 ms (ksatirli@[postgres]:5432) [postgres] > GRANT ALL PRIVILEGES ON DATABASE hashicraft TO hashicraft; GRANT Time: 0.688 ms (ksatirli@[postgres]:5432) [postgres] >
16. \l "hashicraft" (ksatirli@[postgres]:5432) [postgres] > List of databases +------------+------------+----------+---------------------------+ | Name | Owner | Encoding | Access privileges | +------------+------------+----------+---------------------------+ | hashicraft | hashicraft | UTF8 | =Tc/hashicraft +| | | | | hashicraft=CTc/hashicraft | +------------+------------+----------+---------------------------+ (1 row) (ksatirli@[postgres]:5432) [postgres] >
17. TERMINAL \l "hashicraft" (ksatirli@[postgres]:5432) [postgres] > List of databases +------------+------------+----------+---------------------------+ | Name | Owner | Encoding | Access privileges | +------------+------------+----------+---------------------------+ | hashicraft | hashicraft | UTF8 | =Tc/hashicraft +| | | | | hashicraft=CTc/hashicraft | +------------+------------+----------+---------------------------+ (1 row) (ksatirli@[postgres]:5432) [postgres] >
18. TERMINAL \l "hashicraft" (ksatirli@[postgres]:5432) [postgres] > List of databases +------------+------------+----------+---------------------------+ | Name | Owner | Encoding | Access privileges | +------------+------------+----------+---------------------------+ | hashicraft | hashicraft | UTF8 | =Tc/hashicraft +| | | | | hashicraft=CTc/hashicraft | +------------+------------+----------+---------------------------+ (1 row) (ksatirli@[postgres]:5432) [postgres] >
19. \l "hashicraft" (ksatirli@[postgres]:5432) [postgres] > List of databases +------------+------------+----------+---------------------------+ | Name | Owner | Encoding | Access privileges | +------------+------------+----------+---------------------------+ | hashicraft | hashicraft | UTF8 | =Tc/hashicraft +| | | | | hashicraft=CTc/hashicraft | +------------+------------+----------+---------------------------+ (1 row) (ksatirli@[postgres]:5432) [postgres] > Connection closed by foreign host.
20. Gate Privileged Access Controller Transit Secrets Engine Codified Configuration App Deployment Response Automation
21. Run Observe Build Release Delivery
22. Run Observe Response Automation Incident Response Team Incident Triggered ! ALERT
23. Run Observe Response Automation Incident Response Team !? Incident Acknowledged ACKNOWLEDGE
24. Run Observe Response Automation Response Team {} Process Webhook !? Incident Acknowledged Incident
25. Boundary Desktop Postgres Database Grant Access Boundary CLI > _ ___ _ _____ ___ _ ___ __ _ ___ __ __ Incident Resolved {} Process Webhook Incident
26. Response Automation {} Process Webhook Revoke Access Observe Incident Resolved Resolution Relax
27. resource "boundary_host_catalog" "main" { scope_id = var.scope_id type = "static" name = var.host_name } resource "boundary_host" "main" { name = var.host_name address = var.host_address host_catalog_id = boundary_host_catalog.main.id type = "static" } resource "boundary_host_set" "main" { host_catalog_id = boundary_host_catalog.main.id type = "static" Boundary Create Resources for Host, Catalog, and Targets per Service.
28. host_ids = [ boundary_host.main.id ] } resource "boundary_target" "main" { name = var.host_name scope_id = var.scope_id type = "tcp" host_source_ids = [ boundary_host_set.main.id ] Boundary Create Resources for Host, Catalog, and Targets per Service.
29. resource "pagerduty_webhook_subscription" "gate" { type = "webhook_subscription" active = true delivery_method { type = "http_delivery_method" url = true custom_headers = [{ name = "X-Boundary-Project", value = boundary_project.main.id }, { name = "X-Boundary-Targets", value = boundary_target.main.id }] } ... Webhook Custom Headers provide data to map affected service to Boundary Scopes.
30. resource "pagerduty_webhook_subscription" "gate" { ... events = [ "incident.acknowledged", "incident.resolved", ] filter { id = pagerduty_service.gate.id type = "service_reference" } } Webhook Custom Headers provide data to map affected service to Boundary Scopes.
31. to a production database, what: where: Get TCP access when: from 13:04 until 13:15. , why: to resolve incident #34.
32. See this workflow in action in the Dev Lounge during HashiConf. Code will be progressively released via github.com/hashicorp-dev-advocates Resources
33. Sr. Developer Advocate at HashiCorp he / him @ksatirli Kerim Satirli
34. @ksatirli kerim@hashicorp.com Thank you