The Kitchen's Finally Burned Down: DLP Security Bakeoff

(As presented by Zach Lanier and Kelly Lum at Black Hat Asia 2016)

Despite a plethora of data security and protection standards and certifications, companies and their systems are still leaking information like a sieve. For instance, Data Loss Prevention (DLP) solutions have often been touted as the "silver bullet" that will keep corporations from becoming the next headline. With deployment models ranging from a fat agent on an endpoint, to a blinky-lights box surveilling all network traffic, to some unified threat management gateway with DLP secret sauce, these solutions are ripe for bypass -- or worse.

This talk will discuss our previous and current research into a handful of DLP solutions, including their capabilities and their shortcomings. We will demonstrate flaws in administrative and programmatic interfaces and the inspection engines themselves.

Transcript

1. THE KITCHEN'S FINALLY BURNED
   DOWN: DLP SECURITY BAKEOFF
   BLACK HAT ASIA 2016
   ZACH LANIER & KELLY LUM
   

   View Slide

2. AGENDA
   ▸ DLP overview
   ▸ Targets / Component breakdown
   ▸ Assessment criteria/Methodology
   ▸ Findings (by target)
   ▸ Conclusion / Q&A
   

   View Slide

3. WHO ARE *WE*?
   ▸ Zach Lanier, Director of Research at Cylance
   ▸ Old net, web/app, mobile/embedded security research/pen
   test type
   ▸ Co-author, "Android Hacker's Handbook" (Wiley, April
   2014)
   ▸ Kelly Lum, Security Engineer at Tumblr
   ▸ “There’s a bird. Bird. A bird.”
   ▸ Adjunct professor of Application Security at NYU
   

   View Slide

4. WHY DLP?
   ▸ Used to be a hot-button topic
   ▸ Panacea to solve all data leakage woes
   ▸ “Keeps honest people from doing dumb
   things”
   ▸ Data breaches and “files falling off the back of a
   digital truck” spurred DLP
   

   View Slide

5. WHY WE CHOSE TO LOOK AT DLP
   ▸ Curious about attack surface, reliability, etc.
   ▸ Like other security products, DLP agents/appliances often have high privileges
   or are “ideally” situated (i.e. see all the traffic, monitor/access all the things)
   ▸ Testing the “security of security products” is always interesting
   ▸ Big vendor buys small vendor, integrates then shelves them…meaning
   security is often overlooked

   

   View Slide

6. PREVIOUS (NOT US) RESEARCH
   ▸ A bunch of blog posts and whitepapers by Securosis
   ▸ “Defeating DLP”, Matasano, BlackHat USA 2007
   ▸ “Gone in 60 Minutes: Stealing Sensitive Data from Thousands of Systems
   Simultaneously with OpenDLP”, Andrew Gavin, DEFCON 19
   ▸ Many others...
   

   View Slide

7. DLP ARCHITECTURE EXAMPLE
   

   View Slide

8. DLP ARCHITECTURE EXAMPLE
   

   View Slide

9. DLP WORKFLOW EXAMPLE - TREND MICRO
   

   View Slide

10. TARGETS
    

    View Slide

11. ROUND ONE: "BIG"* VENDORS
    Trend Micro DLP Management Appliance 5.6 Linux
    DLP Endpoint Agent 5.6 Windows
    Sophos Astaro UTM Appliance 9.201 Linux
    Sophos Enterprise Console 5.2.1r2 Windows
    Sophos Endpoint Security N/A Windows
    Websense TRITON Management Server 7.8.3 Windows
    Data Protector Endpoint Agent 7.8.3 Windows, Linux, OS X
    Data Security Protector Appliance 7.8.3 Linux
    OpenDLP OpenDLP 0.5.1 Linux
    Vendor Product Version OS
    * - and an open source product, for good measure
    

    View Slide

12. TREND MICRO
    ▸ Windows endpoint agent - monitoring and policy enforcement on client
    machines
    ▸ Acts like a “legitimate” rootkit and hides itself
    ▸ Network agent - virtual appliance; monitors network traffic
    ▸ Remote crawler - for digital assets on machines not on corporate network
    ▸ Management server - Linux-based virtual appliance
    

    View Slide

13. WEBSENSE
    ▸ TRITON management server - unified management console; Apache Coyote on Windows,
    backed by MSSQL DB
    ▸ Windows, OS X, Linux endpoint agents
    ▸ File and network drivers
    ▸ Can also monitor clipboard operations
    ▸ Linux-based “Protector” appliance
    ▸ Restricted “admin” shell
    ▸ Crawler agents can index/identify sensitive documents
    

    View Slide

14. SOPHOS
    ▸ Enterprise Management Console - fat/native, Windows-based unified
    management console
    ▸ Whole lotta .NET…
    ▸ Sophos endpoint security - antivirus + DLP + … (Windows, OS X, Linux)
    

    View Slide

15. OPENDLP
    ▸ Typically Linux virtual appliance
    ▸ Apache + a lot of Perl
    ▸ Windows agent
    ▸ File system crawler and document parser (PCRE-based)
    ▸ SSHFS-based crawler
    ▸ And some Metasploit modules (wtf?)
    

    View Slide

16. ON THE UBIQUITY OF KEYVIEW…
    ▸ “kvoop” binary (“KeyView OOP”) showed up a lot
    ▸ Part of KeyView Filter SDK, used for parsing and normalizing various data and document formats
    ▸ Used in numerous DLP products, messaging servers, and "big data" platforms
    ▸ e.g. “EPClassifier” in Websense spawns kvoop processes to handle documents
    

    View Slide

17. BECAUSE "PR" AND "LEGAL", THE VENDORS/
    PRODUCT NAMES (FOR OUR SECOND ROUND OF
    RESEARCH) HAVE BEEN CHANGED FOR THE
    PURPOSES OF THIS PRESENTATION
    DISCLAIMER :(
    

    View Slide

18. ROUND TWO: NICHE/SMALL(ER) VENDORS
    Alpha Alpha DLP Management Virtual Appliance Linux
    Alpha DLP Endpoint Agent Windows
    Bravo Bravo DLP Management Server Windows
    Bravo DLP Endpoint Agent Windows, OS X
    Charlie Charlie DLP Management Virtual Appliance Linux
    Charlie DLP Agent Windows, Linux, OS X
    Dingus Dingus DLP Central Console Virtual Appliance Linux
    Dingus Network DLP Virtual Appliance Linux
    Dingus DLP Endpoint Agent Windows
    Vendor Product OS
    

    View Slide

19. ALPHA DLP
    ▸ An amalgamation of everything you hate:
    ▸ Previously OSS product, closed after acquisition!
    ▸ Admin panel entirely in Flash!
    ▸ Windows agent is a horrifying Frankenstein of:
    ▸ .NET...Java....and Erlang.
    ▸ Uses Action Message Format for communications!
    ▸ Backend: Apache + Jetty + MySQL
    

    View Slide

20. BRAVO DLP
    ▸ Windows-based management console (MMC snap-in)
    ▸MS SQL DB backend
    ▸ Windows-based content monitoring server (separate service from management console)
    ▸ Windows endpoint agent
    ▸All sorts of drivers (NDIS, TDI, FS, etc.) and hooks
    ▸OS X endpoint agent
    ▸Similar to Windows agent, but with less support for certain operations
    ▸Tons of open source / free libs
    ▸Boost, FreeDCE, etc.
    

    View Slide

21. CHARLIE DLP
    ▸ Linux (Ubuntu) virtual appliance
    ▸ Windows endpoint agent
    ▸File monitor / scanner service
    ▸Clipboard monitor
    ▸Net traffic / URL monitoring
    ▸ Linux endpoint agent (Ubuntu/Debian)
    ▸File monitor / scanner daemon
    ▸GNOME / KDE notification tray thing
    ▸ OS X endpoint agent
    ▸Didn't really evaluate this
    ▸Also does a bunch of MDM-type stuff, but
    didn't look at this
    

    View Slide

22. DINGUS DLP (WHICH IS A TOTALLY HYPOTHETICAL VENDOR/PRODUCT)
    ▸ Linux (CentOS) virtual appliance (management)
    ▸ Dingus Network DLP Appliance
    ▸Linux (CentOS) virtual appliance
    ▸In-line, tap/SPAN, etc. to monitor (or proxy) traffic
    ▸ Windows endpoint agent
    ▸Usual rigamarole - monitor clipboard, filesystem, network traffic/URLs, etc.
    

    View Slide

23. ASSESSMENT
    CRITERIA/
    METHODOLOGY
    

    View Slide

24. METHODOLOGY
    Network Appliance Parsers (docs and configuration) Invalid/mangled files
    Update/Deployment mechanism Protocol analysis; crypto/signing
    Operating System Configuration auditing
    Hardening practices
    Endpoints/Agents Parsers (docs and configuration) Invalid/mangled files
    Update/Deployment mechanism Protocol analysis; crypto/signing
    Drivers and Services Hardening practices/config
    Fuzzing (i.e. IOCTLs, network, etc.)
    Management Server Web Server/Web App OWASP Top 10 type stuff
    Database Configuration auditing
    Sensitive data storage
    Operating System Configuration auditing
    Hardening practices
    Target Component Test(s)
    

    View Slide

25. FINDINGS
    

    View Slide

26. GENERAL OBSERVATIONS
    ▸ Little to no hardening on (Linux) appliances
    ▸ Many services run as root
    ▸ Lack of exploit mitigations (beyond intrinsic/baseline OS mitigations)
    ▸ Highly privileged endpoint agent software out of the box (root, LOCALSYSTEM)
    ▸ General absence of security best practices
    ▸ Comms encryption, webappsec101, etc.
    ▸ Occasional bug inheritance
    ▸Outdated JREs, FreeDCE, etc.
    

    View Slide

27. FINDINGS -
    TREND MICRO
    

    View Slide

28. TREND MICRO - XSS
    

    View Slide

29. TREND MICRO - CSRF
    

    View Slide

30. TREND MICRO - PLAINTEXT CRAWLER COMMS
    

    View Slide

31. FINDINGS -
    SOPHOS
    

    View Slide

32. SOPHOS MANAGER/ENDPOINT: WHAT WE DIDN’T FIND
    ▸ Majority of code implemented in .NET
    ▸ Utilizes most of the MS core libraries,
    which means:
    ▸ DB best practices
    ▸ Contextualized Input/Output
    ▸ Standardized Encryption Libraries
    

    View Slide

33. SOPHOS/ASTARO UTM: NOT A WHOLE LOT…
    ▸ Most services chroot’ed (eh…), drop
    privs
    ▸ Web app fairly clean (just a few really
    low impact “issues”)
    ▸ Tight network- and login-access control
    restrictions
    

    View Slide

34. FINDINGS -
    OPENDLP
    

    View Slide

35. OPENDLP - CSRF
    

    View Slide

36. FINDINGS -
    WEBSENSE
    

    View Slide

37. ON WEBSENSE POLICIES...
    ▸ Websense DLP policy objects include keywords,
    regexes, etc.
    ▸ Regex entries are actually Python pickled
    objects
    ▸ TRITON management server encrypts, bundles
    policies/files, pushes to agents and appliances
    

    View Slide

38. WEBSENSE PROTECTOR & ENDPOINT - CODE EXEC
    Say a local admin on TRITON
    server replaces “.pic” file with
    custom pickled objects…
    Our awful

    pickle POC;

    after

    overwriting a

    “legitimate”

    policy file
    Reverse
    shell from
    Protector

    after policy
    update
    

    View Slide

39. FINDINGS -
    ALPHA DLP
    

    View Slide

40. ALPHA DLP - SERVER/ ADMIN FINDINGS
    ▸ The Good:
    ▸ No CSRF
    ▸ No obvious XSF
    ▸ Proper use of Hibernate - no SQLi
    ▸ The Bad:
    ▸ No authentication on MySQL database
    ▸ User hashes are not salted
    ▸ Frequent crashes
    

    View Slide

41. ALPHA DLP - AGENT FINDINGS
    ▸ Heavy use of marshalling between Java and Erlang
    ▸ Potential heap corruption?
    ▸ Packaged an oooooold JRE

    

    

    ▸ Path manipulation in accessing Erlang rules
    ▸ No assembly signing on .NET assemblies… #waitforit
    

    View Slide

42. ALPHA DLP - AGENT MODIFICATION
    

    View Slide

43. FINDINGS -
    BRAVO DLP
    

    View Slide

44. BRAVO DLP
    ▸ Probably the most complex/sophisticated of all the
    ones we analyzed
    ▸ Also the one we most heavily reversed
    ▸ tl;dr - Spent too much time, found very little, got
    lazy moved on
    ▸ Use of DCERPC for agent<->server comms
    is...good?
    ▸ Benefit from NTLMSSP and "packet
    privacy" (encrypted)
    ▸ Except for OS X agent, which is all plaintext
    comms
    

    View Slide

45. FINDINGS -
    CHARLIE DLP
    

    View Slide

46. CHARLIE DLP - MANAGEMENT SERVER
    ▸ No anti-CSRF tokens
    ▸ Pretty much all operations are done REST-fully
    ▸ Example: CSRF an admin, delete policy by ID
    

    View Slide

47. CHARLIE DLP - MANAGEMENT SERVER
    ▸ Unauthenticated registration of new (or arbitrary) endpoint agents via SOAP API
    ▸ Also injecting CDATA with JS rendered in admin console
    Super amazing
    JS alert() skillz
    

    View Slide

48. FINDINGS -
    DINGUS DLP
    

    View Slide

49. BECAUSE "PR" AND "LEGAL", THE FOLLOWING VENDOR, DESPITE
    BEING REDACTED, SHOULD BE CONSIDERED *TOTALLY HYPOTHETICAL*
    (ALSO, WE HAND DREW SOME SCREENSHOTS BECAUSE, AGAIN,
    *TOTALLY HYPOTHETICAL*)
    ADDITIONAL DISCLAIMER :(
    

    View Slide

50. DINGUS - NETWORK DLP APPLIANCE
    ▸ Support account that's TOTALLY not a
    backdoor
    ▸ Reverse SSHes to some Eastern European-
    based SSH proxy
    ▸ Same (non-password protected) DSA key
    across every appliance
    ▸ Allows "support" to log in as root on
    appliance
    ▸ Invoked from web UI
    ▸ Combined with CSRF = reverse tunnel
    wherever
    

    View Slide

51. DINGUS - NETWORK DLP APPLIANCE
    CSRF + command injection
    Note the semicolon
    

    View Slide

52. DINGUS - NETWORK DLP APPLIANCE
    CSRF + command injection
    

    View Slide

53. DINGUS - NETWORK DLP APPLIANCE
    Numerous whitelisted (NOPASSWD) sudoers entries for apache
    arbitrary package

    installation, anyone?
    

    View Slide

54. DINGUS DLP - CENTRAL CONSOLE
    ▸ Unauthenticated DB
    management script
    ▸ Clear DB, rebuild,
    update, etc.
    

    View Slide

55. DINGUS - NETWORK DLP/CENTRAL CONSOLE
    ▸ Synchronization channel is just
    straight up plaintext PostgreSQL
    comms...with simple auth
    ▸ Just username + DB name
    (HYPOTHETICAL example:
    appliance:appliance)
    ▸ Pushes ACLs and rules down to
    appliance from Console
    (Yes, that is a hand drawn Wireshark “screenshot”)
    

    View Slide

56. A NOTE ON DLP
    BYPASSES
    

    View Slide

57. EVASION IS INEVITABLE
    ▸ Unsupported or obscure file formats, protocols, or
    unexpected network behavior
    ▸ Think: fragment/datagram reassembly ordering
    ▸ Obfuscated or encrypted files
    ▸ Steganography, anyone?
    ▸ How many of your users have elevated (admin,
    sudo/root) privileges?
    ▸ e.g. disable endpoint agents
    

    View Slide

58. ▸ Defenses add weaknesses
    ▸ "Caveat emptor"
    ▸ Every new piece of infrastructure is additional attack
    surface
    ▸ Security companies should know better
    ▸ If a scanner can find it, what’s your excuse?
    ▸ Know what/who you’re defending against
    ▸ An advanced insider probably has own abilities
    BLACK HAT ASIA TAKEAWAYS
    

    View Slide

59. QUESTIONS?
    george.sims@jukt-micronics.com
    @aloria
    zach@n0where.org
    @quine
    

    View Slide