Introduction to Hadoop Security

Hadoop is central to a lot of Enterprises wanting to to store and process their sensitive data. Securing Hadoop deployments is more important than ever before. This talk will illustrate practical and production tested techniques on how you can protect Hadoop data from unauthorized access and modification.

Transcript

1. Introduction to Hadoop Security Tips and Techniques on Protecting your Data Lake
2. Hadoop is everywhere, Security not so much • Hadoop is central to a lot of Enterprises • 50% of Hadoop deployments store sensitive data • However not a lot of deployment implement Security • Even less do it right • Data lakes don’t need to be free for all.
3. Know your ABCs of Security • Authentication • Authorization • Auditing
4. You need a Sentry • Protecting your tables • ACLs for everyone • Knock Knox..
5. HDFS Security: Not an Oxymoron • Fake vs Real • Kerberos is the only option • ACLs are your friend
6. Apache Spark Security • Why Yarn is your friend • Spark Security basics • TLS for Spark • Kerberos and User Impersonation
7. Data Protection • Data in Transit • Data at Rest • Configuration Data • Data Lineaging
8. Hue is your (Security) Daddy ? • Single Sign-on for Hue • Managing Policies using Hue • Groups and Users • Apps and more
9. Kerb(eros) your Enthusiasm • Overview of Kerberos • Usage in Hadoop Authentication • Hadoop impersonation • LDAP based directory server integration
10. HDFS Encryption • Transparent vs Block Level • Know your KMS • Impact on tools and apps • Other Gotchas
11. Data Ingestion Security • Know your Zones • Groups and Others • Default Umask • Sharing Data across Zones.
12. End-user applications Security. • Perimeter Security • Token based Authentication • Search and NoSQL access • Centralized Security Access Manager.
13. Case Study: Data Warehouse • Best Practices • Gotchas • Tools and Apps • Realtime Querying • Visualization tools
14. Conclusion • Security is Hard • Hadoop is not one thing • Different Access patterns, Different Security • Consider Data flow • Consider Threat modeling