Scrubbing PII from Logs in LogStash

ElasticSearch, LogStash and Kibana together create one of the most popular log ingestion and indexing solution. However, the logs being indexed and made available can potentially contain sensitive information such as PII. The talk will explain how to setup masking for such sensitive information(s) present in the logs in LogStash. This would enable any principal who wants to look at the logs to be able to do so while protecting leakage and misuse of sensitive information.

As an example case study, the participants will be shown how to setup filters in LogStash to mask data in the log conditionally based on whether the sensitive field is present in the particluar log entry or not.

Transcript

1. Masking Sensitive Data in Logs with LogStash Ayush (@ayushpriya10), DevSecOps Engineer at Appsecco
2. • What is Sensitive Information? • Problem Statement • Proposed Solution • Demonstration • Impact • Probable Questions • Conclusion Contents
3. • Not just passwords and credentials • Personally Identifiable Information/Context Dependent Information What is Sensitive Information?
4. • Removing PII from Logs for a B2C Internet Unicorn • Why do we want to remove PII from Logs? Problem Statement
5. • Data Masking • What is LogStash? • LogStash Configuration • Input • Filter • Output Proposed Solution
6. LogStash Configuration
7. Demonstration
8. Result
9. • No refactoring required • Can be extended to other applications in the ecosystem • Enabled Dev teams to do their jobs while the problem was solved Impact of the Solution
10. • Why would we log Sensitive Information? • The original logs still have the PII, how does the solution solves the problem for the original logs? Probable Questions
11. • Information can become sensitive based on the context • LogStash allows us to Mutate/Filter logs before being sent to ElasticSearch • The solution enabled Developers to continue their work while ensuring privacy • The aim is to not be a blocker for other teams when solving for security issues Conclusion
12. Thank You! Ayush (ayush@appsecco.com), DevSecOps Engineer at Appsecco Twitter: @ayushpriya10 LinkedIn: https://www.linkedin.com/in/ayushpriya10/