Save 37% off PRO during our Black Friday Sale! »

Scrubbing PII from Logs in LogStash

769a70f67eb5586c3a0c3594a152fb55?s=47 Ayush Priya
November 16, 2021

Scrubbing PII from Logs in LogStash

ElasticSearch, LogStash and Kibana together create one of the most popular log ingestion and indexing solution. However, the logs being indexed and made available can potentially contain sensitive information such as PII. The talk will explain how to setup masking for such sensitive information(s) present in the logs in LogStash. This would enable any principal who wants to look at the logs to be able to do so while protecting leakage and misuse of sensitive information.

As an example case study, the participants will be shown how to setup filters in LogStash to mask data in the log conditionally based on whether the sensitive field is present in the particluar log entry or not.


Ayush Priya

November 16, 2021


  1. Masking Sensitive Data in Logs with LogStash Ayush (@ayushpriya10), DevSecOps

    Engineer at Appsecco
  2. • What is Sensitive Information? • Problem Statement • Proposed

    Solution • Demonstration • Impact • Probable Questions • Conclusion Contents
  3. • Not just passwords and credentials • Personally Identifiable Information/Context

    Dependent Information What is Sensitive Information?
  4. • Removing PII from Logs for a B2C Internet Unicorn

    • Why do we want to remove PII from Logs? Problem Statement
  5. • Data Masking • What is LogStash? • LogStash Configuration

    • Input • Filter • Output Proposed Solution
  6. LogStash Configuration

  7. Demonstration

  8. Result

  9. • No refactoring required • Can be extended to other

    applications in the ecosystem • Enabled Dev teams to do their jobs while the problem was solved Impact of the Solution
  10. • Why would we log Sensitive Information? • The original

    logs still have the PII, how does the solution solves the problem for the original logs? Probable Questions
  11. • Information can become sensitive based on the context •

    LogStash allows us to Mutate/Filter logs before being sent to ElasticSearch • The solution enabled Developers to continue their work while ensuring privacy • The aim is to not be a blocker for other teams when solving for security issues Conclusion
  12. Thank You! Ayush (, DevSecOps Engineer at Appsecco Twitter: @ayushpriya10