Upgrade to Pro — share decks privately, control downloads, hide ads and more …



Ananth Packkildurai

March 25, 2019

More Decks by Ananth Packkildurai

Other Decks in Technology


  1. Me ➢ @ananthdurai ➢ Senior Software Engineer @ Slack ➢

    Passionate about all things related to ethical data management
  2. Public launch: 2014 1000+ employees across 7 countries worldwide HQ

    in San Francisco $841M in capital raised Key investors include Softbank, Accel, a16z, Social Capital, Index, Thrive, GV, Kleiner Perkins, GGV, Horizons, Spark, IVP and DST. Diverse set of industries including software/technology, retail, media, telecom and professional services. About Slack
  3. I have well-defined dimensions & metrics, and I want a

    real-time aggregation and querying. Type 1 (The Analytical pattern) (e.g) ❖ performance analytics ❖ experimentation
  4. I want to slice & dice metrics in real-time. If

    something goes wrong, I would like to see the detail events for the given dimensions. Type 2 (The needle in the haystack pattern) (e.g) ❖ Mysql slow queries ❖ Audit logs ❖ Security logs
  5. I want to dump all the events and explore further

    to understand the pattern. Type 3 (The exploration pattern) (e.g) ❖ Log inspection ❖ Exploratory analysis
  6. Druid Druid i3.2X middle manager node 40+ 10+ Pinot T-Digest

    17 I3 2x Pinot Servers 0 Ingestion nodes Druid -> Pinot Tranquility ingestion nodes. Approximate Histogram aggregator
  7. Did I broke anything while launching an experiment? Is there

    any performance degrade with my experiments? Use case #2: The experimentation framework
  8. Pinot Next Log search support Add support for free text

    search on the dimensions. https://github.com/apache/incubator-pinot/ issues/2798 Integration with visualization tools Ease the integration with the operational intelligence and business intelligence solutions like grafana, superset etc