Machine Learning with Elastic Stack

Slide 1

Slide 1 text

1 Aravind Putrevu @aravindputrevu Engineer | Developer Advocate Machine Learning with Elastic Stack It Catches What You Might Miss, All by Itself

Slide 2

Slide 2 text

2 Takeaways Overview of Holy Grail 1 What is in store from Elastic? 3 What it can/can’t do? 4 Demo Instructions 5 ML Leaps 2

Slide 3

Slide 3 text

3 Takeaways Overview of Holy Grail 1 What is in store from Elastic? 3 What it can/can’t do? 4 Demo Instructions 5 ML Leaps 2

Slide 4

Slide 4 text

4 Takeaways Overview of Holy Grail 1 What is in store from Elastic? 3 What it can/can’t do? 4 Demo Instructions 5 ML Leaps 2

Slide 5

Slide 5 text

5 Takeaways Overview of Holy Grail 1 What is in store from Elastic? 3 What it can/can’t do? 4 Demo Instructions 5 ML Leaps 2

Slide 6

Slide 6 text

6 Takeaways Overview of Holy Grail 1 What is in store from Elastic? 3 What it can/can’t do? 4 Demo Instructions 5 ML Leaps 2

Slide 7

Slide 7 text

7 Elastic Stack No enterprise edition All new versions with 6.2 X-Pack Security Alerting Monitoring Reporting Machine Learning Graph

Slide 8

Slide 8 text

8 “ “ A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E

Slide 9

Slide 9 text

9 Image Credit : Toptal

Slide 10

Slide 10 text

Slide 11

Slide 11 text

11 Datasets Computational Power ThinkTank Hardware

Slide 12

Slide 12 text

Slide 13

Slide 13 text

13 AI and ML

Slide 14

Slide 14 text

14 Algorithmic Approaches • Labelled Data • Driven by objective • NNs, SVMs, Decision Trees • No Label • No objective, only Data • K-means, Apriori • No Label • Driven by objective • Q-Learning, SARSA, DQL Supervised Unsupervised Reinforcement

Slide 15

Slide 15 text

15 Image Credit : Udacity

Slide 16

Slide 16 text

16 How can I use? 1 2 3 Develop your own Using a Framework Ready made solution

Slide 17

Slide 17 text

17 17 ● Online Unsupervised Learning ● Index Visualizer ● Anomaly Detection ● Forecasting Elastic Machine Learning

Slide 18

Slide 18 text

18 Who are you? ● DevOps Engineer ● Security Analyst / SOC Engineer / SecDevOps ● Business Analysts / Application Owner

Slide 19

Slide 19 text

19 What you can do? Operational Analytics Security Analytics Business Analytics • Drop in Orders? • Unusual Traffic • Early hints before things go wrong • MITM detection • Malware? Insider Threat? • Maliciously running processes • Latency in response times • Low click through rate on ads

Slide 20

Slide 20 text

20 Anomaly Detection ● Single or Multi-metric time series ● Outliers in population ● Rare events categorization

Slide 21

Slide 21 text

21 21

Slide 22

Slide 22 text

22 22 Forecasting

Slide 23

Slide 23 text

23 23 Index Visualization

Slide 24

Slide 24 text

DEMO

Slide 25

Slide 25 text

25 What it can’t do?

Slide 26

Slide 26 text

• https://www.elastic.co/products/x-pack/machine-learning • https://www.elastic.co/guide/en/kibana/current/introduction.html • https://www.elastic.co/elasticon/conf/2018/sf/machine-learning-in-the-elastic-stack • https://www.elastic.co/elasticon/conf/2018/sf/the-math-behind-elastic-machine-learning • https://discuss.elastic.co/c/x-pack bit.ly/OracleCode Where else I can look?

Slide 27

Slide 27 text

27 Fin!