Exploratory Seminar: Detecting Anomalies in Time Series Data

This is for introducing Anomaly Detection algorithm, which was developed by folks at Twitter to detect anomaly data in time series data. Also, for demonstrating it with Exploratory’s Analytics view.

Transcript

1. EXPLORATORY
2. Kan Nishida co-founder/CEO Exploratory Summary Beginning of 2016, launched Exploratory, Inc. to make Data Science available for everyone. Prior to Exploratory, Kan was a development director at Oracle leading development teams for building various Data Science products in areas including Machine Learning, BI, Data Visualization, Mobile Analytics, Big Data, etc. While at Oracle, Kan also provided training and consulting services to help organizations transform by data. @KanAugust Instructor
3. Vision Make Data Science available for everyone
4. Data Science is not just for Engineers and Statisticians. Exploratory makes it possible for Everyone to do Data Science. The Third Wave
5. First Wave Second Wave Third Wave Proprietary Open Source UI & Programming Programming 2016 2000 1976 Monetization Commoditization Democratization Statisticians Data Scientists Smart Waves - Machine Learning / AI Algorithms Experience Tools Open Source UI & Automation Business Users Theme Users Exploratory
6. Questions Data Science Workflow Communication Data Access Data Wrangling Data Visualization Machine Learning / Statistics Exploration
7. Questions What you can do with Exploratory Communication Data Access Data Wrangling Visualization Machine Learning / Statistics Exploratory Data Analysis
8. Questions Communication Data Access Data Wrangling Visualization Machine Learning / Statistics Exploratory Data Analysis
9. Anomaly Detection with Time Series Data
12. Anomaly Detection • Detect irregular activities like financial fraud, suspicious access, malfunction of machines, etc. • Find ‘what is trending’ based on web site activities, customer activities, etc.
13. • Developed by Arun Kejariwal and others at Twitter • Employs an algorithm referred to as Seasonal Hybrid ESD (S-H-ESD), which can detect both global as well as local anomalies in the time series data by taking seasonality and trend into account. Anomaly Detection
14. • Local Trend (Hour, Day, etc) • Global Trend (Month, Year, etc) • Basic Trend and Pattern • Seasonality Anomaly Detection
15. Data • Quandl Financial Data • Stock Price Data • Google Analytics Data
16. Bitcoin Price with Quandl
18. Install Quandl Data Source
19. Import Data
20. Find a Code for Bitcoin Price Data
21. Find a Code for Bitcoin Price Data
22. Enter BCHARTS/BITSTAMPUSD Set Start Date 2017-01-01
25. Detect anomalies in the historical bitcoin price movements with weighted price column.
28. Filter to keep only 2017 data.
29. Bitcoin Cash was introduced.
30. Stock Price with TidyQuant
33. Google Analytics
34. Anomaly Detection with Google Analytics 1. Find anomalies in the trend of unique page view counts. 2. Find them in each of the top 5 countries.
35. Anomaly Detection with Google Analytics 1. Find anomalies in the trend of unique page view counts. 2. Find them in each of the top 5 countries.
41. Anomaly Detection with Google Analytics 1. Find anomalies in the trend of unique page view counts. 2. Find them in each of the top 5 countries.
42. Create Other Group Make all the countries other than Top 5 countries to ‘Other’. • Select country column • Select ‘Create Other’ Group for’ -> ‘Least / Most Frequent Values’
43. Keep only the top 5 countries
46. Future Seminars
47. December 19th (Tuesday) • Analytics: An Introduction to K-Means Clustering Planned • Analytics 101 - When to use which algorithms? • Data Wrangling: Handling Date / Time Data https://exploratory.io/online-seminar
48. Contact Email kan@exploratory.io Data Science Training https://exploratory.io/training Twitter @KanAugust Online Seminar https://exploratory.io/online-seminar