Neudesic Geekfest Leith Akkawi 12052017

Data science solution for the DengAI competition using neural networks and time series analysis

Transcript

1. Predicting Dengue Fever Spread Neural Networks & Time Series Analysis By: Leith Akkawi Senior Consultant, Data Science
2. Founded in 2002 Privately held firm headquartered in Irvine, Ca founded by former Microsoft leaders. Neudesic has 12 locations in the US and 3 India based development centers that offers a broad range of strategic and technology services A B O U T N E U D E S I C D E E P T E C H N I C A L P A R T N E R S H I P S B R O A D S T R A T E G I C A N D T E C H N I C A L O F F E R I N G S Our consultants offer a broad range of technical expertise from cloud to mobile, from application architecture to developer operations. G L O B A L R E A C H 2 0 1 5 - 2 0 1 7 M I C R O S O F T A W A R D S 2017 Microsoft US Country Partner of the Year 2017 Microsoft Global Cloud Apps Partner of the Year 2017 Microsoft National Solution Provider of the Year 2016 Microsoft US Data Analytics Partner of the Year 2015 Microsoft Global Data & Analytics Partner of the Year
3. NOW AVAILABLE FOR DOWNLOAD: NEUDESIC.COM/AAEBOOK E-book: A Step-By-Step Guide To Advanced Analytics for Executive Decision Making
4. Leith Akkawi Senior Consultant, Data Science A B O U T L E I T H • Senior Consultant, Data Science • Graduate, Northwestern University- Predictive Analytics • Experience in commercial banking, supply chain, political campaigns, real estate, & human resources Q U I C K I N T R O
5. Agenda W H Y D O E S D A T A S C I E N C E M A T T E R ? • Understand the unknown • Learn the past • Predict the future • Influence the Future A Q U I C K I N T R O T O D A T A S C I E N C E • Data Strategy • Big Data • Machine Learning C A S E S T U D Y A N A L Y S I S • Dengue Fever S O L U T I O N D E V E L O P M E N T A R C H I T E C T U R E • Time Series Analysis • Neural Networks • R - statistical computing software Q & A
6. Data Strategy: The Data or The Objective
7. 7 A Quick Intro to Data Science •Data Strategy •Big Data •Machine Learning
8. A D V A N C E D A N A L Y T I C S T R A D I T I O N A L B I What happened? D I F F I C U L T Y V A L U E D E S C R I P T I V E A N A L Y T I C S Why did it happen? D I A G N O S T I C A N A L Y T I C S What will happen? P R E D I C T I V E A N A L Y T I C S How can we make it happen? P R E S C R I P T I V E A N A L Y T I C S H I N D S I G N T I N S I G H T F O R E S I G H T Why does data science matter?
9. D A T A S C I E N C E A R T O R S C I E N C E ?
10. H O W B I G I S B I G D A T A ? 1 B Y T E = A U N I T O F D A T A T H A T I S E I G H T B I N A R Y D I G I T S L O N G
11. Big Data Ecosystem
12. Machine Learning: Predicting the future!
13. Machine Learning: Classic Predictive Scenarios
14. 14 Case Study •Predictive Modeling 1. Objective: Predict Spread of Dengue Fever 2. Data Acquisition 3. Visual Analysis 1. Exploratory Data Analysis 2. Data Scrubbing 3. Feature Engineering 4. Model Selection 5. Model Comparison
15. Neudesic Advanced Analytics Solution Framework: Introduction to Predictive Modeling Understanding Data Model Creation + Testing Integration in Data Strategy
16. Leadership Management Data Worker Tasks: • Profit driven strategy • Improved business decisions • Low investment to get started Step 1: Defining the Business Objective Soliciting stakeholder buy-in for enterprise impact Tasks: • Investing in the right tools • Insight into progress • Design for adoption • Ability to train the workforce Tasks: • Execution plan • Tools that make their lives easier • Open lines of communication • Tasks infiltrate business operations
17. H T T P S : / / W W W . D R I V E N D A T A . O R G / C O M P E T I T I O N S /
18. Step #2. Data Acquisition D A T A A C Q U I S I T I O N I N T E R N A L D A T A E X T E R N A L D A T A
19. D A T A A C Q U I S I T I O N : D O W N L O A D D A T A Data Set Time Series San Juan Iquitos Train Model Start Week 18, 1990 Week 20, 2000 Train Model End Week 17, 2008 Week 25, 2010 Predict Test Start Week 18, 2008 Week 26, 2010 Predict Test End Week 17, 2013 Week 26, 2013 Time Series Neural Networks
20. Step #3: Visual Analysis How quickly can you count the 5s?
21. Step #3: Visual Analysis How about now?
22. E X P L O R A T O R Y D A T A A N A L Y S I S T O T A L C A S E S P E R W E E K Iquitos, Peru San Juan, Puerto Rico
23. Data Scrubbing O U T L I E R S Temperature = 300 W R O N G V A L U E S Temperature = ABC N U L L S Temperature = Empty
24. F E A T U R E E N G I N E E R I N G Week of Year >= 26 Max Temp <= 35 Min Temp >= 20 Humidity <= 93 & >=73 Dew <= 298 & >= 293 INDEX = week_indicator +temp_max_indicator + temp_min_indicator +humidity_indicator +dew_indiator
25. Step #4: Model Selection
26. Step #4: Model Selection A T O T A L O F 4 5 E X P E R I M E N T S W E R E R U N , S O M E E X A M P L E S O F M O D E L S U S E D : T I M E S E R I E S : • ARIMA • Auto.ARIMA • ETS • GARCH P R E D I C T I O N • GBM • RandomForest • Classifcation & Prediction Model • Neural Networks • Xgboost
27. Step #5. Model Comparison Mean Average Percent Error Root Mean Square Error Variation explained by Predictor How much should we expect to be off? What amount you are comfortable missing by? How much do the features impact the result?
28. Step 5: Model Comparison
29. 29 Solution Development Architecture •Time Series Models •Feed Forward Neural Networks
30. T I M E S E R I E S D E C O M P O S I T I O N
31. E X A M P L E T I M E S E R I E S D E C O M P O S I T I O N T H E M A T H Source: https://www.otexts.org/fpp/8/9
32. A U T O R E G R E S S I V E M O V I N G A V E R A G E
33. T I M E S E R I E S - P R E D I C T I O N
34. Feed Forward Neural Networks
35. H O W D O W E B U I L D A M O D E L ? R - A F R E E S O F T W A R E E N V I R O N M E N T F O R S T A T I S T I C A L C O M P U T I N G A N D G R A P H I C S
37. H O W D O W E G E T T H E A L G O R I T H M S ?
38. T H E B E S T M O D E L : A P A C K A G E T H A T C O N T A I N S A F U N C T I O N T H A T C A N R U N B O T H T I M E S E R I E S & N E U R A L N E T W O R K A L G O R I T H M S
39. P R E D I C T I O N S I Q U I T O S P E R U S A N J U A N P U E R T O R I C O
40. Q & A
41. A D V A N C E D A N A L Y T I C S O F F E R I N G S Architectures designed for local autonomy and global execution for repeatable success Architect for Scaled Execution Strategic details and decisions that are crucial to the success of ML projects ‘Packing List’ Ideation sessions to identify prime ML projects Gather and Rank Implement a roadmap for execution, monitoring and communicating the development of projects Solution Framework www.neudesic.com
42. Thank you