Catflap

Talk by Andreas Hübner on the Datageeks meetup 09.2015

Transcript

1. Ready for cat content? Andreas Hübner - @ahueb cat owner / data scientist Munich DataGeeks - September 2015 Edition
2. I started programming websites as a teenager. Not fully convinced of being a sole software developer, I started studying Business Informatics and later on focused on machine learning and mathematical optimization in Paderborn and Helsinki. My Bachelor and Master theses dealt with outlier detection and recommender systems. In my students job, I helped to design operations research solutions. After finishing master studies, Alex Thamm offered me an fantastic position as a consultant for business analytics projects. Since then I've seen one or the other real world data science problems (including predictive maintenance, search for patterns, anomaly detection and some more), team and project. I enjoy studying and solving data problems (even in my spare time...), keeping their potential effects on business in mind. And I like cats.
3. ? Isis Mika Casimir
4. Mika
7. Maximum size
8. Franzl
9. ?
10. Prototype
11. Magnet Reed sensor (2, in) Reed sensor (1, out)
13. Waiting Collecting Cooldown No sensor activity Sensor active Sensor active No sensor activity for more than 10 seconds Observer
14. 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111100220000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000011111022000000000000000000000000000222000011111111111111111111111111111111100022 Zero and one. And two.
15. 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111100220000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000011111022000000000000000000000000000222000011111111111111111111111111111111100022 In oder Out? Really? ~1 Second (100 checks per second)
16. 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111100220000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000011111022000000000000000000000000000222000011111111111111111111111111111111100022 Longest “Zero- segment” (Passage) In oder Out? “Segment” 1. Last direction before longest zero segment 2. First direction 3. Number of in/out segments before the longest zero segment Criteria “Head” (Flap opens)
17. 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111100220000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 000000000000000011111022000000000000000000000000000222000011111111111111111111111111111111100022 Wirklich? Variables 1. Pattern length 2. Tail length 3. Length of longest zero segment Longest “zero segment” “Tail” (Flap falls back into starting position) “Segment”
18. https://www.coursera.org/course/ml
19. Kmean clustering Are there meaningfull clusters? Center and scale the data! Find out by premutating the whole data set Run some resamples The clusters with the true data should be denser. Isolate the faulty patterns (testing, Mika pring the flap)
20. Identify faulty patterns on the fly Semi-supervised learning (Self Training) Use the cluster labels for patterns Train a model (yes, validate and test…) Apply the model to new patterns Give feedback (send a notification) Store the predicted class Add most confident predictions to train Retrain the model (eventually down- weight by proprensity score and “age”) http://is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/taxo_[0].pdf https://en.wikipedia.org/wiki/Ma% C3%9F#/media/File: Jugg_with_Beer_Loewenbraeu_ one_liter.JPG
21. Mika sends Mail and Notifications
23. Daily pattern
24. Can I haz weekend?
25. Can I haz weekend?
26. Mika comes home before we go to bed and after (or right before!!!) we get up
27. There are no obvious differences between weekday and weekend
28. On weekends, mika hesitates before and rushes after 12am At night, mika is slower and during daytime
29. Seasonal patterns? Yes there is. *y scale is free Mika wakes us at night during the summer months. Here’s the veidence Less passages during summer. Open doors an warm nights!
30. Does Mika hesitate when its cold? No.
31. 3600 passages recorded 32MB Storage used (small data yay!) about 150 faulty patterns Production system since 8/2014
32. WAF++ http://imgur.com/gallery/veoWbmL
35. Questions? Meow! Thank you for your attention! http://www.bbc.co.uk/programmes/b04lcqvq