Making your app see, hear and think with Cognitive Services

Artificial intelligence used to be something super complex but Microsoft is making it simple and available for any developer to integrate it in their apps. In this session we’ll look at how you can use features like computer vision, speech recognition and other AI features of cognitive services in your Xamarin Mobile apps. The session will be packed with hands on demos and live coding. After this session you are ready to implement these cool features in your own apps yourself!

Transcript

1. Make your app see, hear and think with Cognitive Services Roy Cornelissen Xpirit @roycornelissen Level: Intermediate
2. AI is the biggest new hype in software development Should I care when I’m just building mobile apps?
3. AI HYPE
4. AI HYPE
5. But AI is not new
6. AI IS NOT NEW Alan Turing Invented the Turing Test in 1950
7. AI IS NOT NEW Captcha’s Are reverse Turing tests
8. AI IS NOT NEW Marvin Minsky The “father” of AI
9. AI IS NOT NEW 1955 Dartmouth Summer Research Project
10. REASONING KNOWLEDGE REPRESENTATION PLANNING (INCLUDING NAVIGATION) NATURAL LANGUAGE PROCESSING PERCEPTION AI IS NOT NEW – DARTMOUTH RESEARCH PROJECT
11. REASONING KNOWLEDGE REPRESENTATION PLANNING (INCLUDING NAVIGATION) NATURAL LANGUAGE PROCESSING PERCEPTION AI IS NOT NEW – DARTMOUTH RESEARCH PROJECT GENERALIZED INTELLIGENCE
12. General Intelligence is still far away
13. And maybe that’s a good thing
14. Why AI now?
15. WHY AI NOW? The arrival of machine learning Performance for decades-old challenges has been transformed Image recognition Speech recognition October 2016 Microsoft researchers reach human parity in conversational speech recognition
16. Let’s do a small test How many dogs can you find in 2 seconds?
18. How many did you count? Did anyone find all 10?
19. WHY AI NOW? Old way: Rules • Build systems to look for ears, noses, legs, fur… • Hire linguists and write grammar rules New way: 2012 Machine Learning • Huge catalogue of images (way more data) • Huge computing power • Over time it learns patterns rather than need rules
20. BIAS
21. Data-driven bias Bias through interaction Emergent bias e.g. face recognition e.g. Microsoft Tay e.g. “Facebook bubble”
22. AI & Mobile? Sure, I believe the hype but why care as a mobile dev?
23. AI & MOBILE User Experience AI is making natural language and interaction possible Rise of the Chatbots Rise of the Personal Assistants
24. AI & MOBILE Sensors Everywhere But mostly in your phones Mobile is the interface to the intelligent cloud and intelligent edge
25. AI & MOBILE Innovation in hardware & mobile OS Apple’s A11 chip focusing on AI Processing Apple added CoreML APIs to iOS 11 Android supports TensorFlow Lite
26. The Next Wave AMBIENT COMPUTING
27. Microsoft Cognitive Services Let’s see what we can do infuse our apps with AI
28. Machine Learning as a Service Cognitive services are trained models ready to use MICROSOFT COGNITIVE SERVICES
29. MICROSOFT COGNITIVE SERVICES Language LUIS Spell Check Linguistic Analysis Text Analytics Translator Web Language Model Vision Computer Vision Content Moderator Emotion Face Video Video Indexer Speech Speech Speaker Recognition Translator Custom Search Autosuggest Image News Video Web Custom Knowledge Academic Entity Linking Exploration QnA Maker Recommendations Bot Framework
30. Inspirational use cases What kind of ideas can you steal borrow and add to your own apps? MICROSOFT COGNITIVE SERVICES
31. INSPIRATION: VISION UBER: Driver verification using vision API
32. INSPIRATION: SPEECH Speech to Text Build your own personal assistant or voice recorder
33. INSPIRATION: LANGUAGE Spell check user input
34. SHOW ME SOME CODE ALREADY! github.com/roycornelissen/xamarin-cognitive-demo
35. Silicon Valley – Not Hotdog Scene https://www.youtube.com/watch?v=ACmydtFDTGs
36. SHOW ME MORE CODEZ! github.com/roycornelissen/Xamarin-NotHotdog
37. @roycornelissen
38. @roycornelissen blog.roycornelissen.com
39. pages.xpirit.com/magazine9 @roycornelissen blog.roycornelissen.com