Build intelligent bots with IoT on Azure

Bots are becoming more and more part of our daily lives and creating them should be as easy as using them. In this session we create an intelligent bot using the Microsoft Bot Framework and hook it up to Microsoft Cognitive Services and a Raspberry Pi to be able to analyse one's emotions with a webcam. The results are then stream in real-time as Skype Emoji's and to a Power Bi Dashboard

Transcript

1. INTELLIGENT BOTS WITH IOT ON AZURE @FanieReynders
2. A little bit about me @FanieReynders http://reynders.co http://sdn.nl http://fixxup.nl
3. Agenda Azure Bots Artificial Intelligence IoT
4. BOTS
5. CaaS Conversation as a Service
6. Microsoft Bot Framework BotBuilder SDK
7. ByoB Bring your own Bot Your Bot
8. Tools Visual Studio Template • http://aka.ms/bf-bc-vstemplate Bot Framework Emulator • http://aka.ms/bf-bc-emulator
9. Message routing
10. Message structure { "type": "Message", "id": "23IG9F0yL0S", "conversationId": "GZxAXM39a6jdG0n2HQF5TEYL1vGgTG853w2259xn5VhGfs", "created": "2016-03-22T04:19:11.2100568Z", "language": "en", "text": "Hello SDN", "attachments": [ ], "from": { ... }, "to": { ... }, "replyToMessageId": "7TvTPn87HlZ", "participants": [ ... ], "totalParticipants": 2, "mentions": [ ], "channelConversationId": "12345", "hashtags": [ ], "channelData": { ... }, "botConversationData": { ... }, "botPerUserInConversationData": { ... }, "botUserData": { ... }, }
11. Bot Builder Dialog • Conversational • Stateful • LUIS integration FormFlow • Guided conversations • Step navigations • Validations
12. Demo MS Bot Connector
13. ARTIFICIAL INTELLIGENCE
14. Microsoft Cognitive Services Vision • Computer Vision • Emotion • Face • Video Speech • Bing Speech • Custom Recognition • Speaker Recognition Language • Bing Spell Check • Language Understanding • Linguistic Analysis • Text Analysis • WebML Knowledge • Academic • Entity Linking • Knowledge Exploration • Recommendations Search • Bing Web Search • Bing Image Search • Bing Video Search • Bing News Search • Bing Autosuggest
15. Microsoft Cognitive Services Vision • Computer Vision • Emotion • Face • Video Speech • Bing Speech • Custom Recognition • Speaker Recognition Language • Bing Spell Check • Language Understanding • Linguistic Analysis • Text Analysis • WebML Knowledge • Academic • Entity Linking • Knowledge Exploration • Recommendations Search • Bing Web Search • Bing Image Search • Bing Video Search • Bing News Search • Bing Autosuggest
16. Computer Vision • Analyze images • Recognize celebrities • Read text in images • Generate a thumbnail
17. Emotion • Images • Videos • Multiple faces
18. Language Understanding (LUIS) • Create language understanding models • Add conversational intelligence • Pre-built, world class models (Bing & Cortana) • Action fulfillment capabilities • Deploy to HTTP • Activate on any device • Maintain with ease
19. Demo Computer Vison API Emotion API Language Understanding Intelligent Service MS Bot Connector
20. IOT
21. Demo Computer Vison API Emotion API Language Understanding Intelligent Service MS Bot Connector Azure IoT Hub Streaming Analytics
22. THANK YOU Questions? Answers? Comments?