Besser leben mit Maschinen?!

Ein Microsoft Cognitive Serives Einblick - theoretisch wie praktisch!

Transcript

1. besser leben mit Maschinen?! Manu Rink Technical Evangelist @ Ein Microsoft Cognitive Services Einblick ❤ ❤ ❤ ❤
3. With over 1 billion people with disabilities in the world, we’re passionate about ensuring that our products and services are designed for people of all abilities. https://www.microsoft.com/en-us/accessibility
4. Vision Hearing Cognitive Speech Mobility Situational
5. Video Analyse Bilder beschreiben Emotionen erkennen Gesichter lesen Kogni… was? Oh oh, wie alt bin ich? Was kannst du sehen? Was hast du gesagt? Meine Services… … meine Anwendungen
6. Ein kleines bisschen theorie
7. Kognition ist “Kognition ist die von einem verhaltenssteuernden System ausgeführte Umgestaltung von Informationen. Die Bezeichnung leitet sich ab von lateinisch cognoscere ‚erkennen‘, ‚erfahren‘, oder ‚kennenlernen‘. […] Oft ist mit „Kognition“ das Denken in einem umfassenden Sinne gemeint. Zu den kognitiven Fähigkeiten eines Menschen zählen die Wahrnehmung, die Aufmerksamkeit, die Erinnerung, das Lernen, das Problemlösen, die Kreativität, das Planen, die Orientierung, die Imagination, die Argumentation, die Introspektion, der Wille, das Glauben und einige mehr. Auch Emotionen haben einen wesentlichen kognitiven Anteil. […]” Source: https://de.wikipedia.org/wiki/Kognition
8. Vision Speech Language Knowledge Search Computer Vision API Translator Speech API Language Understanding Intelligent Service Recommendations API Bing Autosuggest API Content Moderator Bing Speech API Bing Spell Check API Knowledge Exploration Service Bing News Search API Video API Speaker Recognition PAI Web Language Model API Entity Linking Intelligence Service API Bing Web Search API Video Indexer Custom Speech Service Text Analytics API Academic Knowledge API Bing Entity Search API Face API Translator Text API QnA Maker API Bing Image Search API Emotion API Linguistic Analysis API Custom Decision Service Bing Video Search API Custom Vision Service Bing Custom Search ... was willst du machen?
9. rein ins geschehen!
10. Sehen Hören Lesen Verstehen Sprache Menschen Stimmung Themen Video Indexer
11. http://videoindexer.ai
12. Kompletter API Support
13. Y A X B Demo Video Indexer
14. Computer Vision API Analyse Describe Thumbnail OCR [
15. Computer Vision API Documentation: https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/ API Reference: https://westus.dev.cognitive.microsoft.com/docs/services/56f91f2d778daf23d8ec6739/operations/56f91f2e778daf14a499e1fa Analyse an image https://$region.api.cognitive.microsoft.com/vision/v1.0[?visualFeatures][&details] URL parameters Visual features Categories, Tags, Description, Faces, ImageType, Color, Adult Details Currently just “Celebrities” is supported Header Content-Type application/json, application/octet-stream, multipart/form-data Ocp-Apim-Subscription-Key Get your key from “My account” at https://www.microsoft.com/cognitive-services/. You might have to create an account first.
16. Computer Vision API JSON result for analyzed image • Details “Celebrities” • Visual features “Categories, Tags, Description, Adult”
17. Computer Vision API 1.0 Describe an image https://$region.api.cognitive.microsoft.com/vision/v1.0/describe[?maxCandidates] URL parameters - maxCandidates Header - see “Analyse” Get Thumbnail https://$region.api.cognitive.microsoft.com/vision/v1.0/generateThumbnail[?width][&height][&smartCropping] URL parameters – width, height, smartCropping Header - see “Analyse” OCR https://$region.api.cognitive.microsoft.com/vision/v1.0/ocr[?language][&detectOrientation ] URL parameters – language, detectOrientation Header - see “Analyse”
18. Computer Vision API On the left: JSON result for OCR On the right: JSON result for Describe
19. Recognition in images Recognition in videos Recognition with rectangles [ Emotion API
20. emotion API beta Documentation: https://azure.microsoft.com/en-us/services/cognitive-services/emotion/ API Reference https://westus.dev.cognitive.microsoft.com/docs/services/5639d931ca73072154c1ce89/operations/563b31ea778daf121cc3a5fa Emotion Recognition https://$region.api.cognitive.microsoft.com/emotion/v1.0/recognize Header Content-Type application/json, application/octet-stream, multipart/form-data Ocp-Apim-Subscription-Key Get your key from “My account” at https://www.microsoft.com/cognitive-services/. You might have to create an account first.
21. emotion API beta JSON result for Emotion Recognition of an image. For every detected face the API returns • the face rectangle • the list of emotions with scores
22. Emotion API beta Emotion Recognition in videos https://$region.api.cognitive.microsoft.com/emotion/v1.0/recognizeinvideo[?outputStyle] URL parameters – outputStyle [aggregate, perFrame] Header - see “Recognition” Result on 202 - video operation status/result as URL Emotion Recognition with Face Rectangles https://$region.api.cognitive.microsoft.com/emotion/v1.0/recognize?faceRectangles={faceRectangles} URL parameters – faceRectangles (left, top, width, height) Header - see “Recognition” Recognition in Video Operation Result https://$region.api.cognitive.microsoft.com/emotion/v1.0/operations/{oid}] URL parameters – oid (URL from Emotion Recognition in videos) Header - see “Recognition” Result: Status of recognition operation. On SUCCEEDED -> JSON can be retrieved from processingResult field. https://www.microsoft.com/cognitive-services/en-us/emotion-api/documentation/howtocallemotionforvideo
23. Face API Detect Verify Identify Find Similar [
24. Face API 1.0 Documentation: https://azure.microsoft.com/en-us/services/cognitive-services/face/ API Reference https://westus.dev.cognitive.microsoft.com/docs/services/563879b61984550e40cbbe8d/operations/563879b61984550f30395236 Detect Faces https://$region.api.cognitive.microsoft.com/face/v1.0/detect[?returnFaceId][&returnFaceLandmarks][&returnFaceAttributes] URL parameters returnFaceId faceId needed if face should later be attached to a person returnFaceLandmarks get position of e.g. eyes, pupils, nose, eyebrows,… returnFaceAttributes get attributes “age, gender, smile, facialHair, headPose, glasses” for a face Header Content-Type application/json, application/octet-stream, multipart/form-data Ocp-Apim-Subscription-Key Get your key from “My account” at https://www.microsoft.com/cognitive-services/. You might have to create an account first.
25. Face API 1.0 JSON result for Face Detection of an image. For every detected face the API returns • the faceId • the list of faceLandmarks • requested attributes of the face
26. Face API 1.0 Find Similar Faces https://$region.api.cognitive.microsoft.com/face/v1.0/findsimilars URL parameters – faceId, faceListId, faceIds, maxNumOfCandidatesReturned, mode [matchPerson, matchFace] Header - see “Detect” Verify a face https://$region.api.cognitive.microsoft.com/face/v1.0/verify Request Body Face2Face Verification: faceId1, faceId2 Face2Person Verification: faceId, personGroupId, personId Header - see “Detect” Identify a face https://$region.api.cognitive.microsoft.com/face/v1.0/identify Request Body – faceIds, personGroupId, maxNumOfCandidatesReturned, confidenceThreshold Header - see “Detect”
27. Face API 1.0 APIs for creating, populating, training and maintaining persons, person groups and face lists
28. Face API 1.0 So..., how exactly does the verification and identification of faces to persons work? [*13] [*14]
29. Y A X B Demo . Cognitive Services
30. Kann man das auch einfach nutzen?
32. Presentation Translator https://translator.microsoft.com/help/presentation-translator/
33. http://translate.it
35. Lasst uns loslegen, und Technologie uns Menschen verstehen lernen! [*vid1] Disney Research baute einen Telepresenzroboter, der menschlich agieren soll. Link to the paper: https://s3-us-west- 1.amazonaws.com/disneyresearch/wp- content/uploads/20160503162533/A-Hybrid- Hydrostatic-Transmission-and-Human-Safe-Haptic- Telepresence-Robot-Paper.pdf
36. wie kann ich nun loslegen?
37. if (! self.haveAzureAccount) { openWebpage(https://azure.microsoft.com ) } else { let pass = Manu.instance.grabCredit(“AzurePass”) self.azureAccount.redeem(pass) } openWebpage(https://azure.microsoft.com/en- us/services/cognitive-services) self.doEpicShit()
38. Merci :) Manu Rink Technical Evangelist marink@microsoft.com @codeprincess github.com/codeprincess says