• What are Digital Characters and Avatars? • Applications • Tasks :: Create a Character • Tasks :: Make it Intelligent • Tasks :: Make it Alive • Summary Agenda
Applications Media & Entertainment Digital Twins Interactive Avatars Communication Games Movies VFX / CGI Digital Factory / City Drive Simulation Synthetic Data for AI Training Digital Assistants Education Healthcare Social Media Avatars Video Conferences Enterprise Telepresence
Tasks in Digital Avatars Domain ✏️ Create a Character 🧠 Make it Intelligent Using Reconstruction Using Generative AI Conversational AI Environment Awareness
Tasks in Digital Avatars Domain ✏️ Create a Character 🧠 Make it Intelligent 😃 Make it Alive Using Reconstruction Using Generative AI Conversational AI Environment Awareness Rendering / Materials Animation / Physics
✏️ Creating Digital Avatars Source: Chan et al, “Efficient Geometry-aware 3D Generative Adversarial Networks” Input Photo Inverted GAN Output 3D Reconstruction by GAN Inversion
🧠 Environment Awareness 3D Scene Understanding Planning and Navigation 3D Object Classification Scene Parsing / Segmentation Building a Scene Graph Affordance Detection Task-Based Semantic Planning Route Construction “Take the cup from the table and put in on the shelf next to the TV” Go To Table #2 Take Cup #5 Go To Shelf #3 Put Shelf #3 1 2 3 4 laptop bed table chest 1 3 4 2
😃 Animation Audio-Driven Gesture and Body Animation Generator or Motion Matching Speech Audio Emotion / Style Retargeting Template Skeleton Animation Final Character Final Character Source: NVIDIA Omniverse Audio2Gesture
Animation AI Conversational AI Vision AI Recommender AI Audio2Face Audio2Emotion NLP ASR, TTS Computer Vision Video Analytics Recommender Systems NVIDIA Omniverse Avatar Cloud Engine Cloud-Based AI Microservices
Summary ✏️ Create a Character 🧠 Make it Intelligent 😃 Make it Alive Using Reconstruction Using Generative AI Face / Body / Texture Conversation Speech Vision 3D Scene Understanding Planning and Navigation Rendering Materials Physics Facial Animation Gestures Locomotion Audio / Video-based Interaction with Objects Task-Based Animation