Artificial Perception for 3D and kinesthetic senses

Kudan briefing deck
- Artificial Perception
- SLAM algorithms

Transcript

1. Artificial Perception “3D and kinesthetic senses” BRIEFING DOCUMENT
2. Kudan = Right-brained “Artificial perception” Artificial Intelligence (= left brain) Stored big-data based “learning” for semantic extraction • Pattern recognition • Semantic/ language processing Camera / image sensor (= eye / retina) Physical sensing input Algorithms software to perceive 3D and kinesthetic senses, as the complementary pair to Artificial Intelligence Artificial Perception (= right brain) Real-time sensing “instinct” for geo- metric acquisition • Space recognition • Sense of control/ motion
3. Tech overview Kudan’s Artificial Perception (AP) tech is capable to perceive and interpret 3D and kinesthetic senses from all the related sensor inputs 3 Sense Sensor Vision Depth Inertia § Camera § ToF § LiDAR § IMU § Gyroscope Mechanical odometry § Motor § Actuator Position § Beacon § GPS Kudan’s capabilities (AP) § Spatial mapping/ reconstruction § Position tracking/ odometry § 3D recognition § Orientation/ posture tracking § Scale detection § Azimuth/ vertical detection Sophisticated algorithms to perceive rich sensor data Advanced interfaces to interpret and integrate simple sensor data Rich data Simple data
4. Tech highlight: KudanSLAM 4 Input Output Demo movie: https://www.youtube.com/watch?v=Yif99vjvFTg § 2D images from multiple view points along the moves of the image sensor § Real-time 6DoF trajectory of the view point § Real-time 3D map of the feature points Kudan’s SLAM (Simultaneous Localization and Mapping) enables real-time 3D mapping and position tracking
5. Applications Kudan’s Artificial Perception has been adapted in break-through applications highly scalable with significant industrial/commercial opportunities 5 https://www.youtube.com/watch?v=GDJ6aFsPWN4&t=14s https://www.youtube.com/watch?v=EE-QvVTMTdY AR/VR Robotics/ drone Auto/ transport https://www.youtube.com/watch?v=g2SFJGDz9cQ § Indoor and outdoor AR/MR (mobile/glasses) § VR (mobile/HMD) § Industrial/personal robot § Robot cleaner § Drone § ADAS § Autonomous driving § AGV system (automated guided vehicle) Application examples Demo movies
6. Technical features of KudanSLAM Kudan licenses fully functional SLAM algorithms, provided as well as a modular framework for use in other solutions, to be embedded on chipset to versatilely support future devices 6 Versatile Modularized Practicable • Portable to any processing architecture • Flexible to camera setup and peripherals • Configurable to required use cases • A stack of 50+ modular frameworks (e.g. point matching, image blurring) as plug-in for other solutions • Fast and constant consumption • Accurate mapping and tracking • Robust in unpredictable movements and re- localization
7. Technical features of KudanSLAM (1/3): versatility KudanSLAM is a versatile technology which is applicable to required hardware setups and use cases 7 Features Setup examples that KudanSLAM is applicable Flexibility to camera Integration with sensors Capability for cross-hardware § Camera configuration: mono, stereo, multiple § Shutter: rolling, global § Lens: fisheye, omnidirectional § Tracking sensor: IMU, GPS, Beacon § Depth sensor: RGB-D, ToF, LiDAR § Mapping and Localisation across different hardware: cameras, lenses, depth sensors § Allocation of computing and data flow: local, edge, cloud Portability across platform § OS: Android, iOS, Linux, OSX, Windows and any other required § Processing architecture: CPU (ARM/Intel), GPU, DSP and any other required Configurability to use cases § Performance: accuracy, speed, robustness, data size § Output: position (localization/re-localization), point cloud density (sparse/dense)
8. Technical features of KudanSLAM (2/3): practicality KudanSLAM performs with high speed/low consumption, high accuracy as the ONLY commercial SLAM algorithm 8 1: Test performed on a Macbook Pro (quad 2.2GHz i7) 2: Default ORB-SLAM2 settings for EuRoC 3: Identical compiler version and optimization settings (LLVM -03) Tracking time; ms Accuracy; RMSE mm KudanSLAM vs ORB-SLAM2 on EuRoC datasets1, 2, 3 KudanSLAM ORB-SLAM2 Sequence 55 39 91 62 49 53 56 22 19 43 15 17 19 21 MH01 MH03 MH05 V1_01 V1_01 V2_02 V1_02 62 61 65 62 62 69 67 7 7 7 7 7 7 7
9. Technical features of KudanSLAM (3/3): modulalization KudanSLAM has fully versatile modular frameworks modules, can be used as plug-in in other solutions 9 KudanSLAM High level frameworks Multi-threaded bundle adjustment Re- localization Point tracking Pose estimation Loop detection and closure Intermediate level frameworks Point matching Plane finding Stereo matching RANSAC/ PROSAC Epipolar triangulation low frameworks Descriptor generation Image blurring Feature detection Image warping Sub-pixel template matching 10+ 15+ 25+ 50+ … … …
10. 10 Communication Navigation Entertainment Education Healthcare Security Construction Industry Transportation Mobile Wearable Drone Robotics Automotive CCTV Service Hardware/ system on chip IP Algorithm architecture Processing architecture § S/W Libraries § H/W IPs § S/W Libraries § H/W IPs Future vision Kudan architecture of Artificial Perception algorithms will be the fundamental technology platform, which is cross-device at the bottom of the industry to support all the vision related solutions