Background: Many people suffer from diabetes worldwide ● Avoiding excessive sugar intake is important to prevent worsening diabetes ● If you realize the sugar content, it is easier to make the right choice - 2 - https://www.cureus.com/articles/3625-drug-targets-for-oxidative-podocyte-injury-in-diabetic-nephropathy Global prevalence of diabetes (2014)
Idea Sketch: Sugar Visualizer - 3 - x y z MONSTER: 0.99 Tea: 0.91 ← Sugar cube ● Visualize the amount of sugar in a beverage as sugar cubes ! “What a sugary drink !?” after drinking
Implementation: 4 steps - 7 - 1. detects each drinks using object detection 1. calculates world coordinates of detected drinks by bounding boxes 1. gets the number of sugar cubes contained from the detected labels 1. recognizes the AR marker & stack the sugar cubes next to each drinks virtually ! ①, ②, ③ ④ Client Server
②Algorithm (Python-Side): Transform to world coordinates - 9 - ● We want is Moore–Penrose inverse We used (x,z) of the bounding box and substitute y coordinate of the sugar for that of the AR marker How to estimate locations of the sugar? https://www.researchgate.net/figure/The-extrinsic-parameters-consisting-of-a-translation-vector-t-and-a-rotation-matrix-R_fig20_312376910
④ Algorithm (C#-Side): Stack the sugar cubes - 10 - ● prepares 4 different ways of stacking the sugar with the number of sugar ● divides the number of sugar cubes N into binary numbers ○ example: 5 → 1 + 4, 14 → 2 + 4 + 8 }1-15 sugars can be stacked in a pyramid shape
AR Marker: ArUco marker + Paripi Komei - 11 - ● ArUco marker for OpenCV low detection accuracy in Unity ● + distinctive logos make it easy for Unity to detect! ↑ can be easily used by both OpenCV and Unity 👀 improved
Conclusion: Ciki Ciki Bam Bam (Komei Lab) Background ✓ Avoiding excessive sugar intake is important to prevent worsening diabetes Proposal ✓ Object detection of beverages using ✓ Coordinates transformation by rotation matrix ✓ 3D falling motion of sugar cubes Result ✓ Real-time display of sugar cubes using a single AR marker - 12 -