Team13: Ciki Ciki Bam Bam (Komei Lab) XXXXXX Yuiga Wada YYYYYYY Ryosuke Korekata ZZZZZZZ Takumi Komatsu Sugar Visualizer Mixed Reality 

Background: Many people suffer from diabetes worldwide ● Avoiding excessive sugar intake is important to prevent worsening diabetes ● If you realize the sugar content, it is easier to make the right choice - 2 - https://www.cureus.com/articles/3625-drug-targets-for-oxidative-podocyte-injury-in-diabetic-nephropathy Global prevalence of diabetes (2014) 

Idea Sketch: Sugar Visualizer - 3 - x y z MONSTER: 0.99 Tea: 0.91 ← Sugar cube ● Visualize the amount of sugar in a beverage as sugar cubes ! “What a sugary drink !?” after drinking 

Demo1: Coca-Cola (Sugar Cubes ≒ 13) - 4 - 

Demo2: Red Bull (Sugar Cubes ≒ 5) - 5 - ← The same AR marker can be used for any beverage 

Demo3: Tea (Sugar Cubes ≒ 0) - 6 - 

Implementation: 4 steps - 7 - 1. detects each drinks using object detection 1. calculates world coordinates of detected drinks by bounding boxes 1. gets the number of sugar cubes contained from the detected labels 1. recognizes the AR marker & stack the sugar cubes next to each drinks virtually ! ①, ②, ③ ④ Client Server 

① Object detection: fine-tuned YOLOv5 ● collected original dataset including 3 beverages ○ annotated with ● GPU: GeForce RTX 3090 - 8 - Sample predictions label “cola” “redbull” “tea” images 30 30 30 Statistics of collected dataset Learning curves 

②Algorithm (Python-Side): Transform to world coordinates - 9 - ● We want is Moore–Penrose inverse We used (x,z) of the bounding box and substitute y coordinate of the sugar for that of the AR marker How to estimate locations of the sugar? https://www.researchgate.net/figure/The-extrinsic-parameters-consisting-of-a-translation-vector-t-and-a-rotation-matrix-R_fig20_312376910 

④ Algorithm (C#-Side): Stack the sugar cubes - 10 - ● prepares 4 different ways of stacking the sugar with the number of sugar ● divides the number of sugar cubes N into binary numbers ○ example: 5 → 1 + 4, 14 → 2 + 4 + 8 }1-15 sugars can be stacked in a pyramid shape 

AR Marker: ArUco marker + Paripi Komei - 11 - ● ArUco marker for OpenCV low detection accuracy in Unity ● + distinctive logos make it easy for Unity to detect! ↑ can be easily used by both OpenCV and Unity 👀 improved 

Conclusion: Ciki Ciki Bam Bam (Komei Lab) Background ✓ Avoiding excessive sugar intake is important to prevent worsening diabetes Proposal ✓ Object detection of beverages using ✓ Coordinates transformation by rotation matrix ✓ 3D falling motion of sugar cubes Result ✓ Real-time display of sugar cubes using a single AR marker - 12 -