【授業スライド】Sugar Visualizer

慶應義塾⼤学 B4 和田唯我 / Yuiga Wada
授業『複合現実感』のスライドです. (院先取り)

- About me
- Blog

Transcript

1. Team13: Ciki Ciki Bam Bam (Komei Lab) XXXXXX Yuiga Wada YYYYYYY Ryosuke Korekata ZZZZZZZ Takumi Komatsu Sugar Visualizer Mixed Reality
2. Background: Many people suffer from diabetes worldwide • Avoiding excessive sugar intake is important to prevent worsening diabetes • If you realize the sugar content, it is easier to make the right choice - 2 - https://www.cureus.com/articles/3625-drug-targets-for-oxidative-podocyte-injury-in-diabetic-nephropathy Global prevalence of diabetes (2014)
3. Idea Sketch: Sugar Visualizer - 3 - x y z MONSTER: 0.99 Tea: 0.91 ← Sugar cube • Visualize the amount of sugar in a beverage as sugar cubes ! “What a sugary drink !?” after drinking
4. Demo1: Coca-Cola (Sugar Cubes ≒ 13) - 4 -
5. Demo2: Red Bull (Sugar Cubes ≒ 5) - 5 - ← The same AR marker can be used for any beverage
6. Demo3: Tea (Sugar Cubes ≒ 0) - 6 -
7. Implementation: 4 steps - 7 - 1. detects each drinks using object detection 1. calculates world coordinates of detected drinks by bounding boxes 1. gets the number of sugar cubes contained from the detected labels 1. recognizes the AR marker & stack the sugar cubes next to each drinks virtually ! ①, ②, ③ ④ Client Server
8. ① Object detection: fine-tuned YOLOv5 • collected original dataset including 3 beverages ◦ annotated with • GPU: GeForce RTX 3090 - 8 - Sample predictions label “cola” “redbull” “tea” images 30 30 30 Statistics of collected dataset Learning curves
9. ②Algorithm (Python-Side): Transform to world coordinates - 9 - • We want is Moore–Penrose inverse We used (x,z) of the bounding box and substitute y coordinate of the sugar for that of the AR marker How to estimate locations of the sugar? https://www.researchgate.net/figure/The-extrinsic-parameters-consisting-of-a-translation-vector-t-and-a-rotation-matrix-R_fig20_312376910
10. ④ Algorithm (C#-Side): Stack the sugar cubes - 10 - • prepares 4 different ways of stacking the sugar with the number of sugar • divides the number of sugar cubes N into binary numbers ◦ example: 5 → 1 + 4, 14 → 2 + 4 + 8 }1-15 sugars can be stacked in a pyramid shape
11. AR Marker: ArUco marker + Paripi Komei - 11 - • ArUco marker for OpenCV low detection accuracy in Unity • + distinctive logos make it easy for Unity to detect! ↑ can be easily used by both OpenCV and Unity 👀 improved
12. Conclusion: Ciki Ciki Bam Bam (Komei Lab) Background ✓ Avoiding excessive sugar intake is important to prevent worsening diabetes Proposal ✓ Object detection of beverages using ✓ Coordinates transformation by rotation matrix ✓ 3D falling motion of sugar cubes Result ✓ Real-time display of sugar cubes using a single AR marker - 12 -