【授業スライド】Sugar Visualizer

慶應義塾⼤学 B4 和田唯我 / Yuiga Wada
授業『複合現実感』のスライドです. (院先取り)

- About me
- Blog

Transcript

1. Team13: Ciki Ciki Bam Bam (Komei Lab)
   XXXXXX Yuiga Wada
   YYYYYYY Ryosuke Korekata
   ZZZZZZZ Takumi Komatsu
   Sugar Visualizer
   Mixed Reality
   

   View Slide

2. Background: Many people suffer from diabetes worldwide
   ● Avoiding excessive sugar intake is important to prevent worsening diabetes
   ● If you realize the sugar content,
   it is easier to make the right choice
   - 2 -
   https://www.cureus.com/articles/3625-drug-targets-for-oxidative-podocyte-injury-in-diabetic-nephropathy
   Global prevalence of diabetes (2014)
   

   View Slide

3. Idea Sketch: Sugar Visualizer
   - 3 -
   x
   y
   z
   MONSTER: 0.99
   Tea: 0.91
   ← Sugar cube
   ● Visualize the amount of sugar
   in a beverage as sugar cubes !
   “What a sugary drink !?”
   after
   drinking
   

   View Slide

4. Demo1: Coca-Cola (Sugar Cubes ≒ 13)
   - 4 -
   

   View Slide

5. Demo2: Red Bull (Sugar Cubes ≒ 5)
   - 5 -
   ← The same AR marker
   can be used
   for any beverage
   

   View Slide

6. Demo3: Tea (Sugar Cubes ≒ 0)
   - 6 -
   

   View Slide

7. Implementation: 4 steps
   - 7 -
   1. detects each drinks using object detection
   1. calculates world coordinates of detected drinks by bounding boxes
   1. gets the number of sugar cubes contained from the detected labels
   1. recognizes the AR marker & stack the sugar cubes next to each drinks virtually !
   ①, ②, ③
   ④
   Client Server
   

   View Slide

8. ① Object detection: fine-tuned YOLOv5
   ● collected original dataset including 3 beverages
   ○ annotated with
   ● GPU: GeForce RTX 3090
   - 8 -
   Sample predictions
   label “cola” “redbull” “tea”
   images 30 30 30
   Statistics of collected dataset
   Learning curves
   

   View Slide

9. ②Algorithm (Python-Side): Transform to world coordinates
   - 9 -
   ● We want
   is Moore–Penrose inverse
   We used (x,z) of the bounding box and substitute
   y coordinate of the sugar for that of the AR marker
   How to estimate locations of the sugar?
   https://www.researchgate.net/figure/The-extrinsic-parameters-consisting-of-a-translation-vector-t-and-a-rotation-matrix-R_fig20_312376910
   

   View Slide

10. ④ Algorithm (C#-Side): Stack the sugar cubes
    - 10 -
    ● prepares 4 different ways of stacking the sugar
    with the number of sugar
    ● divides the number of sugar cubes N into binary numbers
    ○ example: 5 → 1 + 4, 14 → 2 + 4 + 8
    }1-15 sugars can be
    stacked in a pyramid
    shape
    

    View Slide

11. AR Marker: ArUco marker + Paripi Komei
    - 11 -
    ● ArUco marker for OpenCV
    low detection accuracy in Unity
    ● + distinctive logos
    make it easy for Unity to detect!
    ↑ can be easily used by both OpenCV and Unity 👀
    improved
    

    View Slide

12. Conclusion: Ciki Ciki Bam Bam (Komei Lab)
    Background
    ✓ Avoiding excessive sugar intake is important to prevent worsening diabetes
    Proposal
    ✓ Object detection of beverages using
    ✓ Coordinates transformation by rotation matrix
    ✓ 3D falling motion of sugar cubes
    Result
    ✓ Real-time display of sugar cubes using a single AR marker
    - 12 -
    

    View Slide