Upgrade to Pro — share decks privately, control downloads, hide ads and more …

【授業スライド】Sugar Visualizer

【授業スライド】Sugar Visualizer

慶應義塾⼤学 B4 和田唯我 / Yuiga Wada
授業『複合現実感』のスライドです. (院先取り)

- About me
- Blog

Yuiga Wada (和田唯我)

October 10, 2022
Tweet

More Decks by Yuiga Wada (和田唯我)

Other Decks in Technology

Transcript

  1. Team13: Ciki Ciki Bam Bam (Komei Lab)
    XXXXXX Yuiga Wada
    YYYYYYY Ryosuke Korekata
    ZZZZZZZ Takumi Komatsu
    Sugar Visualizer
    Mixed Reality

    View Slide

  2. Background: Many people suffer from diabetes worldwide
    ● Avoiding excessive sugar intake is important to prevent worsening diabetes
    ● If you realize the sugar content,
    it is easier to make the right choice
    - 2 -
    https://www.cureus.com/articles/3625-drug-targets-for-oxidative-podocyte-injury-in-diabetic-nephropathy
    Global prevalence of diabetes (2014)

    View Slide

  3. Idea Sketch: Sugar Visualizer
    - 3 -
    x
    y
    z
    MONSTER: 0.99
    Tea: 0.91
    ← Sugar cube
    ● Visualize the amount of sugar
    in a beverage as sugar cubes !
    “What a sugary drink !?”
    after
    drinking

    View Slide

  4. Demo1: Coca-Cola (Sugar Cubes ≒ 13)
    - 4 -

    View Slide

  5. Demo2: Red Bull (Sugar Cubes ≒ 5)
    - 5 -
    ← The same AR marker
    can be used
    for any beverage

    View Slide

  6. Demo3: Tea (Sugar Cubes ≒ 0)
    - 6 -

    View Slide

  7. Implementation: 4 steps
    - 7 -
    1. detects each drinks using object detection
    1. calculates world coordinates of detected drinks by bounding boxes
    1. gets the number of sugar cubes contained from the detected labels
    1. recognizes the AR marker & stack the sugar cubes next to each drinks virtually !
    ①, ②, ③

    Client Server

    View Slide

  8. ① Object detection: fine-tuned YOLOv5
    ● collected original dataset including 3 beverages
    ○ annotated with
    ● GPU: GeForce RTX 3090
    - 8 -
    Sample predictions
    label “cola” “redbull” “tea”
    images 30 30 30
    Statistics of collected dataset
    Learning curves

    View Slide

  9. ②Algorithm (Python-Side): Transform to world coordinates
    - 9 -
    ● We want
    is Moore–Penrose inverse
    We used (x,z) of the bounding box and substitute
    y coordinate of the sugar for that of the AR marker
    How to estimate locations of the sugar?
    https://www.researchgate.net/figure/The-extrinsic-parameters-consisting-of-a-translation-vector-t-and-a-rotation-matrix-R_fig20_312376910

    View Slide

  10. ④ Algorithm (C#-Side): Stack the sugar cubes
    - 10 -
    ● prepares 4 different ways of stacking the sugar
    with the number of sugar
    ● divides the number of sugar cubes N into binary numbers
    ○ example: 5 → 1 + 4, 14 → 2 + 4 + 8
    }1-15 sugars can be
    stacked in a pyramid
    shape

    View Slide

  11. AR Marker: ArUco marker + Paripi Komei
    - 11 -
    ● ArUco marker for OpenCV
    low detection accuracy in Unity
    ● + distinctive logos
    make it easy for Unity to detect!
    ↑ can be easily used by both OpenCV and Unity 👀
    improved

    View Slide

  12. Conclusion: Ciki Ciki Bam Bam (Komei Lab)
    Background
    ✓ Avoiding excessive sugar intake is important to prevent worsening diabetes
    Proposal
    ✓ Object detection of beverages using
    ✓ Coordinates transformation by rotation matrix
    ✓ 3D falling motion of sugar cubes
    Result
    ✓ Real-time display of sugar cubes using a single AR marker
    - 12 -

    View Slide