Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Detect Body and Hand Pose with Vision
Search
Koki Hirokawa
July 01, 2020
Technology
980
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Detect Body and Hand Pose with Vision
Koki Hirokawa
July 01, 2020
Transcript
32 slides
Koki Hirokawa Detect Body and Hand Pose With Vision CA.swift
#12
ኍ߉ل / Koki Hirokawa / ͽʔͻΖ GitHub -> KokiHirokawa Twitter
-> @pihero13
ਓؒͷࢹ֮γεςϜʹ͓͍ͯߦ͑ΔλεΫΛίϯϐϡʔλͰ࣮ݱ͢Δ͜ͱΛٻ͢Δݚڀ Computer Vision ͱʁ
iOS 11 Ͱಋೖ͞ΕͨϑϨʔϜϫʔΫ إͷϥϯυϚʔΫݕग़ɾςΩετݕग़ɾόʔίʔυೝࣝɾը૾ೝࣝ ྨମݕग़ʹಠࣗͷ Core ML ϞσϧΛ༻͢Δ͜ͱՄೳ iOS /
Vision ͱʁ
ମͱखͷϙʔζΛݕͰ͖ΔΑ͏ʹͳͬͨ Detect Body and Hand Pose with Vision
͏ࣗࡱΓ͍Βͳ͍ ಛఆͷδΣενϟʔΛݕͯ͠γϟολʔΛΔ ׆༻ྫᶃ
δΣενϟʔͷྨ ֆจࣈΛΦʔόʔϨΠ ׆༻ྫᶄ
࣮ࡍͲͷ͘Β͍ͷύϑΥʔϚϯε͕ ग़ͤΔͷʁ
ͪ͜ΒαϯϓϧΞϓϦͱͯ͠͞Ε͍ͯ·͢ ۭதͰจࣈΛॻ͚ΔΞϓϦͰݕূ
ྲྀੴʹֆݫ͍͠Μ͡Όͳ͍ʁ
ԿΛඳ͍͍ͯΔ͔νϟοτͰͯͯΈͯԼ͍͞
ࢥͬͨΑΓઢ͕៉ྷʹॻ͚Δʂ ࢦΛ͍Α͘ಈ͔ͯ͠ݕͯ͘͠ΕΔʂ Beta ൛͚ͩͲɺ҆ఆ͍ͯ͠Δɻ ͨͩɺશͯͷࢦ͕ݟ͑ΔΑ͏ʹ͠ͳ͍ͱ্ख͘ݕͯ͘͠Εͳ͍ɻ ମݧͯ͠Έͨॴײ
ը໘ͷͰ্ख͘ػೳ͠ͳ͍͜ͱ͕͋Δ Χϝϥͷࢹํʹରͯ͠ฒߦͳखݕ͕͍͠ खାΛ͍ͯ͠Δ߹ݕ͕͍͠ Λखͱצҧ͍ͯ͠͠·͏͜ͱ͕͋Δ σόΠεݻఆ͞ΕΔඞཁ͕͋Δ ਫ਼ΛߴΊΔͨΊͷߟྀࣄ߲
࣮໘ͷʹҠΓ·͢
Request ͷ࡞ Handler ͷ࡞ & Handler ʹ Request Λ͢ ݁Ռͱͯ͠
Observation Λऔಘ͢Δ λεΫͷྲྀΕ
var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler( cmSampleBuffer: sampleBuffer,
orientation: .up, options: [:] ) try! handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation
var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler( cmSampleBuffer: sampleBuffer,
orientation: .up, options: [:] ) try! handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation
var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler( cmSampleBuffer: sampleBuffer,
orientation: .up, options: [:] ) try! handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation
var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler( cmSampleBuffer: sampleBuffer,
orientation: .up, options: [:] ) try! handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation
public func captureOutput( _ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from
connection: AVCaptureConnection ) { var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer, …) try handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation } AVCaptureVideoDataOutputSampleBufferDelegate AVCaptureSession AVCaptureVideoDataOutput
֤ϥϯυϚʔΫͷऔಘ - VNRecognizedPointGroupKey Thumb Index Finger Middle Finger Ring Finger
Little Finger
֤ϥϯυϚʔΫͷऔಘ - VNRecognizedPoint
VNRecognizedPoint • ࠲ඪͷ৴པ • x ࠲ඪ • y ࠲ඪ •
ϥϕϧ
let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb) let thumbTipPoints = thumbPoints[.handLandmarkKeyThumbTIP]!
let x = thumbTipPoints.location.x let y = thumbTipPoints.location.y
let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb) let thumbTipPoints = thumbPoints[.handLandmarkKeyThumbTIP]!
let x = thumbTipPoints.location.x let y = thumbTipPoints.location.y
let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb) let thumbTipPoints = thumbPoints[.handLandmarkKeyThumbTIP]!
let x = thumbTipPoints.location.x let y = thumbTipPoints.location.y
None
VNDetectHumanHandPoseRequest ͷ maximumHandCount Λௐ͢Δ͜ͱͰෳͷख Λݕ͢Δ͜ͱ͕Մೳ σϑΥϧτ 2 ύϥϝʔλͷΛେ͖͘͢Εͦͷ͚ͩϨΠςϯγʹӨڹ͕ੜ͡Δ ͘ઃఆ͢ΕɺͦͷҎ্ͷखݕग़͞Εͳ͘ͳΓύϑΥʔϚϯε্͕Δ ϨΠςϯγʹؔ͢Δҙࣄ߲
BODY POSE ΄΅ಉ͡ॲཧʹͳΓ·͢
ARKit ͷϞʔγϣϯΩϟϓνϟʔͱͷҧ͍
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
ग़య ʮDetect Body and Hand Pose with Visionʯ ؔ࿈ηογϣϯ ʮCreate
ML Λ༻ͨ͠ΞΫγϣϯྨࢠͷ࡞ʯ ʮComputer Vision API ͷ୳ٻʯ
More Decks by Koki Hirokawa
See All by Koki Hirokawa
KMP の Swift export
kokihirokawa
0
960
SwiftUI移行のためのインプレッショントラッキング基盤の構築
kokihirokawa
0
460
App内課金におけるトラブルを劇的に減らすための取り組み
kokihirokawa
2
1.3k
ABEMAのリアーキテクチャ
kokihirokawa
5
5.6k
Other Decks in Technology
See All in Technology
思考過程から秘密が漏れる:Amazon Bedrock上のKimi K3を5つの方法で攻めたら、25回中22回漏洩した話
yama3133
1
170
少ない人手 限られた予算 それでも守る 消せない・見つける・戻れる オールハザード型BCP on AWS / 20261107 Mitsutoshi Matsuo
shift_evolve
PRO
0
400
E2E Test というハーネス — AI時代の自動テスト民主化
daishukojima
9
4.3k
Kiro CLI 入門 (ターミナルと CI で使う) | 目指せ Kiro マスター 03
inariku
PRO
2
170
[関西Kaggler会 20261113] 続・コミュニティコンペのススメ
nejumi
0
110
小さなエージェントから始めてみよう!実践コンテキストエンジニアリング
iwamot
PRO
0
130
DevOps Agentにあらゆるセキュリティインシデントのトリアージを任せた件
jirtosterone
1
170
AIネイティブ組織の作り方。1人のPOとAIから始める、自社での実験
ama_ch
2
1.1k
Frontier Teams — Amazon は AI で「開発のやり方」をどう変えたのか
kmotohas
0
280
Go初心者のクラウドエンジニアが、デプロイを通して旨味を知っていく話
ryudai
1
230
仕様と現実の間で落とし所を探るリモートMCPの認証・認可 — Sansanの認証基盤における課題と実践
sansantech
PRO
2
330
おうち環境でも「なるべく止めたくない!」 ― おうち環境の機器冗長化の理想と現実
yuutosi
1
500
Featured
See All Featured
<Decoding/> the Language of Devs - We Love SEO 2024
nikkihalliwell
1
380
Claude Code どこまでも/ Claude Code Everywhere
nwiizo
68
60k
Lessons Learnt from Crawling 1000+ Websites
charlesmeaden
PRO
1
1.7k
Context Engineering - Making Every Token Count
addyosmani
9
1.3k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.9k
Docker and Python
trallard
47
4.3k
Producing Creativity
orderedlist
PRO
348
41k
How to Align SEO within the Product Triangle To Get Buy-In & Support - #RIMC
aleyda
2
1.9k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
10k
世界の人気アプリ100個を分析して見えたペイウォール設計の心得
akihiro_kokubo
PRO
74
43k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
25
2.2k
Building the Perfect Custom Keyboard
takai
2
960