Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Detect Body and Hand Pose with Vision
Koki Hirokawa
July 01, 2020
Technology
0
710
Detect Body and Hand Pose with Vision
Koki Hirokawa
July 01, 2020
Tweet
Share
More Decks by Koki Hirokawa
See All by Koki Hirokawa
ABEMAのリアーキテクチャ
kokihirokawa
5
3.5k
Other Decks in Technology
See All in Technology
400種類のWeb APIをサポートしているデータパイプラインツールにおけるWeb APIとの共存戦略
cdataj
0
160
OpenShiftのリリースノートを整理してみた
loftkun
2
260
PCL (Point Cloud Library)の基本となぜ点群処理か_2023年_第2版.pdf
cvmlexpertguide
0
140
AWS re:Invent 2022で発表された新機能を試してみた ~Cloud OperationとSecurity~ / New Cloud Operation and Security Features Announced at AWS reInvent 2022
yuj1osm
1
180
Astroで始める爆速個人サイト開発
takanorip
12
8.5k
Hasuraの本番運用に向けて
nori3tsu
0
290
プログラミング支援AI GitHub Copilot すごいの話
moyashi
0
290
Akiba-dot-SaaS-ExtraHop
sakaitakeshi
1
110
20230121_データ分析系コミュニティ_サテライト企画
doradora09
0
520
Airdrop for Open Source Projects
epicsdao
0
490
地方自治体業務あるある ーアナログ最適化編-
y150saya
1
120
OCI DevOps 概要 / OCI DevOps overview
oracle4engineer
PRO
0
480
Featured
See All Featured
Statistics for Hackers
jakevdp
785
210k
Web Components: a chance to create the future
zenorocha
304
40k
Design by the Numbers
sachag
271
18k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
31
20k
How to Ace a Technical Interview
jacobian
270
21k
The Power of CSS Pseudo Elements
geoffreycrofte
52
4.3k
Bash Introduction
62gerente
601
210k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
152
13k
JazzCon 2018 Closing Keynote - Leadership for the Reluctant Leader
reverentgeek
175
9.1k
Designing the Hi-DPI Web
ddemaree
273
32k
Building Flexible Design Systems
yeseniaperezcruz
314
35k
How STYLIGHT went responsive
nonsquared
89
4.2k
Transcript
Koki Hirokawa Detect Body and Hand Pose With Vision CA.swift
#12
ኍ߉ل / Koki Hirokawa / ͽʔͻΖ GitHub -> KokiHirokawa Twitter
-> @pihero13
ਓؒͷࢹ֮γεςϜʹ͓͍ͯߦ͑ΔλεΫΛίϯϐϡʔλͰ࣮ݱ͢Δ͜ͱΛٻ͢Δݚڀ Computer Vision ͱʁ
iOS 11 Ͱಋೖ͞ΕͨϑϨʔϜϫʔΫ إͷϥϯυϚʔΫݕग़ɾςΩετݕग़ɾόʔίʔυೝࣝɾը૾ೝࣝ ྨମݕग़ʹಠࣗͷ Core ML ϞσϧΛ༻͢Δ͜ͱՄೳ iOS /
Vision ͱʁ
ମͱखͷϙʔζΛݕͰ͖ΔΑ͏ʹͳͬͨ Detect Body and Hand Pose with Vision
͏ࣗࡱΓ͍Βͳ͍ ಛఆͷδΣενϟʔΛݕͯ͠γϟολʔΛΔ ׆༻ྫᶃ
δΣενϟʔͷྨ ֆจࣈΛΦʔόʔϨΠ ׆༻ྫᶄ
࣮ࡍͲͷ͘Β͍ͷύϑΥʔϚϯε͕ ग़ͤΔͷʁ
ͪ͜ΒαϯϓϧΞϓϦͱͯ͠͞Ε͍ͯ·͢ ۭதͰจࣈΛॻ͚ΔΞϓϦͰݕূ
ྲྀੴʹֆݫ͍͠Μ͡Όͳ͍ʁ
ԿΛඳ͍͍ͯΔ͔νϟοτͰͯͯΈͯԼ͍͞
ࢥͬͨΑΓઢ͕៉ྷʹॻ͚Δʂ ࢦΛ͍Α͘ಈ͔ͯ͠ݕͯ͘͠ΕΔʂ Beta ൛͚ͩͲɺ҆ఆ͍ͯ͠Δɻ ͨͩɺશͯͷࢦ͕ݟ͑ΔΑ͏ʹ͠ͳ͍ͱ্ख͘ݕͯ͘͠Εͳ͍ɻ ମݧͯ͠Έͨॴײ
ը໘ͷͰ্ख͘ػೳ͠ͳ͍͜ͱ͕͋Δ Χϝϥͷࢹํʹରͯ͠ฒߦͳखݕ͕͍͠ खାΛ͍ͯ͠Δ߹ݕ͕͍͠ Λखͱצҧ͍ͯ͠͠·͏͜ͱ͕͋Δ σόΠεݻఆ͞ΕΔඞཁ͕͋Δ ਫ਼ΛߴΊΔͨΊͷߟྀࣄ߲
࣮໘ͷʹҠΓ·͢
Request ͷ࡞ Handler ͷ࡞ & Handler ʹ Request Λ͢ ݁Ռͱͯ͠
Observation Λऔಘ͢Δ λεΫͷྲྀΕ
var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler( cmSampleBuffer: sampleBuffer,
orientation: .up, options: [:] ) try! handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation
var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler( cmSampleBuffer: sampleBuffer,
orientation: .up, options: [:] ) try! handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation
var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler( cmSampleBuffer: sampleBuffer,
orientation: .up, options: [:] ) try! handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation
var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler( cmSampleBuffer: sampleBuffer,
orientation: .up, options: [:] ) try! handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation
public func captureOutput( _ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from
connection: AVCaptureConnection ) { var handPoseRequest = VNDetectHumanHandPoseRequest() let handler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer, …) try handler.perform([handPoseRequest]) let observation = handPoseRequest.results?.first as! VNRecognizedPointsObservation } AVCaptureVideoDataOutputSampleBufferDelegate AVCaptureSession AVCaptureVideoDataOutput
֤ϥϯυϚʔΫͷऔಘ - VNRecognizedPointGroupKey Thumb Index Finger Middle Finger Ring Finger
Little Finger
֤ϥϯυϚʔΫͷऔಘ - VNRecognizedPoint
VNRecognizedPoint • ࠲ඪͷ৴པ • x ࠲ඪ • y ࠲ඪ •
ϥϕϧ
let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb) let thumbTipPoints = thumbPoints[.handLandmarkKeyThumbTIP]!
let x = thumbTipPoints.location.x let y = thumbTipPoints.location.y
let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb) let thumbTipPoints = thumbPoints[.handLandmarkKeyThumbTIP]!
let x = thumbTipPoints.location.x let y = thumbTipPoints.location.y
let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb) let thumbTipPoints = thumbPoints[.handLandmarkKeyThumbTIP]!
let x = thumbTipPoints.location.x let y = thumbTipPoints.location.y
None
VNDetectHumanHandPoseRequest ͷ maximumHandCount Λௐ͢Δ͜ͱͰෳͷख Λݕ͢Δ͜ͱ͕Մೳ σϑΥϧτ 2 ύϥϝʔλͷΛେ͖͘͢Εͦͷ͚ͩϨΠςϯγʹӨڹ͕ੜ͡Δ ͘ઃఆ͢ΕɺͦͷҎ্ͷखݕग़͞Εͳ͘ͳΓύϑΥʔϚϯε্͕Δ ϨΠςϯγʹؔ͢Δҙࣄ߲
BODY POSE ΄΅ಉ͡ॲཧʹͳΓ·͢
ARKit ͷϞʔγϣϯΩϟϓνϟʔͱͷҧ͍
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
ग़య ʮDetect Body and Hand Pose with Visionʯ ؔ࿈ηογϣϯ ʮCreate
ML Λ༻ͨ͠ΞΫγϣϯྨࢠͷ࡞ʯ ʮComputer Vision API ͷ୳ٻʯ