Slide 1
Slide 1 text
i O S 1 5 ͷ
ϐΫ ν ϟ ɾ Π ϯ ɾ ϐΫ ν ϟ
Λ ͬ ͯ
͖ ͳ U I Λ ද ࣔ ͞ ͤ Α ͏ ʂ
᠃ ڮ ɹ ྋ
ಈը͚ͩ͡Όͳ͍ʂ
iOSDC Japan 2022
1
Slide 2
Slide 2 text
@ t s u z u k i 8 1 7
🐸ࣗݾհ🐸
Yahoo JAPAN!
PayPayϑϦϚ iOS։ൃ
• Q.UIKitͱSwiftUIͲ͕͖ͬͪʁ
RYO TSUZUKIHASHI
ϛχΤϐιʔυ
2019৽ଔͷ࣌ʹॳΊͯiOSDCʹࢀՃ
͍͔ͭࣗొஃ͍ͨ͠ͱಌΕΛ๊͘Α
͏ʹͳΓɺҎ߱ຖࢀՃ
ͦͯ͠4ͷ2022ʹ೦ئͷొஃ✌
Slide 3
Slide 3 text
@ t s u z u k i 8 1 7
🐸ࣗݾհ🐸
Yahoo JAPAN!
PayPayϑϦϚ iOS։ൃ
• Q.UIKitͱSwiftUIͲ͕͖ͬͪʁ
A. ͲͪΒ͖
• Q. झຯ?
RYO TSUZUKIHASHI
ϛχΤϐιʔυ
2019৽ଔͷ࣌ʹॳΊͯiOSDCʹࢀՃ
͍͔ͭࣗొஃ͍ͨ͠ͱಌΕΛ๊͘Α
͏ʹͳΓɺҎ߱ຖࢀՃ
ͦͯ͠4ͷ2022ʹ೦ئͷొஃ✌
Slide 4
Slide 4 text
@ t s u z u k i 8 1 7
🐸ࣗݾհ🐸
Yahoo JAPAN!
PayPayϑϦϚ iOS։ൃ
• Q.UIKitͱSwiftUIͲ͕͖ͬͪʁ
A. ͲͪΒ͖
• Q. झຯ?
A. ݸਓΞϓϦ։ൃ
• Q. ͖ͳۦಈ։ൃʁ
RYO TSUZUKIHASHI
ϛχΤϐιʔυ
2019৽ଔͷ࣌ʹॳΊͯiOSDCʹࢀՃ
͍͔ͭࣗొஃ͍ͨ͠ͱಌΕΛ๊͘Α
͏ʹͳΓɺҎ߱ຖࢀՃ
ͦͯ͠4ͷ2022ʹ೦ئͷొஃ✌
Slide 5
Slide 5 text
@ t s u z u k i 8 1 7
🐸ࣗݾհ🐸
Yahoo JAPAN!
PayPayϑϦϚ iOS։ൃ
• Q.UIKitͱSwiftUIͲ͕͖ͬͪʁ
A. ͲͪΒ͖
• Q. झຯ?
A. ݸਓΞϓϦ։ൃ
• Q. ͖ͳۦಈ։ൃʁ
A. ςετۦಈ։ൃ
RYO TSUZUKIHASHI
ϛχΤϐιʔυ
2019৽ଔͷ࣌ʹॳΊͯiOSDCʹࢀՃ
͍͔ͭࣗొஃ͍ͨ͠ͱಌΕΛ๊͘Α
͏ʹͳΓɺҎ߱ຖࢀՃ
ͦͯ͠4ͷ2022ʹ೦ئͷొஃ✌
Slide 6
Slide 6 text
͓ॻ͖
•ࣄྫհ
•Picture In Picture֓
•࡞Γͳ͕ΒֶͿPicture In Picture
•౾tips
6
Slide 7
Slide 7 text
ࣄྫհ
• ྲྀΕΔϝϞா
• ҙͷจࣈྻ
• PictIn
• ҙͷը૾
• PIPMP
• ࠶ੜதͷԻָใ
Slide 8
Slide 8 text
ྲྀΕΔϝϞா PictIn PIPMP
Slide 9
Slide 9 text
ϐΫνϟɾΠϯɾ
ϐΫνϟ(PiP)ͱ
P i P Ͱ Ͱ ͖ Δ ͜ ͱ
Slide 10
Slide 10 text
PiPͱ
• ଞͷΞϓϦΛ༻͠ͳ͕ΒFaceTimeΛͬͨΓ
ϏσΦΛࢹௌͨ͠ΓͰ͖Δػೳͷ͜ͱ
• ΟϯυΛϐϯνΦʔϓϯɾϐϯνΫϩʔζ͢
Δ͜ͱͰαΠζΛมߋ͢Δ͜ͱ͕Ͱ͖Δ
• ଞʹҠಈɾඇදࣔɾআ
• ಈըͷ࠶ੜɾఀࢭɾ15sec ͷૣૹΓͱר͖͠
Ҿ༻J1IPOFͷϐΫνϟɾΠϯɾϐΫνϟΛ༻ͨ͠ϚϧνλεΫ
IUUQTTVQQPSUBQQMFDPNKBKQHVJEFJQIPOFJQIDDCEJPT
Slide 11
Slide 11 text
iOS 15͔ΒͷPiP
Կ ͕ Ͱ ͖ Δ Α ͏ ʹ ͳ Δ ͷ ͔
Slide 12
Slide 12 text
iOS 15͔ΒͷPicture In Picture
• ~ iOS 14·Ͱ
PiP͢Δ͜ͱ͕Ͱ͖ΔͷجຊతʹಈըͷΈ
• iOS 15͔Β
• AVPictureInPictureController.ContentSourceͷՃ
→ɹAVSampleBufferDisplayLayerͷίϯςϯπΛPicture In Pictureαϙʔτ
• ͦͷଞPiP༻ͷϝιουͷՃ
Slide 13
Slide 13 text
AVSampleBufferDisplayLayer
• ѹॖɾඇѹॖͷVideoFrameΛදࣔ͢ΔΦϒδΣΫτ (iOS 8 ~)
• CMSampleBufferΛ༩͑Δ͜ͱʹΑͬͯ ಈըΛ࠶ੜ͢Δ͜ͱ͕Ͱ͖Δ
CMSampleBuffer
• ө૾ɺԻɺ͋Δ͍ͦͷ྆ํɺϝσΟΞσʔλΛ࣋ͪӡͿͨΊͷΦϒδΣΫτ
• UIView͔Β࡞ΕΔ
↓
Slide 14
Slide 14 text
AVSampleBufferDisplayLayer
• ѹॖɾඇѹॖͷVideoFrameΛදࣔ͢ΔΦϒδΣΫτ (iOS 8 ~)
• CMSampleBufferΛ༩͑Δ͜ͱʹΑͬͯ ಈըΛ࠶ੜ͢Δ͜ͱ͕Ͱ͖Δ
CMSampleBuffer
• ө૾ɺԻɺ͋Δ͍ͦͷ྆ํɺϝσΟΞσʔλΛ࣋ͪӡͿͨΊͷΦϒδΣΫτ
• UIView͔Β࡞ΕΔ
↓
͖ͳUIΛPicture In Pictureͤ͞Δ͜ͱ͕Ͱ͖Δʂ
Slide 15
Slide 15 text
࣮ફPiP
Ͳ ͷ Α ͏ ʹ P i P Λ ࣮ ݱ ͞ ͤ Δ ͷ ͔
Slide 16
Slide 16 text
࣮ػͷ༻ҙ
• ࣮ػϏϧυͰͳ͍ͱPiP͕ࢼͤͳ͍
γϛϡϨʔλͰಈ͔ͳ͍
Slide 17
Slide 17 text
αϯϓϧϓϩδΣΫτ
• ࣮ػϏϧυ͕Ͱ͖ΕࢼͤΔ https://github.com/tsuzukihashi/sample-pip
• PiPManager
PiPΛཧ͢ΔΫϥεɺγϯάϧτϯ
• UIViewExtension
UIViewΛCMSampleBufferʹม͢ΔExtension
• ContentView
ϝΠϯͷView ϘλϯͳͲΛஔ
• ContentViewModel
PiPManagerΛอ࣋͠Πϕϯτͷड͚͠Λ͢Δ
• PiPContainerView
SwiftUIͰAVSampleBufferDisplayLayerΛඳը͢ΔͨΊͷView
Slide 18
Slide 18 text
࣮ͷલʹඞཁͳઃఆ
• όοΫάϥϯυͰͷΦʔσΟΦ࠶ੜʹରԠ͢ΔΑ͏ʹΞϓϦΛઃఆ͢Δ
CapabilityͰ Background ModesΛՃ
Audio, AirPlay, and Picture in PictureΛνΣοΫ
Slide 19
Slide 19 text
࣮ͷલʹඞཁͳઃఆ
• AudioSessionΛ։࢝͢Δ
PiPΛىಈͰ͖ɺ࠶ੜதͷԻָΛࢭΊͣʹࡁΉΈ߹Θͤ
override init() {
let session = AVAudioSession.sharedInstance()
do {
try session.setCategory(.playAndRecord, mode: .moviePlayback)
try session.setActive(true)
} catch {
print("Failed to set AVAudioSession: \(error)")
}
}
PiPManagerͷॳظԽͰߦ͍ͬͯΔ
Slide 20
Slide 20 text
UIViewΛPiP͢Δ·ͰͷಓͷΓ
6*7JFX
1J1͍ͤͨ͞7JFX
"74BNQMF#V FS
%JTQMBZ-BZFS
6*7JFX
ΞϓϦʹදࣔ͢Δ
"71JDUVSF*O1JDUVSF
$POUSPMMFS
1J1Λཧ͢Δ
Slide 21
Slide 21 text
UIViewΛPiP͢Δ·ͰͷಓͷΓ
6*7JFX
1J1͍ͤͨ͞7JFX
"74BNQMF#V FS
%JTQMBZ-BZFS
6*7JFX
ΞϓϦʹදࣔ͢Δ
"71JDUVSF*O1JDUVSF
$POUSPMMFS
1J1Λཧ͢Δ
PiPManager
ContentView
Slide 22
Slide 22 text
UIViewΛPiP͢Δ·ͰͷಓͷΓ
6*7JFX
1J1͍ͤͨ͞7JFX
"74BNQMF#V FS
%JTQMBZ-BZFS
6*7JFX
ΞϓϦʹදࣔ͢Δ
"71JDUVSF*O1JDUVSF
$POUSPMMFS
1J1Λཧ͢Δ
Slide 23
Slide 23 text
PiP͍ͤͨ͞Viewͷ࡞
• UIView
AVSampleBufferDisplayLayerʹදࣔ͢ΔͨΊͷUIViewΛ࡞Δ
ࠓճ؆қతʹUILabelΛ͏
ݱࡏ࣌ࠁΛදࣔ
Slide 24
Slide 24 text
PiP͍ͤͨ͞View
private let dateLabel: UILabel = {
let label = UILabel()
label.frame = .init(
origin: .zero,
size: .init(
width: UIScreen.main.bounds.width,
height: 120
)
)
label.font = .monospacedSystemFont(ofSize: 24, weight: .medium)
label.textAlignment = .center
label.textColor = .black
label.backgroundColor = .white
return label
}()
PiPManagerͰఆٛ
Slide 25
Slide 25 text
UIViewΛPiP͢Δ·ͰͷಓͷΓ
6*7JFX
1J1͍ͤͨ͞7JFX
"74BNQMF#V FS
%JTQMBZ-BZFS
6*7JFX
ΞϓϦʹදࣔ͢Δ
"71JDUVSF*O1JDUVSF
$POUSPMMFS
1J1Λཧ͢Δ
Slide 26
Slide 26 text
AVSampleBufferDisplayLayerΛඳը͢Δʹ
• CMSampleBuffer͕ඞཁ
enqueue(_:)ϝιουͰCMSampleBufferΛ͢
func enqueue(_ sampleBuffer: CMSampleBuffer)
Slide 27
Slide 27 text
CMSampleBufferΛ࡞
• UIViewΛCMSampleBufferʹม͢ΔExtension
https://gist.github.com/tsuzukihashi/97e379a42e32cc0647aa7a4770d2d9a6
do {
return try CMSampleBuffer(
imageBuffer: pixelBuffer,
formatDescription: formatDescription,
sampleTiming: getCMSampleTimingInfo()
)
} catch {
assertionFailure("Failed to create CMSampleBuffer: \(error)")
return nil
}
Slide 28
Slide 28 text
UIView →CVPixelBuffer
• CVPixelBufferͱ
ϝΠϯϝϞϦʹϐΫηϧΛอ࣋͢ΔΠϝʔδόοϑΝ
ϑϨʔϜΛੜಈըͷѹॖ·ͨղౚɺCoreImageΛར༻͢ΔΞϓϦͰΘΕΔ
• CMPixelBufferCreate
CVPixelBufferΛ࡞͢Δϝιου
https://developer.apple.com/documentation/corevideo/cvpixelbuffer-q2e
Slide 29
Slide 29 text
UIView →CVPixelBuffer
var pixelBuffer: CVPixelBuffer?
let createPixelBufferResult: OSStatus = CVPixelBufferCreate(
kCFAllocatorDefault,
Int(size.width),
Int(size.height),
kCVPixelFormatType_32ARGB,
[
kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue!,
kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue!,
kCVPixelBufferIOSurfacePropertiesKey: [:] as CFDictionary,
] as CFDictionary,
&pixelBuffer
)
Slide 30
Slide 30 text
CVPixelBuffer →CGContext
• CGContextͱ
Quartz 2DͷඳըઌΛද͢
ඳըύϥϝʔλͱϖʔδ্ͷϖΠϯτΛѼઌʹϨϯμϦϯά͢ΔͨΊʹඞཁͳͯ͢ͷ
σόΠεݻ༗ใؚ͕·Ε͍ͯΔ
• Quartz 2Dͱ
iOS, tvOS, macOSͷΞϓϦ։ൃͰར༻Ͱ͖Δ2࣍ݩඳըΤϯδϯ
ϨϕϧͰܰྔͳ2DϨϯμϦϯάػೳΛఏڙ
https://developer.apple.com/documentation/coregraphics/cgcontext
Slide 31
Slide 31 text
CVPixelBuffer →CGContext
guard let context: CGContext = .init(
data: CVPixelBufferGetBaseAddress(pixelBuffer),
width: Int(size.width),
height: Int(size.height),
bitsPerComponent: 8,
bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer),
space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue
) else { return nil }
Slide 32
Slide 32 text
CVPixelBuffer →CMFormatDescription
• CMFormatDescriptionͱ
αϯϓϧόοϑΝͷαϯϓϧΛهड़͢ΔϝσΟΞϑΥʔϚοτهड़ࢠ
ΦʔσΟΦɺϏσΦɺ͓Αͼ Muxed ϝσΟΞσʔλͳͲɺϝσΟΞλΠϓʹͱΒΘΕ
ͳ͍ͷͱϝσΟΞʹಛԽͨ͠ͷ͕͋Δ
• CMVideoFormatDescriptionCreateForImageBuffer
ΠϝʔδόοϑΝΛ༻ͯ͠ϏσΦϝσΟΞετϦʔϜ༻ͷFormatDesciptionΛ࡞͢
Δ
https://developer.apple.com/documentation/coremedia/cmformatdescription-u8g
Slide 33
Slide 33 text
CVPixelBuffer →CMFormatDescription
var formatDescription: CMFormatDescription?
let createImageBufferResult: OSStatus =
CMVideoFormatDescriptionCreateForImageBuffer(ɹ
allocator: kCFAllocatorDefault,
imageBuffer: pixelBuffer,
formatDescriptionOut: &formatDescription
)
Slide 34
Slide 34 text
CMSampleTimingInfo
• CMSampleTimingInfoͱ
CMSampleBufferͷͯ͢ͷαϯϓϧͷλΠϛϯάใΛूΊͨͷ
• init(duration:presentationTimeStamp:decodeTimeStamp:)
• duration
αϯϓϧͷظؒ
• presentationTimeStamp
αϯϓϧ͕දࣔ͞ΕΔ࣌ؒ
• decodeTimeStamp
αϯϓϧ͕σίʔυ͞ΕΔ࣌ؒ
https://developer.apple.com/documentation/coremedia/cmsampletiminginfo
Slide 35
Slide 35 text
CMSampleTimingInfo
let currentTime: CMTime = .init(
seconds: CACurrentMediaTime(),
preferredTimescale: 60
)
let timingInfo: CMSampleTimingInfo = .init(
duration: .init(seconds: 1, preferredTimescale: 60),ɹ
presentationTimeStamp: currentTime,
decodeTimeStamp: currentTime
)
Slide 36
Slide 36 text
CMSampleBufferΛ࡞
• UIViewΛCMSampleBufferʹม͢ΔExtension
https://gist.github.com/tsuzukihashi/97e379a42e32cc0647aa7a4770d2d9a6
do {
return try CMSampleBuffer(
imageBuffer: pixelBuffer,
formatDescription: formatDescription,
sampleTiming: getCMSampleTimingInfo()
)
} catch {
assertionFailure("Failed to create CMSampleBuffer: \(error)")
return nil
}
Slide 37
Slide 37 text
UIViewΛPiP͢Δ·ͰͷಓͷΓ
6*7JFX
1J1͍ͤͨ͞7JFX
"74BNQMF#V FS
%JTQMBZ-BZFS
6*7JFX
ΞϓϦʹදࣔ͢Δ
"71JDUVSF*O1JDUVSF
$POUSPMMFS
1J1Λཧ͢Δ
Slide 38
Slide 38 text
AVSampleBufferDisplayLayerΛSwiftUI͔Β͑ΔΑ͏ʹ͢Δ
struct PiPContainerView: UIViewRepresentable {
let bufferDisplayLayer: AVSampleBufferDisplayLayer
let frame: CGRect
func makeUIView(context: Context)
->
UIView {
let view = UIView()
view.frame = frame
bufferDisplayLayer.frame = view.bounds
bufferDisplayLayer.videoGravity = .resizeAspect
view.layer.addSublayer(bufferDisplayLayer)
return view
}
func updateUIView(_ uiView: UIView, context: Context) {}ɹ
}
Slide 39
Slide 39 text
UIViewΛPiP͢Δ·ͰͷಓͷΓ
6*7JFX
1J1͍ͤͨ͞7JFX
"74BNQMF#V FS
%JTQMBZ-BZFS
6*7JFX
ΞϓϦʹදࣔ͢Δ
"71JDUVSF*O1JDUVSF
$POUSPMMFS
1J1Λཧ͢Δ
Slide 40
Slide 40 text
AVPictureInPicture
Controller
P i P Λ ཧ ͢ Δ
Slide 41
Slide 41 text
AVPictureInPictureController
• PiPΛཧ͢Δίϯτϩʔϥ
• AVPictureInPictureController.ContentSourceʹAVSampleBufferDisplayLayerͱdelegate
Λ͢
pipController = AVPictureInPictureController(
contentSource: AVPictureInPictureController.ContentSource(ɹɹ
sampleBufferDisplayLayer: bufferDisplayLayer,
playbackDelegate: self
)
)
pipController
?
.
delegate = self
Slide 42
Slide 42 text
AVPictureInPictureController
• PiPΛཧ͢Δίϯτϩʔϥ
• AVPictureInPictureController.ContentSourceʹAVSampleBufferDisplayLayerͱdelegate
Λ͢
pipController = AVPictureInPictureController(
contentSource: AVPictureInPictureController.ContentSource(ɹɹ
sampleBufferDisplayLayer: bufferDisplayLayer,
playbackDelegate: self
)
)
pipController
?
.
delegate = self
PiPManagerΛAVPictureInPictureControllerDelegateͱ
AVPictureInPictureSampleBufferPlaybackDelegateʹ४ڌͤ͞Δ
Slide 43
Slide 43 text
func prepare() {
let timerBlock: ((Timer)
->
Void) = { [weak self] timer in
guard let buffer: CMSampleBuffer = self
? .
nextBuffer() else { return }
self
?
.
bufferDisplayLayer.enqueue(buffer)
}
let timer = Timer(timeInterval: 1, repeats: true, block: timerBlock)
self.timer = timer
RunLoop.main.add(timer, forMode: .default)
pipController = …
pipController
?
.
delegate = self
}
PiPManager
Slide 44
Slide 44 text
private func nextBuffer()
- >
CMSampleBuffer? {
if bufferDisplayLayer.status
= =
.failed {
bufferDisplayLayer.flush()
}
dateLabel.text = Date().formatted(date: .numeric, time: .complete)
return dateLabel.toCMSampleBuffer()
}
PiPManager
• flush()
iPhoneΛεϦʔϓঢ়ଶ͔Β෮ؼͨ͠ͱ͖ͳͲ
AVQueuedSampleBufferRenderingStatus͕.failedʹͳΓΤϥʔʹͳΔ
͜ͷϝιουΛݺͼอཹதͷαϯϓϧόοϑΝΛഁغ͢Δ
Ignoring enqueueSampleBuffer: because status is “failed"
Slide 45
Slide 45 text
AVPictureInPicture
ControllerDelegate
P i P ͷ Πϕ ϯ τ Λ ݕ ͢ Δ ͨ Ί ͷ ϓ ϩ τ ί ϧ
Slide 46
Slide 46 text
PiPͷΠϕϯτΛݕ͢ΔͨΊͷϓϩτίϧ
• pictureInPictureControllerWillStartPictureInPicture(_:)
Picture in Pictureͷ։࢝͞ΕΔ͜ͱΛ௨
• pictureInPictureControllerDidStartPictureInPicture(_:)
Picture in Picture͕։࢝͞Εͨ͜ͱ௨
• pictureInPictureController(_:failedToStartPictureInPictureWithEr
ror:)
Picture in Pictureͷىಈʹࣦഊͨ͜͠ͱΛ௨
• pictureInPictureControllerWillStopPictureInPicture(_:)
Picture in Picture͕ఀࢭ͢Δ͜ͱΛ௨ʢ໌ࣔత, Ϣʔβʔ, γεςϜΘͣʣ
• pictureInPictureControllerDidStopPictureInPicture(_:)
Picture in Picture͕ఀࢭͨ͜͠ͱΛ௨
Slide 47
Slide 47 text
AVPictureInPicture
SampleBuffer
PlaybackDelegate
AV S a m p l e B u f f e r D i s p l a y L a y e r ͔ Β ੍ ޚ ͢ Δ
Slide 48
Slide 48 text
pictureInPictureControllerTimeRangeForPlayback(_:)
• ࠶ੜՄೳͳ࣌ؒଳΛදࣔ͢Δͷʹར༻
• CMTimeRangeΛ͏
• LIVEϞʔυͷ࣌ skipϘλϯ͕disabledʹͳΔ
https://developer.apple.com/documentation/avkit/avpictureinpicturesamplebufferplaybackdelegate/3750337-pictureinpicturecontrollertimera
Slide 49
Slide 49 text
LIVEίϯςϯπ
• ਖ਼ͷແݶେͷܧଓ࣌ؒΛ࣋ͭ࣌ؒൣғΛฦ͢ͱLIVEίϯςϯπʹͳΔ
func pictureInPictureControllerTimeRangeForPlayback(
_ pictureInPictureController: AVPictureInPictureController
)
-
>
CMTimeRange {
return CMTimeRange(
start: .negativeInfinity,
duration: .positiveInfinity
)
}
Slide 50
Slide 50 text
ඇLIVEίϯςϯπ
• ίϯςϯπͷ࣌ؒΛฦ͢͜ͱͰඇϥΠϒίϯςϯπ͢Δ͜ͱ͕Ͱ͖Δ
• skipϘλϯ͕׆ੑԽ͢Δ
ɹɹfunc pictureInPictureControllerTimeRangeForPlayback(
_ pictureInPictureController: AVPictureInPictureController
)
-
>
CMTimeRange {
return CMTimeRange(
start: .indefinite,
duration: CMTime(
seconds: 300,
preferredTimescale: CMTimeScale(1)
)
)
}
Slide 51
Slide 51 text
pictureInPictureController(_:skipByInterval:completion:)
• Ϣʔβ͕ࢦఆ͞Εִ͚ͨ࣌ؒؒͩલํ·ͨޙํʹҠಈ͢Δ͜ͱ͑Δ
• skipInterval15 or -15Ͱฦ٫͞ΕΔ
func pictureInPictureController(
_ pictureInPictureController: AVPictureInPictureController,
skipByInterval skipInterval: CMTime,
completion completionHandler: @escaping ()
->
Void
) {
completionHandler()
}
Slide 52
Slide 52 text
pictureInPictureControllerIsPlaybackPaused(_:)
• ࠶ੜঢ়گʹΑΒͣɺҰ࣌ఀࢭ͍ͯ͠ΔΑ͏ͳUIΛදࣔ͢Δ͔Ͳ͏͔Λཁٻ
• Ұ࣌ఀࢭঢ়ଶʹ͍ͨ͠ͳΒ true, ͦΕҎ֎ false
func pictureInPictureControllerIsPlaybackPaused(
_ pictureInPictureController: AVPictureInPictureController
)
-
>
Bool {
return false
}
true false
Slide 53
Slide 53 text
pictureInPictureController(_:setPlaying:)
• Ϣʔβ͕࠶ੜ։࢝·ͨҰ࣌ఀࢭΛཁٻͨ͜͠ͱΛ௨
ԻָΞϓϦͷ࠶ੜɺఀࢭ͜ͷϝιουΛ੍͍ޚ͢Δ
func pictureInPictureController(
_ pictureInPictureController: AVPictureInPictureController,
setPlaying playing: Bool
) {
print(playing)
}
true false
→ →
Slide 54
Slide 54 text
pictureInPictureController(_:didTransitionToRenderSize:)
• PiPͷαΠζ͕มߋ͞Εͨ͜ͱͱͦͷαΠζΛ௨
func pictureInPictureController(
_ pictureInPictureController: AVPictureInPictureController,
didTransitionToRenderSize newRenderSize: CMVideoDimensions
) {
dateLabel.text = "w: \(newRenderSize.width) h: \(newRenderSize.height)"
if let sampleBuffer = dateLabel.toCMSampleBuffer() {
bufferDisplayLayer.enqueue(sampleBuffer)
}
}
Slide 55
Slide 55 text
AVPictureInPictureController
• startPictureInPicture()
ՄೳͰ͋ΕPiPΛ։࢝͢Δ
• stopPictureInPicture()
PiP͕ΞΫςΟϒঢ়ଶͳΒఀࢭ͢Δ
• isPictureInPictureActive
ݱࡏɺPicture in Picture͕༗ޮͰ͋Δ͔Ͳ͏͔
func swapPictureInPicture() {
if pipController
?.
isPictureInPictureActive
==
true {
pipController
?.
stopPictureInPicture()
} else {
pipController
?.
startPictureInPicture()
}
}
Slide 56
Slide 56 text
ContentViewModel
final class ContentViewModel: ObservableObject {
@Published var isReady: Bool = false
let pipManager: PiPManager = .shared
func didTapMainButton() {
if isReady {
pipManager.reset()
} else {
pipManager.prepare()
}
isReady.toggle()
}
func didTapPiPSwap() {
pipManager.swapPictureInPicture()
}
}
Slide 57
Slide 57 text
tips
ҙ ͢ ͖ ͜ ͱ
Slide 58
Slide 58 text
⚠ϦδΣΫτ͞ΕΔةݥ⚠
• όοΫάϥϯυͰͷΦʔσΟΦ࠶ੜʹରԠ͢ΔΑ͏ʹ͍ͯ͠Δ͕ɺ࣮ࡍʹԻྲྀΕͯ
͍ͳ͍ͨΊϦδΣΫτ
• App Store Connect্ͰPiPͷͨΊʹํͳ͍ͬͯ͘ΔͱϝϞʹॻ͍ͯಈ࡞͍ͯ͠
ΔՕॴΛಈըͰఴͨ͠ΒϦδΣΫτճආ 🎉
Slide 59
Slide 59 text
PiPͰͰ͖ͳ͍͜ͱ
• Ϣʔβ͔ΒͷϨϏϡʔͰPiPதʹεϦʔϓঢ়ଶͯ͠ཉ͍͠ͱ͍͏ཁ
PiPಈըΛ࠶ੜ͍ͯ͠Δͱಉ༷ͳͷͰɺεϦʔϓঢ়ଶʹͰ͖ͳ͍
• PiPͷࣗಈىಈ
PiPΛTimerͰࢦఆͨ࣌ؒ͠ʹऴྃͤ͞Δ͜ͱՄೳ͕ͩɺࢦఆͨ࣌ؒ͠ʹࣗಈىಈ
Ͳ͏ΒͰ͖ͳͦ͏ͩͬͨ(Ͱ͖ͨΑͬͯํڭ͍͑ͯͩ͘͞🙏)
• ҐஔΛࣗ༝ʹม͑Δ͜ͱ
্Լ࢛۱ͷ4Օॴʹ͔͠ಈ͔ͤͳ͍
Slide 60
Slide 60 text
·ͱΊ
• iOS 15͔Β͖ͳUIΛPicture In PictureͰදࣔͤ͞Δ͜ͱ͕Ͱ͖
ΔΑ͏ʹͳͬͨ
• CMSampleDisplayBuffer͕ॏཁ
• Picture In PictureϢʔβʔ·ͩෆ׳ΕͳͨΊɺखް͘αϙʔτ
͢Δͱྑ͍
• جຊతͳ͍ํ
• ࣗಈΦϑઃఆͷՃͳͲ
Slide 61
Slide 61 text
·ͱΊ
• iOS 15͔Β͖ͳUIΛPicture In PictureͰදࣔͤ͞Δ͜ͱ͕Ͱ͖
ΔΑ͏ʹͳͬͨ
• CMSampleDisplayBuffer͕ॏཁ
• Picture In PictureϢʔβʔ·ͩෆ׳ΕͳͨΊɺखް͘αϙʔτ
͢Δͱྑ͍
• جຊతͳ͍ํ
• ࣗಈΦϑઃఆͷՃͳͲ
·ͩݟ͵PiPͷΞϓϦΛ࡞ͬͯΈ͍ͯͩ͘͞🙏
Slide 62
Slide 62 text
iOS/Android
Ξ
プ
ϦΤϯ
ジ
χΞ࠾༻͍ͯ͠·͢!
Slide 63
Slide 63 text
ࢀߟURL
AppleͷυΩϡϝϯτ
https://developer.apple.com/documentation/avkit/accessing_the_camera_while_multitasking
৴ίϝϯτόʔ ʙ iOS15 Ͱ࣮ݱ͢Δ৽͍͠ PiP ମݧ
https://tech.mirrativ.stream/entry/2021/11/26/114002
iOS Ͱҙͷ UIView ΛϐΫνϟʔΠϯϐΫνϟʔ͢Δ
https://zenn.dev/uakihir0/articles/211128-uipip
UIViewͷදࣔ༰ΛCMSampleBu
ff
erʹ͢Δ
https://soranoba.net/programming/uiview-to-cmsamplebu
ff
er