動画だけじゃない！iOS 15のピクチャ・イン・ピクチャを使って好きなUIを表示させよう！

iOS 14まではピクチャ・イン・ピクチャ（以下PiP）を表示させるには動画コンテンツが必要でした。
しかし、新しくiOS 15でPiPのAPIが追加されたことにより動画コンテンツが無いただのUIViewもPiPとして表示させることが可能になりました！

これまでPiPを利用したアプリを3つリリースしてきた経験から、PiPを利用したアプリの開発からリリースするまでについて話したいと思います。

・PiPに好きなUIを表示させる仕組みと実装
・より良いPiP体験の提供
・PiPでできないこと
・Appleの審査を通過する

PiPを使うことでユーザーにより良い体験を与えることができるアプリはたくさんあると自分は感じています。
ぜひこのセッションで得た情報をもとにPiPを使った良いアプリが増えれば良いと願っています！

この資料はiOSDC2022 の補足としてお使いください

Transcript

1. i O S 1 5 ͷ 
 ϐΫ ν ϟ ɾ Π ϯ ɾ ϐΫ ν ϟ 
 Λ ࢖ ͬ ͯ 
 ޷ ͖ ͳ U I Λ ද ࣔ ͞ ͤ Α ͏ ʂ ᠃ ڮ ɹ ྋ ಈը͚ͩ͡Όͳ͍ʂ iOSDC Japan 2022 1
2. @ t s u z u k i 8 1 7 🐸ࣗݾ঺հ🐸 Yahoo JAPAN! PayPayϑϦϚ iOS։ൃ • Q.UIKitͱSwiftUIͲ͕ͬͪ޷͖ʁ 
 RYO TSUZUKIHASHI ϛχΤϐιʔυ 2019೥৽ଔͷ࣌ʹॳΊͯiOSDCʹࢀՃ ͍͔ͭࣗ෼΋ొஃ͍ͨ͠ͱಌΕΛ๊͘Α ͏ʹͳΓɺҎ߱ຖ೥ࢀՃ ͦͯ͠4೥໨ͷ2022೥ʹ೦ئͷొஃ✌
3. @ t s u z u k i 8 1 7 🐸ࣗݾ঺հ🐸 Yahoo JAPAN! PayPayϑϦϚ iOS։ൃ • Q.UIKitͱSwiftUIͲ͕ͬͪ޷͖ʁ 
 A. ͲͪΒ΋޷͖ • Q. झຯ͸? 
 RYO TSUZUKIHASHI ϛχΤϐιʔυ 2019೥৽ଔͷ࣌ʹॳΊͯiOSDCʹࢀՃ ͍͔ͭࣗ෼΋ొஃ͍ͨ͠ͱಌΕΛ๊͘Α ͏ʹͳΓɺҎ߱ຖ೥ࢀՃ ͦͯ͠4೥໨ͷ2022೥ʹ೦ئͷొஃ✌
4. @ t s u z u k i 8 1 7 🐸ࣗݾ঺հ🐸 Yahoo JAPAN! PayPayϑϦϚ iOS։ൃ • Q.UIKitͱSwiftUIͲ͕ͬͪ޷͖ʁ 
 A. ͲͪΒ΋޷͖ • Q. झຯ͸? 
 A. ݸਓΞϓϦ։ൃ • Q. ޷͖ͳۦಈ։ൃ͸ʁ 
 RYO TSUZUKIHASHI ϛχΤϐιʔυ 2019೥৽ଔͷ࣌ʹॳΊͯiOSDCʹࢀՃ ͍͔ͭࣗ෼΋ొஃ͍ͨ͠ͱಌΕΛ๊͘Α ͏ʹͳΓɺҎ߱ຖ೥ࢀՃ ͦͯ͠4೥໨ͷ2022೥ʹ೦ئͷొஃ✌
5. @ t s u z u k i 8 1 7 🐸ࣗݾ঺հ🐸 Yahoo JAPAN! PayPayϑϦϚ iOS։ൃ • Q.UIKitͱSwiftUIͲ͕ͬͪ޷͖ʁ 
 A. ͲͪΒ΋޷͖ • Q. झຯ͸? 
 A. ݸਓΞϓϦ։ൃ • Q. ޷͖ͳۦಈ։ൃ͸ʁ 
 A. ςετۦಈ։ൃ RYO TSUZUKIHASHI ϛχΤϐιʔυ 2019೥৽ଔͷ࣌ʹॳΊͯiOSDCʹࢀՃ ͍͔ͭࣗ෼΋ొஃ͍ͨ͠ͱಌΕΛ๊͘Α ͏ʹͳΓɺҎ߱ຖ೥ࢀՃ ͦͯ͠4೥໨ͷ2022೥ʹ೦ئͷొஃ✌
6. ͓඼ॻ͖ •ࣄྫ঺հ •Picture In Picture֓࿦ •࡞Γͳ͕ΒֶͿPicture In Picture •౾tips 6
7. ࣄྫ঺հ • ྲྀΕΔϝϞா • ೚ҙͷจࣈྻ • PictIn • ೚ҙͷը૾ • PIPMP • ࠶ੜதͷԻָ৘ใ
8. ྲྀΕΔϝϞா PictIn PIPMP
9. ϐΫνϟɾΠϯɾ ϐΫνϟ(PiP)ͱ͸ P i P Ͱ Ͱ ͖ Δ ͜ ͱ
10. PiPͱ͸ • ଞͷΞϓϦΛ࢖༻͠ͳ͕ΒFaceTimeΛ࢖ͬͨΓ ϏσΦΛࢹௌͨ͠ΓͰ͖Δػೳͷ͜ͱ • ΢Οϯυ΢ΛϐϯνΦʔϓϯɾϐϯνΫϩʔζ͢ Δ͜ͱͰαΠζΛมߋ͢Δ͜ͱ͕Ͱ͖Δ • ଞʹ΋Ҡಈɾඇදࣔɾ࡟আ • ಈըͷ࠶ੜɾఀࢭɾ15sec ͷૣૹΓͱר͖໭͠ Ҿ༻
    J1IPOFͷϐΫνϟɾΠϯɾϐΫνϟΛ࢖༻ͨ͠ϚϧνλεΫ
    IUUQTTVQQPSUBQQMFDPNKBKQHVJEFJQIPOFJQIDDCEJPT
11. iOS 15͔ΒͷPiP Կ ͕ Ͱ ͖ Δ Α ͏ ʹ ͳ Δ ͷ ͔
12. iOS 15͔ΒͷPicture In Picture • ~ iOS 14·Ͱ 
 PiP͢Δ͜ͱ͕Ͱ͖Δͷ͸جຊతʹಈըͷΈ • iOS 15͔Β • AVPictureInPictureController.ContentSourceͷ௥Ճ 
 →ɹAVSampleBufferDisplayLayerͷίϯςϯπΛPicture In Pictureαϙʔτ • ͦͷଞPiP༻ͷϝιουͷ௥Ճ
13. AVSampleBufferDisplayLayer • ѹॖɾඇѹॖͷVideoFrameΛදࣔ͢ΔΦϒδΣΫτ (iOS 8 ~) • CMSampleBufferΛ༩͑Δ͜ͱʹΑͬͯ ಈըΛ࠶ੜ͢Δ͜ͱ͕Ͱ͖Δ CMSampleBuffer • ө૾ɺԻ੠ɺ͋Δ͍͸ͦͷ྆ํ౳ɺϝσΟΞσʔλΛ࣋ͪӡͿͨΊͷΦϒδΣΫτ • UIView͔Β࡞ΕΔ ↓
14. AVSampleBufferDisplayLayer • ѹॖɾඇѹॖͷVideoFrameΛදࣔ͢ΔΦϒδΣΫτ (iOS 8 ~) • CMSampleBufferΛ༩͑Δ͜ͱʹΑͬͯ ಈըΛ࠶ੜ͢Δ͜ͱ͕Ͱ͖Δ CMSampleBuffer • ө૾ɺԻ੠ɺ͋Δ͍͸ͦͷ྆ํ౳ɺϝσΟΞσʔλΛ࣋ͪӡͿͨΊͷΦϒδΣΫτ • UIView͔Β࡞ΕΔ ↓ ޷͖ͳUIΛPicture In Pictureͤ͞Δ͜ͱ͕Ͱ͖Δʂ
15. ࣮ફPiP Ͳ ͷ Α ͏ ʹ P i P Λ ࣮ ݱ ͞ ͤ Δ ͷ ͔
16. ࣮ػ୺຤ͷ༻ҙ • ࣮ػϏϧυͰͳ͍ͱPiP͕ࢼͤͳ͍ 
 γϛϡϨʔλͰ͸ಈ͔ͳ͍
17. αϯϓϧϓϩδΣΫτ • ࣮ػϏϧυ͕Ͱ͖Ε͹ࢼͤΔ https://github.com/tsuzukihashi/sample-pip • PiPManager 
 PiPΛ؅ཧ͢ΔΫϥεɺγϯάϧτϯ • UIViewExtension 
 UIViewΛCMSampleBufferʹม׵͢ΔExtension • ContentView 
 ϝΠϯͷView ϘλϯͳͲΛ഑ஔ • ContentViewModel 
 PiPManagerΛอ࣋͠Πϕϯτͷड͚౉͠Λ͢Δ • PiPContainerView 
 SwiftUIͰAVSampleBufferDisplayLayerΛඳը͢ΔͨΊͷView
18. ࣮૷ͷલʹඞཁͳઃఆ • όοΫάϥ΢ϯυͰͷΦʔσΟΦ࠶ੜʹରԠ͢ΔΑ͏ʹΞϓϦΛઃఆ͢Δ 
 CapabilityͰ Background ModesΛ௥Ճ 
 Audio, AirPlay, and Picture in PictureΛνΣοΫ
19. ࣮૷ͷલʹඞཁͳઃఆ • AudioSessionΛ։࢝͢Δ 
 <category: .playAndRecord, mode: .moviePlayback> 
 PiPΛىಈͰ͖ɺ࠶ੜதͷԻָΛࢭΊͣʹࡁΉ૊Έ߹Θͤ override init() { let session = AVAudioSession.sharedInstance() do { try session.setCategory(.playAndRecord, mode: .moviePlayback) try session.setActive(true) } catch { print("Failed to set AVAudioSession: \(error)") } } PiPManagerͷॳظԽͰߦ͍ͬͯΔ
20. UIViewΛPiP͢Δ·ͰͷಓͷΓ 6*7JFX
    1J1͍ͤͨ͞7JFX "74BNQMF#V FS %JTQMBZ-BZFS 6*7JFX
    ΞϓϦʹදࣔ͢Δ "71JDUVSF*O1JDUVSF
    $POUSPMMFS
    1J1Λ؅ཧ͢Δ
21. UIViewΛPiP͢Δ·ͰͷಓͷΓ 6*7JFX
    1J1͍ͤͨ͞7JFX "74BNQMF#V FS %JTQMBZ-BZFS 6*7JFX
    ΞϓϦʹදࣔ͢Δ "71JDUVSF*O1JDUVSF
    $POUSPMMFS
    1J1Λ؅ཧ͢Δ PiPManager ContentView
22. UIViewΛPiP͢Δ·ͰͷಓͷΓ 6*7JFX
    1J1͍ͤͨ͞7JFX "74BNQMF#V FS %JTQMBZ-BZFS 6*7JFX
    ΞϓϦʹදࣔ͢Δ "71JDUVSF*O1JDUVSF
    $POUSPMMFS
    1J1Λ؅ཧ͢Δ
23. PiP͍ͤͨ͞Viewͷ࡞੒ • UIView 
 AVSampleBufferDisplayLayerʹදࣔ͢ΔͨΊͷUIViewΛ࡞Δ 
 ࠓճ͸؆қతʹUILabelΛ࢖͏ 
 ݱࡏ࣌ࠁΛදࣔ
24. PiP͍ͤͨ͞View private let dateLabel: UILabel = { let label = UILabel() label.frame = .init( origin: .zero, size: .init( width: UIScreen.main.bounds.width, height: 120 ) ) label.font = .monospacedSystemFont(ofSize: 24, weight: .medium) label.textAlignment = .center label.textColor = .black label.backgroundColor = .white return label }() PiPManager಺Ͱఆٛ
25. UIViewΛPiP͢Δ·ͰͷಓͷΓ 6*7JFX
    1J1͍ͤͨ͞7JFX "74BNQMF#V FS %JTQMBZ-BZFS 6*7JFX
    ΞϓϦʹදࣔ͢Δ "71JDUVSF*O1JDUVSF
    $POUSPMMFS
    1J1Λ؅ཧ͢Δ
26. AVSampleBufferDisplayLayerΛඳը͢Δʹ͸ • CMSampleBuffer͕ඞཁ 
 enqueue(_:)ϝιουͰCMSampleBufferΛ౉͢ func enqueue(_ sampleBuffer: CMSampleBuffer)
27. CMSampleBufferΛ࡞੒ • UIViewΛCMSampleBufferʹม׵͢ΔExtension https://gist.github.com/tsuzukihashi/97e379a42e32cc0647aa7a4770d2d9a6 do { return try CMSampleBuffer( imageBuffer: pixelBuffer, formatDescription: formatDescription, sampleTiming: getCMSampleTimingInfo() ) } catch { assertionFailure("Failed to create CMSampleBuffer: \(error)") return nil }
28. UIView →CVPixelBuffer • CVPixelBufferͱ͸ 
 ϝΠϯϝϞϦʹϐΫηϧΛอ࣋͢ΔΠϝʔδόοϑΝ 
 ϑϨʔϜΛੜ੒΍ಈըͷѹॖ·ͨ͸ղౚɺCoreImageΛར༻͢ΔΞϓϦͰ࢖ΘΕΔ • CMPixelBufferCreate 
 CVPixelBufferΛ࡞੒͢Δϝιου https://developer.apple.com/documentation/corevideo/cvpixelbuffer-q2e
29. UIView →CVPixelBuffer var pixelBuffer: CVPixelBuffer? let createPixelBufferResult: OSStatus = CVPixelBufferCreate( kCFAllocatorDefault, Int(size.width), Int(size.height), kCVPixelFormatType_32ARGB, [ kCVPixelBufferCGImageCompatibilityKey: kCFBooleanTrue!, kCVPixelBufferCGBitmapContextCompatibilityKey: kCFBooleanTrue!, kCVPixelBufferIOSurfacePropertiesKey: [:] as CFDictionary, ] as CFDictionary, &pixelBuffer )
30. CVPixelBuffer →CGContext • CGContextͱ͸ 
 Quartz 2DͷඳըઌΛද͢ 
 ඳըύϥϝʔλͱϖʔδ্ͷϖΠϯτΛѼઌʹϨϯμϦϯά͢ΔͨΊʹඞཁͳ͢΂ͯͷ σόΠεݻ༗৘ใؚ͕·Ε͍ͯΔ • Quartz 2Dͱ͸ 
 iOS, tvOS, macOSͷΞϓϦ։ൃͰར༻Ͱ͖Δ2࣍ݩඳըΤϯδϯ 
 ௿ϨϕϧͰܰྔͳ2DϨϯμϦϯάػೳΛఏڙ https://developer.apple.com/documentation/coregraphics/cgcontext
31. CVPixelBuffer →CGContext guard let context: CGContext = .init( data: CVPixelBufferGetBaseAddress(pixelBuffer), width: Int(size.width), height: Int(size.height), bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer), space: CGColorSpaceCreateDeviceRGB(), bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue ) else { return nil }
32. CVPixelBuffer →CMFormatDescription • CMFormatDescriptionͱ͸ 
 αϯϓϧόοϑΝ಺ͷαϯϓϧΛهड़͢ΔϝσΟΞϑΥʔϚοτهड़ࢠ 
 ΦʔσΟΦɺϏσΦɺ͓Αͼ Muxed ϝσΟΞσʔλͳͲɺϝσΟΞλΠϓʹͱΒΘΕ ͳ͍΋ͷͱϝσΟΞʹಛԽͨ͠΋ͷ͕͋Δ • CMVideoFormatDescriptionCreateForImageBuffer 
 ΠϝʔδόοϑΝΛ࢖༻ͯ͠ϏσΦϝσΟΞετϦʔϜ༻ͷFormatDesciptionΛ࡞੒͢ Δ https://developer.apple.com/documentation/coremedia/cmformatdescription-u8g
33. CVPixelBuffer →CMFormatDescription var formatDescription: CMFormatDescription? let createImageBufferResult: OSStatus = CMVideoFormatDescriptionCreateForImageBuffer(ɹ allocator: kCFAllocatorDefault, imageBuffer: pixelBuffer, formatDescriptionOut: &formatDescription )
34. CMSampleTimingInfo • CMSampleTimingInfoͱ͸ 
 CMSampleBuffer಺ͷ͢΂ͯͷαϯϓϧͷλΠϛϯά৘ใΛूΊͨ΋ͷ • init(duration:presentationTimeStamp:decodeTimeStamp:) • duration 
 αϯϓϧͷظؒ • presentationTimeStamp 
 αϯϓϧ͕දࣔ͞ΕΔ࣌ؒ • decodeTimeStamp 
 αϯϓϧ͕σίʔυ͞ΕΔ࣌ؒ https://developer.apple.com/documentation/coremedia/cmsampletiminginfo
35. CMSampleTimingInfo let currentTime: CMTime = .init( seconds: CACurrentMediaTime(), preferredTimescale: 60 ) let timingInfo: CMSampleTimingInfo = .init( duration: .init(seconds: 1, preferredTimescale: 60),ɹ presentationTimeStamp: currentTime, decodeTimeStamp: currentTime )
36. CMSampleBufferΛ࡞੒ • UIViewΛCMSampleBufferʹม׵͢ΔExtension https://gist.github.com/tsuzukihashi/97e379a42e32cc0647aa7a4770d2d9a6 do { return try CMSampleBuffer( imageBuffer: pixelBuffer, formatDescription: formatDescription, sampleTiming: getCMSampleTimingInfo() ) } catch { assertionFailure("Failed to create CMSampleBuffer: \(error)") return nil }
37. UIViewΛPiP͢Δ·ͰͷಓͷΓ 6*7JFX
    1J1͍ͤͨ͞7JFX "74BNQMF#V FS %JTQMBZ-BZFS 6*7JFX
    ΞϓϦʹදࣔ͢Δ "71JDUVSF*O1JDUVSF
    $POUSPMMFS
    1J1Λ؅ཧ͢Δ
38. AVSampleBufferDisplayLayerΛSwiftUI͔Β࢖͑ΔΑ͏ʹ͢Δ struct PiPContainerView: UIViewRepresentable { let bufferDisplayLayer: AVSampleBufferDisplayLayer let frame: CGRect func makeUIView(context: Context) -> UIView { let view = UIView() view.frame = frame bufferDisplayLayer.frame = view.bounds bufferDisplayLayer.videoGravity = .resizeAspect view.layer.addSublayer(bufferDisplayLayer) return view } func updateUIView(_ uiView: UIView, context: Context) {}ɹ }
39. UIViewΛPiP͢Δ·ͰͷಓͷΓ 6*7JFX
    1J1͍ͤͨ͞7JFX "74BNQMF#V FS %JTQMBZ-BZFS 6*7JFX
    ΞϓϦʹදࣔ͢Δ "71JDUVSF*O1JDUVSF
    $POUSPMMFS
    1J1Λ؅ཧ͢Δ
40. AVPictureInPicture Controller P i P Λ ؅ ཧ ͢ Δ
41. AVPictureInPictureController • PiPΛ؅ཧ͢Δίϯτϩʔϥ • AVPictureInPictureController.ContentSourceʹAVSampleBufferDisplayLayerͱdelegate Λ౉͢ pipController = AVPictureInPictureController( contentSource: AVPictureInPictureController.ContentSource(ɹɹ sampleBufferDisplayLayer: bufferDisplayLayer, playbackDelegate: self ) ) pipController ? . delegate = self
42. AVPictureInPictureController • PiPΛ؅ཧ͢Δίϯτϩʔϥ • AVPictureInPictureController.ContentSourceʹAVSampleBufferDisplayLayerͱdelegate Λ౉͢ pipController = AVPictureInPictureController( contentSource: AVPictureInPictureController.ContentSource(ɹɹ sampleBufferDisplayLayer: bufferDisplayLayer, playbackDelegate: self ) ) pipController ? . delegate = self PiPManagerΛAVPictureInPictureControllerDelegateͱ AVPictureInPictureSampleBufferPlaybackDelegateʹ४ڌͤ͞Δ
43. func prepare() { let timerBlock: ((Timer) -> Void) = { [weak self] timer in guard let buffer: CMSampleBuffer = self ? . nextBuffer() else { return } self ? . bufferDisplayLayer.enqueue(buffer) } let timer = Timer(timeInterval: 1, repeats: true, block: timerBlock) self.timer = timer RunLoop.main.add(timer, forMode: .default) pipController = … pipController ? . delegate = self } PiPManager
44. private func nextBuffer() - > CMSampleBuffer? { if bufferDisplayLayer.status = = .failed { bufferDisplayLayer.flush() } dateLabel.text = Date().formatted(date: .numeric, time: .complete) return dateLabel.toCMSampleBuffer() } PiPManager • flush() 
 iPhoneΛεϦʔϓঢ়ଶ͔Β෮ؼͨ͠ͱ͖ͳͲ 
 AVQueuedSampleBufferRenderingStatus͕.failedʹͳΓΤϥʔʹͳΔ 
 ͜ͷϝιουΛݺͼอཹதͷαϯϓϧόοϑΝΛഁغ͢Δ Ignoring enqueueSampleBuffer: because status is “failed"
45. AVPictureInPicture ControllerDelegate P i P ͷ Πϕ ϯ τ Λ ݕ ஌ ͢ Δ ͨ Ί ͷ ϓ ϩ τ ί ϧ
46. PiPͷΠϕϯτΛݕ஌͢ΔͨΊͷϓϩτίϧ • pictureInPictureControllerWillStartPictureInPicture(_:) 
 Picture in Pictureͷ։࢝͞ΕΔ͜ͱΛ௨஌ • pictureInPictureControllerDidStartPictureInPicture(_:) 
 Picture in Picture͕։࢝͞Εͨ͜ͱ௨஌ • pictureInPictureController(_:failedToStartPictureInPictureWithEr ror:) 
 Picture in Pictureͷىಈʹࣦഊͨ͜͠ͱΛ௨஌ • pictureInPictureControllerWillStopPictureInPicture(_:) 
 Picture in Picture͕ఀࢭ͢Δ͜ͱΛ௨஌ʢ໌ࣔత, Ϣʔβʔ, γεςϜ໰Θͣʣ • pictureInPictureControllerDidStopPictureInPicture(_:) 
 Picture in Picture͕ఀࢭͨ͜͠ͱΛ௨஌
47. AVPictureInPicture SampleBuffer PlaybackDelegate AV S a m p l e B u f f e r D i s p l a y L a y e r ͔ Β ੍ ޚ ͢ Δ
48. pictureInPictureControllerTimeRangeForPlayback(_:) • ࠶ੜՄೳͳ࣌ؒଳΛදࣔ͢Δͷʹར༻ • CMTimeRangeΛ࢖͏ • LIVEϞʔυͷ࣌͸ skipϘλϯ͕disabledʹͳΔ https://developer.apple.com/documentation/avkit/avpictureinpicturesamplebufferplaybackdelegate/3750337-pictureinpicturecontrollertimera
49. LIVEίϯςϯπ • ਖ਼ͷແݶେͷܧଓ࣌ؒΛ࣋ͭ࣌ؒൣғΛฦ͢ͱLIVEίϯςϯπʹͳΔ func pictureInPictureControllerTimeRangeForPlayback( _ pictureInPictureController: AVPictureInPictureController ) - > CMTimeRange { return CMTimeRange( start: .negativeInfinity, duration: .positiveInfinity ) }
50. ඇLIVEίϯςϯπ • ίϯςϯπͷ࣌ؒΛฦ͢͜ͱͰඇϥΠϒίϯςϯπ͢Δ͜ͱ͕Ͱ͖Δ • skipϘλϯ͕׆ੑԽ͢Δ ɹɹfunc pictureInPictureControllerTimeRangeForPlayback( _ pictureInPictureController: AVPictureInPictureController ) - > CMTimeRange { return CMTimeRange( start: .indefinite, duration: CMTime( seconds: 300, preferredTimescale: CMTimeScale(1) ) ) }
51. pictureInPictureController(_:skipByInterval:completion:) • Ϣʔβ͕ࢦఆ͞Εִ͚ͨ࣌ؒؒͩલํ·ͨ͸ޙํʹҠಈ͢Δ͜ͱ఻͑Δ • skipInterval͸15 or -15Ͱฦ٫͞ΕΔ func pictureInPictureController( _ pictureInPictureController: AVPictureInPictureController, skipByInterval skipInterval: CMTime, completion completionHandler: @escaping () -> Void ) { completionHandler() }
52. pictureInPictureControllerIsPlaybackPaused(_:) • ࠶ੜঢ়گʹΑΒͣɺҰ࣌ఀࢭ͍ͯ͠ΔΑ͏ͳUIΛදࣔ͢Δ͔Ͳ͏͔Λཁٻ • Ұ࣌ఀࢭঢ়ଶʹ͍ͨ͠ͳΒ true, ͦΕҎ֎͸ false func pictureInPictureControllerIsPlaybackPaused( _ pictureInPictureController: AVPictureInPictureController ) - > Bool { return false } true false
53. pictureInPictureController(_:setPlaying:) • Ϣʔβ͕࠶ੜ։࢝·ͨ͸Ұ࣌ఀࢭΛཁٻͨ͜͠ͱΛ௨஌ 
 ԻָΞϓϦͷ࠶ੜɺఀࢭ͸͜ͷϝιουΛ࢖੍͍ޚ͢Δ func pictureInPictureController( _ pictureInPictureController: AVPictureInPictureController, setPlaying playing: Bool ) { print(playing) } true false → →
54. pictureInPictureController(_:didTransitionToRenderSize:) • PiPͷαΠζ͕มߋ͞Εͨ͜ͱͱͦͷαΠζΛ௨஌ func pictureInPictureController( _ pictureInPictureController: AVPictureInPictureController, didTransitionToRenderSize newRenderSize: CMVideoDimensions ) { dateLabel.text = "w: \(newRenderSize.width) h: \(newRenderSize.height)" if let sampleBuffer = dateLabel.toCMSampleBuffer() { bufferDisplayLayer.enqueue(sampleBuffer) } }
55. AVPictureInPictureController • startPictureInPicture() 
 ՄೳͰ͋Ε͹PiPΛ։࢝͢Δ • stopPictureInPicture() 
 PiP͕ΞΫςΟϒঢ়ଶͳΒఀࢭ͢Δ • isPictureInPictureActive 
 ݱࡏɺPicture in Picture͕༗ޮͰ͋Δ͔Ͳ͏͔ func swapPictureInPicture() { if pipController ?. isPictureInPictureActive == true { pipController ?. stopPictureInPicture() } else { pipController ?. startPictureInPicture() } }
56. ContentViewModel final class ContentViewModel: ObservableObject { @Published var isReady: Bool = false let pipManager: PiPManager = .shared func didTapMainButton() { if isReady { pipManager.reset() } else { pipManager.prepare() } isReady.toggle() } func didTapPiPSwap() { pipManager.swapPictureInPicture() } }
57. tips ஫ ҙ ͢΂ ͖ ͜ ͱ
58. ⚠ϦδΣΫτ͞ΕΔةݥ⚠ • όοΫάϥ΢ϯυͰͷΦʔσΟΦ࠶ੜʹରԠ͢ΔΑ͏ʹ͍ͯ͠Δ͕ɺ࣮ࡍʹԻ੠ྲྀΕͯ ͍ͳ͍ͨΊϦδΣΫτ • App Store Connect্ͰPiPͷͨΊʹ࢓ํͳ͘࢖͍ͬͯΔͱϝϞʹॻ͍ͯಈ࡞͍ͯ͠ ΔՕॴΛಈըͰఴ෇ͨ͠ΒϦδΣΫτճආ 🎉
59. PiPͰͰ͖ͳ͍͜ͱ • Ϣʔβ͔ΒͷϨϏϡʔͰPiPதʹεϦʔϓঢ়ଶͯ͠ཉ͍͠ͱ͍͏ཁ๬ 
 PiP͸ಈըΛ࠶ੜ͍ͯ͠Δͱಉ༷ͳͷͰɺεϦʔϓঢ়ଶʹͰ͖ͳ͍ • PiPͷࣗಈىಈ 
 PiPΛTimerͰࢦఆͨ࣌ؒ͠ʹऴྃͤ͞Δ͜ͱ͸Մೳ͕ͩɺࢦఆͨ࣌ؒ͠ʹࣗಈىಈ͸ Ͳ͏΍ΒͰ͖ͳͦ͏ͩͬͨ(Ͱ͖ͨΑͬͯํڭ͍͑ͯͩ͘͞🙏) • ҐஔΛࣗ༝ʹม͑Δ͜ͱ 
 ্Լ࢛۱ͷ4Օॴʹ͔͠ಈ͔ͤͳ͍
60. ·ͱΊ • iOS 15͔Β޷͖ͳUIΛPicture In PictureͰදࣔͤ͞Δ͜ͱ͕Ͱ͖ ΔΑ͏ʹͳͬͨ • CMSampleDisplayBuffer͕ॏཁ • Picture In Picture͸Ϣʔβʔ΋·ͩෆ׳ΕͳͨΊɺखް͘αϙʔτ ͢Δͱྑ͍ • جຊతͳ࢖͍ํ • ࣗಈΦϑઃఆͷ௥ՃͳͲ
61. ·ͱΊ • iOS 15͔Β޷͖ͳUIΛPicture In PictureͰදࣔͤ͞Δ͜ͱ͕Ͱ͖ ΔΑ͏ʹͳͬͨ • CMSampleDisplayBuffer͕ॏཁ • Picture In Picture͸Ϣʔβʔ΋·ͩෆ׳ΕͳͨΊɺखް͘αϙʔτ ͢Δͱྑ͍ • جຊతͳ࢖͍ํ • ࣗಈΦϑઃఆͷ௥ՃͳͲ ·ͩݟ͵PiPͷΞϓϦΛ࡞ͬͯΈ͍ͯͩ͘͞🙏
62. iOS/Android Ξ プ ϦΤϯ ジ χΞ࠾༻͍ͯ͠·͢!
63. ࢀߟURL AppleͷυΩϡϝϯτ https://developer.apple.com/documentation/avkit/accessing_the_camera_while_multitasking ഑৴ίϝϯτόʔ ʙ iOS15 Ͱ࣮ݱ͢Δ৽͍͠ PiP ମݧ https://tech.mirrativ.stream/entry/2021/11/26/114002 iOS Ͱ೚ҙͷ UIView ΛϐΫνϟʔΠϯϐΫνϟʔ͢Δ https://zenn.dev/uakihir0/articles/211128-uipip UIViewͷදࣔ಺༰ΛCMSampleBu ff erʹ͢Δ https://soranoba.net/programming/uiview-to-cmsamplebu ff er