空間オーディオを活用しよう！

Slide 1

Slide 1 text

ۭؒΦʔσΟΦΛ׆༻͠Α͏ʂ ߦ໦ ઍय़ʢͳΊ͖ ͪ͸ΔʣGitHub, X @Ridwy Streaming Client Team, גࣜձࣾ AbemaTV iOSDC JAPAN 2023, day1 (9/2)

Slide 2

Slide 2 text

גࣜձࣾϐΫηϥ Mac, iOS޲͚TVνϡʔφʔ, ಈըฤूΞϓϦ   nana music ΦʔσΟΦΤϯδϯ, ΤϑΣΫλ։ൃ   גࣜձࣾ AbemaTV Streaming Client Team ABEMA͕ରԠ͢ΔશͯͷσόΠεͷ   ϓϨΠϠʔͷ࣮૷ͱ࠶ੜ඼࣭ͷ୲อ ܦྺ

Slide 3

Slide 3 text

ۭؒΦʔσΟΦ 2020/12 AirPods Max ൃද 2021/06 Apple Music ͕ Dolby AtmosʹରԠ 2021/10 AirPodsʢୈ̏ੈ୅ʣൃද

Slide 4

Slide 4 text

“ ۭؒΦʔσΟΦ͸ɺ3࣍ݩͷ·ΔͰөըؗͷΑ͏ͳମݧΛੜΈग़͠ɺ ۭؒͷ͋ΒΏΔ৔ॴʹԻΛԾ૝తʹ഑ஔ͢Δ͜ͱ͕Ͱ͖… ” 2021/10/18 AirPodsʢୈ̏ੈ୅ʣͷൃද࣌ͷϓϨεϦϦʔεΑΓ

Slide 5

Slide 5 text

͔͋ͨ΋Իݯ͕ͦ͜ʹ͋Δ͔ͷΑ͏ͳମݧ Ծ૝తͳঢ়گʹϦΞϦςΟΛ࣋ͨͤΒΕΔ ήʔϜ Ծ૝ੈքʹೖΓࠐΉ өը ·ΔͰөըؗͷΑ͏ͳ ΦʔσΟΦମݧ Իָɾεϙʔπதܧ ຅ೖײɾྟ৔ײ iOS 15Ҏ߱ FaceTime ࿩ऀΛ۠ผ͠΍͍͢

Slide 6

Slide 6 text

ϔουτϥοΩϯάʹରԠͨ͠AirPodsͳͲΛ࢖͑͹ɺΑΓମݧͷઆಘྗ͕૿͢

Slide 7

Slide 7 text

xRͱͷ૬ੑ͸ൈ܈ Apple Vision Pro ΑΓ

Slide 8

Slide 8 text

AppleͷۭؒΦʔσΟΦͱ͸ ιϑτ΢ΣΞɾϋʔυ΢ΣΞͷ૊Έ߹ΘͤʹΑͬͯ ԻΛԾ૝తʹ഑ஔ͠ɺͦΕʹΑͬͯ຅ೖײ͋ΔମݧΛ࣮ݱͨ͠΋ͷ

Slide 9

Slide 9 text

։ൃऀ໨ઢͰ ΋͏গ͠ৄ͘͠ཧղ͍ͨ͠

Slide 10

Slide 10 text

ΦʔσΟΦػثͷൃల

Slide 11

Slide 11 text

Ϟϊϥϧ εϐʔΧʔ1ͭ ࠷΋γϯϓϧͳߏ੒

Slide 12

Slide 12 text

εςϨΦ εϐʔΧʔ2ͭͰ ࠨӈͷ޿͕Γ(Pan) Λදݱ

Slide 13

Slide 13 text

γϯϓϧͳPanͷख๏ Equal Power Panning ࠨӈͷεϐʔΧʔʹׂΓৼΔԻྔΛม͑Δ͜ͱͰ Ծ૝తͳԻͷҐஔ͕มԽ ৼ෯ ԻݯͷҐஔ L R

Slide 14

Slide 14 text

Ϛϧννϟϯωϧ өըԻڹͱڞʹൃల L C R S 1970೥୅ 4 ch

Slide 15

Slide 15 text

Ϛϧννϟϯωϧ өըԻڹͱڞʹൃల L C R SL 5 ch SR αϥ΢ϯυ͕εςϨΦԽ

Slide 16

Slide 16 text

Ϛϧννϟϯωϧ өըԻڹͱڞʹൃల L C R SL 5.1 ch SR ௿Ի੒෼͕෼཭ 1990೥୅ LFE

Slide 17

Slide 17 text

5.1 chͷ .1 ͸௿Իઐ༻νϟϯωϧ ௿Իͷಛ௃ • ฉ͖औΓʹ͍͘ ௿Ի͕͔ͬ͠Γௌ͑͜ΔΑ͏ʹ͢Δʹ͸ɺৼ෯Λେ͖͘ه࿥͢Δඞཁ͕͋Δ  தԻҬҎ্ʹదͨ͠ઃఆͰ͸ԻׂΕͯ͠͠·͏ • ࢦ޲ੑ͕͋·Γͳ͘෦԰શମʹڹ͘ ௿Ի੒෼Λ෼཭ͯ͠ઐ༻ͷ̍νϟϯωϧʹ

Slide 18

Slide 18 text

Ϛϧννϟϯωϧ өըԻڹͱڞʹൃల L C R SL 6.1 ch SR LFE BS

Slide 19

Slide 19 text

Ϛϧννϟϯωϧ өըԻڹͱڞʹൃల L C R BSL 7.1 ch BSR LFE SL SR

Slide 20

Slide 20 text

ͨ͘͞ΜͷεϐʔΧʔΛઃஔ͢Δͷ͕େม ՈఉͰ͸͞Βʹ೉қ౓͕ߴ͍ • ਖ਼֬ͳҐஔʹઃஔ͠ͳ͚Ε͹ɺਖ਼͘͠Ի͕࠶ݱ͞Εͳ͍ • ۤ࿑ͷׂʹεΠʔτεϙοτʢૉ੖Β͍͠Իڹମݧ͕Ͱ͖ΔΤϦΞʣ͕ڱ͍ ϚϧννϟϯωϧΦʔσΟΦͷݱ࣮తͳ໰୊ өըؗʹ΋ ݶք͸͋ΔΑ😂 εϖʔεͱࢿۚ💰͕ඞཁ

Slide 21

Slide 21 text

ΦϒδΣΫτϕʔεΦʔσΟΦొ৔

Slide 22

Slide 22 text

෺ମ͕ൃ͢ΔԻ 🕊 + ෺ମͷҐஔͳͲͷϝλ৘ใ ΦϒδΣΫτͷԻͱҐஔΛه࿥

Slide 23

Slide 23 text

࠶ੜ࣌͸ͦΕͧΕͷߏ੒ʹ߹ΘͤͯεϐʔΧʔ͔Βग़ΔԻΛ ϦΞϧλΠϜʹܭࢉʢ=ϨϯμϦϯάʣ 🕊 🕊 🕊

Slide 24

Slide 24 text

εϐʔΧʔ޲͚ϨϯμϦϯά p l1 l2 l3 p = g1l1 + g2l2 + g3l3 3ͭͷεϐʔΧʔͰͷ3D-VBAP, Pulkki 1997

Slide 25

Slide 25 text

εϐʔΧʔʹνϟϯωϧΛׂΓ౰ͯΔैདྷͷํ๏ͩͱ খ͞ͳσόΠεʹεςϨΦεϐʔΧʔΛ෇͚ͯ΋ Իͷ޿͕Γ͕͍·͍ͪ ৼ෯ ԻݯͷҐஔ L R Իͷ޿͕Γ͸ εϐʔΧʔͷִ͕ؒ࠷େ 😕 Equal Power Panning

Slide 26

Slide 26 text

σόΠεͷ෯Ҏ্ͷ޿͕ΓΛײ͡ΒΕΔΑ͏ʹ ϨϯμϦϯά͢Δ͜ͱ΋Մೳʂ

Slide 27

Slide 27 text

εςϨΦԻݯΛͦͷ··ϔουϗϯͰௌ͘ͱ Իݯ͕಄ͷதʹ͋ΔΑ͏ͳ…ʁ ಄಺ఆҐ 🎸 ϔουϗϯ޲͚ϨϯμϦϯά 㽈 ♪

Slide 28

Slide 28 text

ਓ͸࠷ऴతʹ2ͭͷࣖͰԻΛฉ͘ ྆ࣖͷೖΓޱͷԻΛ࠶ݱͰ͖Ε͹ɺۭؒతͳ޿͕Γ͕ײ͡ΒΕΔ͸ͣ ϔουϗϯ޲͚ϨϯμϦϯά

Slide 29

Slide 29 text

ϔουϗϯ޲͚ϨϯμϦϯά ڑ཭ʹΑΔݮਰɾ஗Ԇ ํ޲ʹΑΔ ࠨӈͷฉ͑͜ํͷҧ͍ + ಄෦ʹΑΔ ःṭޮՌ ಄෦఻ୡؔ਺ͰγϛϡϨʔτ HRTF: Head Related Transfer Function

Slide 30

Slide 30 text

ϔουϗϯ޲͚ϨϯμϦϯά ΦϒδΣΫτͷҐஔ৘ใ ڑ཭3 ํ֯П В ΦϒδΣΫτͷԻ ڑ཭ʹԠͯ͡ ݮਰɾ஗Ԇ )35'@-FGU )35'@3JHIU 3 П В .*9&3 ΦϒδΣΫτͷҐஔ৘ใ ڑ཭3 ํ֯П В ΦϒδΣΫτͷԻ ڑ཭ʹԠͯ͡ ݮਰɾ஗Ԇ )35'@-FGU )35'@3JHIU 3 П В ʜ 3 -

Slide 31

Slide 31 text

ैདྷ͔Βͷϔουϗϯ޲͚ͷख๏ όΠϊʔϥϧ࿥Ի μϛʔϔου or ࿥ԻऀͷࣖʹϚΠΫΛ૷ண ಄෦఻ୡؔ਺·ͰՃຯͨ͠Ի͕࿥ԻͰ͖Δ NEUMANN KU100

Slide 32

Slide 32 text

ීٴ͍ͯ͠ΔΦϒδΣΫτΦʔσΟΦͷܗࣜ • Dolby Atmos • DTS:X • 360 Reality Audio

Slide 33

Slide 33 text

Dolby Atmos • Dolby Laboratories, Inc.ʹΑͬͯөըԻڹͷͨΊʹ։ൃ͞Εͨ • ࠷େ 7.1.2 ch ͷϕου + 118ݸͷΦϒδΣΫτ 3ͭ໨ͷ਺ࣈ.2͸্ํͷνϟϯωϧ਺ • ैདྷͷԻ੠ه࿥ํࣜ Dolby Digital Plus (E-AC-3) ΍Dolby TrueHDͷ   ֦ுྖҬʹه࿥͞ΕԼҐޓ׵ੑ͕͋Δ ↑ ഑৴޲͚ ↑ ϒϧʔϨΠ޲͚ Lossless

Slide 34

Slide 34 text

ैདྷͷػث΁ͷԼҐޓ׵ੑͷ࢓૊Έ • Dolby AtmosඇରԠͷσίʔμʔ͸֦ுྖҬΛແࢹ͢Δ • ΦϒδΣΫτͷԻ΋͋Β͔͡ΊνϟϯωϧʹϨϯμϦϯά͞Ε͍ͯΔͷͰ   ඇରԠͷػثͰ΋ௌ͚Δ • ରԠػث͸֦ுྖҬͷ৘ใ΋ซͤͯ࢖͏ࣄͰΦϒδΣΫτͱຊདྷͷϕουͷԻΛ࠶ݱ %PMCZ%JHJUBM1MVT PS %PMCZ5SVF)% %PMCZ"UNPT ֦ுྖҬ νϟϯωϧͷ೾ܗ Joint Object Coding

Slide 35

Slide 35 text

DTS:X • DTSࣾʹΑΔϑΥʔϚοτ • ൺֱతॊೈͳεϐʔΧʔϨΠΞ΢τ • ैདྷͷνϟϯωϧϕʔεͷԻ੠ه࿥ํࣜ DTS-HD Master Audioͷ   ֦ுྖҬʹه࿥͞ΕΔ • Ի࣭Λॏࢹ͠ѹॖ཰͸௿͍ͨΊɺ༰ྔʹ༨༟ͷ͋ΔBlu-rayσΟεΫ޲͖ %54%JHJUBM4VSSPVOE %54)%.BTUFS"VEJP %549

Slide 36

Slide 36 text

360 Reality Audio • SonyʹΑָͬͯۂͷετϦʔϛϯά༻్Ͱ࡞ΒΕͨ • ଍ݩΛؚΉશํҐ͔ΒͷԻʹରԠ • MPEG-H 3D AudioΛϕʔεʹ   ഑৴ϏοτϨʔτͳͲͷऔΓܾΊΛ௥Ճͨ͠΋ͷ • σίʔυͱϨϯμϦϯά͕Ͱ͖ΔϥΠϒϥϦ͕ఏڙ͞Ε͍ͯΔ αϯϩΫϚϧ

Slide 37

Slide 37 text

ۭؒΦʔσΟΦ͸ Ϛϧννϟϯωϧͱ ΦϒδΣΫτϕʔεΦʔσΟΦ ͷར఺Λੜ࣮ͯ͠ݱ͞Ε͍ͯΔ

Slide 38

Slide 38 text

🕊 ͔͋ͨ΋ͦ͜ʹԻݯ͕͋Δ͔ͷΑ͏ͳମݧ ϨϯμϦϯά

Slide 39

Slide 39 text

ഭྗ͋Δ௿Ի ϚϧννϟϯωϧͰͷ௿Իͷ෼཭

Slide 40

Slide 40 text

iPhone΍MacBookͷ಺ଂεϐʔΧʔ͔Βͷ ޿͕Γ͋Δ๛͔ͳα΢ϯυ ࠶ੜػثʹ߹ΘͤͨϨϯμϦϯά

Slide 41

Slide 41 text

ΞϓϦͰۭؒΦʔσΟΦΛ ར༻͢Δʹ͸

Slide 42

Slide 42 text

ۭؒΦʔσΟΦͷϢʔεέʔε ಈըɾԻָ ίϯςϯπ࠶ੜ ήʔϜ FaceTimeͳͲͷΞϓϦ 🕊

Slide 43

Slide 43 text

ಈըɾԻָίϯςϯπ࠶ੜ༻API • AVPlayer • AVSampleBufferAudioRenderer   AVSampleBufferRenderSynchronizerͱͷ૊Έ߹ΘͤʹΑΔΧελϜϓϨΠϠʔ • ϒϥ΢βͷ Media Source Extension λά Ϛϧννϟϯωϧ΍Dolby Atmosͷ࠶ੜʹରԠ͍ͯ͠ΔϓϨΠϠʔ

Slide 44

Slide 44 text

allowedAudioSpatializationFormats ϓϩύςΟ ίϯςϯπΛ“ۭؒԽ”ͯ͠ྑ͍͔Ͳ͏͔ AVPlayerItem, AVSampleBu ff erAudioRenderer .monoAndStereo ϞϊϥϧͱεςϨΦͷۭؒԽΛڐՄ .multichannel ϚϧννϟϯωϧͷΈۭؒԽΛڐՄ Իָͷ৔߹ͷσϑΥϧτ .monoStereoAndMultichannel ϞϊϥϧɾεςϨΦɾϚϧννϟϯωϧͷ ۭؒԽΛڐՄ ಈըͷ৔߹ͷσϑΥϧτ

Slide 45

Slide 45 text

ۭؒԽ (spatialization)ॲཧͱ͸ʁ ⚠ ίϯιʔϧϩά͔ΒͷਪଌͰ͢ Decoder AUScotty 2ch → 5.1ch Up Mix AUSpatial Mixer Stereo L R ηϯαʔ஋ Decoder AUSpatial Mixer Multichannel L R ηϯαʔ஋ 👆ίίʂ(ͨͿΜ)

Slide 46

Slide 46 text

ۭؒԽ͠ͳ͍ͱ… Decoder Stereo L R Decoder Multichannel L R ❌ ैདྷ௨ΓͷεςϨΦ LͱR͔͠ฉ͑͜ͳ͍ʂ item.allowedAudioSpatializationFormats = []

Slide 47

Slide 47 text

Ի੠ίϯςϯπͷσϑΥϧτ͸.multichannel Decoder Stereo L R ैདྷ௨ΓͷεςϨΦ Decoder AUSpatial Mixer Multi-channel L R ηϯαʔ஋ όΠϊʔϥϧ࿥Ի͸͢Ͱʹ྆ࣖͷೖΓޱͷԻʹ ͳ͍ͬͯΔͨΊۭؒԽ͢Δ΂͖Ͱ͸ͳ͍

Slide 48

Slide 48 text

HTTP Live StreamingͱۭؒԽ • ෯޿͍σόΠεΛαϙʔτ͢Δඞཁ͕͋ΔͨΊɺଟ͘ͷVariantΛ༻ҙ͢Δ • ࠶ੜ୺຤͸ࣗ਎͕࠶ੜͰ͖ΔVariantΛ௨৴ঢ়ଶΛՃຯͯ͠ಈతʹબͿ HLSʹΑΔετϦʔϜ഑৴ AAC BINAURAL AC3 5.1ch EAC3 w/JOC Dolby Atmos AAC 2ch • allowedAudioSpatializationFormatsͰ   ڐՄ͞Ε͍ͯͳ͍΋ͷ͸બ͹Εͳ͍ͷͰ   جຊ .monoStereoAndMultichannel • όΠϊʔϥϧ൛͸ઃఆʹΑΒۭͣؒԽ͞Εͳ͍   CHANNEL=“2/-/BINAURAL” VariantؒͷԻͷେ͖͞Λἧ͑ΔͨΊ ϝλσʔλΛೖΕΔͷ͕๬·͍͠

Slide 49

Slide 49 text

ۭؒΦʔσΟΦͷग़ྗ͕Մೳ͔ʁ AVAudioSessionPortDescription isSpatialAudioEnabled iOS, tvOS ϋʔυ΢ΣΞͷೳྗ ੾Γସ͑ΒΕΔ৔߹ ۭؒԽΦϯ +

Slide 50

Slide 50 text

ग़ྗઌ ۭؒΦʔσΟΦ isSpatialAudioEnabled iPhone 14 Pro ಺ଂεϐʔΧʔ ੾Γସ͑ෆՄ true iPhone 5s ಺ଂεϐʔΧʔ ੾Γସ͑ෆՄ false AirPods (ୈ̏ੈ୅) Φϑ false AirPods (ୈ̏ੈ୅) ݻఆ, ϔουτϥοΩϯά true SOUNDPEATS BluetoothΠϠϗϯ ੾Γସ͑ෆՄ false isSpatialAudioEnabledͷ஋ͷྫ ಺ଂεϐʔΧʔͷ ۭؒΦʔσΟΦରԠ͸ iPhone XS ͔Βʂ

Slide 51

Slide 51 text

Control CenterͰͷۭؒԽͷ੾Γସ͑ • NowPlayingͷରԠΛߦ͏   • બ୒࣌ʹ௨஌͕དྷΔ     AVAudioSession   .spatialPlaybackCapabilitiesChangedNotification     w/ AVAudioSessionSpatialAudioEnabledKey

Slide 52

Slide 52 text

• ͜ͷ෦෼͸AVPlayer͕දࣔΛ؅ཧ • AVPlayerҎ֎Λར༻࣌͸     AVAudioSession  setSupportsMultichannelContent(_:)    Ͱઃఆ  

Slide 53

Slide 53 text

• AURemoteIO • AudioQueue NowPlayingͷରԠΛ͢Ε͹ɺόοϑΝΛ௚઀ग़ྗ͢Δέʔε΋ۭؒԽͰ͖Δ

Slide 54

Slide 54 text

ۭؒΦʔσΟΦ”ରԠ”ͷ iPhone ෳ਺ཁૉͷ૊Έ߹ΘͤͰ࣮ݱ͞Ε͍ͯΔͷͰ ඞཁͳػೳͷαϙʔτൣғΛνΣοΫʂ https://support.apple.com/ja-jp/HT211775 AirPods ΍ Beats ͰۭؒΦʔσΟΦΛମݧ͢Δ https://developer.dolby.com/platforms/apple/ios/device-support/ Dolby Audio Apple iOS Device Support Dolby Developer iPhone 5s Dolby Digital, Dolby Digital Plus ͷσίʔμ౥ࡌ ※ Appleͷࢿྉʹ͸هࡌͳ͠ iPhone 7 AirPodsͱ࿈ܞՄ iPhone XS Dolby AtmosͷσίʔμΛ౥ࡌ ಺ଂεϐʔΧۭؒΦʔσΟΦରԠ ͪͳΈʹMacͷ಺ଂεϐʔΧʔ͸AppleγϦίϯ͔ΒରԠ ͔ͳΓߴ඼ҐͳͷͰͥͻDolbyͷαϯϓϧͳͲΛฉ͍ͯΈͯ

Slide 55

Slide 55 text

ۭؒΦʔσΟΦରԠͷΠϠϗϯɾϔουϗϯ • AirPods (ୈ̏ੈ୅) • AirPods Pro • AirPods MAX • BeatsʢAppleࡿԼʣͷରԠϞσϧ ΋͏গ͋͠Δͱ͍͍ͳ͊ 🤔 ඇରԠͷBluetoothΠϠϗϯʹ޲͚ͯग़དྷΔ͜ͱ • όΠϊʔϥϧ൛Λఏڙ • ΞϓϦ಺ͰϨϯμϦϯά·Ͱߦͬͯ͠·͏

Slide 56

Slide 56 text

ήʔϜ΍FaceTimeͷΑ͏ͳػೳͷͨΊͷAPI • PHASE   Physical Audio Spatialization Engine • RealityKit, Reality Composer Pro • SceneKit 🕊 WWDC21 ~ 23 ʹ ଟ਺ηογϣϯ͋Γ😀 ҎԼͷFrameworkͰαϙʔτ

Slide 57

Slide 57 text

Recap • ۭؒΦʔσΟΦʹΑͬͯಘΒΕΔମݧ • ϚϧννϟϯωϧͱΦϒδΣΫτϕʔεΦʔσΟΦͷར఺ • ΞϓϦͰ׆༻͢ΔͨΊͷAPIͱͦͷڍಈ

Slide 58

Slide 58 text

f ind a key moment WWDC23 Principles of spatial designΑΓ ฉ͑͜ํͷಛ௃ 1ͭ1ͭͷԻݯͷԻ͕ࠞ͟Βͳ͍ ۭؒͷ೚ҙͷҐஔʹ഑ஔͰ͖Δ ࿩ऀ΍ָثͷύʔτͳͲΛฉ͖෼͚΍͍͢ ॏ௿Ի΋ײ͡΍͍͢ …

Slide 59

Slide 59 text

ࢀߟʹͨ͠WWDCηογϣϯ WWDC21 Immerse your app in Spatial Audio νϟϨϯδ: ۭؒΦʔσΟΦͷα΢ϯυεέʔϓ ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠