飛び道具ではないMetal #iOSDC

19d7ae634445d4bd9b10c7961a462260?s=47 shu223
September 17, 2017

飛び道具ではないMetal #iOSDC

iOSDC 2017での登壇資料です。

[概要]

WWDC17でもMetalのセッション会場は閑古鳥が鳴いてました。Metalって3Dゲームとかカメラアプリとかのトリッキーなエフェクト用でしょ、うちには関係ないなぁ・・・と思われている方も多いかもしれません。しかし、UIKitの下回りもMetalですし、色んな所で実はMetalは暗躍しています。そんなMetalにもっと多くの開発者に興味を持ってもらうため、非ゲーム・非カメラアプリ開発者にも関係しそうな切り口からMetalやGPUについて紹介してみたいと思います。

19d7ae634445d4bd9b10c7961a462260?s=128

shu223

September 17, 2017
Tweet

Transcript

  1. iOSDC 2017 Shuichi Tsutsumi @shu223 ඈͼಓ۩Ͱ͸ͳ͍

  2. None
  3. WWDC 2014

  4. ͦ͏ʮΞϨʯ͕ൃද͞Εͨ೥

  5. None
  6. None
  7. SwiftͱMetal͸ಉ͍೥!

  8. WWDC2014ͰͷMetalͷσϞ

  9. WWDC2014ͰͷSwiftൃද

  10. ! !

  11. Metalʢ࠶ʣ

  12. " "

  13. ʰMetalɾɾɾ͏ͪʹ͸ؔ܎ͳ͍ͳ͊ʱ

  14. γΣʔμͰ͜Μͳʮදݱʯ΋Ͱ͖·͢ʂ .FUBMγΣʔμ͜ͱ͸͡Ί8FC(-(-4-ͷ๛෋ͳαϯϓϧΛࢀߟʹ͢Δ2JJUB ※ಈը΍ը૾Ͱ͸ͳ͘ɺ͢΂ͯϓϩάϥϜͰੜ੒͍ͯ͠·͢

  15. ʢઈରϓϩμΫτͰ͸΍Βͳ͍ɾɾɾʣ

  16. None
  17. Metal GPU • Core ML • Vision • Core Image

    • SceneKit • SpriteKit • MapKit
  18. Metal GPU • Core ML • Vision • Core Image

    • SceneKit • SpriteKit • MapKit • Core Animation • UIKit
  19. 143,000 MetalͷAPIΛ௚઀ར༻͍ͯ͠ΔΞϓϦͷ਺

  20. 143,000 17,000,000 MetalͷAPIΛ௚઀ར༻͍ͯ͠ΔΞϓϦͷ਺ MetalͷAPIΛ҉໧తʹར༻͍ͯ͠ΔΞϓϦͷ਺

  21. ͋ͳͨͷΞϓϦͷԼͰ΋ Metal͕҉༂͍ͯ͠Δʂ

  22. ඈͼಓ۩Ͱ͸ͳ͍.FUBM

  23. ఻͍͑ͨ͜ͱ ✓Metal͸ؔ܎ͳ͘ͳ͍ ✓Metalͷجૅ ✓Metalͷ࣮૷ํ๏

  24. imageView.image = image

  25. None
  26. Կ͕ى͖͍ͯΔͷ͔ʁ

  27. σΟεϓϨΠ ϓϩηοα ϑϨʔϜ
 όοϑΝ

  28. σΟεϓϨΠ ϓϩηοα ϑϨʔϜ
 όοϑΝ ը໘෼ͷ ϐΫηϧσʔλ

  29. σΟεϓϨΠ ϓϩηοα ϑϨʔϜ
 όοϑΝ ը໘෼ͷ ϐΫηϧσʔλ ϐΫηϧσʔλ
 දࣔ

  30. σΟεϓϨΠ ϓϩηοα ϑϨʔϜ
 όοϑΝ ը໘෼ͷ ϐΫηϧσʔλ ඵؒʹճ ॻ͖ࠐΉ ϐΫηϧσʔλ
 දࣔ

  31. σΟεϓϨΠ ϓϩηοα ϑϨʔϜ
 όοϑΝ ը໘෼ͷ ϐΫηϧσʔλ ඵؒʹճ ॻ͖ࠐΉ ϐΫηϧσʔλ
 දࣔ

    ղ૾౓͕ߴ͘ͳΔ΄Ͳେม
  32. CPUͱGPUͷੑ࣭ͷҧ͍ CPU͸εϙʔπΧʔ • ͱʹ͔͘଎͍ • Ұ౓ʹଟ͘ͷਓΛӡ΂ͳ͍ GPU͸όε • ಉ͡໨త஍ͷਓΛҰ౓ʹଟ͘ӡ΂Δ •

    ଎౓ࣗମ͸CPUʹ͸ٴ͹ͳ͍
  33. • CPU͸଎ͯ͘ɺԿͰ΋Ͱ͖Δʢ൚༻ϓϩηοαʣ - ͔ͱ͍ͬͯԿͰ΋΍ΒͤΔͱɺ͙͢ʹՔಇ཰100%ʹ • GPU͕ಘҙͳ࢓ࣄʢʹฒྻԋࢉʣ͸GPUʹ΍ΒͤΔ - ಉ͡ܭࢉΛߦ͏ॲཧ - ௖఺ͷܭࢉʢ࠲ඪม׵ʣ

    - ϥΠςΟϯά - ͦͷଞϐΫηϧ͝ͱͷॲཧ
  34. σΟεϓϨΠ ϓϩηοα ϑϨʔϜ
 όοϑΝ 1ඵؒʹ60ճ ॻ͖ࠐΉ • CPU͸৭Μͳ͜ͱͰ๩͍͠ • ϐΫηϧ͝ͱͷܭࢉɺϑϨʔϜόοϑΝ

    ΁ͷॻ͖ࠐΈ͸ฒྻॲཧOK ղ૾౓͕ߴ͘ͳΔ΄Ͳେม 1ը໘෼ͷϐΫηϧσʔλ
  35. σΟεϓϨΠ ϓϩηοα ϑϨʔϜ
 όοϑΝ 1ඵؒʹ60ճ ॻ͖ࠐΉ • CPU͸৭Μͳ͜ͱͰ๩͍͠ • ϐΫηϧ͝ͱͷܭࢉɺϑϨʔϜόοϑΝ

    ΁ͷॻ͖ࠐΈ͸ฒྻॲཧOK GPU ղ૾౓͕ߴ͘ͳΔ΄Ͳେม 1ը໘෼ͷϐΫηϧσʔλ
  36. మଇ: GPU͕ಘҙͳ࢓ࣄ͸GPUʹ

  37. GPUΛૢΔ

  38. None
  39. GPU΁ͷΞΫηεΛఏڙ GPU Your app ???

  40. OpenGLͱͷҧ͍

  41. OpenGL • Ҏલ͔Β͋ΔάϥϑΟοΫεAPI • ֤ࣾͷϋʔυ΢ΣΞɾϓϥοτϑΥʔϜʹରԠ • Appleͷϋʔυ΢ΣΞ͚ͩʹ࠷దԽ͞Ε͍ͯΔΘ͚Ͱ͸ ͳ͍

  42. Metal • Apple͕ಠࣗʹࡦఆ • Appleͷϋʔυ΢ΣΞʹ࠷దԽ • OpenGLΑΓ࠷େ10ഒߴ଎ • ࠓ͸OpenGL࣮૷ͷԼճΓ΋Metal

  43. ͷ࣮૷

  44. imageView.image = image

  45. None
  46. Metal APIΛʮ໌ࣔతʹʯୟ͍ͯ ը૾Λඳը͢Δ৔߹ͷ࠷খ࣮૷

  47. func draw(in view: MTKView) { guard let drawable = view.currentDrawable

    else {return} guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalError()} 
 guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fatalError()} let targetW = min(texture.width, drawable.texture.width) let targetH = min(texture.height, drawable.texture.height) blitEncoder.copy(from: texture, sourceSlice: 0, sourceLevel: 0, sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0), sourceSize: MTLSizeMake(targetW, targetH, texture.depth), to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0)) blitEncoder.endEncoding() commandBuffer.present(drawable) commandBuffer.commit() commandBuffer.waitUntilCompleted() } private let device = MTLCreateSystemDefaultDevice()! 
 private func setup() { commandQueue = device.makeCommandQueue() let textureLoader = MTKTextureLoader(device: device) texture = try! textureLoader.newTexture(name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil) mtkView.device = device mtkView.delegate = self mtkView.colorPixelFormat = texture.pixelFormat }
  48. ʮ࠷খݶʯͷ࣮૷Ͱొ৔͢ΔΫϥε • MTLDevice • MTLCommandQueue • MTLCommandBuffer • MTLCommandEncoder •

    MTLTexture • MTKView • MTKTextureLoader • etc…
  49. None
  50. $ ͔͕ͨը૾Λදࣔ͢Δ͙Β͍Ͱͳͥ͜Μͳʹɻɻ

  51. ཧղͷखҾ͖

  52. None
  53. $16͸ɺ(16΁ͷ໋ྩΛ
 ίϚϯυόοϑΝͱ͍͏ܗͰ࡞੒

  54. $16͸ɺ(16΁ͷ໋ྩΛ
 ίϚϯυόοϑΝͱ͍͏ܗͰ࡞੒ (16ʹ౉͢

  55. $16͸ɺ(16΁ͷ໋ྩΛ
 ίϚϯυόοϑΝͱ͍͏ܗͰ࡞੒ (16͸౉͞ΕͨίϚ ϯυΛ࣮ߦ͢Δ (16ʹ౉͢

  56. $16͸ɺ(16΁ͷ໋ྩΛ
 ίϚϯυόοϑΝͱ͍͏ܗͰ࡞੒ $16ͱ(16͸྆ํ͔ΒΞΫη εͰ͖ΔϝϞϦྖҬΛ࣋ͭ (16͸౉͞ΕͨίϚ ϯυΛ࣮ߦ͢Δ (16ʹ౉͢

  57. MTLCommandBuffer MTLTexture MTLDevice

  58. MTLCommandBuffer MTLTexture MTLDevice MTLCommandEncoder

  59. MTLCommandBuffer MTLTexture MTLDevice MTLCommandQueue MTLCommandEncoder

  60. MTLCommandBuffer MTLTexture MTLDevice MTLCommandQueue MTKTextureLoader MTLCommandEncoder

  61. ۩ମతʹݟ͍͖ͯ·͠ΐ͏

  62. MTLDevice let device = MTLCreateSystemDefaultDevice()! ※Τϥʔॲཧ౳͸লུ͍ͯ͠·͢

  63. MTLTexture, MTKTextureLoader // ϩʔμʔΛॳظԽ loader = MTKTextureLoader(device: device) // ςΫενϟΛΞηοτΧλϩά͔Βڞ༗ϝϞϦྖҬʹϩʔυ

    texture = try! loader.newTexture(
 name: “hoge", 
 scaleFactor: view.contentScaleFactor, 
 bundle: nil) ※Τϥʔॲཧ౳͸লུ͍ͯ͠·͢
  64. MTLCommandQueue, MTLCommandBuffer // ίϚϯυΩϡʔ͔ΒίϚϯυόοϑΝΛ࡞੒ let commandBuffer = commandQueue.makeCommandBuffer()! // …

    ίϚϯυόοϑΝʹίϚϯυ௥Ճ // ίϚϯυόοϑΝΛίϛοτʢʹΤϯΩϡʔ͞ΕΔʣ commandBuffer.commit() ※Τϥʔॲཧ౳͸লུ͍ͯ͠·͢ // ίϚϯυΩϡʔΛॳظԽ commandQueue = device.makeCommandQueue()
  65. MTLCommandEncoder • ίϚϯυόοϑΝʹίϚϯυΛ௥ՃʢΤϯίʔυʣ͢Δ MTLRenderCommandEncoder άϥϑΟοΫεϨϯμϦϯάίϚϯυΛΤϯίʔυ͢Δ MTLComputeCommandEncoder ฒྻԋࢉॲཧΛΤϯίʔυ͢Δ MTLBlitCommandEncode όοϑΝɾςΫενϟؒͷίϐʔॲཧΛΤϯίʔυ͢Δ

  66. MTLCommandEncoder (MTLBlitCommandEncoder) let blitEncoder = commandBuffer.makeBlitCommandEncoder()! blitEncoder.copy(from: texture, sourceSlice: 0,

    sourceLevel: 0, sourceOrigin: srcOrigin, sourceSize: srcSize, to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin:dstOrigin) blitEncoder.endEncoding() ※Τϥʔॲཧ౳͸লུ͍ͯ͠·͢
  67. MTLCommandEncoder (MTLBlitCommandEncoder) let blitEncoder = commandBuffer.makeBlitCommandEncoder()! blitEncoder.copy(from: texture, sourceSlice: 0,

    sourceLevel: 0, sourceOrigin: srcOrigin, sourceSize: srcSize, to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin:dstOrigin) blitEncoder.endEncoding() ※Τϥʔॲཧ౳͸লུ͍ͯ͠·͢ ίϐʔݩ ίϐʔઌ
  68. // ίϚϯυΩϡʔ͔ΒίϚϯυόοϑΝΛ࡞੒ let commandBuffer = commandQueue.makeCommandBuffer()! // ίϚϯυΤϯίʔμΛ࡞੒ let blitEncoder

    = commandBuffer.makeBlitCommandEncoder()! blitEncoder.copy(from: texture, sourceSlice: 0, sourceLevel: 0, sourceOrigin: srcOrigin, sourceSize: srcSize, to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin:dstOrigin) // ίϚϯυ௥Ճ׬ྃ blitEncoder.endEncoding() // ίϚϯυόοϑΝΛίϛοτʢʹΤϯΩϡʔ͞ΕΔʣ commandBuffer.commit() ※Τϥʔॲཧ౳͸লུ͍ͯ͠·͢
  69. private let device = MTLCreateSystemDefaultDevice()! private func setup() { commandQueue

    = device.makeCommandQueue() let textureLoader = MTKTextureLoader(device: device) texture = try! textureLoader.newTexture(name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil) mtkView.device = device mtkView.delegate = self mtkView.colorPixelFormat = texture.pixelFormat } func draw(in view: MTKView) { guard let drawable = view.currentDrawable else {return} guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalError()} 
 guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fatalError()} let targetW = min(texture.width, drawable.texture.width) let targetH = min(texture.height, drawable.texture.height) blitEncoder.copy(from: texture, sourceSlice: 0, sourceLevel: 0, sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0), sourceSize: MTLSizeMake(targetW, targetH, texture.depth), to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0)) blitEncoder.endEncoding() commandBuffer.present(drawable) commandBuffer.commit() commandBuffer.waitUntilCompleted() }
  70. private let device = MTLCreateSystemDefaultDevice()! private func setup() { commandQueue

    = device.makeCommandQueue() let textureLoader = MTKTextureLoader(device: device) texture = try! textureLoader.newTexture(name: "highsierra", scaleFactor: view.contentScaleFactor, bundle: nil) mtkView.device = device mtkView.delegate = self mtkView.colorPixelFormat = texture.pixelFormat } ίϚϯυόοϑΝ࡞੒ → ΤϯΩϡʔ →ʢGPU΁ૹ৴ʣ →ʢGPUͰίϚϯυ࣮ߦʣ →ʢඳըʣ func draw(in view: MTKView) { guard let drawable = view.currentDrawable else {return} guard let commandBuffer = commandQueue.makeCommandBuffer() else {fatalError()} 
 guard let blitEncoder = commandBuffer.makeBlitCommandEncoder() else {fatalError()} let targetW = min(texture.width, drawable.texture.width) let targetH = min(texture.height, drawable.texture.height) blitEncoder.copy(from: texture, sourceSlice: 0, sourceLevel: 0, sourceOrigin: MTLOrigin(x: 0, y: 0, z: 0), sourceSize: MTLSizeMake(targetW, targetH, texture.depth), to: drawable.texture, destinationSlice: 0, destinationLevel: 0, destinationOrigin: MTLOrigin(x: 0, y: 0, z: 0)) blitEncoder.endEncoding() commandBuffer.present(drawable) commandBuffer.commit() commandBuffer.waitUntilCompleted() } ɾɾɾίϚϯυΩϡʔੜ੒ ςΫενϟΛϩʔυ ϏϡʔͷηοτΞοϓ ɾɾɾσόΠεੜ੒
  71. % എܠʹ͋Δ֓೦Λ౿·͑Δͱҙ֎ͱ؆୯

  72. Powered by

  73. GPUΛૢΕΔΑ͏ʹʂ GPU Your app ???

  74. ૿͑ଓ͚ΔMetal׆༻ͷ৔ • ARKitͷΧελϜϨϯμϦϯά • MPS CNN / RNNΛ༻͍ͨػցֶश • Core

    ImageͷΧελϜϑΟϧλ
  75. ࢀߟॻ੶ GPU Metal

  76. ·ͱΊ • Metal͸ؔ܎ͳ͘ͳ͍ - UIKitͷԼճΓͱͯ͠΋҉༂ • GPU͕ಘҙͳ࢓ࣄ͸GPUʹ೚ͤΑ͏ • Metalͷجૅ -

    Metalͱ͸ / OpenGLͱͷൺֱ • Metalͷ࣮૷ - എܠʹ͋Δ֓೦Λཧղ͢Δͱ؆୯
  77. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ https://github.com/shu223