No Shaders? No Worries Let’s Talk about Metal Render Pipeline

KWDC25 Presentation

Transcript

1. Rei (Kim MinGuk), September 9th, 2025 No Shaders? No Worries Let’s Talk about Metal Render Pipeline
2. Rei Kim MinGuk nekodoko315 rei315
3. Agenda • What is Metal? • How can Metal be used in Non-Gaming Apps? • Understanding the Components of the Metal Pipeline and their Behavior • Building Efficient Pipelines with the Latest in Metal 4
4. What is Metal ?
5. !"QQMF
6. !"QQMF
7. What is Metal • Low-Level access to the GPU • Graphics rendering • Parallel computation
8. !"QQMF
9. How can Metal be used in Non-Gaming Apps?
10. Why do we use Metal ?
11. !"QQMF
12. !"QQMF
13. Better Performance
14. Example
15. • Rendering Engine with Metal • Opens a 1000-page scanned book in an instant PDF Expert
16. • Express wind radar using Metal Yahoo ఱؾ @After iOSDC Japan 2023 @https://www.docswell.com/s/ydnjp/KYW9DL-2023-09-14-143000#p1
17. • Display subtitles for live video in pip mode χίχί(niconico)
18. • Alpha packed Video Player • VideoToolBox + Metal • Broadcast Filter E ff ects • Recently, I have considered introducing UI animation as well • etc Pococha
19. • Holographic • Pixel • Fluid • Marquee • etc Custom UI Effects with SwiftUI
20. Understanding the Components of the Metal Pipeline and their Behavior
21. GPU Basic
22. Point Line Strip Line Triangle Triangle Strip
23. Vertex Vertices Vertices
24. Vertex processing // shader.metal #include <metal_stdlib> using namespace metal; vertex float4 vertexFunction( uint vid [[vertex_id]], constant float2* vertices [[buffer(0)]] ) { return float4(vertices[vid], 0.0, 1.0); } // render.swift let vertices: [Float] = [ -0.5, -0.5, 0.5, -0.5, 0.0, 0.5 ]
25. Rasterization x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
26. Fragment processing // shader.metal #include <metal_stdlib> using namespace metal; fragment float4 fragmentFunction() { return float4(0.5, 0.5, 0.5, 1.0); }
27. Present
28. Metal Components
29. • MTLDevice • MTLCommandQueue • MTLCommandBuffer • MTLRenderPipelineState • MTLRenderPipelineDescriptor • MTLRenderPassDescriptor • MTLRenderCommandEncoder ɾ ɾ ɾ • MTLDevice • MTL4CommandQueue • MTL4CommandBuffer • MTL4RenderPipelineState • MTL4RenderPipelineDescriptor • MTL4RenderPassDescriptor • MTL4RenderCommandEncoder ɾ ɾ ɾ Metal Metal 4 (OS 26+)
30. MTLDevice • Abstraction of the physical GPU • Every Metal object-command queue, bu ff er, etc- originates from in • App’s entire interaction with the GPU funnels through a single MTLDevice instance
31. MTL4CommandQueue • Stores all the command bu ff ers and allow the application to control the order of execution of all commands. • MTLCommandQueue → MTL4CommandQueue Updated
32. MTL4CommandBuffer • Stores translated hardware commands for use by GPUs • A single command bu ff er can contain many di ff erent kinds of encoded commands • MTLCommandBu ff er → MTL4CommandBu ff er Updated
33. MTL4CommandAllocator • Explicit memory management for your application's command bu ff er memory use • Reuse command bu ff er memory and avoid dynamic allocations while encoding New
34. MTL4RenderPipelineState • Immutable, precompiled object in Metal that encapsulates all the GPU state needed to execute a graphics draw call. • De fi nes the entire rendering pipeline con fi guration, including shader features, pixel format, blending, and other fi xed-function settings • MTLRenderPipelineState → MTL4RenderPipelineState Updated
35. MTL4RenderPipelineDescriptor • Mutable con fi guration object used to describe the state of a Metal render pipeline • Con fi guration objects that contain settings necessary for GPU rendering, such as shader, render target format, blending, vertex layout, etc • MTLRenderPipelineDescriptor → MTL4RenderPipelineDescriptor Updated
36. MTL4RenderPassDescriptor • De fi nes how render targets (color, depth, stencil) are con fi gured and used in a render pass • To render directly to the current frame bu ff er (drawable), the currentMTL4RenderPassDescriptor provided by MTKView could be used • MTLRenderPassDescriptor → MTL4RenderPassDescriptor Updated
37. Pipeline RenderPass
38. MTL4RenderCommandEncoder • Responsible of translating rendering commands into hardware commands • Encodes rendering commands in the command bu ff er for a single rendering pass (MTL4RenderPassDescriptor) • MTLRenderCommandEncoder → MTL4RenderCommandEncoder Updated
39. MTL4ComputeCommandEncoder • Encoder used to encode GPU compute workloads in Metal • Encode all compute operations, including kernel dispatches, blits, and acceleration structure builds, within a single compute encoder • MTLComputeCommandEncoder → MTL4ComputeCommandEncoder Updated
40. MTL4ArgumentTable • Stores the binding points your app/scene/view/object etc needs • Create tables with a size based on the bind points New
41. 􀔪 Buffer (Generic Data) 􀏅 Texture *NBHF
    'PSNBU
    Data
42. !"QQMF
43. !"QQMF
44. MTLResidencySet • Specify the resources that Metal should make resident • Ensuring they are accessible to the hardware
45. Barrier API • Metal 4 is concurrency by default • Need to sync each stage with Barrier API New
46. Compute Render
47. Compute Render 􀀳
48. Compute Render
49. 􀁣 Barrier Compute Render
50. Metal Render Pipeline Behavior
52. MTKView CommandBu ff er CommandQueue Device RenderPipelineState RenderPipelineDescriptor RenderPassDescriptor RenderCommandEncoder Library
53. MTKView Device RenderPassDescriptor Library CommandBu ff er RenderCommandEncoder CommandQueue RenderPipelineState RenderPipelineDescriptor makeDefaultLibrary
54. MTKView CommandBu ff er CommandQueue Device RenderPipelineState RenderPipelineDescriptor RenderPassDescriptor RenderCommandEncoder Library device.makeRenderPipelineState( descriptor: pipelineDescriptor ) vertexFunction fragmentFunction colorAttachments[0].pixelFormat colorAttachments[0].~~~~ vertexFunction fragmentFunction
55. MTKView CommandBu ff er CommandQueue Device RenderPipelineState RenderPipelineDescriptor RenderPassDescriptor RenderCommandEncoder Library makeMTL4CommandQueue()
56. MTKView CommandBu ff er CommandQueue Device RenderPipelineState RenderPipelineDescriptor RenderPassDescriptor RenderCommandEncoder Library makeCommandBuffer()
57. MTKView CommandBu ff er CommandQueue Device RenderPipelineState RenderPipelineDescriptor RenderPassDescriptor RenderCommandEncoder Library commandBuffer.beginCommandBuffer( allocator: commandAllocator ) commandBuffer.makeRenderCommandEncoder( descriptor: currentMTL4RenderPassDescriptor ) renderCommandEncoder.setRenderPipelineState renderCommandEncoder.setArgumentTable residencySet.addAllocations([]) residencySet.commit() argumentTable.setTexture argumentTable.setSamplerState argumentTable.setAddress argumentTable.setResource renderCommandEncoder.drawPrimitives renderCommandEncoder.endEncoding
58. MTKView CommandBu ff er CommandQueue Device RenderPipelineState RenderPipelineDescriptor RenderPassDescriptor RenderCommandEncoder Library commandBuffer.endCommandBuffer() commandQueue.commit([commandBuffer]) commandQueue.signalDrawable(currentDrawable) currentDrawable.present() currentDrawable
59. MTKView CommandBu ff er CommandQueue Device RenderPipelineState RenderPipelineDescriptor RenderPassDescriptor RenderCommandEncoder Library
60. Building Efficient Pipelines with the Latest in Metal 4
61. Alpha Video Player • H.265 • Video Alpha Channel with 3 Planes (YCbCr + Alpha) • AVPlayer for control video playback • AVPlayerItemVideoOutput for CVPixelBu ff er • Metal for Rendering • Using Metal4
62. Metal4 Implementation
79. Performance Optimization
80. Allocate Resource Storage Up Front • Create MTLBu ff er or MTLTexture as early as possible • Reuse as much as possible • Avoid creating new resource during loop
81. Triple Buffering !"QQMF
82. Hold a Drawable as Briefly as Possible • Always acquire a drawable as late as possible • If possible, just before encoding the render path on the screen • Always release a drawable as soon as possible • If possible, immediately after completing the CPU operation of the frame.
83. Choose an Appropriate Load Action for render targets The previous content on the render target .dontCare • The previous content is discarded. The GPU does not preserve the existing data .clear • The render target is cleared to a speci fi ed clear color or value .load • The previous content is preserved and loaded for use in the render pass
84. Choose an Appropriate Store Action for render targets What to do with the rendered content after the render pass .dontCare • The rendered content is discarded. Use this if the output is not needed afterward .store • The rendered content is preserved and stored in memory for later use .storeAndMultisampleResolve • Resolves a multisample render target and stores the result .multisampleResolve • Resolves a multisample render target, but does not store the result
95. Other Optimizations
96. Other Optimizations Argument Bu ff ers Indirect Bu ff ers Resource Storage Mode Reuse Pipeline Specify compilation timing and thread priority ɾ ɾ ɾ
97. Apple Metal Best Practices Guide Metal Sample Code Library
98. Thank you for watching