Poking Metal with s̶t̶i̶c̶k̶ swift

By Andrew Khacrhyshyn - https://www.facebook.com/andrew.kharchyshyn
Lightning intro to 3D graphics. Metal API overview. Going through all required steps to get 3D object rendered to the screen. Discussing how Metal works (and doesn’t work) with Swift.

Transcript

1. Poking Metal with Swift
2. About myself: • iOS Dev • RayWenderlich.com tutorial team member • One of WWDC15 scholarship winners • Hackathon enthusiast • Student @haawa799
3. 3D Graphics basics Everything is a triangle
4. 3D Graphics basics
9. 3D Graphics basics Rendering - process of getting 2D image from 3D scene.
10. 3D Graphics basics Rendering steps:
11. 3D Graphics basics Rendering steps: 1. Set scene geometry
12. 3D Graphics basics Rendering steps: 1. Set scene geometry 2. Apply transformations: Translate
13. 3D Graphics basics Rendering steps: 1. Set scene geometry 2. Apply transformations: Translate Rotate
14. 3D Graphics basics Rendering steps: 1. Set scene geometry 2. Apply transformations: Translate Rotate Scale
15. 3D Graphics basics Rendering steps: 1. Set scene geometry 2. Apply transformations 3. Map textures
16. 3D Graphics basics Rendering steps: 1. Set scene geometry 2. Apply transformations 3. Map textures 4. Add lighting
17. Fixed-Function Pipeline Programmable pipeline vs.
18. Programmable pipeline Allows developers to write small programs called shaders directly on the GPU
19. GPU
20. GPU =
21. GPU = =
22. Programmable pipeline Allows developers to write small programs called shaders directly on the GPU
23. Shaders
24. Shaders
25. Shaders Vertex shader
26. Shaders vertex VertexOut vertexShader(const Vertex vertexIn [[stage_in]], constant Uniforms &uniformBuffer [[buffer(1)]], unsigned int vid [[vertex_id]]) { float4x4 projectionMatrix = uniformBuffer.projectionMatrix; float4x4 modelViewMatrix = uniformBuffer.modelViewMatrix; float3 position = vertexIn.position; float4 fragmentPosition = float4(position, 1.0); VertexOut vertexOut; vertexOut.position = projectionMatrix * modelViewMatrix * fragmentPosition; vertexOut.occlusion = vertexIn.occlusion; vertexOut.texCoords = vertexIn.texCoords; vertexOut.color = float4(1.0,1.0,1.0,1.0); vertexOut.occlusionEnabled = uniformBuffer.occlusionEnabled; vertexOut.textureEnabled = uniformBuffer.textureEnabled; return vertexOut; }
27. Rasterisation
28. Shaders Fragment shader
29. Shaders fragment float4 fragmentShader(VertexOut interpolated [[stage_in]], texture2d<float> tex2D [[texture(0)]], sampler sampler2D [[sampler(0)]]) { float4 finalColor = interpolated.color; if (interpolated.occlusionEnabled) { finalColor *= interpolated.occlusion; } if (interpolated.textureEnabled) { finalColor *= tex2D.sample(sampler2D, interpolated.texCoords); } return finalColor; }
30. Metal GPU Core Animations Core Graphics SpriteKit & SceneKit OpenGL
31. Metal GPU Core Animations Core Graphics SpriteKit & SceneKit Metal iOS 9. OSX El Cap *
32. Metal GPU Core Animations Core Graphics SpriteKit & SceneKit Metal iOS 9. OSX El Cap *
33. Metal MTLDevice
34. Metal <MTLDevice>
35. Metal <MTLDevice> Abstraction over GPU Used to create different objects that Metal needs
36. Metal let device = MTLCreateSystemDefaultDevice() Abstraction over GPU Used to create different objects that Metal needs
37. Metal let metalLayer = CAMetalLayer() metalLayer.device = device metalLayer.pixelFormat = .BGRA8Unorm Connecting Metal and UIKIt
38. Metal drawable 0 drawable 1 drawable 2 let drawable = layer.nextDrawable()
39. Metal drawable 0 drawable 1 drawable 2 let drawable = layer.nextDrawable()
40. Metal drawable 0 drawable 1 drawable 2 let drawable = layer.nextDrawable()
41. Metal drawable 0 drawable 1 drawable 2 let drawable = layer.nextDrawable()
42. Metal drawable 0 drawable 2 drawable 1 let drawable = layer.nextDrawable()
43. Metal drawable 0 drawable 2 drawable 1 let drawable = layer.nextDrawable()
44. Metal let drawable = view.currentDrawable drawable 0 drawable 2 drawable 1
45. Metal MTKView : UIView MTKViewDelegate func mtkView(view: MTKView, drawableSizeWillChange size: CGSize) func drawInMTKView(view: MTKView)
46. Metal Render Pass Descriptor guard let renderPassDescriptor = view.currentRenderPassDescriptor else {return} renderPassDescriptor.colorAttachments[0].loadAction = .Clear renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColor(red: 1.0, green: 0.4, blue: 0.6, alpha: 1.0) renderPassDescriptor.colorAttachments[0].storeAction = MTLStoreAction.Store Frame buffer associated with MTKView
47. Metal Command queue let commandQ = device.newCommandQueue()
48. Metal Command queue let commandQ = device.newCommandQueue() command buffer command buffer command buffer command buffer command buffer command buffer GPU
49. Metal Command buffer let commandBuffer = commandQ.commandBuffer() command buffer command buffer command buffer command buffer command buffer command buffer GPU
50. Metal Command encoder Render, Compute, Blit
51. Metal Command encoder let renderEncoder = commandBuffer.renderCommandEncoderWithDescriptor(renderPassDescriptor) renderEncoder.drawPrimitives(.Triangle, vertexStart:0, vertexCount:3) commandBuffer.presentDrawable(drawable) commandBuffer.commit()
52. DEMO
53. Metal Pros and cons
54. More resources on Metal: • RayWenderlich.com • metalbyexample.com • flexmonkey.blogspot.com • WWDC 14 videos Q&A @haawa799 Sample code @ https://github.com/haawa799/ RamOnMetal