An iOS Developer's OpenGL Primer

Slides from my talk at NSScotland in October 2011. An explanatory introduction to OpenGL for iOS programmers.

More information here: http://www.blog.montgomerie.net/slides-from-an-ios-developers-opengl-primer

A word of warning: the slides are designed as an aid to the talk, not the other way around. There’s lots of information that was in the talk that’s not on the slides, and that might make them confusing or hard to follow. If there was video from the talk, that would be much better than slides, but there’s not, so slides it is.

Transcript

1. An iOS Developer’s OpenGL Primer Things Made Out Of Other Things Jamie Montgomerie @th_in_gs
7. OpenGL ES 2.0
8. A transparent, lightweight, interface to modern graphics hardware. OpenGL ES 2.0
9. Why Use OpenGL?
10. Why Use OpenGL? Flexibility
11. Why Use OpenGL? Flexibility Performance
12. How OpenGL uses your hardware to put stuff on the screen.
13. How you can use OpenGL to put your stuff on the screen.
14. Demo
15. Demo
16. What makes all that stuff happen?
17. A transparent, lightweight, interface to modern graphics hardware.
18. A transparent, lightweight, interface to modern graphics hardware.
19. A transparent, lightweight, interface to modern graphics hardware.
20. Where do the pixels come from?
21. Demo
22. Where do the pixels come from?
23. Where do the pixels come from? • Fragment shader.
24. Where do the pixels come from? • Fragment shader. • Simple program that runs on the GPU.
25. Where do the pixels come from? • Fragment shader. • Simple program that runs on the GPU. • Computes the shade for each fragment, or pixel.
26. Where do the pixels come from? • Fragment shader. • Simple program that runs on the GPU. • Computes the shade for each fragment, or pixel. • Run once for every pixel simultaneously.
27. Where do the pixels come from? • Fragment shader. • Simple program that runs on the GPU. • Computes the shade for each fragment, or pixel. • Run once for every fragment independently.
28. Where do the pixels come from? • Fragment shader. • Simple program that runs on the GPU. • Computes the shade for each fragment, or pixel. • Run once for every pixel simultaneously.
29. OpenGL ES 2.0 Shading Language
30. OpenGL ES 2.0 Shading Language uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
31. Scalars float int bool Vectors [i,b]vec2 [i,b]vec3 [i,b]vec4 Matrices mat2 mat3 mat4 Samplers sampler2d samplerCube Shading Language Basic Types
32. float i = 1.0; Shading Language Variable Initialisation
33. float i = 1.0; const float too = 2.0; Shading Language Variable Initialisation
34. float i = 1.0; const float too = 2.0; Shading Language Variable Initialisation
35. float i = 1.0; const float too = 2.0; int j = 1; Shading Language Variable Initialisation
36. Shading Language Variable Initialisation 1 2 3 4 v =
37. vec4 v = vec4(1.0, 2.0, 3.0, 4.0); Shading Language Variable Initialisation
38. vec4 v = vec4(1.0, 2.0, 3.0, 4.0); mat2 m = mat2(1.0, 0.0, // 1st Column 0.0, 1.0); // 2nd Column Shading Language Variable Initialisation
39. OpenGL ES 2.0 Shading Language uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
40. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
41. Simples.fsh void main() { gl_FragColor = vec4(0.0, 0.0, 0.0, 1.0); }
42. Simples.fsh void main() { gl_FragColor = vec4(0.0, 0.0, 0.0, 1.0); }
43. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
44. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
45. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
46. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
47. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
48. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
49. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
50. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
51. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
52. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
53. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
54. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
55. Demo
56. Demo
57. Where do the pixels come from?
58. Where do the pixels come from?
59. Where do the pixels come from?
60. Where do the pixels come from? pixels
61. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }
62. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate;
63. Colours.fsh uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; Where are these inputs coming from?
64. Where are these inputs coming from?
65. Where are these inputs coming from? • uniforms
66. Where are these inputs coming from? • uniforms • Your app code
67. Where are these inputs coming from? • uniforms • Your app code • Objective-C (or other language - C, C++, Java etc.)
68. Where are these inputs coming from?
69. Where are these inputs coming from? • varyings
70. Where are these inputs coming from? • varyings • Vertex shader
71. Where are these inputs coming from? • varyings • Vertex shader • Simple program that runs on the GPU.
72. Where are these inputs coming from? • varyings • Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code.
73. Where are these inputs coming from? • varyings • Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously.
74. Where are these inputs coming from? • varyings • Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex independently.
75. Where are these inputs coming from? • varyings • Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously.
76. What’s a Vertex?
77. What’s a Vertex? • A vertex is a point
78. What’s a Vertex? • A vertex is a point • Represents
79. What’s a Vertex? • A vertex is a point • Represents • Disconnected point
80. What’s a Vertex? • A vertex is a point • Represents • Disconnected point • Line end point
81. What’s a Vertex? • A vertex is a point • Represents • Disconnected point • Line end point • Triangle corner point
82. What’s a Vertex? • A vertex is a point • Represents • Disconnected point • Line end point • Triangle corner point • Two or three dimensional (vec2, vec3)
83. What’s a Triangle Strip?
84. What’s a Triangle Strip? • An array of points...
85. What’s a Triangle Strip? • An array of points... ...an array of vec2s or vec3s, in Shading Language terms.
86. What’s a Triangle Strip?
89. 1 2
90. 1 2 3
91. 1 2 3
92. 1 2 3 4
93. 1 2 3 4
94. 1 2 3 4
95. 5 1 2 3 4
96. 5 6 1 2 3 4
97. 7 5 6 1 2 3 4
98. 8 7 5 6 1 2 3 4
99. 8 7 5 6 1 2 3 4 9
100. Where do the Fragment Shader inputs come from? • varyings • Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously.
101. Where do the Fragment Shader inputs come from? • varyings • Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously. • Calculates the position of the vertex...
102. Where do the Fragment Shader inputs come from? • varyings • Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously. • Calculates the position of the vertex... • ...and arbitrary other inputs to pass to the fragment shaders.
103. Vertex Shader
104. Vertex Shader attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }
105. Colours.vsh attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }
106. Colours.vsh attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }
107. Colours.vsh attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }
108. Colours.vsh attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }
109. Colours.vsh attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }
110. Colours.vsh attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }
111. Colours.vsh attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }
112. Demo
113. Demo
114. Demo
115. Where do the pixels come from? pixels
116. Where do the fragment shader inputs come from?
117. Where do the fragment shader inputs come from?
118. Where do the fragment shader inputs come from? varyings
119. Where do the shader inputs come from? varyings
120. Where do the shader inputs come from? varyings
121. Where do the shader inputs come from? varyings
122. Where do the shader inputs come from? varyings uniforms uniforms
123. Where do the shader inputs come from? varyings uniforms uniforms & attributes
124. The OpenGL ES 2.0 Pipeline varyings uniforms uniforms & attributes
125. Modern Graphics Hardware Pipeline varyings uniforms uniforms & attributes
126. The OpenGL ES 2.0 Pipeline varyings uniforms uniforms & attributes
127. What does the app code do?
128. What does the app code do?
129. What does the app code do? • OpenGL content is presented in a CALayer...
130. What does the app code do? • OpenGL content is presented in a CALayer... • ...specifically, a CAEAGLLayer.
131. What does the app code do? • OpenGL content is presented in a CALayer... • ...specifically, a CAEAGLLayer. • Use a UIView backed by a CAEAGLLayer...
132. What does the app code do? • OpenGL content is presented in a CALayer... • ...specifically, a CAEAGLLayer. • Use a UIView backed by a CAEAGLLayer... • ...like Xcode template and sample code’s EAGLView.
133. What does the app code do? • OpenGL content is presented in a CALayer... • ...specifically, a CAEAGLLayer. • Use a UIView backed by a CAEAGLLayer... • ...like Xcode template and sample code’s EAGLView. • Use the C OpenGL API to draw.
134. Integrating with UIKit
135. Integrating with UIKit • You can’t use the setNeedsDisplay model.
136. Integrating with UIKit • You can’t use the setNeedsDisplay model. • Draw explicitly when you have something to display.
137. Integrating with UIKit • You can’t use the setNeedsDisplay model. • Draw explicitly when you have something to display. • Draw on a timer (e.g. 30 frames per second).
138. Integrating with UIKit • You can’t use the setNeedsDisplay model. • Draw explicitly when you have something to display. • Draw on a timer (e.g. 30 frames per second). • Use the CADisplayLink timer class.
139. C OpenGL API
140. C OpenGL API • Context based
141. C OpenGL API • Context based • Like CoreGraphics’ CGContext.
142. C OpenGL API • Context based • Like CoreGraphics’ CGContext. • No state save/restore.
143. C OpenGL API • Context based • Like CoreGraphics’ CGContext. • No state save/restore. • Not thread safe.
144. C OpenGL API • Context based • Like CoreGraphics’ CGContext. • No state save/restore. • Not thread safe. • Use from one thread at a time.
145. C OpenGL API • Context based • Like CoreGraphics’ CGContext. • No state save/restore. • Not thread safe. • Use from one thread at a time. • Use share pools to share resources between threads.
146. C OpenGL API
147. C OpenGL API • Functions to upload things to the GPU, e.g.:
148. C OpenGL API • Functions to upload things to the GPU, e.g.: • Shader programs
149. C OpenGL API • Functions to upload things to the GPU, e.g.: • Shader programs • Textures
150. C OpenGL API • Functions to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes
151. C OpenGL API • Functions to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes • Functions to set context state, e.g.:
152. C OpenGL API • Functions to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes • Functions to set context state, e.g.: • Active texture
153. C OpenGL API • Functions to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes • Functions to set context state, e.g.: • Active texture • Clear colour
154. C OpenGL API • Functions to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes • Functions to set context state, e.g.: • Active texture • Clear colour • Functions to perform rendering
155. Demo
156. Demo
157. Demo
158. Demo
159. Why Use OpenGL?
160. Why Use OpenGL? Flexibility
161. Why Use OpenGL? Flexibility Performance
162. Why Use OpenGL? Flexibility Performance 3D Stuff?
163. What about 3D stuff?
164. What about 3D stuff? • Upload models as in triangle strips made of 3D vec3 points.
165. What about 3D stuff?
166. What about 3D stuff? • Do positioning maths in the vertex shaders
167. What about 3D stuff? • Do positioning maths in the vertex shaders • One uniform mat4 to position each strip - a model matrix.
168. What about 3D stuff? • Do positioning maths in the vertex shaders • One uniform mat4 to position each strip - a model matrix. • One uniform mat4 to position the world around the camera, calculate perspective - a projection matrix.
169. What about 3D stuff?
170. What about 3D stuff? • Do lighting maths in the vertex shaders.
171. What about 3D stuff? • Do lighting maths in the vertex shaders. • Upload
172. What about 3D stuff? • Do lighting maths in the vertex shaders. • Upload • light information
173. What about 3D stuff? • Do lighting maths in the vertex shaders. • Upload • light information • arrays of vertex normals
174. What about 3D stuff? • Do lighting maths in the vertex shaders. • Upload • light information • arrays of vertex normals • Calculate per-vertex lighting brightness, color
175. What about 3D stuff? • Do lighting maths in the vertex shaders. • Upload • light information • arrays of vertex normals • Calculate per-vertex lighting brightness, color • Formulas online
176. What about 3D stuff? • Do lighting maths in the vertex shaders. • Upload • light information • arrays of vertex normals • Calculate per-vertex lighting brightness, color • Formulas online • Pass as varying to fragment shader.
177. What about 3D stuff?
178. What about 3D stuff? • Do texturing, other per-pixel effects in the fragment shaders.
179. What about 3D stuff? • Do texturing, other per-pixel effects in the fragment shaders. • Multiply the pixel colour with the light colour.
180. Learning More
181. Learning More • OpenGL ES Programming Guide for iOS; Apple
182. Learning More • OpenGL ES Programming Guide for iOS; Apple • OpenGL ES 2.0 Programming Guide; Munshi, Ginsburg & Shreiner
183. Learning More • OpenGL ES Programming Guide for iOS; Apple • OpenGL ES 2.0 Programming Guide; Munshi, Ginsburg & Shreiner • PowerVR Insider web site
184. Learning More • OpenGL ES Programming Guide for iOS; Apple • OpenGL ES 2.0 Programming Guide; Munshi, Ginsburg & Shreiner • PowerVR Insider web site • Watch out for great new things in iOS 5!
185. github.com/th-in-gs/Colours Jamie Montgomerie @th_in_gs jamie@montgomerie.net www.blog.montgomerie.net th.ingsmadeoutofotherthin.gs Things Made Out Of Other Things