iOS OpenGL - Speaker Deck

Slide 1

Slide 1 text

Elastic Image Software LLC iOS OpenGL Douglass Turner Elastic Image Software email: [email protected] mobile: 781 775 3708

Slide 2

Slide 2 text

Elastic Image Software LLC 2 A Talk in Three Acts

Slide 3

Slide 3 text

Elastic Image Software LLC • Introduction to Computer Graphics • Computer Graphics in OpenGL • OpenGL and CocoaTouch Playing Nice Together 3

Slide 4

Slide 4 text

Elastic Image Software LLC 4 Introduction to Computer Graphics A Trip Down the Graphics Pipeline

Slide 5

Slide 5 text

Elastic Image Software LLC Copyright Steve Jubinville 5 Build and Tesselate Model into Facets

Slide 6

Slide 6 text

Elastic Image Software LLC Copyright Steve Jubinville 6 Pose Model

Slide 7

Slide 7 text

Elastic Image Software LLC Aim Camera 7

Slide 8

Slide 8 text

Elastic Image Software LLC 8 Design the Look

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Elastic Image Software LLC 12 Render Scene

Slide 13

Slide 13 text

Elastic Image Software LLC 13 Clip scene to the bounds of the viewing frustrum.

Slide 14

Slide 14 text

Elastic Image Software LLC Rasterize Scene 14

Slide 15

Slide 15 text

Elastic Image Software LLC 15 for each object in the scene ! determine its corresponding pixels ! for each corresponding pixel ! ! calculate depth of object at pixel ! ! if depth is closer than any object yet seen ! ! ! put this object in the pixel ! ! endif ! end end The GPU implements a z-buffer algorithm in hardware

Slide 16

Slide 16 text

Elastic Image Software LLC In OpenGL, the Look is evaluated at the same time that depth comparison is performed. Together these are the core activities of computer graphics. They are the raison d'être of the GPU. Compute intensive, Highly optimized and parallelized. 16

Slide 17

Slide 17 text

Elastic Image Software LLC 17 Update Model State. This could entail geometric transformation or perhaps taking a time step in the physic engine.

Slide 18

Slide 18 text

Elastic Image Software LLC 18 Rinse, repeat.

Slide 19

Slide 19 text

Elastic Image Software LLC 19 Example. Beautiful Panoramas iPad app. In the App Store: http://bit.ly/QVIDgg

Slide 20

Slide 20 text

Elastic Image Software LLC 20 Computer Graphics in OpenGL

Slide 21

Slide 21 text

Elastic Image Software LLC OpenGL Cocoa 21

Slide 22

Slide 22 text

Elastic Image Software LLC OpenGL is Old School 22

Slide 23

Slide 23 text

Elastic Image Software LLC You are ﬂipping levers on a large state machine. 23

Slide 24

Slide 24 text

Elastic Image Software LLC OpenGL is brutally spare. Just the essentials. float oldSchool[] glPushMatrix() glPopMatrix()! 24

Slide 25

Slide 25 text

Elastic Image Software LLC Keep in mind OpenGL is a dragster, not a Prius. So, just buckle up ... 25

Slide 26

Slide 26 text

Elastic Image Software LLC ... and enjoy the ride, baby! 26

Slide 27

Slide 27 text

Elastic Image Software LLC 27 Posing models. Positioning light sources. Aiming the camera. All these task require and understanding of transformation matrices.

Slide 28

Slide 28 text

Elastic Image Software LLC 28 Transformation matrix. Coordinate frame World space, Screen space, etc. All are different ways of thinking about the same underlying concept.

Slide 29

Slide 29 text

Elastic Image Software LLC 29 When we move an object we apply a transformation via a transformation matrix. This transformation is deﬁned in terms of a coordinate frame. A speciﬁc location on the transformed object can be interpreted in terms of any coordinate frame we choose.

Slide 30

Slide 30 text

Elastic Image Software LLC 30 The motivation for a coordinate frame transformation is not only motivated by the need to pose a model or aim a camera as in the physical world. In computer graphics land it often has to do with the convenience of performing a particular calculation in one space rather then another.

Slide 31

Slide 31 text

Elastic Image Software LLC 31 We refer to the transformation pipeline: the successive application of a series of transformations as a 3D model wends its way from application land to GPU land.

Slide 32

Slide 32 text

Elastic Image Software LLC Hierarchical modeling is based on the hierarchical organization of transformations matrices. Many objects - trees, robots, cars - form natural hierarchies. 32

Slide 33

Slide 33 text

Elastic Image Software LLC 33 The UIView class supports all the key concepts of transformations and hierarchy. It is a nice, friendly way to gain intuition about facility with this powerful concept.

Slide 34

Slide 34 text

Elastic Image Software LLC 34 Hierarchical modeling in Cocoa Touch

Slide 35

Slide 35 text

Elastic Image Software LLC 35 struct CGAffineTransform { CGFloat a, b, c, d; CGFloat tx, ty; }; struct CATransform3D { CGFloat m11, m12, m13, m14; CGFloat m21, m22, m23, m24; CGFloat m31, m32, m33, m34; CGFloat m41, m42, m43, m44; }; Transforms

Slide 36

Slide 36 text

Elastic Image Software LLC 36 UIView properties // Coordinate frame in parent space @property(nonatomic) CGRect frame; // Dimensions of view in object space @property(nonatomic) CGRect bounds; // View center in parent space @property(nonatomic) CGPoint center; // View transformation matrix @property(nonatomic) CGAffineTransform transform; // Parent view @property(nonatomic,readonly) UIView *superview; // Child views @property(nonatomic,readonly,copy) NSArray *subviews;

Slide 37

Slide 37 text

Elastic Image Software LLC 37 UIView methods // Transform 2D point from/to another view’s coordinate frame - (CGPoint)convertPoint:(CGPoint)point toView:(UIView *)view; - (CGPoint)convertPoint:(CGPoint)point fromView:(UIView *)view; // Transform rectangle from/to another view’s coordinate frame - (CGRect)convertRect:(CGRect)rect toView:(UIView *)view; - (CGRect)convertRect:(CGRect)rect fromView:(UIView *)view; // Manipulate view hierarchy - (void)removeFromSuperview; - (void)insertSubview:(UIView *)view atIndex:(NSInteger)index; - (void)addSubview:(UIView *)view; - (void)insertSubview:(UIView *)view belowSubview:(UIView *)siblingSubview; - (void)insertSubview:(UIView *)view aboveSubview:(UIView *)siblingSubview;

Slide 38

Slide 38 text

Elastic Image Software LLC 38 - (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions { [window addSubview:viewController.view]; [window makeKeyAndVisible]; return YES; } View hierarchy in iOS

Slide 39

Slide 39 text

Elastic Image Software LLC 39 Transformation matrices in iOS - (void)setTransform:(CGAffineTransform)newValue { // Scale along the y-axis only CGAffineTransform constrainedTransform = CGAffineTransformScale(CGAffineTransformIdentity, 1.0, newValue.a); newValue.a = constrainedTransform.a; newValue.b = constrainedTransform.b; newValue.c = constrainedTransform.c; newValue.d = constrainedTransform.d; [super setTransform:newValue]; }

Slide 40

Slide 40 text

Elastic Image Software LLC HelloTeapot Demo and Code Walkthrough http://github.com/turner/HelloTeapot 40

Slide 41

Slide 41 text

Elastic Image Software LLC In what follows, we will be referring to the ES1 “ﬁxed-function” transformation pipeline provided by the GPU. In ES2 (GLSL) the transformation pipeline must be handled entirely in application space. 41

Slide 42

Slide 42 text

Elastic Image Software LLC 42

Slide 43

Slide 43 text

Elastic Image Software LLC 43 A camera with a spotlight attached to it orbits around a stationary teapot model lit by three stationary colored point light sources.

Slide 44

Slide 44 text

Elastic Image Software LLC In iOS we never explicitly say “render scene”. Rather we inform the system that now would be a good time to redisplay the scene: [myView setNeedsDisplay] and [myView drawRect] 44

Slide 45

Slide 45 text

Elastic Image Software LLC - (void)startAnimation { animationTimer = [NSTimer scheduledTimerWithTimeInterval:animationInterval target:self selector:@selector(drawView) userInfo:nil repeats:YES]; } 45 In OpenGL we manage rendering ourselves

Slide 46

Slide 46 text

Elastic Image Software LLC 46 In OpenGL we manage rendering ourselves - (void)startAnimation { ! if (!self.isAnimating) { ! ! self.displayLink = [NSClassFromString(@"CADisplayLink") displayLinkWithTarget:self selector:@selector(drawView:)]; [self.displayLink setFrameInterval:animationFrameInterval]; [self.displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; ! ! ! self.animating = YES; ! ! } // if (!self.isAnimating) ! }

Slide 47

Slide 47 text

Elastic Image Software LLC ! Position the camera ! glMatrixMode(GL_MODELVIEW);! ! glLoadIdentity(); ! glLoadMatrixf(_openGLCameraInverseTransform); 47 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Slide 48

Slide 48 text

Elastic Image Software LLC Position spotlight at camera location glPushMatrix(); ! glMultMatrixf(_cameraTransform);! ! ! // A white spotlight for a camera headlight! GLfloat spotLightColor[] = { 1.0, 1.0, 1.0, 1.0 }; GLfloat spotLightPosition[] = { 0.0, 0.0, 2.0, 1.0 }; GLfloat spotLightDirection[] = { 0.0, 0.0, -1.0, 1.0 }; GLfloat spotCutOff! ! ! = 3.5; ! ! glEnable(GL_LIGHT3); ! glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLightColor);! ! glLightfv(GL_LIGHT3, GL_POSITION, spotLightPosition); ! glLightfv(GL_LIGHT3, GL_SPOT_DIRECTION, spotLightDirection); ! glLightf(GL_LIGHT3, GL_SPOT_CUTOFF, spotCutOff); ! ! glPopMatrix();! 48 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Slide 49

Slide 49 text

Elastic Image Software LLC 49 glPushMatrix & glPopMatrix These functions push/pop a transformation matrix on/off the transformation stack. This stack is the key data structure for presenting a hierarchical model to the GPU in fixed-function OpenGL.

Slide 50

Slide 50 text

Elastic Image Software LLC 50 glPushMatrix - Create a new matrix and push it on the stack. Subsequent matrix concatenation is done with stack.top. stack.top is the current coordinate frame. glPopMatrix - Pop the stack. The pre-existing matrix is now stack.top. We revert to the coordinate frame the existed prior to the push. ! glPushMatrix(); ! glMultMatrixf(_cameraTransform); ! ! glEnable(GL_LIGHT3); ! glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight); ! glPopMatrix();!

Slide 51

Slide 51 text

Elastic Image Software LLC 51 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m Position the red stationary light source ! const GLfloat redLightColor[]! ! = { 1.0f, 0.0f, 0.0f, 1.0f }; ! const GLfloat redLightPosition[]! = { 10.0f, 0.0f, 0.0f, 1.0f }; ! glEnable(GL_LIGHT0); ! glLightfv(GL_LIGHT0, GL_DIFFUSE, redLightColor);! ! glLightfv(GL_LIGHT0, GL_POSITION, redLightPosition);

Slide 52

Slide 52 text

Elastic Image Software LLC 52 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m Position the green stationary light source ! const GLfloat greenLightColor[]! = { 0.0f, 1.0f, 0.0f, 1.0f }; ! const GLfloat greenLightPosition[]!= { 0.0f, 10.0f, 0.0f, 1.0f }; ! glEnable(GL_LIGHT0); ! glLightfv(GL_LIGHT0, GL_DIFFUSE, greenLightColor);! ! glLightfv(GL_LIGHT0, GL_POSITION, greenLightPosition);

Slide 53

Slide 53 text

Elastic Image Software LLC 53 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m Position the blue stationary light source ! const GLfloat blueLightColor[]! ! = { 0.0f, 0.0f, 1.0f, 1.0f }; ! const GLfloat blueLightPosition[]! = { 0.0f, 0.0f, 10.0f, 1.0f }; ! glEnable(GL_LIGHT0); ! glLightfv(GL_LIGHT0, GL_DIFFUSE, blueLightColor);! ! glLightfv(GL_LIGHT0, GL_POSITION, blueLightPosition);

Slide 54

Slide 54 text

Elastic Image Software LLC 54 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m Position and draw the teapot ! glPushMatrix(); ! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); ! JLMMatrix3DSetScaling(scale, sx, sy, sz); ! JLMMatrix3DMultiply(rotation, scale, concatenation); ! glMultMatrixf(concatenation); ! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { ! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] ); ! } // for (num_teapot_indices) ! glPopMatrix();

Slide 55

Slide 55 text

Elastic Image Software LLC 55 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m Position and draw the teapot ! glPushMatrix(); ! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); ! JLMMatrix3DSetScaling(scale, sx, sy, sz); ! JLMMatrix3DMultiply(rotation, scale, concatenation); ! glMultMatrixf(concatenation); ! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { ! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] ); ! } // for (num_teapot_indices) ! glPopMatrix();

Slide 56

Slide 56 text

Elastic Image Software LLC 56 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m Position and draw the teapot ! glPushMatrix(); ! JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); ! JLMMatrix3DSetScaling(scale, sx, sy, sz); ! JLMMatrix3DMultiply(rotation, scale, concatenation); ! glMultMatrixf(concatenation); ! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { ! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] ); ! } // for (num_teapot_indices) ! glPopMatrix();

Slide 57

Slide 57 text

Elastic Image Software LLC 57 Notice the absence of GL_CAMERA? You may be surprised to learn there is no concept of camera in OpenGL. Camera posing is equivalent to inverting the camera transform and applying it to the scene observed by the camera.

Slide 58

Slide 58 text

Elastic Image Software LLC camera teapot world wTt wTc 58

Slide 59

Slide 59 text

Elastic Image Software LLC camera teapot glMatrixMode(GL_MODELVIEW); glLoadMatrixf(_openGLCameraInverseTransform); ? cTt 59

Slide 60

Slide 60 text

Elastic Image Software LLC camera teapot world wTt (wTc) cTt cTt wTt cTw * = -1 60

Slide 61

Slide 61 text

Elastic Image Software LLC cTt wTt cTw * = 61

Slide 62

Slide 62 text

Elastic Image Software LLC - (void)placeCameraAtLocation:(M3DVector3f)location target:(M3DVector3f)target up:(M3DVector3f)up; 62 http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.m

Slide 63

Slide 63 text

Elastic Image Software LLC The vertices comprising the 3D geometry of the teapot float teapot_vertices [] = { 0.0663, 0.1178, 0.0, 0.0672, 0.1152, 0.0, 0.0639, 0.1178, 0.0178043, ... }; The surface normals associated with each vertex // The list of surface normals corresponding to the vertices float teapot_normals[] = { -0.987635, -0.156768, 0, -0.902861, -0.429933, 0, -0.953562, -0.156989, -0.257047, ... }; 63

Slide 64

Slide 64 text

Elastic Image Software LLC The teapot model is defined as a collection of triangle stripes that combine to form the teapot surface. short indicies[] = { // how many vertices in vertex strip 26, // vertex strip indices 1122, 1243, 1272, 1242, ... ,1283, 1199, ... }; 64

Slide 65

Slide 65 text

Elastic Image Software LLC glEnableClientState(GL_VERTEX_ARRAY); glVertexPointer(3 ,GL_FLOAT, 0, teapot_vertices); glEnableClientState(GL_NORMAL_ARRAY); glNormalPointer(GL_FLOAT, 0, teapot_normals); glEnable(GL_NORMALIZE);! 65 ! for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) { ! ! glDrawElements( GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1] ); ! }

Slide 66

Slide 66 text

Elastic Image Software LLC Design the Look 66

Slide 67

Slide 67 text

Elastic Image Software LLC 67 We now begin leaving behind the ﬁxed- function pipeline of ES1 and focus on ES2, GLSL, and visual coolness in general.

Slide 68

Slide 68 text

Elastic Image Software LLC The Look: the lighting, illumination model, and surface shaders that in combination achieve the desired look of a model in a scene. 68

Slide 69

Slide 69 text

Elastic Image Software LLC Lights 69

Slide 70

Slide 70 text

Elastic Image Software LLC 70 Point Light Directional Light Spot Light Area Light

Slide 71

Slide 71 text

Elastic Image Software LLC 71

Slide 72

Slide 72 text

Elastic Image Software LLC Illumination Model 72

Slide 73

Slide 73 text

Elastic Image Software LLC Ambient Diffuse Specular Diffuse + Specular 73

Slide 74

Slide 74 text

Elastic Image Software LLC Diffuse(NormalVector, LightVector) Ambient(ConstantReﬂectance) Specular(LightReﬂectionVector, EyeVector) Shiny(Specular, Diffuse) 74

Slide 75

Slide 75 text

Elastic Image Software LLC Surface Shaders 75

Slide 76

Slide 76 text

Elastic Image Software LLC Surface shaders were introduced to the ﬁeld of computer graphics by Pixar with its rendering API and shading language called RenderMan. A shader is a small functions evaluated at every location on a being rendered. GLSL borrows heavily from the RenderMan model. 76

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Elastic Image Software LLC ZBrush Modeling Daniel Lieske Material Design Ralf Stumpf 79

Slide 80

Slide 80 text

Elastic Image Software LLC Copyright: Stephen Molyneaux 80

Slide 81

Slide 81 text

Elastic Image Software LLC Hello Shader Demo and Code Walkthrough https://github.com/turner/HelloShader 81

Slide 82

Slide 82 text

Elastic Image Software LLC 82 Before we dive into shaders lets get a lay of the land. There are two ﬂavors of shaders in GLSL. A vertex shader is evaluated at each vertex. A fragment shader is evaluated at each screen space pixel corresponding to a sampled on the facet being rasterized.

Slide 83

Slide 83 text

Elastic Image Software LLC 83 Vertex and fragment shaders work together in a pipeline fashion. Vertex attributes - color, surface normal, texture coordinate - are evaluated in the vertex shader then passed on to the fragment shader where those values are interpolated across the surface.

Slide 84

Slide 84 text

Elastic Image Software LLC 84 ShowST. A shader that visualizes the texture coordinate parameterization of a surface.

Slide 85

Slide 85 text

Elastic Image Software LLC 85 https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh // ShowST.vsh attribute highp vec4 myVertexXYZ; attribute highp vec2 myVertexST; // P * V * M - Projection space uniform mediump mat4 myProjectionViewModelMatrix; varying vec2 v_st; void main() { gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST; } ShowST - vertex shader

Slide 86

Slide 86 text

Elastic Image Software LLC 86 attribute vec2 myVertexST - Vertex attribute varying vec2 v_st - Varies across the surface uniform sampler2D myTexture_0 - Invariant throughout rendering cycle Shader variables come in different ﬂavors

Slide 87

Slide 87 text

Elastic Image Software LLC 87 https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh Vertex attribute myVertexST is a texture coordinate passed to the fragment shader via varying variable v_st. // ShowST.vsh attribute highp vec4 myVertexXYZ; attribute highp vec2 myVertexST; // P * V * M - Projection space uniform mediump mat4 myProjectionViewModelMatrix; varying vec2 v_st; void main() { gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST; }

Slide 88

Slide 88 text

Elastic Image Software LLC 88 https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/ShowST.fsh // ShowST.fsh varying vec2 v_st; void main() { // Visualize the s-t parameterization of the underlying surface gl_FragColor.r = v_st.s; gl_FragColor.g = v_st.t; gl_FragColor.b = 0.0; gl_FragColor.a = 1.0; } ShowST - fragment shader

Slide 89

Slide 89 text

Elastic Image Software LLC 89 http://github.com/turner/HelloiPadGLSL/blob/master/Shaders/ShowST.fsh // ShowST.fsh varying vec2 v_st; void main() { // Visualize the s-t parameterization of the underlying surface gl_FragColor.r = v_st.s; gl_FragColor.g = v_st.t; gl_FragColor.b = 0.0; gl_FragColor.a = 1.0; } The texture coordinate values v_st.s and v_st.t are for the red and green channels of the shader color.

Slide 90

Slide 90 text

Elastic Image Software LLC 90 TexturePairShader. A shader that mixes the colors of a pair of textures using the tweaked alpha channel of one of the pair. Fail Whale over Mandril

Slide 91

Slide 91 text

Elastic Image Software LLC 91 TexturePairShader - vertex shader https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh // TEITexturePairShader.vsh attribute mediump vec2 myVertexST; attribute highp vec4 myVertexXYZ; // P * V * M - Projection space uniform mediump mat4 myProjectionViewModelMatrix; varying mediump vec2 v_st; void main() { gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST; }

Slide 92

Slide 92 text

Elastic Image Software LLC 92 TexturePairShader - vertex shader // TEITexturePairShader.vsh attribute mediump vec2 myVertexST; attribute highp vec4 myVertexXYZ; // P * V * M - Projection space uniform mediump mat4 myProjectionViewModelMatrix; varying mediump vec2 v_st; void main() { gl_Position = myProjectionViewModelMatrix * myVertexXYZ; v_st = myVertexST; } https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh

Slide 93

Slide 93 text

Elastic Image Software LLC 93 TexturePairShader - fragment shader // TEITexturePairShader.fsh varying mediump vec2 v_st; uniform sampler2D myTexture_0; uniform sampler2D myTexture_1; void main() { vec4 rgba_0 = texture2D(myTexture_0, v_st); vec4 rgba_1 = texture2D(myTexture_1, v_st); // Interpolate between the two textures using the alpha // channel of texture 0 as the interpolant. float interpolate = rgba_0.a; gl_FragColor.r = rgba_0.r + (1.0 - interpolate) * rgba_1.r; gl_FragColor.g = rgba_0.g + (1.0 - interpolate) * rgba_1.g; gl_FragColor.b = rgba_0.b + (1.0 - interpolate) * rgba_1.b; gl_FragColor.a = rgba_0.a + (1.0 - interpolate) * rgba_1.a; } https://github.com/turner/HelloShader/blob/master/HelloShader/Shaders/TEITexturePairShader.vsh

Slide 94

Slide 94 text

Elastic Image Software LLC 94 attribute vec2 myVertexST - Vertex attribute varying vec2 v_st - Varies across the surface uniform sampler2D myTexture_0 - Invariant throughout rendering cycle Shader variables come in different ﬂavors

Slide 95

Slide 95 text

Elastic Image Software LLC 95 An application communicates with its shaders in the follow ways: • attributes - geometry, color, normal, texture coordinate • uniforms - texture sampler, matrix, eye vector, light vector • textures - texture channels - r, g, b, a - are not just for color attribute vec2 myVertexST - Vertex attribute varying vec2 v_st - Varies across the surface uniform sampler2D myTexture_0 - Invariant throughout rendering cycle

Slide 96

Slide 96 text

Elastic Image Software LLC 96 Lets look at how we wire our iOS app together with our shaders. We will use a texture shading example. https://github.com/turner/HelloShader/blob/master/HelloShader/Classes/Renderer/GLRenderer.m

Slide 97

Slide 97 text

Elastic Image Software LLC 97 First make a texture object ... TEITexture *t = [[ [TEITexture alloc] initWithImageFile:@"twitter_fail_whale_red_channnel_knockout" extension:@"png" mipmap:YES ] autorelease]; [glView.renderer.rendererHelper.renderables setObject:t forKey:@"texture_0"];

Slide 98

Slide 98 text

Elastic Image Software LLC 98 Load shaders, Bind shader attribute to application handle. // Vertex shader NSString *vertShaderPathname = [[NSBundle mainBundle] pathForResource:@"TEITextureShader" ofType:@"vsh"]; [self compileShader:&vertShader type:GL_VERTEX_SHADER file:vertShaderPathname]; ! // Fragment shader NSString *fragShaderPathname = [[NSBundle mainBundle] pathForResource:@"TEITextureShader" ofType:@"fsh"]; [self compileShader:&fragShader type:GL_FRAGMENT_SHADER file:fragShaderPathname]; ! // Bind attribute glBindAttribLocation(m_program, VertexSTAttributeHandle,! "myVertexST");

Slide 99

Slide 99 text

Elastic Image Software LLC 99 Associate shader uniform with application object instance ... TEITexture *t = (TEITexture *)[self.rendererHelper.renderables objectForKey:@"texture_0"]; t.location = glGetUniformLocation(m_program, "myTexture_0"); glActiveTexture( GL_TEXTURE0 ); glBindTexture(GL_TEXTURE_2D, t.name); glUniform1i(t.location, 0); Activate a texture unit, bind it to a texture object, and assign a number to the corresponding texture sampler used in the fragment shader

Slide 100

Slide 100 text

Elastic Image Software LLC 100 We now have a linkage between our application texture object and a texture sampler in our fragment shader. Rock.

Slide 101

Slide 101 text

Elastic Image Software LLC 101

Slide 102

Slide 102 text

Elastic Image Software LLC 102

Slide 103

Slide 103 text

Elastic Image Software LLC 103 Lets Talk Textures

Slide 104

Slide 104 text

Elastic Image Software LLC 104 Texture mapping is the fundamental tool for creating visual complexity

Slide 105

Slide 105 text

Elastic Image Software LLC 105 By establishing a mapping between a surface and a texture we can “attach” the texture to the surface. The interpretation of the a can go far beyond that of a decal to be applied to a surface. Bumps, opacity, displacement, and much more can be designed with a texture.

Slide 106

Slide 106 text

Elastic Image Software LLC 106

Slide 107

Slide 107 text

Elastic Image Software LLC 107 Shaders can have it both ways: Compute surface color (or surface roughness, or opacity) algorithmically or look it up in a texture. Or a bit of both.

Slide 108

Slide 108 text

Elastic Image Software LLC OpenGL and iOS Playing Nice Together 108

Slide 109

Slide 109 text

Elastic Image Software LLC • OpenGL is mostly M • Keep V dumb • Lots of chatter between M and C MVC 109

Slide 110

Slide 110 text

Elastic Image Software LLC 110 Lets take a look at how iOS and OpenGL interact. We will us Hello iPad OpenGL as our guide. https://github.com/turner/HelloShader

Slide 111

Slide 111 text

Elastic Image Software LLC 111 -(id)initializeEAGL { ! ! CAEAGLLayer *eaglLayer = (CAEAGLLayer *)self.layer; ! ! eaglLayer.opaque = TRUE; ! eaglLayer.drawableProperties = ! [NSDictionary dictionaryWithObjectsAndKeys: ! [NSNumber numberWithBool:FALSE], kEAGLDrawablePropertyRetainedBacking, ! kEAGLColorFormatRGBA8, kEAGLDrawablePropertyColorFormat, ! nil]; ! ! self.renderer = [[[ES2Renderer alloc] init] autorelease]; ! ! m_animating = FALSE; ! animationFrameInterval = 1; ! m_displayLink = nil; ! ! return self; ! } The view’s layer is used as the rendering surface

Slide 112

Slide 112 text

Elastic Image Software LLC 112 Rendering frame rate is sync-ed to the screen refresh rate. The actual rendering is handled via a selector. - (void)startAnimation { if (!self.isAnimating) { self.displayLink = [NSClassFromString(@"CADisplayLink") displayLinkWithTarget:self selector:@selector(drawView:)]; [self.displayLink setFrameInterval:animationFrameInterval]; [self.displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode]; self.animating = YES; } // if (!self.isAnimating) }

Slide 113

Slide 113 text

Elastic Image Software LLC 113 // framebuffer glGenFramebuffers(1, &m_framebuffer); glBindFramebuffer(GL_FRAMEBUFFER, m_framebuffer); // rgb buffer glGenRenderbuffers(1, &m_colorbuffer); glBindRenderbuffer(GL_RENDERBUFFER, m_colorbuffer); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, m_colorbuffer); [m_context renderbufferStorage:GL_RENDERBUFFER fromDrawable:layer]; glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &m_backingWidth); glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &m_backingHeight); // z-buffer glGenRenderbuffers(1, &m_depthbuffer); glBindRenderbuffer(GL_RENDERBUFFER, m_depthbuffer); glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, m_backingWidth, m_backingHeight); glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, m_depthbuffer); A framebuffer object is created. A color and depth buffer are attached.

Slide 114

Slide 114 text

Elastic Image Software LLC 114 - (void) render { ! ! [EAGLContext setCurrentContext:m_context]; glBindFramebuffer(GL_FRAMEBUFFER, m_framebuffer); // transform, light, shade, etc. glBindRenderbuffer(GL_RENDERBUFFER, m_colorbuffer); [m_context presentRenderbuffer:GL_RENDERBUFFER]; } The render loop. Draw to the colorbuffer then present to the display. This is the classic “ping pong” between back buffer and front buffer.

Slide 115

Slide 115 text

Elastic Image Software LLC Thank You Douglass Turner Elastic Image Software email: [email protected] mobile: 781 775 3708 115