DroidconUK 2017 - OpenGL - A noob's guide for Android developers

OpenGL is often regarded as too complex, obscure or even too low level to be considered when developing an application. Most people will even try to go around it and develop what they want using Views and Canvas to avoid developing with it. Nevertheless, it's not always possible nor the best solution.

As someone who have been going through all the learning of OpenGL recently, Benjamin has encountered a lot of frustration but also a lot of “aha!“ moment. In this talk, Benjamin will try to demystified OpenGL and share the basics of what it is and how to use it in Android to make applications perform better. Also, you will learn all the pitfalls and key concepts that you should know when starting with OpenGL.

Transcript

1. OpenGL A noob's guide for Android developers Benjamin Monjoie - @Xzan
2. The promise https://www.turbosquid.com/3d-models/3d-pine-tree-model/700221
3. The reality http://www.script-tutorials.com/demos/145/img1.jpg 300 lines of code later ...
5. The challenge
6. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
7. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
8. What is OpenGL ? http://infos-reportages.com/wp-content/uploads/2016/10/phone_hero-module_hero-image_144 0_2x.png http://fr.ubergizmo.com/wp-content/uploads/2013/02/chromebook-pixel.png http://www.samsung.com/us/explore/family-hub-refrigerator/assets/images/glance/food-manag ement-slide01-mb.png http://www.samsung.com/us/2012-smart-blu-ray-player/img/tv-front.png
9. OpenGL & Android • OpenGL ES 1.0 & 1.1 since Android 1.0 (API 4)
10. OpenGL & Android • OpenGL ES 1.0 & 1.1 since Android 1.0 (API 4) • OpenGL ES 2.0 since Android 2.2 (API 8)
11. OpenGL & Android • OpenGL ES 1.0 & 1.1 since Android 1.0 (API 4) • OpenGL ES 2.0 since Android 2.2 (API 8) • OpenGL ES 3.0 since Android 4.3 (API 18) (almost)
12. OpenGL & Android • OpenGL ES 1.0 & 1.1 since Android 1.0 (API 4) • OpenGL ES 2.0 since Android 2.2 (API 8) • OpenGL ES 3.0 since Android 4.3 (API 18) (almost) • OpenGL ES 3.1 since Android 5.0 (API 21)
13. OpenGL & Android • OpenGL ES 1.0 & 1.1 since Android 1.0 (API 4) • OpenGL ES 2.0 since Android 2.2 (API 8) • OpenGL ES 3.0 since Android 4.3 (API 18) (almost) • OpenGL ES 3.1 since Android 5.0 (API 21)
14. Graphics driver http://international.download.nvidia.com/webassets/en_US/shared/images/articles/gtx-680m/gtx-680m-chip.png
15. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
16. What about with Android ? GLSurfaceView & GLSurfaceView.Renderer onSurfaceCreated (GL10 gl, EGLConfig config) onSurfaceChanged (GL10 gl, int width, int height) onDrawFrame (GL10 gl)
17. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
18. OpenGL’s internals https://www.ntu.edu.sg/home/ehchua/programming/opengl/CG_BasicsTheory.html
19. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
20. Shaders (GLSL)
21. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix; attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
22. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix; attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
23. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix; attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
24. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix; attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
25. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix; attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
26. Fragment shader - Example precision mediump float; uniform sampler2D uTexture; varying vec2 position; void main() { gl_FragColor = texture2D(uTexture, position); }
27. Fragment shader - Example precision mediump float; uniform sampler2D uTexture; varying vec2 position; void main() { gl_FragColor = texture2D(uTexture, position); }
28. Fragment shader - Example precision mediump float; uniform sampler2D uTexture; varying vec2 position; void main() { gl_FragColor = texture2D(uTexture, position); }
29. Fragment shader - Example precision mediump float; uniform sampler2D uTexture; varying vec2 position; void main() { gl_FragColor = texture2D(uTexture, position); }
30. Attribute vs Uniform vs Varying Attribute Attribute of a vertex (position, color, etc). Uniform Value passed by the application to the shader. Constant during a draw call. Varying Information passed from the vertex shader to the fragment shader.
31. Attribute vs Uniform vs Varying Attribute Uniform Varying Vertex Shader ✔ ✔ ✔ Fragment Shader ✔ ✔ Can change ~
32. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
33. Enter the matrix https://developer.android.com/guide/topics/graphics/opengl.html#coordinate-mapping
34. The strip https://en.wikipedia.org/wiki/Triangle_strip
35. The strip POSITION_MATRIX = { -1,-1, 1, // X1,Y1,Z1 1,-1, 1, // X2,Y2,Z2 -1, 1, 1, // X3,Y3,Z3 1, 1, 1, // X4,Y4,Z4 }; TEXTURE_COORDS[] = { 0, 1, // X1,Y1 1, 1, // X2,Y2 0, 0, // X3,Y3 1, 0, // X4,Y4 };
36. The strip 0,0 1,0 0,1 1,1 https://pbs.twimg.com/profile_images/616076655547682816/6gMRtQyY.jpg
37. The strip 0,0 1,0 0,1 1,1
38. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
39. Advices • Abstract as much as possible
40. Advices • Abstract as much as possible • Crash often but crash well !
41. Advices • Abstract as much as possible • Crash often but crash well ! • Test, a lot !
42. Advices • Abstract as much as possible • Crash often but crash well ! • Test, a lot ! • Use Google’s Play Store’s Alpha & Beta channels
43. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
44. Roadmap • What is OpenGL ? • What about with Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • GPUEffect
45. GPUEffect • Inspired by CyberAgent’s GPUImage
46. GPUEffect • Inspired by CyberAgent’s GPUImage • Written in Kotlin
47. GPUEffect • Inspired by CyberAgent’s GPUImage • Written in Kotlin • As close as possible to OpenGL
48. GPUEffect • Inspired by CyberAgent’s GPUImage • Written in Kotlin • As close as possible to OpenGL • Fail often and fail early
49. GPUImage ? • Applies filters to images • Works with Picasso, Glide and Fresco • Works with SurfaceView • Not maintained for several years & too opinionated
50. Written in Kotlin • Allows to write shaders with “““ • Properties for cleaner code • Operators overloading (“+=”, “+”, ..) • Infix
51. Written in Kotlin
52. As close as possible to OpenGL • Interfaces like : Texture, Program and FrameBuffer • Classes like : BitmapTexture, GLSLProgram, FBOTexture
53. Sample - ImageView
54. Sample - SurfaceView (1)
55. Sample - SurfaceView (2)
56. OpenGL A noob's guide for Android developers https://medium.com/@Xzan https://twitter.com/Xzan https://medium.com/@xzan/opengl-a-noobs-guide-for-android-developers-5eed724e07ad https://medium.com/@xzan/why-we-created-gpueffect-e94fb31b0d15