DroidconUK 2017 - OpenGL - A noob's guide for Android developers

DroidconUK 2017 - OpenGL - A noob's guide for Android developers

OpenGL is often regarded as too complex, obscure or even too low level to be considered when developing an application. Most people will even try to go around it and develop what they want using Views and Canvas to avoid developing with it. Nevertheless, it's not always possible nor the best solution.

As someone who have been going through all the learning of OpenGL recently, Benjamin has encountered a lot of frustration but also a lot of “aha!“ moment. In this talk, Benjamin will try to demystified OpenGL and share the basics of what it is and how to use it in Android to make applications perform better. Also, you will learn all the pitfalls and key concepts that you should know when starting with OpenGL.

75aea27d01bec65f5a925132fe7ac81c?s=128

Benjamin Monjoie

October 26, 2017
Tweet

Transcript

  1. OpenGL A noob's guide for Android developers Benjamin Monjoie -

    @Xzan
  2. The promise https://www.turbosquid.com/3d-models/3d-pine-tree-model/700221

  3. The reality http://www.script-tutorials.com/demos/145/img1.jpg 300 lines of code later ...

  4. None
  5. The challenge

  6. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
  7. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
  8. What is OpenGL ? http://infos-reportages.com/wp-content/uploads/2016/10/phone_hero-module_hero-image_144 0_2x.png http://fr.ubergizmo.com/wp-content/uploads/2013/02/chromebook-pixel.png http://www.samsung.com/us/explore/family-hub-refrigerator/assets/images/glance/food-manag ement-slide01-mb.png http://www.samsung.com/us/2012-smart-blu-ray-player/img/tv-front.png

  9. OpenGL & Android • OpenGL ES 1.0 & 1.1 since

    Android 1.0 (API 4)
  10. OpenGL & Android • OpenGL ES 1.0 & 1.1 since

    Android 1.0 (API 4) • OpenGL ES 2.0 since Android 2.2 (API 8)
  11. OpenGL & Android • OpenGL ES 1.0 & 1.1 since

    Android 1.0 (API 4) • OpenGL ES 2.0 since Android 2.2 (API 8) • OpenGL ES 3.0 since Android 4.3 (API 18) (almost)
  12. OpenGL & Android • OpenGL ES 1.0 & 1.1 since

    Android 1.0 (API 4) • OpenGL ES 2.0 since Android 2.2 (API 8) • OpenGL ES 3.0 since Android 4.3 (API 18) (almost) • OpenGL ES 3.1 since Android 5.0 (API 21)
  13. OpenGL & Android • OpenGL ES 1.0 & 1.1 since

    Android 1.0 (API 4) • OpenGL ES 2.0 since Android 2.2 (API 8) • OpenGL ES 3.0 since Android 4.3 (API 18) (almost) • OpenGL ES 3.1 since Android 5.0 (API 21)
  14. Graphics driver http://international.download.nvidia.com/webassets/en_US/shared/images/articles/gtx-680m/gtx-680m-chip.png

  15. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
  16. What about with Android ? GLSurfaceView & GLSurfaceView.Renderer onSurfaceCreated (GL10

    gl, EGLConfig config) onSurfaceChanged (GL10 gl, int width, int height) onDrawFrame (GL10 gl)
  17. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
  18. OpenGL’s internals https://www.ntu.edu.sg/home/ehchua/programming/opengl/CG_BasicsTheory.html

  19. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
  20. Shaders (GLSL)

  21. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix;

    attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
  22. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix;

    attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
  23. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix;

    attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
  24. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix;

    attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
  25. Vertex shader - Example precision mediump float; uniform mat4 uMVPMatrix;

    attribute vec4 vPosition; attribute vec4 vTextureCoordinate; varying vec2 position; void main() { gl_Position = uMVPMatrix * vPosition; position = vTextureCoordinate.xy; }
  26. Fragment shader - Example precision mediump float; uniform sampler2D uTexture;

    varying vec2 position; void main() { gl_FragColor = texture2D(uTexture, position); }
  27. Fragment shader - Example precision mediump float; uniform sampler2D uTexture;

    varying vec2 position; void main() { gl_FragColor = texture2D(uTexture, position); }
  28. Fragment shader - Example precision mediump float; uniform sampler2D uTexture;

    varying vec2 position; void main() { gl_FragColor = texture2D(uTexture, position); }
  29. Fragment shader - Example precision mediump float; uniform sampler2D uTexture;

    varying vec2 position; void main() { gl_FragColor = texture2D(uTexture, position); }
  30. Attribute vs Uniform vs Varying Attribute Attribute of a vertex

    (position, color, etc). Uniform Value passed by the application to the shader. Constant during a draw call. Varying Information passed from the vertex shader to the fragment shader.
  31. Attribute vs Uniform vs Varying Attribute Uniform Varying Vertex Shader

    ✔ ✔ ✔ Fragment Shader ✔ ✔ Can change ~
  32. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
  33. Enter the matrix https://developer.android.com/guide/topics/graphics/opengl.html#coordinate-mapping

  34. The strip https://en.wikipedia.org/wiki/Triangle_strip

  35. The strip POSITION_MATRIX = { -1,-1, 1, // X1,Y1,Z1 1,-1,

    1, // X2,Y2,Z2 -1, 1, 1, // X3,Y3,Z3 1, 1, 1, // X4,Y4,Z4 }; TEXTURE_COORDS[] = { 0, 1, // X1,Y1 1, 1, // X2,Y2 0, 0, // X3,Y3 1, 0, // X4,Y4 };
  36. The strip 0,0 1,0 0,1 1,1 https://pbs.twimg.com/profile_images/616076655547682816/6gMRtQyY.jpg

  37. The strip 0,0 1,0 0,1 1,1

  38. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
  39. Advices • Abstract as much as possible

  40. Advices • Abstract as much as possible • Crash often

    but crash well !
  41. Advices • Abstract as much as possible • Crash often

    but crash well ! • Test, a lot !
  42. Advices • Abstract as much as possible • Crash often

    but crash well ! • Test, a lot ! • Use Google’s Play Store’s Alpha & Beta channels
  43. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • Bonus
  44. Roadmap • What is OpenGL ? • What about with

    Android ? • OpenGL’s internals • Shaders • Enter the matrix • Advices • GPUEffect
  45. GPUEffect • Inspired by CyberAgent’s GPUImage

  46. GPUEffect • Inspired by CyberAgent’s GPUImage • Written in Kotlin

  47. GPUEffect • Inspired by CyberAgent’s GPUImage • Written in Kotlin

    • As close as possible to OpenGL
  48. GPUEffect • Inspired by CyberAgent’s GPUImage • Written in Kotlin

    • As close as possible to OpenGL • Fail often and fail early
  49. GPUImage ? • Applies filters to images • Works with

    Picasso, Glide and Fresco • Works with SurfaceView • Not maintained for several years & too opinionated
  50. Written in Kotlin • Allows to write shaders with “““

    • Properties for cleaner code • Operators overloading (“+=”, “+”, ..) • Infix
  51. Written in Kotlin

  52. As close as possible to OpenGL • Interfaces like :

    Texture, Program and FrameBuffer • Classes like : BitmapTexture, GLSLProgram, FBOTexture
  53. Sample - ImageView

  54. Sample - SurfaceView (1)

  55. Sample - SurfaceView (2)

  56. OpenGL A noob's guide for Android developers https://medium.com/@Xzan https://twitter.com/Xzan https://medium.com/@xzan/opengl-a-noobs-guide-for-android-developers-5eed724e07ad

    https://medium.com/@xzan/why-we-created-gpueffect-e94fb31b0d15