Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Traitement d'image simple et rapide avec Renderscript

Traitement d'image simple et rapide avec Renderscript

5e1f9aa9b3acb8eb8d230294e3b7340a?s=128

Quentin Menini

April 24, 2018
Tweet

Other Decks in Programming

Transcript

  1. RenderScript Traitement d’image simple et rapide sur Android

  2. Traitement d’image : pourquoi c’est chiant ?

  3. Traitement d’image : pourquoi c’est chiant ?

  4. Traitement d’image : pourquoi c’est chiant ?

  5. Traitement d’image : pourquoi c’est chiant ? 4032 3024

  6. Traitement d’image : pourquoi c’est chiant ? 4032 3024 .

    12 192 768 Pixels .
  7. Traitement d’image : pourquoi c’est chiant ? 4032 3024 .

    12 192 768 Pixels . for (i in 0 .. 12192768)
  8. Traitement d’image : pourquoi c’est chiant ? for (i in

    0 .. 12192768)
  9. Traitement d’image : pourquoi c’est chiant ? for (i in

    0 .. 12192768) ➔
  10. Traitement d’image : pourquoi c’est chiant ? for (i in

    0 .. 12192768) 4 368 ms ➔
  11. Traitement d’image : pourquoi c’est chiant ? ➔ 12 192

    768 Pixels ➔ 12 192 768 int
  12. Traitement d’image : pourquoi c’est chiant ? ➔ 12 192

    768 Pixels ➔ 46,5 Mo
  13. Traitement d’image : pourquoi c’est chiant ? ➔ 46,5 Mo

    12 192 768 Pixels ➔ 46,5 Mo
  14. Traitement d’image : pourquoi c’est chiant ? ➔ 46,5 Mo

    46,5 Mo 12 192 768 Pixels ➔ 46,5 Mo
  15. Traitement d’image : pourquoi c’est chiant ? ➔ Out of

    Memory 46,5 Mo 46,5 Mo 12 192 768 Pixels ➔ 46,5 Mo
  16. ➔ 46,5 Mo 46,5 Mo Traitement d’image : pourquoi c’est

    chiant ? Out of Memory 12 192 768 Pixels ➔ 46,5 Mo
  17. Traitement d’image : des solutions qui font souffrir

  18. JNI/C/C++ (CPU)

  19. JNI/C/C++ (CPU) Cross compilation

  20. JNI/C/C++ (CPU) Cross compilation Interface C++/Java

  21. JNI/C/C++ (CPU) Cross compilation Interface C++/Java Plus lent pour le

    calcul parallèle (CPU vs GPU)
  22. OpenGL (GPU)

  23. OpenGL (GPU) Coder en OpenGL c’est compliqué

  24. OpenGL (GPU) Coder en OpenGL c’est compliqué GL_MAX_TEXTURE_SIZE

  25. OpenGL (GPU) Coder en OpenGL c’est compliqué GL_MAX_TEXTURE_SIZE Fait pour

    du render, pas du calcul parallèle
  26. RenderScript

  27. RenderScript

  28. RenderScript Fonctionne sur GPU, CPU et/ou DSP

  29. RenderScript Fonctionne sur GPU, CPU et/ou DSP « Write once,

    run anywhere »
  30. RenderScript Fonctionne sur GPU, CPU et/ou DSP « Write once,

    run anywhere » Parallélisation gérée par RenderScript
 

  31. RenderScript Fonctionne sur GPU, CPU et/ou DSP « Write once,

    run anywhere » Parallélisation gérée par RenderScript
 
 for (int i = 0; i < 12192768; i++) { out[i] = doSomething(in[i]) }
  32. Temps de calcul du blur Java 4 368 ms

  33. Temps de calcul du blur Java 4 368 ms Renderscript

    516 ms
  34. Temps de calcul du blur Java 4 368 ms Renderscript

    516 ms 8x plus rapide
  35. Renderscript pour les nuls : blur

  36. None
  37. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  38. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  39. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  40. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  41. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  42. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  43. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  44. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  45. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  46. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  47. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  48. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  49. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  50. Blur.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptIntrinsicBlur script = ScriptIntrinsicBlur.create(rs, Element.U8_4(rs)); script.setRadius(radius); script.setInput(in); script.forEach(out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  51. Les plus simples - ScriptIntrinsicBlur - ScriptIntrinsicResize - ScriptIntrinsicColorMatrix -

    ScriptIntrinsicBlend ScriptIntrinsics
  52. Les plus simples - ScriptIntrinsicBlur - ScriptIntrinsicResize - ScriptIntrinsicColorMatrix -

    ScriptIntrinsicBlend Les moins simples - ScriptIntrinsicConvolve3x3 - ScriptIntrinsicConvolve5x5 - ScriptIntrinsicYuvToRGB - ScriptIntrinsicLUT - ScriptIntrinsic3DLUT ScriptIntrinsics
  53. Renderscript like a boss : effet vignette à la Instagram

  54. None
  55. None
  56. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); }
  57. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); }
  58. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); }
  59. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); }
  60. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); }
  61. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); }
  62. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); } static float computeDistance(uint32_t x, uint32_t y) {...}
  63. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); } static float4 computeColor(...) {...}
  64. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); }
  65. vignette.rs int width; int height; uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t

    x, uint32_t y) { float4 f4 = rsUnpackColor8888(in); float red = f4.r; float green = f4.g; float blue = f4.b; float distance = computeDistance(x, y); float4 newColor = computeColor(red, green, blue, distance); return rsPackColorTo8888(newColor.r, newColor.g, newColor.b, f4.a); }
  66. Mask.java RenderScript rs = RenderScript.create(context); Allocation in = Allocation.createFromBitmap(rs, bitmap);

    Type t = in.getType(); Allocation out = Allocation.createTyped(rs, t); ScriptC_vignette script = ScriptC_vignette(rs); script.setWidth(bitmap.width); script.setHeight(bitmap.height); script.forEach_root(in, out); out.copyTo(bitmap); in.destroy(); out.destroy(); script.destroy(); rs.destroy(); return bitmap;
  67. Générations à partir du fichier .rs RenderScript ➔ Java

  68. Générations à partir du fichier .rs script.rs ScriptC_script.java RenderScript ➔

    Java
  69. Générations à partir du fichier .rs script.rs ScriptC_script.java Typedef struct

    Struct ScriptField_Struct.java RenderScript ➔ Java
  70. Générations à partir du fichier .rs script.rs ScriptC_script.java Typedef struct

    Struct ScriptField_Struct.java Struct_t* my_struct; .bind_my_struct(...); RenderScript ➔ Java
  71. Générations à partir du fichier .rs script.rs ScriptC_script.java Typedef struct

    Struct ScriptField_Struct.java Struct_t* my_struct; .bind_my_struct(...); int value; .set_value(int i); .get_value(); RenderScript ➔ Java
  72. Générations à partir du fichier .rs script.rs ScriptC_script.java Typedef struct

    Struct ScriptField_Struct.java Struct_t* my_struct; .bind_my_struct(...); int value; .set_value(int i); .get_value(); void function(); .invoke_function(); RenderScript ➔ Java
  73. Générations à partir du fichier .rs script.rs ScriptC_script.java Typedef struct

    Struct ScriptField_Struct.java Struct_t* my_struct; .bind_my_struct(...); int value; .set_value(int i); .get_value(); void function(); .invoke_function(); uchar4 __attribute__((kernel)) root(uchar4 in, uint32_t x, uint32_t y) .forEach_root(Allocation in, Allocation out); RenderScript ➔ Java
  74. Renderscript godlike : amélioration automatique d’image

  75. None
  76. None
  77. None
  78. None
  79. None
  80. None
  81. None
  82. None
  83. None
  84. None
  85. None
  86. None
  87. None
  88. None
  89. Temps de calcul Java >15 s

  90. Temps de calcul Java >15 s Renderscript < 1s

  91. Temps de calcul Java >15 s Renderscript < 1s 15x

    plus rapide
  92. ElevationImageView

  93. Merci @QMenini @qhutch @qhutch