Using the Qt Quick Scene Graph API

Recording available at https://youtu.be/cNE6Jabxxxo
Code available at https://github.com/woboq/using_the_qt_quick_scene_graph_api

The scene graph is the main change under the hood of QtQuick 2 that justified a new major version of the technology, while the rest initially remained similar to QtQuick 1. The extra performance is provided automatically for applications to enable smooth panning and scaling, but there are some situations where you can take advantage of using the scene graph API directly to smoothen your custom graphical item animations.

This talk presents how the lower-level scene graph API in QtQuick can, with the help of a few lines of C++ code, allow you to reduce the overhead of QML abstractions and offload some transform computations to the GPU. In addition this presents:

- What the scene graph is and how it relates to a QML component or QQuickItem.
- How it is different from a QGraphicsScene.
- How it interacts with OpenGL and the GPU.
- How you can write code to run safely in the render thread and taking advantage of the geometry batching done by the renderer.
- What APIs like QQuickItem::updatePaintNode, QSGNode, QSGGeometry, QSGMaterial and QSGTexture are used for.

Transcript

1. Using the Qt Quick
 Scene Graph API by Jocelyn Turcotte Woboq GmbH @woboq
2. Hello! • I’m Jocelyn Turcotte • Working at Woboq GmbH, Berlin • Worked on Qt WebKit and Qt WebEngine
 at Nokia / Digia / The Qt Company • Accelerated compositing integration of WebKit2 and Chromium with the Qt Quick Scene Graph 2
3. Agenda 3 1. Origins of the Qt Quick Scene Graph 2. Writing your own QQuickItem:
 Using built-in QSGGeometryNodes 3. Writing your own QSGGeometryNode:
 Custom geometries and shaders
4. 1. Origins of the
 Qt Quick Scene Graph
5. Qt Quick 1 vs. Qt Quick 2 • Qt Quick 1: • Built on top of Qt Script and Qt Widgets • Uses QGraphicsView and QPainter for rendering • Qt Quick 2: • Built directly on top of Qt Gui • Uses QWindow and QOpenGL* for rendering 5
7. Rendering in Qt Quick 1 7
8. Rendering in Qt Quick 2 8
9. 2. Writing your own QQuickItem: Using built-in QSGGeometryNodes
10. Let's start with a Rectangle (demo)
11. Who owns an Item's paint node tree? • The Item temporarily owns its paint node tree during updatePaintNode() • There, the Item can update or destroy the tree • Items live on the GUI thread, QSGNodes live on the SG thread 11
12. The Scene Graph Thread • Avoids blocking the GUI thread doing VSync • The scene graph thread deals with OpenGL • QQuick* objects live in the GUI thread,
 QSG* in the scene graph thread • QQuickItem::updatePaintNode() is the portal between the threads • Don't use mutexes! Transfer all states in updatePaintNode() instead 12
13. A pathological QML example (demo)
14. Effects of the animation on the scene graph For each of all N points: • The "opacity" and "y" property bindings are updated • update() is called • Their underlying QSGTransformNode and a QSGOpacityNode are updated by QQuickItem 14
15. But must QML be involved here? • We don't need direct control over each point in QML (no event handling, no use by other items) • The points data originally comes from C++ • The text and revealProgress properties are the same for the whole QR code 15
16. Let’s animate in updatePaintNode() 16
17. It’s faster! But with 100% CPU usage of the GUI thread,
 the GPU is bored.
18. 3. Writing your own QSGGeometryNode: Custom geometries and shaders
19. First: Alternatives to consider • QML ShaderEffect
 Consider first, less flexible but easier to use • QML Canvas / QQuickPaintedItem
 Use when imperative painting is desired • QQuickFramebufferObject
 For standard OpenGL code, uses an FBO, preserves Z-order • QQuickWindow::(before|after)Rendering
 For standard OpenGL code, doesn't go through an FBO 19
21. Could we animate in the vertex shader? 21
22. If it doesn't work • Read the “Qt Quick Scene Graph Renderer” documentation • Read the code! The scene graph only a framework at that level, OpenGL is your platform • Have a look at QQuickWindow, QSGThreadedRenderLoop and QSGBatchRenderer • Ask Woboq 22
23. Conclusion • QML property bindings have limited scalability • Using too many QSGGeometryNodes can also be a scalability issue (because of algorithmic pressure) • Using opaque geometry materials reduces the worst case QSGGeometryNodes complexity • Computing in vertex/fragment shaders will be >20x faster and frees the GUI thread to do other work 23
24. Thank you! jturcotte@woboq.com
 Twitter: @woboq
25. 4. Extra material
26. Batching • All vertices go in the same vertex buffer • The only GL state change between element draws of a batch is their transform 26
27. Condition for a batch: • Geometries have the same drawingMode and same vertex attributes • Using the same QSGMaterial type • QSGMaterial::compare returns 0 • Same computed opacity • If transparent: bounds not overlapping 27
28. Merged batches • All transformed are flattened to a chosen root node • One single glDrawElements call • Root promotion heuristic: try to find a scrolling layer with many children 28
29. Batching debug variables • QSG_RENDERER_DEBUG=dump
 Shows the complete QSGNode tree • QSG_RENDERER_DEBUG=render
 Gives an overview of all rendered batches • QSG_RENDERER_DEBUG=build
 Shows batch contents • QSG_RENDERER_DEBUG=roots
 Display which transform node is promoted as root for merged batches • QSG_VISUALIZE
 http://doc.qt.io/qt-5/qtquick-visualcanvas-scenegraph-renderer.html#visualizing 29
30. Blending is off by default because • Overlapping transparent geometries MUST be rendered back to front • Can't render batches in an arbitrary z-order to use the depth buffer • Blending requires an extra work for the GPU 30
31. Clipping isn't free • Nodes across clip nodes can't be batched together • Creates additional OpenGL state changes and performance costs 31
32. QSGNode::preprocess() • Called on every frame, not just when QQuickItem::update() is called • Runs on the SG thread, after updatePaintNode() but before the scene’s rendering with OpenGL • The GUI thread isn't blocked at that point, can be used to do CPU work that we can’t do on the GPU while reducing the main thread’s event loop latency • Usually used to: • Render FBOs • Update a QSGDynamicTexture 32
33. Textures • The QSGTexture must be owned by a node in the tree • Must be created in updatePaintNode() using window()- >createTextureFromImage() • Used by calling QSGTexture::bind in your QSGMaterial::updateState • Textures behind an Image, Canvas or ShaderEffectSource are available through the QSGTextureProvider API 33
34. Using the index buffer • Our squares have 2 triangles onto 4 vertices, 2 vertices are duplicated • Will use more memory • The vertex shader might run 6 times instead of 4 • Need degenerate triangles with GL_TRIANGLES_STRIP, but takes even more memory • Solution: Use an index buffer to separate the vertex position and attribute from the geometry assembly • Trigger by passing indexCount >0 to QSGGeometry 34