Using the Qt Quick Scene Graph API

Recording available at https://youtu.be/cNE6Jabxxxo
Code available at https://github.com/woboq/using_the_qt_quick_scene_graph_api

The scene graph is the main change under the hood of QtQuick 2 that justified a new major version of the technology, while the rest initially remained similar to QtQuick 1. The extra performance is provided automatically for applications to enable smooth panning and scaling, but there are some situations where you can take advantage of using the scene graph API directly to smoothen your custom graphical item animations.

This talk presents how the lower-level scene graph API in QtQuick can, with the help of a few lines of C++ code, allow you to reduce the overhead of QML abstractions and offload some transform computations to the GPU. In addition this presents:

- What the scene graph is and how it relates to a QML component or QQuickItem.
- How it is different from a QGraphicsScene.
- How it interacts with OpenGL and the GPU.
- How you can write code to run safely in the render thread and taking advantage of the geometry batching done by the renderer.
- What APIs like QQuickItem::updatePaintNode, QSGNode, QSGGeometry, QSGMaterial and QSGTexture are used for.

Transcript

1. Using the Qt Quick

   Scene Graph API
   by Jocelyn Turcotte
   Woboq GmbH
   @woboq
   

   View Slide

2. Hello!
   • I’m Jocelyn Turcotte
   • Working at Woboq GmbH, Berlin
   • Worked on Qt WebKit and Qt WebEngine

   at Nokia / Digia / The Qt Company
   • Accelerated compositing integration of WebKit2
   and Chromium with the Qt Quick Scene Graph
   2
   

   View Slide

3. Agenda
   3
   1. Origins of the Qt Quick Scene Graph
   2. Writing your own QQuickItem:

   Using built-in QSGGeometryNodes
   3. Writing your own QSGGeometryNode:

   Custom geometries and shaders
   

   View Slide

4. 1. Origins of the

   Qt Quick Scene Graph
   

   View Slide

5. Qt Quick 1 vs. Qt Quick 2
   • Qt Quick 1:
   • Built on top of Qt Script and Qt Widgets
   • Uses QGraphicsView and QPainter for rendering
   • Qt Quick 2:
   • Built directly on top of Qt Gui
   • Uses QWindow and QOpenGL* for rendering
   5
   

   View Slide

6. View Slide

7. Rendering in Qt Quick 1
   7
   

   View Slide

8. Rendering in Qt Quick 2
   8
   

   View Slide

9. 2. Writing your own QQuickItem:
   Using built-in QSGGeometryNodes
   

   View Slide

10. Let's start with a Rectangle
    (demo)
    

    View Slide

11. Who owns an Item's paint node tree?
    • The Item temporarily owns its paint node tree
    during updatePaintNode()
    • There, the Item can update or destroy the tree
    • Items live on the GUI thread, QSGNodes live on
    the SG thread
    11
    

    View Slide

12. The Scene Graph Thread
    • Avoids blocking the GUI thread doing VSync
    • The scene graph thread deals with OpenGL
    • QQuick* objects live in the GUI thread,

    QSG* in the scene graph thread
    • QQuickItem::updatePaintNode() is the portal between the threads
    • Don't use mutexes! Transfer all states in updatePaintNode() instead
    12
    

    View Slide

13. A pathological QML example
    (demo)
    

    View Slide

14. Effects of the animation on the scene graph
    For each of all N points:
    • The "opacity" and "y" property bindings are
    updated
    • update() is called
    • Their underlying QSGTransformNode and a
    QSGOpacityNode are updated by QQuickItem
    14
    

    View Slide

15. But must QML be involved here?
    • We don't need direct control over each point in
    QML (no event handling, no use by other items)
    • The points data originally comes from C++
    • The text and revealProgress properties are the
    same for the whole QR code
    15
    

    View Slide

16. Let’s animate in updatePaintNode()
    16
    

    View Slide

17. It’s faster!
    But with 100% CPU usage of the GUI thread,

    the GPU is bored.
    

    View Slide

18. 3. Writing your own
    QSGGeometryNode: Custom
    geometries and shaders
    

    View Slide

19. First: Alternatives to consider
    • QML ShaderEffect

    Consider first, less flexible but easier to use
    • QML Canvas / QQuickPaintedItem

    Use when imperative painting is desired
    • QQuickFramebufferObject

    For standard OpenGL code, uses an FBO, preserves Z-order
    • QQuickWindow::(before|after)Rendering

    For standard OpenGL code, doesn't go through an FBO
    19
    

    View Slide

20. View Slide

21. Could we animate in the vertex shader?
    21
    

    View Slide

22. If it doesn't work
    • Read the “Qt Quick Scene Graph Renderer”
    documentation
    • Read the code! The scene graph only a framework at
    that level, OpenGL is your platform
    • Have a look at QQuickWindow,
    QSGThreadedRenderLoop and QSGBatchRenderer
    • Ask Woboq
    22
    

    View Slide

23. Conclusion
    • QML property bindings have limited scalability
    • Using too many QSGGeometryNodes can also be a
    scalability issue (because of algorithmic pressure)
    • Using opaque geometry materials reduces the worst
    case QSGGeometryNodes complexity
    • Computing in vertex/fragment shaders will be >20x
    faster and frees the GUI thread to do other work
    23
    

    View Slide

24. Thank you!
    jturcotte@woboq.com

    Twitter: @woboq
    

    View Slide

25. 4. Extra material
    

    View Slide

26. Batching
    • All vertices go in the same vertex buffer
    • The only GL state change between element
    draws of a batch is their transform
    26
    

    View Slide

27. Condition for a batch:
    • Geometries have the same drawingMode and same
    vertex attributes
    • Using the same QSGMaterial type
    • QSGMaterial::compare returns 0
    • Same computed opacity
    • If transparent: bounds not overlapping
    27
    

    View Slide

28. Merged batches
    • All transformed are flattened to a chosen root
    node
    • One single glDrawElements call
    • Root promotion heuristic: try to find a scrolling
    layer with many children
    28
    

    View Slide

29. Batching debug variables
    • QSG_RENDERER_DEBUG=dump

    Shows the complete QSGNode tree
    • QSG_RENDERER_DEBUG=render

    Gives an overview of all rendered batches
    • QSG_RENDERER_DEBUG=build

    Shows batch contents
    • QSG_RENDERER_DEBUG=roots

    Display which transform node is promoted as root for merged batches
    • QSG_VISUALIZE

    http://doc.qt.io/qt-5/qtquick-visualcanvas-scenegraph-renderer.html#visualizing
    29
    

    View Slide

30. Blending is off by default because
    • Overlapping transparent geometries MUST be
    rendered back to front
    • Can't render batches in an arbitrary z-order to
    use the depth buffer
    • Blending requires an extra work for the GPU
    30
    

    View Slide

31. Clipping isn't free
    • Nodes across clip nodes can't be batched
    together
    • Creates additional OpenGL state changes and
    performance costs
    31
    

    View Slide

32. QSGNode::preprocess()
    • Called on every frame, not just when QQuickItem::update() is called
    • Runs on the SG thread, after updatePaintNode() but before the scene’s
    rendering with OpenGL
    • The GUI thread isn't blocked at that point, can be used to do CPU work
    that we can’t do on the GPU while reducing the main thread’s event loop
    latency
    • Usually used to:
    • Render FBOs
    • Update a QSGDynamicTexture
    32
    

    View Slide

33. Textures
    • The QSGTexture must be owned by a node in the tree
    • Must be created in updatePaintNode() using window()-
    >createTextureFromImage()
    • Used by calling QSGTexture::bind in your
    QSGMaterial::updateState
    • Textures behind an Image, Canvas or ShaderEffectSource
    are available through the QSGTextureProvider API
    33
    

    View Slide

34. Using the index buffer
    • Our squares have 2 triangles onto 4 vertices, 2
    vertices are duplicated
    • Will use more memory
    • The vertex shader might run 6 times instead of 4
    • Need degenerate triangles with
    GL_TRIANGLES_STRIP, but takes even more
    memory
    • Solution: Use an index buffer to separate the vertex
    position and attribute from the geometry assembly
    • Trigger by passing indexCount >0 to QSGGeometry
    34
    

    View Slide