$30 off During Our Annual Pro Sale. View Details »

Using the Qt Quick
Scene Graph API

Using the Qt Quick
Scene Graph API

Recording available at https://youtu.be/cNE6Jabxxxo
Code available at https://github.com/woboq/using_the_qt_quick_scene_graph_api

The scene graph is the main change under the hood of QtQuick 2 that justified a new major version of the technology, while the rest initially remained similar to QtQuick 1. The extra performance is provided automatically for applications to enable smooth panning and scaling, but there are some situations where you can take advantage of using the scene graph API directly to smoothen your custom graphical item animations.

This talk presents how the lower-level scene graph API in QtQuick can, with the help of a few lines of C++ code, allow you to reduce the overhead of QML abstractions and offload some transform computations to the GPU. In addition this presents:

- What the scene graph is and how it relates to a QML component or QQuickItem.
- How it is different from a QGraphicsScene.
- How it interacts with OpenGL and the GPU.
- How you can write code to run safely in the render thread and taking advantage of the geometry batching done by the renderer.
- What APIs like QQuickItem::updatePaintNode, QSGNode, QSGGeometry, QSGMaterial and QSGTexture are used for.

Jocelyn Turcotte

October 07, 2015
Tweet

Other Decks in Programming

Transcript

  1. Using the Qt Quick

    Scene Graph API
    by Jocelyn Turcotte
    Woboq GmbH
    @woboq

    View Slide

  2. Hello!
    • I’m Jocelyn Turcotte
    • Working at Woboq GmbH, Berlin
    • Worked on Qt WebKit and Qt WebEngine

    at Nokia / Digia / The Qt Company
    • Accelerated compositing integration of WebKit2
    and Chromium with the Qt Quick Scene Graph
    2

    View Slide

  3. Agenda
    3
    1. Origins of the Qt Quick Scene Graph
    2. Writing your own QQuickItem:

    Using built-in QSGGeometryNodes
    3. Writing your own QSGGeometryNode:

    Custom geometries and shaders

    View Slide

  4. 1. Origins of the

    Qt Quick Scene Graph

    View Slide

  5. Qt Quick 1 vs. Qt Quick 2
    • Qt Quick 1:
    • Built on top of Qt Script and Qt Widgets
    • Uses QGraphicsView and QPainter for rendering
    • Qt Quick 2:
    • Built directly on top of Qt Gui
    • Uses QWindow and QOpenGL* for rendering
    5

    View Slide

  6. View Slide

  7. Rendering in Qt Quick 1
    7

    View Slide

  8. Rendering in Qt Quick 2
    8

    View Slide

  9. 2. Writing your own QQuickItem:
    Using built-in QSGGeometryNodes

    View Slide

  10. Let's start with a Rectangle
    (demo)

    View Slide

  11. Who owns an Item's paint node tree?
    • The Item temporarily owns its paint node tree
    during updatePaintNode()
    • There, the Item can update or destroy the tree
    • Items live on the GUI thread, QSGNodes live on
    the SG thread
    11

    View Slide

  12. The Scene Graph Thread
    • Avoids blocking the GUI thread doing VSync
    • The scene graph thread deals with OpenGL
    • QQuick* objects live in the GUI thread,

    QSG* in the scene graph thread
    • QQuickItem::updatePaintNode() is the portal between the threads
    • Don't use mutexes! Transfer all states in updatePaintNode() instead
    12

    View Slide

  13. A pathological QML example
    (demo)

    View Slide

  14. Effects of the animation on the scene graph
    For each of all N points:
    • The "opacity" and "y" property bindings are
    updated
    • update() is called
    • Their underlying QSGTransformNode and a
    QSGOpacityNode are updated by QQuickItem
    14

    View Slide

  15. But must QML be involved here?
    • We don't need direct control over each point in
    QML (no event handling, no use by other items)
    • The points data originally comes from C++
    • The text and revealProgress properties are the
    same for the whole QR code
    15

    View Slide

  16. Let’s animate in updatePaintNode()
    16

    View Slide

  17. It’s faster!
    But with 100% CPU usage of the GUI thread,

    the GPU is bored.

    View Slide

  18. 3. Writing your own
    QSGGeometryNode: Custom
    geometries and shaders

    View Slide

  19. First: Alternatives to consider
    • QML ShaderEffect

    Consider first, less flexible but easier to use
    • QML Canvas / QQuickPaintedItem

    Use when imperative painting is desired
    • QQuickFramebufferObject

    For standard OpenGL code, uses an FBO, preserves Z-order
    • QQuickWindow::(before|after)Rendering

    For standard OpenGL code, doesn't go through an FBO
    19

    View Slide

  20. 20

    View Slide

  21. Could we animate in the vertex shader?
    21

    View Slide

  22. If it doesn't work
    • Read the “Qt Quick Scene Graph Renderer”
    documentation
    • Read the code! The scene graph only a framework at
    that level, OpenGL is your platform
    • Have a look at QQuickWindow,
    QSGThreadedRenderLoop and QSGBatchRenderer
    • Ask Woboq
    22

    View Slide

  23. Conclusion
    • QML property bindings have limited scalability
    • Using too many QSGGeometryNodes can also be a
    scalability issue (because of algorithmic pressure)
    • Using opaque geometry materials reduces the worst
    case QSGGeometryNodes complexity
    • Computing in vertex/fragment shaders will be >20x
    faster and frees the GUI thread to do other work
    23

    View Slide

  24. Thank you!
    [email protected]
    Twitter: @woboq

    View Slide

  25. 4. Extra material

    View Slide

  26. Batching
    • All vertices go in the same vertex buffer
    • The only GL state change between element
    draws of a batch is their transform
    26

    View Slide

  27. Condition for a batch:
    • Geometries have the same drawingMode and same
    vertex attributes
    • Using the same QSGMaterial type
    • QSGMaterial::compare returns 0
    • Same computed opacity
    • If transparent: bounds not overlapping
    27

    View Slide

  28. Merged batches
    • All transformed are flattened to a chosen root
    node
    • One single glDrawElements call
    • Root promotion heuristic: try to find a scrolling
    layer with many children
    28

    View Slide

  29. Batching debug variables
    • QSG_RENDERER_DEBUG=dump

    Shows the complete QSGNode tree
    • QSG_RENDERER_DEBUG=render

    Gives an overview of all rendered batches
    • QSG_RENDERER_DEBUG=build

    Shows batch contents
    • QSG_RENDERER_DEBUG=roots

    Display which transform node is promoted as root for merged batches
    • QSG_VISUALIZE

    http://doc.qt.io/qt-5/qtquick-visualcanvas-scenegraph-renderer.html#visualizing
    29

    View Slide

  30. Blending is off by default because
    • Overlapping transparent geometries MUST be
    rendered back to front
    • Can't render batches in an arbitrary z-order to
    use the depth buffer
    • Blending requires an extra work for the GPU
    30

    View Slide

  31. Clipping isn't free
    • Nodes across clip nodes can't be batched
    together
    • Creates additional OpenGL state changes and
    performance costs
    31

    View Slide

  32. QSGNode::preprocess()
    • Called on every frame, not just when QQuickItem::update() is called
    • Runs on the SG thread, after updatePaintNode() but before the scene’s
    rendering with OpenGL
    • The GUI thread isn't blocked at that point, can be used to do CPU work
    that we can’t do on the GPU while reducing the main thread’s event loop
    latency
    • Usually used to:
    • Render FBOs
    • Update a QSGDynamicTexture
    32

    View Slide

  33. Textures
    • The QSGTexture must be owned by a node in the tree
    • Must be created in updatePaintNode() using window()-
    >createTextureFromImage()
    • Used by calling QSGTexture::bind in your
    QSGMaterial::updateState
    • Textures behind an Image, Canvas or ShaderEffectSource
    are available through the QSGTextureProvider API
    33

    View Slide

  34. Using the index buffer
    • Our squares have 2 triangles onto 4 vertices, 2
    vertices are duplicated
    • Will use more memory
    • The vertex shader might run 6 times instead of 4
    • Need degenerate triangles with
    GL_TRIANGLES_STRIP, but takes even more
    memory
    • Solution: Use an index buffer to separate the vertex
    position and attribute from the geometry assembly
    • Trigger by passing indexCount >0 to QSGGeometry
    34

    View Slide