Using the Qt Quick
 Scene Graph API by Jocelyn Turcotte Woboq GmbH @woboq

Hello! • I’m Jocelyn Turcotte • Working at Woboq GmbH, Berlin • Worked on Qt WebKit and Qt WebEngine
 at Nokia / Digia / The Qt Company • Accelerated compositing integration of WebKit2 and Chromium with the Qt Quick Scene Graph 2

Agenda 3 1. Origins of the Qt Quick Scene Graph 2. Writing your own QQuickItem:
 Using built-in QSGGeometryNodes 3. Writing your own QSGGeometryNode:
 Custom geometries and shaders

1. Origins of the
 Qt Quick Scene Graph

Qt Quick 1 vs. Qt Quick 2 • Qt Quick 1: • Built on top of Qt Script and Qt Widgets • Uses QGraphicsView and QPainter for rendering • Qt Quick 2: • Built directly on top of Qt Gui • Uses QWindow and QOpenGL* for rendering 5

No content

Rendering in Qt Quick 1 7

Rendering in Qt Quick 2 8

2. Writing your own QQuickItem: Using built-in QSGGeometryNodes

Let's start with a Rectangle (demo)

Who owns an Item's paint node tree? • The Item temporarily owns its paint node tree during updatePaintNode() • There, the Item can update or destroy the tree • Items live on the GUI thread, QSGNodes live on the SG thread 11

The Scene Graph Thread • Avoids blocking the GUI thread doing VSync • The scene graph thread deals with OpenGL • QQuick* objects live in the GUI thread,
 QSG* in the scene graph thread • QQuickItem::updatePaintNode() is the portal between the threads • Don't use mutexes! Transfer all states in updatePaintNode() instead 12

A pathological QML example (demo)

Effects of the animation on the scene graph For each of all N points: • The "opacity" and "y" property bindings are updated • update() is called • Their underlying QSGTransformNode and a QSGOpacityNode are updated by QQuickItem 14

But must QML be involved here? • We don't need direct control over each point in QML (no event handling, no use by other items) • The points data originally comes from C++ • The text and revealProgress properties are the same for the whole QR code 15

Let’s animate in updatePaintNode() 16

It’s faster! But with 100% CPU usage of the GUI thread,
 the GPU is bored.

3. Writing your own QSGGeometryNode: Custom geometries and shaders

First: Alternatives to consider • QML ShaderEffect
 Consider first, less flexible but easier to use • QML Canvas / QQuickPaintedItem
 Use when imperative painting is desired • QQuickFramebufferObject
 For standard OpenGL code, uses an FBO, preserves Z-order • QQuickWindow::(before|after)Rendering
 For standard OpenGL code, doesn't go through an FBO 19

20

Could we animate in the vertex shader? 21

If it doesn't work • Read the “Qt Quick Scene Graph Renderer” documentation • Read the code! The scene graph only a framework at that level, OpenGL is your platform • Have a look at QQuickWindow, QSGThreadedRenderLoop and QSGBatchRenderer • Ask Woboq 22

Conclusion • QML property bindings have limited scalability • Using too many QSGGeometryNodes can also be a scalability issue (because of algorithmic pressure) • Using opaque geometry materials reduces the worst case QSGGeometryNodes complexity • Computing in vertex/fragment shaders will be >20x faster and frees the GUI thread to do other work 23

Thank you! jturcotte@woboq.com
 Twitter: @woboq

4. Extra material

Batching • All vertices go in the same vertex buffer • The only GL state change between element draws of a batch is their transform 26

Condition for a batch: • Geometries have the same drawingMode and same vertex attributes • Using the same QSGMaterial type • QSGMaterial::compare returns 0 • Same computed opacity • If transparent: bounds not overlapping 27

Merged batches • All transformed are flattened to a chosen root node • One single glDrawElements call • Root promotion heuristic: try to find a scrolling layer with many children 28

Batching debug variables • QSG_RENDERER_DEBUG=dump
 Shows the complete QSGNode tree • QSG_RENDERER_DEBUG=render
 Gives an overview of all rendered batches • QSG_RENDERER_DEBUG=build
 Shows batch contents • QSG_RENDERER_DEBUG=roots
 Display which transform node is promoted as root for merged batches • QSG_VISUALIZE
 http://doc.qt.io/qt-5/qtquick-visualcanvas-scenegraph-renderer.html#visualizing 29

Blending is off by default because • Overlapping transparent geometries MUST be rendered back to front • Can't render batches in an arbitrary z-order to use the depth buffer • Blending requires an extra work for the GPU 30

Clipping isn't free • Nodes across clip nodes can't be batched together • Creates additional OpenGL state changes and performance costs 31

QSGNode::preprocess() • Called on every frame, not just when QQuickItem::update() is called • Runs on the SG thread, after updatePaintNode() but before the scene’s rendering with OpenGL • The GUI thread isn't blocked at that point, can be used to do CPU work that we can’t do on the GPU while reducing the main thread’s event loop latency • Usually used to: • Render FBOs • Update a QSGDynamicTexture 32

Textures • The QSGTexture must be owned by a node in the tree • Must be created in updatePaintNode() using window()- >createTextureFromImage() • Used by calling QSGTexture::bind in your QSGMaterial::updateState • Textures behind an Image, Canvas or ShaderEffectSource are available through the QSGTextureProvider API 33

Using the index buffer • Our squares have 2 triangles onto 4 vertices, 2 vertices are duplicated • Will use more memory • The vertex shader might run 6 times instead of 4 • Need degenerate triangles with GL_TRIANGLES_STRIP, but takes even more memory • Solution: Use an index buffer to separate the vertex position and attribute from the geometry assembly • Trigger by passing indexCount >0 to QSGGeometry 34