LibreOffice Core Unit Testing & Well Documented Tests

Collabora Online Team Time Trainings 2025 (https://collaboraonline.github.io/post/communicate/#tea-time-trainings)

Transcript

1. Miklos Vajna Software Engineer Core Unit Testing Well Documented Tests vmiklos@collabora.com 2025-10-17
2. 2 Motivation (via catgirlsfor.science)
3. 3 Motivation (via catgirlsfor.science) Kohei's advice (slides) after the Calc storage rework:  One unit test is worth 20 future bug fixes.  A bug fix does not finish until you write a test.  Writing unit test is courtesy for your fellow developers.  We really don’t have a choice.
4. Core Testing
5. 5 No fix without a test Would be ideal, but not all changes are testable:  Testable, as in with a reasonable amount of effort  File import/export fixes are reasonably straightforward to test, don’t skip it  Most LOK changes are testable in the tiledrendering suites  If the change is not testable, do mention that in the commit message, so it’s explicit you considered it  We focus on regression testing here, not e.g. proactive testing based on code coverage
6. 6 Unit testing: one function/class The only time we really do unit testing, not integration testing  E.g. OUString concatenation tests, checkConcat(): https://github.co m/libreoffice/core/blob/87613ae13cf3260f82d12afd35e67c1a7e7 5984c/sal/qa/rtl/strings/test_oustring_concat.cxx#L37-L70  Test the changed function directly  Not practical in many cases, your test code may want to be closer to the user description of the problem
7. 7 File import tests Load the document and assert some results  Example: testTdf117246, https://github.com/libreoffice/core/blob/ 87613ae13cf3260f82d12afd35e67c1a7e75984c/sw/qa/extras/rtfi mport/rtfimport.cxx#L130-L135  Asserting the resulting document model, typically using the UNO API or code that’s internal to the current module (sw/, sc/, sd/, etc.)
8. 8 File export tests Typically import, export, reimport, assert  Example, testDMLSolidfillAlpha: https://github.com/libreoffice/cor e/blob/87613ae13cf3260f82d12afd35e67c1a7e75984c/sw/qa/extr as/ooxmlexport/ooxmlexport6.cxx#L141-L151  Alternative approach: if the export format has some low-level parser, work with that, e.g. XML, HTML  Ideal: build the model from code, export, assert (not practical in many cases)
9. 9 XPath assertions Idea: work with XML, map GDIMetafile or Writer layout to XML  Example, testBehinddoc(): https://github.com/libreoffice/core/blo b/87613ae13cf3260f82d12afd35e67c1a7e75984c/sw/qa/extras/o oxmlexport/ooxmlexport6.cxx#L715-L721  Also works for drawinglayer primitives, useful for WMF/EMF/EMF+ fixes
10. 10 Asserting pixels in a bitmap Only use it as a last resort: • Example, testFdo77229: https://github.com/libreoffice/core/blob/ 87613ae13cf3260f82d12afd35e67c1a7e75984c/cppcanvas/qa/extr as/emfplus/emfplus.cxx#L51-L60 • In most cases there is a better, more reliable way to test the rendering result, that doesn’t hardcode screen coordinates
11. 11 Layout dump test Writer layout has a very deep tree, the inner parts are private  Solution: dump it to XML, then use XPath assert  “Make sure the textframe containing the text "third" appears on the 3rd page.” • Example, testFdo52052: https://github.com/libreoffice/core/blob/ 87613ae13cf3260f82d12afd35e67c1a7e75984c/sw/qa/extras/rtfi mport/rtfimport.cxx#L461-L467
12. 12 LOK test This tests core code, driven by LOK API calls from online code  Most importantly: turns on comphelper::LibreOfficeKit::isActive(), which is off everywhere else  Allows testing user input like pressing a key as if it would be real keyboard input  Example: testTdf89954, https://github.com/libreoffice/core/blob/8 7613ae13cf3260f82d12afd35e67c1a7e75984c/sw/qa/extras/uiwrit er/uiwriter7.cxx#L2456-L2468
13. 13 Python UI test Cppunit tests run in headless mode, and sometimes that’s not good  Enter the world of core-side UI tests  Example: test_insert_page_number, https://github.com/libreoffice /core/blob/87613ae13cf3260f82d12afd35e67c1a7e75984c/sw/qa /uitest/writer_tests3/pageNumber.py#L16-L38  See https://wiki.documentfoundation.org/Development/UITests fore more info
14. 14 Performance test A weak test type  Example: ScPerfObj::testFTest(), https://github.com/libreoffice/cor e/blob/87613ae13cf3260f82d12afd35e67c1a7e75984c/sc/qa/perf /scperfobj.cxx#L284-L287  Uses callgrind to measure pseudo-cycles, not part of ‘make check’  Results have to be interpreted manually, though some charts are available at https://perf.libreoffice.org/
15. 15 Discovering a test strategy This may be obvious, but...  Attach gdb  Insert a breakpoint in a relevant part of the the bug fix  Manually do the thing that used to cause the bug and inspect the backtrace  In your cppunit test you invoke the function similarly to how you see it invoked in the backtrace
16. 16 Run a single test at a time core.git make check takes ~1h  A single test suite takes a couple of minutes  A single test case ideally takes a couple of seconds at most  make -C sw -sr CppunitTest_sw_writerfilter_dmapper CPPUNIT_TEST_NAME=testDOCXFloatingTableHidemark  Additional CPPUNITTRACE="gdb --args" at the end if you need gdb
17. 17 Demo How to test deleting tables works in Writer? A possible way:  Have a document that has two tables in it already  Go to the first table  Delete the table  Assert that we now only have a single table
18. Documenting your tests
19. 19 Given, when, then Documenting tests helps the next reader  Setup the context: “given a document with two tables”  Execute the interesting operation: “when deleting the second table”  Assert the result: “there is one table in the document”  Separating these 3 steps help understanding the intent of the test when it fails or during refactor  Document these with source code comments  Document how the test failed without the fix: if the value is actually 101 and we expected 100, it does matter if it was 2 or 101 when the test originally failed without the fix
20. Final thoughts
21. 21 Writing tests is worth it Don’t just skip it  Test Driven Development (TDD) may be too extreme for our codebase, but writing tests for newly created fixes is totally doable most of the time  Do check if the test fails without the fix, ideally in some readable way  Document how the test failed without the fix  More info: https://wiki.documentfoundation.org/Development/Cpp_Unit _Tests