CorelDraw Import filter in LibreOffice: a history of an incremental reverse-engineering

Transcript

1. 1 CorelDraw Import filter in LibreOffice CorelDraw Import filter in LibreOffice a history of an incremental reverse-engineering Fridrich Štrba, LibreOffice Developer, SUSE
2. 2 CorelDraw Import filter in LibreOffice Agenda Present the CorelDraw import filter in LibreOffice Reveal a bit how the reverse-engineering work is done Tools Methods Present some theories that allowed us to work fast Incremental reverse-engineering Preconditions of success
3. 3 CorelDraw Import filter in LibreOffice LibreOffice and CorelDraw filter
4. 4 CorelDraw Import filter in LibreOffice Flagship feature of the 3.6 release Work started slowly at the end of 2011 Verification of methods used for Visio import filter Standalone library libcdr released under dual MPL1.1+ and LGPL 2.1+ Reuse and improvement of the ODG generation framework Already used for the WordPerfect Graphics and Visio work The fun continued in 2012 with MS Publisher import filter
5. 5 CorelDraw Import filter in LibreOffice Why CorelDraw Several times a Google Summer of Code idea Never any student chose it :) Still one of the top TODOs in the filter realm Valek's reverse-engineering work cdr_explorer Some of it reused in sk1 project, which is currently dormant An interesting challenge after the success of libvisio Continuation of a fruitful collaboration
6. 6 CorelDraw Import filter in LibreOffice State of the import filter (1) Support for all CorelDraw file-formats Starting from version 1 (code Waldo) Ending by CorelDraw x6 released in March 2012 Supported features Line styles, fill styles, embedded bitmaps, raster fills,... Spot and process colours from several colour spaces Thanks to the use of littlecms2 Basic support for text Still needs more reverse-engineering and implementing
7. 7 CorelDraw Import filter in LibreOffice State of the import filter (2) Work-in-progress features Better gradient support Limitation is the ODG and LibreOffice gradient capacity CorelDraw gradients are more expressive More text features Complete text properties in the 32-bit formats CDR 6 to x6 Finish reverse-engineering and implementing for 16-bit formats CDR 1 to 5 We are accepting patches!
8. 8 CorelDraw Import filter in LibreOffice Reverse-engineering? Reverse? What?
9. 9 CorelDraw Import filter in LibreOffice Pre-existing documentation Almost none cdr_explorer with Valek's reverse-engineering work RIFF based Covering CorelDraw 7 to x3 versions Document explaining a bit the CMX exchange format similar concepts understanding of some enums
10. 10 CorelDraw Import filter in LibreOffice Development method Focus on getting “some” result early Seeing real pictures imported is encouraging First embedded raster images LibreOffice is able to render them without further processing Next graphic primitives “Everything is just a path” Some code is better then an abstract design Possibility to find and fix real bugs Verification of theories in almost real time
11. 11 CorelDraw Import filter in LibreOffice Tools used oletoy File introspection tool for different file-formats colupatr Intelligent hexadecimal viewer Important role for discovering patterns visually Develop tools along the implementation Introspection tool improved constantly Driven by the need of the implementation Reflecting growing understanding of file-format Don't solve problems that don't exist
12. 12 CorelDraw Import filter in LibreOffice Incremental reverse-engineering
13. 13 CorelDraw Import filter in LibreOffice Extending the file-format coverage (1) Departing point Support for versions 7 to x3 Basically the knowledge from cdr_explorer Extending the coverage upwards x4 and x5 Support for RIFF documents inside structured ZIP storage x6 More complicated structure inside the ZIP storage
14. 14 CorelDraw Import filter in LibreOffice Evolution of ZIP-based CDR file-formats (1) Version x4 -rw---- 4.5 fat 2610 bx stor 12-Mar-05 05:19 content/riffData.cdr -rw---- 4.5 fat 196662 bx defN 12-Mar-05 05:19 metadata/thumbnails/thumbnail.bmp -rw---- 4.5 fat 184374 bx defN 12-Mar-05 05:19 metadata/thumbnails/page1.bmp -rw---- 4.5 fat 5860 bx defN 12-Mar-05 05:19 metadata/metadata.xml -rw---- 4.5 fat 667 bx defN 12-Mar-05 05:19 metadata/textinfo.xml -rw---- 4.5 fat 53 bx defN 12-Mar-05 05:19 links.xml Version x5 -rw---- 4.5 fat 2608 bx stor 12-Mar-05 05:19 content/riffData.cdr -rw---- 4.5 fat 196662 bx defN 12-Mar-05 05:19 metadata/thumbnails/thumbnail.bmp -rw---- 4.5 fat 184374 bx defN 12-Mar-05 05:19 metadata/thumbnails/page1.bmp -rw---- 4.5 fat 252 bx defN 12-Mar-05 05:19 color/color.xml -rw---- 4.5 fat 5860 bx defN 12-Mar-05 05:19 metadata/metadata.xml -rw---- 4.5 fat 667 bx defN 12-Mar-05 05:19 metadata/textinfo.xml -rw---- 4.5 fat 103 bx defN 12-Mar-05 05:19 color/docPalette.xml -rw---- 4.5 fat 53 bx defN 12-Mar-05 05:19 links.xml
15. 15 CorelDraw Import filter in LibreOffice Evolution of ZIP-based CDR file-formats (2) Version x6 -rw---- 4.5 fat 14315 bx defN 12-Mar-05 05:18 content/data/data1.dat -rw---- 4.5 fat 445 bx defN 12-Mar-05 05:18 content/data/masterPage.dat -rw---- 4.5 fat 3355 bx defN 12-Mar-05 05:18 content/data/page1.dat -rw---- 4.5 fat 34 bx defN 12-Mar-05 05:18 content/dataFileList.dat -rw---- 4.5 fat 1332 bx defN 12-Mar-05 05:18 content/root.dat -rw---- 4.5 fat 196662 bx defN 12-Mar-05 05:18 metadata/thumbnails/thumbnail.bmp -rw---- 4.5 fat 184374 bx defN 12-Mar-05 05:18 metadata/thumbnails/page1.bmp -rw---- 4.5 fat 252 bx defN 12-Mar-05 05:18 color/color.xml -rw---- 4.5 fat 5860 bx defN 12-Mar-05 05:18 metadata/metadata.xml -rw---- 4.5 fat 667 bx defN 12-Mar-05 05:18 metadata/textinfo.xml -rw---- 4.5 fat 103 bx defN 12-Mar-05 05:18 color/docPalette.xml -rw---- 4.5 fat 15377 bx defN 12-Mar-05 05:18 styles/document.cdss -rw---- 4.5 fat 53 bx defN 12-Mar-05 05:18 links.xml
16. 16 CorelDraw Import filter in LibreOffice Extending the file-format coverage (2) Extending the coverage downwards Version 6 (first 32-bit version) Only some RIFF names different Versions 4 and 5 (16-bit versions) Different way to express coordinates The CMX exchange format document has some hints 16-bit coordinates are specified in 1/1 000 of an inch, while 32-bit coordinates are specified in 1/10 000 000 of a millimeter. Conditional coordinate reading according to the version
17. 17 CorelDraw Import filter in LibreOffice Extending the file-format coverage (3) Extending the coverage downwards (cont'ed) Version 3 First RIFF based CDR file-format but we did not know it by then Fill and outline information embedded inside the shape Shape transform does not accumulate group transforms Versions 2 and 1 Not RIFF based at all Version 2 more structured With some exception handling both can be parsed alike A header with pointers to different sequences of chunks Implementation of linked list (“type 1”) and shape information (“type 2”) Embedded raster (“type 3” and “6”), group transforms (“type 7”), arrow information (“type 8”),
18. 18 CorelDraw Import filter in LibreOffice Lessons learned (1) Progressive development of file-formats Nobody reinvents a wheel from scratch It is useful to know the release dates of different versions when doing incremental reverse-engineering Common DNA Two subsequent versions of the same file-format will have many things in common Similar structure of the shape information between versions 2 and 3
19. 19 CorelDraw Import filter in LibreOffice Lesson learned (2) Design parser to be able to parse lower and higher versions Opened version conditions Guard assumptions by exceptions and be verbose in debug mode Try to parse lower or higher version using the existing parser Important to have version conditions opened Fix issues as they appear Small number of reference documents covering many features
20. 20 CorelDraw Import filter in LibreOffice Reverse-engineering – the never ending story The hypotheses and assumptions are challenged with every new document that “does not convert correctly” Even a complex feature document is easily beaten by real life documents. Heise example Not a very good review of the CDR import filter in c't “Ein neuer Import-Filter in Draw öffnet jetzt auch CorelDraw-Dateien, was uns im Test allerdings nur mit sehr einfachen Zeichnungen fehlerfrei gelang. In dieser Form ist er schlicht unbrauchbar.” “A new import filter in Draw opens now also CorelDraw files, which we managed to do without errors only with very simple drawings. In this form, it is rather unusable.”
21. 21 CorelDraw Import filter in LibreOffice We hear our users even when they cry in the wilderness Before LibreOffice 3.6.2.2 After LibreOffice 3.6.3.1
22. 22 CorelDraw Import filter in LibreOffice All text and image content in this document is licensed under the Creative Commons Attribution-Share Alike 3.0 License (unless otherwise specified). "LibreOffice" and "The Document Foundation" are registered trademarks. Their respective logos and icons are subject to international copyright laws. The use of these therefore is subject to the trademark policy. Questions and Stones!