Building an interface for and with Climb - Slides

Transcript

1. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Building an interface for and with Climb Laurent Senta LRDE Laboratoire de Recherche et Développement d’EPITA July 11, 2012 http://lrde.epita.fr/ 1 / 28 Laurent Senta
2. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion 2 / 28 Laurent Senta
3. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Introduction Graph Based images Preview Morpher GUI Conclusion 3 / 28 Laurent Senta
4. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Climb Generic Image Processing Written in Common Lisp Still considered as a prototype 4 / 28 Laurent Senta
5. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Building a GUI with and for Climb load-image algorithm1 algorithm2 algorithm3 Section1 algorithm1 algorithm2 algorithm3 Section2 load-image save-image ... setting1 setting2 setting3 An excuse to use the library as an end-user Check that the library can handle new types Use Climb genericity outside Image Processing 5 / 28 Laurent Senta
6. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Conclusion 6 / 28 Laurent Senta
7. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Generic Image Definition Low level image access: matrix[x, y] = value graph.getNode(label) = value model[x, y, z] = value Generic access: image(site) = value In Climb: (setf (iref image site) value) 7 / 28 Laurent Senta
8. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Generic Image Definition Low level image access: matrix[x, y] = value graph.getNode(label) = value model[x, y, z] = value Generic access: image(site) = value In Climb: (setf (iref image site) value) 7 / 28 Laurent Senta
9. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Generic Image Definition Low level image access: matrix[x, y] = value graph.getNode(label) = value model[x, y, z] = value Generic access: image(site) = value In Climb: (setf (iref image site) value) 7 / 28 Laurent Senta
10. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Generic Image Definition Low level image access: matrix[x, y] = value graph.getNode(label) = value model[x, y, z] = value Generic access: image(site) = value In Climb: (setf (iref image site) value) 7 / 28 Laurent Senta
11. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Graph-based images set of vertices connected through edges values can be stored on nodes or edges A D C B Figure: Segmented image to graph 8 / 28 Laurent Senta
12. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Graph-based images Fixed grid images vs graph-based images ... 9 / 28 Laurent Senta
13. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Graph-based images Fixed grid images vs graph-based images ... Most Climb algorithms should work on graphs Is the library generic enough to allow it ? 9 / 28 Laurent Senta
14. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Issue In 2 dimensional images, neighborhoods are shifted site-set (windows). 3x3 window site(4; 5) (site+ site-i site(4; 5)) -1;-1 0;-1 1;-1 -1;0 0;0 1;0 -1;1 0;1 1;1 Figure: Shift using site − set − window 10 / 28 Laurent Senta
15. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Issue In 2 dimensional images, neighborhoods are shifted site-set (windows). 3x3 window site(4; 5) (site+ site-i site(4; 5)) -1;-1 0;-1 1;-1 -1;0 0;0 1;0 -1;1 0;1 1;1 Figure: Shift using site − set − window However, in graph we can’t infer a neighborhood from such fixed set. 10 / 28 Laurent Senta
16. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Issue example in dilation site-set like neighborhoods are defined and manipulated independently from the image. ( defun d i l a t i o n ( image window ) ; ; f o r each s i t e of the image domain ( do−sites ( s i t e image ) ; ; c e n t e r the window on the c u r r e n t s i t e ( l e t (( n e i g h b o r s ( site−set−window window s i t e ) ) ( v a l u e ( i r e f image s i t e ) ) ) ; ; browse a l l the neighbors , p i c k the maximal v a l u e ( do−sites ( neighbor n e i g h b o r s ) . . . ) ) ) ) 11 / 28 Laurent Senta
17. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Issue example in dilation site-set like neighborhoods are defined and manipulated independently from the image. ( defun d i l a t i o n ( image window ) ; ; f o r each s i t e of the image domain ( do−sites ( s i t e image ) ; ; c e n t e r the window on the c u r r e n t s i t e ( l e t (( n e i g h b o r s ( site−set−window window s i t e ) ) ( v a l u e ( i r e f image s i t e ) ) ) ; ; browse a l l the neighbors , p i c k the maximal v a l u e ( do−sites ( neighbor n e i g h b o r s ) . . . ) ) ) ) Rewrite current algorithms ? Extend Climb model with a new concept ? 11 / 28 Laurent Senta
18. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Unbound sites Inherits from site Abstract concept (e.g. ith neighbor) Must be bound to an image to produce a concrete coordinate site-set centering binding 1st neighbor 1st neighbor of node A A B D C image B 12 / 28 Laurent Senta
19. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Dilation ( d i l a t i o n image2d (8− c o n n e c t i v i t y )) ( d i l a t i o n graph ( s i t e − s e t − f i r s t − n e i g h b o r s )) 13 / 28 Laurent Senta
20. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Introduction Graph Based images Preview Morpher GUI Conclusion 14 / 28 Laurent Senta
21. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Climb Morphers Modify read and written value on images Transforms how an image is seen by the outside world (a) Morpher Grayscale (b) Morpher Mask 15 / 28 Laurent Senta
22. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Preview morpher Morphers with side effects on reading and writing Holds a GTK context (a windows and a surface) Updates the GTK surface along with the Climb image GTK Context Climb Image Morpher Preview Writing Reading Figure: Morpher Preview 16 / 28 Laurent Senta
23. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Real Time Visualization command real-time preview ( s e t f morphed ( image−load . . . ) ) ( d i l a t i o n morphed ( box2d−1 1)) ( e r o s i o n morphed ( box2d−1 1)) 17 / 28 Laurent Senta
24. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Introduction Graph Based images Preview Morpher GUI Description Model View Controller Conclusion 18 / 28 Laurent Senta
25. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Inspiration Climb provides a DSL that simplify chains of algorithm development. ( $ ( image−load " images / l e n a . png" ) ( otsu ) (// ( ( e r o s i o n (4− c o n n e c t i v i t y ) ) ) ( ( d i l a t i o n (4− c o n n e c t i v i t y ) ) ) ) ( d i f f #1 #2) ( image−save " output . png" ) ) Figure: $-macro Avoid low-level manipulation (temporary variables,...) Enhance readability Little knowledge in programming needed 19 / 28 Laurent Senta
26. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Climb’s graphical interface No programming knowledge needed Dynamic preview Iterative construction of complex operators load-image algorithm1 algorithm2 algorithm3 Section1 algorithm1 algorithm2 algorithm3 Section2 load-image save-image ... setting1 setting2 setting3 20 / 28 Laurent Senta
27. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion MVC Design Model View Controller data query notifications transformations user interactions view selection Figure: MVC pattern Model holds knowledge, data and processing View shows the model Controller links the user and the system 21 / 28 Laurent Senta
28. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Model Holds the processing chain built by the user. Processing chain: Climb graph-based image Node values: function call Edge values: parameter dependency image-load file=”image.png” value: erosion window=(4-connectivity) value: to-parameter: ‘image1 value: 22 / 28 Laurent Senta
29. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Model execution Execution is defined as a Climb graph algorithm: ( defun execute ( graph i n i t i a l s c o n t r o l l e r ) ( do−sites ( s i t e ( make−node−set−breadth graph i n i t i a l s ) ) ; r e t r i e v e the node v a l u e : f o r c a l l = ( i r e f graph s i t e ) ; c a l l the a l g o r i t h m update , the preview : f o r r e s u l t s = ( a l g o − c a l l c a l l c o n t r o l l e r ) ; update dependent nodes : do ( algo−sons−update graph s i t e r e s u l t s ) ) ) Climb’s tools on graph can be used The GUI benefits from the library’s enhancements 23 / 28 Laurent Senta
30. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion View The processing chain is displayed using Climb tools. Boxes: graph values displayed using Climb Thumbnails: Climb images (morpher-resize is used here) 24 / 28 Laurent Senta
31. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Controller Links the user and the system. Few functionality provided currently (model transformations) Additions to Climb may be used in the Controller Example: Loading and saving graph loading and saving GUI’s workspace 25 / 28 Laurent Senta
32. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Introduction Graph Based images Preview Morpher GUI Conclusion 26 / 28 Laurent Senta
33. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Conclusion graph, unbound-site Climb’s model can be extended easily preview-morpher, GUI Climb’s genericity can be used outside Image Processing GUI Climb now provides a graphical editor Future work GUI’s Processing Chain extension (cycles, hierarchy) New types and use case 27 / 28 Laurent Senta
34. Building an interface for and with Climb Introduction Graph Based images Image Definition Unbound site Usage Preview Morpher GUI Description Model View Controller Conclusion Questions ? 28 / 28 Laurent Senta