1 25-27 NOVEMBER SOFTWARE TESTING, MACHINE LEARNING AND COMPLEX PROCESS ANALYSIS Optic Flow approximated by a homogeneous, three- dimensional Point Renewal Process Aim: to create prerequisites for effective preparation of a directed experiment with a minimum number of simulation iterations.

2 Authors 1Tomsk State University of Control Systems and Radioelectronics E-mail: dmitrii.v.dubinin@tusur.ru, 2National Research Tomsk Polytechnic University E-mail: kaicc@tpu.ru, kocheg@mail.ru 3Baden-Wuerttemberg Cooperative State University E-mail: geringer@DHBW-Ravensburg.de D.V. Dubinin1, A.I. Kochegurov2, E.A. Kochegurova2 , V.E. Geringer3

3 Urgency of the problem and the state of the art Currently, evaluation of image processing algorithms has been disadvantaged by the use of real images as reference images. This makes it difficult to assess the quality of the algorithms, to refine and compare them. Problems: • Non-repeatability of the experiment • Selection and optimization of algorithms • Selection of algorithm parameters • Adaptation of algorithms to a subject area • Subjectivity of assessments, results and recommendations • Difficulty of porting to other subject areas

4 Solutions: Analytical solution Experimental research a) Fast testing of hypothesis b) Simplified calculations c) Detailed analysis of the results d) Increased number of alternatives e) Improved quality of the research subject f) Precise prediction of consequences Urgency of the problem and the state of the art

5 Approaches to solving the problem

6 Approaches to solving the problem The use of artificial reference images (image streams) synthesized based on the probabilistic factor

7 Approaches to solving the problem • free use in different subject areas • elimination of the subjective factor in the selection of test images • introduction of the probabilistic factor into the analysis of the selected image processing algorithms • introduction of performance indicators for the processing algorithm Advantages :

8 Timeline: 1978 Publication in Automatic Control and Computer Sciences : Sergeev V.V., Soifer V.A. Image simulation model and data compression method, Automation and computing. - 1978. - no 3. - pp.76-78. V.A. Soifer V.V. Sergeev

9 Timeline: 1981 - 1985 A number of publications in Autometria : 1) On the statistics of palm fields, Autometry. 1981. No. 6. pp. 13-18. 2) Statistical analysis of correlations in the palm field, Autometry. 1981. No. 6. pp. 87-89. 3) Investigation of autocorrelation of images by scaling, rotations and shifts, Autometry. 1982. No. 1. pp. 84-87. A.G. Buimov

10 Recovery flow model: Basic alphabet on the left (allows building morphologies of types A, B, C, D, E ) 0 34 170 10 42 138 40 130 162 168 160 136 17 68 85 5 20 65 80 21 69 81 84 r 00 r 01 r 02 r 05 r 12 a c e g b f d h a c d e g h b f a c e g d h b f a c e g b f d h a c e g h f b d a c e g h b f d a c e g d f b h a c e g d b f h a c e g h b f d a c e g f b d h a c e g d b f h a c e g b f d h d h b f g c a e d h b f e a c g d h b f c a e g d h b f e g c a d h b f g a c e d h b f e c a g d h b f a c e g d h b f g c a e d h b f e c a g d h b f c a e g d h b f a c e g Basis ABC (Gittertypen A, B, C, D, E) Erweiterung (Gittertypen F, FB, FC, FD, FE) Gruppe 00 Gruppe 01 Gruppe 02 Gruppe 05 Gruppe 12 0 34 170 10 42 138 40 130 162 168 160 136 17 68 85 5 20 65 80 21 69 81 84 r 00 r 01 r 02 r 05 r 12 a c e g b f d h a c d e g h b f a c e g d h b f a c e g b f d h a c e g h f b d a c e g h b f d a c e g d f b h a c e g d b f h a c e g h b f d a c e g f b d h a c e g d b f h a c e g b f d h d h b f g c a e d h b f e a c g d h b f c a e g d h b f e g c a d h b f g a c e d h b f e c a g d h b f a c e g d h b f g c a e d h b f e c a g d h b f c a e g d h b f a c e g Basis ABC (Gittertypen A, B, C, D, E) Erweiterung (Gittertypen F, FB, FC, FD, FE) Gruppe 00 Gruppe 01 Gruppe 02 Gruppe 05 Gruppe 12 in the work of A.G. Buimov state space of 12 elements v L

11 Recovery flow model: 0 34 170 10 42 138 40 130 162 168 160 136 r 00 r 01 r 02 r 05 r 12 a c e g b f d h a c d e g h b f a c e g d h b f a c e g b f d h a c e g h f b d a c e g h b f d a c e g d f b h a c e g d b f h a c e g h b f d a c e g f b d h a c e g d b f h a c e g b f d h Basis ABC (Gittertypen A, B, C, D, E) 0 34 170 10 42 138 40 130 162 168 160 136 r 00 r 01 r 02 r 05 r 12 a c e g b f d h a c d e g h b f a c e g d h b f a c e g b f d h a c e g h f b d a c e g h b f d a c e g d f b h a c e g d b f h a c e g h b f d a c e g f b d h a c e g d b f h a c e g b f d h Basis ABC (Gittertypen A, B, C, D, E) - Group probability (no contour) - Group probability (The contour runs: vertically, horizontally) - Group probability (The contour runs: in 3 directions) - Group probability (Contour passes: reversing direction by 90 0) - Group probability (The contour runs: in two directions) State space related by group probabilities

12 Recovery flow model: Neighborhood of point Z Eight possible directions (a, b, c, d, e, f, g, h)

13 0 1 2 3 4 5 6 7 2 2 2 2 2 2 2 2 Z a b c d e f g h = ⋅ + ⋅ + ⋅ + ⋅ + ⋅ + ⋅ + ⋅ + ⋅ Neighborhood of point Z Recovery flow model:

14 00 r − 01 r − 12 r − 1 1 1 0 0 0 ( ) ( ) ( ) f g h P abcdefgh P fgh abcde P abcdefgh = = = = ∑∑∑ Recovery flow model:

15 0 1 2 5 12 [0] 00 [34] [136] 01 [170] 02 [10] [40] [130] [160] 05 [42] [ Normalization conditions: 1 Relationship between the probability of individual letters and group probabilities : 2 4 r r r r r P r P P r P r P P P P r P P + + + + = = = = = = = = = = 138] [162] [168] 12 4 P P r = = = The uniformity of the structure (as well as the stationarity of the field properties) is achieved due to the following: 1 2 Recovery flow model:

16 3 05 0 00 05 01 05 12 00 00 2 4 2 4 2 4 4 r p r r r r r r r  = +       + = + + +       The following conditions must be met [from the works by A.G. Buimov]: The uniformity of the structure (as well as the stationarity of the field properties) is achieved due to the following: Recovery flow model:

17 4 5 The uniformity of the structure (as well as the stationarity of the field properties) is achieved due to the following : The transition probability matrix of the point renewal flow is derived based on Bayes' theorem for all possible mosaic structures (Types: A, B, C, D, E, F, FB, FC, FD, FE): 05 00 01 05 12 01 05 12 05 12 02 4 2 4 4 2 4 4 4 2 m r r r r r p r r r r r r   +       + +   =     + +       + +     Vector of final probabilities: Recovery flow model:

18 General view of the transition probability matrix for mosaic structures: A, B, C, D, E, F, FB, FC, FD, FE. ( ) 00 05 00 05 05 12 02 0 05 01 12 [4 4] 01 05 12 01 05 12 05 12 02 01 05 12 01 05 12 01 05 12 05 12 02 05 02 12 12 00 05 01 05 12 01 05 12 05 12 00 01 10 11 4 0 0 4 2 4 2 0 2 2 2 4 2 0 2 2 2 4 4 4 2 2 2 4 m r r r r r r r r r r П r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r r = × + + + = + + + + + + + + + + + + + + + + + + + 1 02 00 01 10 11 m X m X +                                      Recovery flow model:

19 ( 0, 193, 34 ) j = 2 a 3 3 3 b c h 3 g 3 f3 e 3 d 3 a = 0 3 b = 1 3 c = 0 3 d = 0 3 e = 0 3 f = 1 3 g = 0 3 h = 0 3 N = 1 2 + 1 2 = 34 3 1 j = 3 j = 1 j = 3 5 a 2 2 2 b c h 2 g 2 f2 e 2 d 2 j = 2 a = 1 2 b = 0 2 c = 0 2 d = 0 2 e = 0 2 f = 0 2 g = 1 2 h = 1 2 N = 1 2 + 1 2 + 1 2 = 193 2 0 6 7 a = 0 1 b = 0 1 c = 0 1 d = 0 1 e = 0 1 f = 0 1 g = 0 1 h = 0 1 N = 0 1 j = 1 a 1 1 1 b c h 1 g f1 e 1 d 1 1 Problem formalization where j is the layer number 0 1 2 3 4 5 6 7 2 2 2 2 2 2 2 2 j j j j j j j j j N a b c d e f g h = ⋅ + ⋅ + ⋅ + ⋅ + ⋅ + ⋅ + ⋅ + ⋅

20 a 1 h 1 z 1 g 1 z 2 z 4 h 3 g 3 z 4 a 1 b 1 c 1 h 1 z 1 d 1 g 1 f 1 e 1 z 2 z 4 z 2 z 1 X Y t 6 t 5 t 4 t 3 t 2 t 1 a 1 h 1 z 1 g 1 a 1 z 2 b 1 c 1 z 2 h 1 g 1 z 4 z 2 a 1 1 1 z 1 c 2 z 2 a 2 a 3 b 3 c 3 3 z 4 h 3 b c 1 1 b c h 1 g 1 g 2 g 3 f 1 e 1 d 1 a 1 1 1 z 1 c 2 z 2 a 2 a 3 b 3 c 3 z 4 h 3 b c h 1 g 1 g 2 g 3 f 1 e 1 d 1 z 2 b 3 c 2 a 3 c 3 T Image plane Z´ a 2 a 3 g 2 a 2 a 3 b 1 c 1 a 2 a 1 b 1 c 1 d 1 e 1 a 2 a 3 a 1 a 1 a 2 a 3 a 1 a 2 a 3 g 2 g 3 3D Neighborhood Z 3D Neighborhood Z ( 0, 193, 34 ) Possible elements ( 0, 0, 0 ) Problem formalization

21 X Y t 3 t 1 T Image plane Z´ (1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4) t 2 Image projection onto the Z' plane at t1 S (t) 5 S (t) 7 S (t) 8 S (t) 6 S (t) 9 S (t) 2 S (t) 1 S (t) 3 S (t) 4 Image projection onto the Z' plane at t2 Image projection onto the Z' plane at t2 where K S = number of subsets. ( ) ( ) ( ) ( ) { } 1 2 , , : , ,....., K I x y t S S t S t S t = Problem formalization

22 K connected, piecewise-linear subsets on the image plane Z´ (t) formed on the basis of a random vector field. The boundaries of piecewise-linear subsets are built on the basis of point recovery: where K is the number of subsets for a certain moment in time t: - random, scalar, piecewise- constant function ( ) ( ) ( ) ( ) { } 1 2 , , : , ,....., K I x y t S S t S t S t = ( ) K S t = ( ) ( ) ( ) ( ) { } 1 2 , ,....., K S t S t S t S t = { } { } 1 1 2 2 1 1 1 1 , ,.... 0, , m m m m m m m m m m v m P X n X n X n X n P X n X n k n L − − − − − − = = = = = = = ∀ ≥ ∈ – discrete random variable with discrete set of state space m X 1 2 n – denotes the discrete moment of the iteration step Problem formalization

23 3 4 the resulting set of piecewise-linear subsets on the image plane must be coherent and meet the condition: the following homogeneity condition for a single subset holds: ( ) ( ) i LP S t true i = ∀ ( ) ( ) 1 K i i Z t S t = ′ = ∪ each point on the image plane belongs to a certain area ( ) ( ) ( ) ( ) { } 1 2 , ,....., K S t S t S t S t = 5 ( ) ( ) { } i j S t S t i j = ∅ ∀ ≠ ∩ Problem formalization

24 Perspectives and further research to develop quality criteria to evaluate algorithms based on Optical Flow. to create criteria based on binary classification. to create a generalized methodology for testing algorithms.

25 Summary / Conclusion • eliminating the subjectivity of evaluating the efficiency of image processing algorithms • introducing a probabilistic factor into the analysis of the selected image processing algorithms • creating prerequisites (foundation) for factor analysis aimed to identify the patterns of processing specific formative elements Using a homogeneous, single-level point-to-point renewal flow to obtain a sequence of random optical images will allow:

26 Publications in Russian and foreign journals 1. Algorithm of one-level Markovian fields construction, V. Laevski // Bulletin of the Tomsk Polytechnic University (ISSN 1684-8519), 2006. № 8. Т 309, pp. 32-36 2. The method of suboptimal estimation of operating the algorithms for obtaining pattern outline drawing, V. Laevski // Bulletin of the Tomsk Polytechnic University (ISSN 1684-8519), 2009. № 5. Vol. 314, pp. 126-131 3. The Technique of Evaluating Algorithms Functioning Quality for Obtaining Edge Image of Objects in Patterns Approximated by Homogeneous Markov Random Fields, D. Dubinin, V. Laevski, A. Kochegurov // Bulletin of the Tomsk Polytechnic University (ISSN 1684-8519), 2010. № 5. Vol. 317, pp. 130–134 4. Modelling Technique for the Random Brightness Fields Approximated by Uniform Single- Level Markovian Chains, D. Dubinin, A. Kochegurov, V. Geringer // Problems of informatics (ISSN: 2073-0667), Novosibirsk: ICM&MG SB RAS publishers, 2011. № 4(12). pp. 35–40

27 5. Modellbildung und stochastische Computersimulation: Objektive Beurteilung von Edge- Detektor-Algorithmen auf Grund der Ergebnisse der Objektextraktion in 2D- stochastischen Helligkeitsfeldern, V. Laevski, V. Denisov, D. Dubinin // LAP- LAMBERT Academic Publishing, Saarbrücken, 2011, ISBN: 978-3-8433-2569-1, 123 S 6. Eine Methode zur Erzeugung stochastischer Helligkeitsfelder durch homogene, einstufige Markoff-Ketten, V. Geringer, D. Dubinin, A. Kochegurov // Oldenbourg Wissenschaftsverlag, Technisches Messen (ISSN 0171-8096), 2012, №5, s. 271-276 7. On the Statistics of Space-Time Signals Created by a Two-Dimensional Markov Renewal Process, V. Geringer, D. Dubinin, A. Kochegurov // Bulletin of the Tomsk Polytechnic University (ISSN 1684-8519), Tomsk, 2012, № 5, Vol. 321, pp. 194-198 Publications in Russian and foreign journals

28 8. Bundled Software for Simulation Modeling, V. Geringer (В.Е. Лаевский), D. Dubinin, A. Kochegurov, K. Reif // In : Proceedings of the International Symposium on Signals, Circuits and Systems (ISSCS 2013), Romania, Iasi: ISSCS Press, 2013. ISBN: 978-1-4673-6141-5, IEEE Catalog Number: CFP13816-CDR, pp. 1-4. 9. Ein stochastischer Algorithmus zur Bildgenerierung durch einen zweidimensionalen Markoff-Erneuerungsprozess, D. Dubinin, V. Geringer, A. Kochegurov, K. Reif // Oldenbourg Wissenschaftsverlag, In: at- Automatisierungstechnik (ISSN: 0178-2312), Band 62 (Heft 1), 2014. S. 57-64 10. On the Statistics of Space-Time Signals Created by a Two-Dimensional Markov Renewal Process, V. Geringer, D. Dubinin, A. Kochegurov // Bulletin of the Tomsk Polytechnic University (ISSN 1684-8519), Tomsk, 2012, № 5, Vol. 321, pp. 194-198 Publications in Russian and foreign journals

29 11. The Results of the Investigation of the Integrated Performance Evaluation Method of Edge Detection Based on the two-dimensional Renewal Stream, V. Denisov, D. Dubinin, A. Kochegurov, V. Geringer // Vestnik of Siberian State Aerospace University named after academician M.F. Reshetnev (ISSN 1816-9724), 2015. № 2. Vol. 16, pp. 300-309. 12. The Results of a Complex Analysis of the Modified Pratt-Yaskorskiy Performance Metrics Based on the Two-Dimensional Markov-Renewal-Process, Geringer V., Dubinin D., Kochegurov A. // Springer: Lecture Notes in Computer Science. – 2016. – v. 9875. – pp. 187– 196. 13. Estimation of the Efficiency of Contour Detectors on the Basis of a Point Recovery Flow, A.I. Kochegurov, D. Dubinin, V. Geringer, K. Reif // Bulletin of the Tomsk Polytechnic University (ISSN 1684-8519), Tomsk, 2019. № 3. Vol. 330, pp. 204–216. Publications in Russian and foreign journals

30 Thank You! Follow TMPA on Facebook TMPA-2021 Conference