OpenTalks.AI - Сергей Терехов, Tensor Decompositions in Statistical Estimation and Modelling

Transcript

1. Tensor Decompositions in Statistical Estimation and Modelling Tensor Decompositions in Statistical Estimation and Modelling Serge A. Terekhov, Svyaznoy Math Methods in Control, Head
2. Tensor Decompositions in Statistical Estimation and Modelling In brief Multi-way data Statistical estimation and tensor decompositions Econometrics and tensors: Sales potential in large retail network HR and tensors: Most effective store for each salesman From tensors to Tensor Train Neural Networks (TTNN) Industrial AI and TTNN: key part between deep sensors and deep actuators
3. Tensor Decompositions in Statistical Estimation and Modelling Multi-way business data Traditional form: “object-property” tables (99% of todays’ research and applications). Multi-modality: “object - context of type A - context of type A - . . . - time - . . . - control of type U - control of type V - . . . ” Measurements - events counts or utility level (economic effect, max joy, total reward, and alike). 6W: Who does What for What by What Where and When. x - y - z - t - f(x, y, z, t). State 1 - State 2 - Year - Treaty type - Scale - Effect.
4. Tensor Decompositions in Statistical Estimation and Modelling Multi-way data is everywhere Sandia Research Magazine (http://www.sandia.gov/news/publications/ research_magazine/index.html)
5. Tensor Decompositions in Statistical Estimation and Modelling Data and parameters as tensors and statistical estimation Data: (very) sparse random measurements, organized in multi-way arrays (tensors) Estimation problem: Optimal values of statistical distribution parameters (e.g. β in censored zero-inflate Poisson). Common base for supervised, unsupervised, and reinforcement learning. Goal: Optimal statistical decisions based on comparison of alternatives, in time.
6. Tensor Decompositions in Statistical Estimation and Modelling Econometrics: Sales potential Oserved sales only for certain SKU, in subset of stores, and in particular days. Very sparse. Observations are subject of censoring by items availability (censored by control process). Statistical mixture of censored and non-censored Poisson counts. Result: Estimation of sales potential for all SKU, all stores, and time. Items redistribution and logistics algorithm. More tensor dimensions: Marketing actions, prices, motivation, competitors.
7. Tensor Decompositions in Statistical Estimation and Modelling HR: Salesmen-in-the-store potential Different sales people performance only partially measured in subset of stores. People vary in their skills, education, team context, etc. Who is better where? Result: Estimation of personal salesman potential in various working conditions. Recommendations to HR, better planning (and hunting).
8. Tensor Decompositions in Statistical Estimation and Modelling What’s the point? Data is sparse, random, noisy, and inclined by (our) control. Data cannot be used in decision making per se. Statistical Estimates are stable and complete (cover all contexts). Statistical models also cover unobserved cases.
9. Tensor Decompositions in Statistical Estimation and Modelling How? Tensor decompositions. Generalization of SVD (NMF) from 2D-matrices to multi-way tensors. Maximum likelihood with stochastic gradient descent (ADAM+) T.Kolda Math to start: T.G. Kolda, B. W. Bader. “Tensor Decompositions and Applications.” SIAM Review, 51(3), pp.455-500.
10. Tensor Decompositions in Statistical Estimation and Modelling
11. Tensor Decompositions in Statistical Estimation and Modelling Go Deep: Tensor train Decomposition in the form of diagonal matrices. Matrices can be arbitrary: Tensor Train (Ivan Oseledets, Institute of Numerical Mathematics, Russia, 2011)
12. Tensor Decompositions in Statistical Estimation and Modelling Almost done: Matrix compression and deep learning A. Novikov, A, Rodomanov, A. Osokin, Dmitry Vetrov, ICML 2014
13. Tensor Decompositions in Statistical Estimation and Modelling Critical chord: Tensor Train Neural Network Neural layers instead of matrix multiplications! Tensor Train Neural Network (TTNN, S.Terekhov, Neuroinformatics 2017, MIFI, Moscow) Between sensors and actuators: Learning of large amount of small neural networks (hyper-graph, K. Anokhin) instead of huge deep nets.
14. Tensor Decompositions in Statistical Estimation and Modelling Practical Industrial AI Key element is TTNN associative mediation (“mind”) between sensor systems and control actions. Formula: DLactuators = f(DLsensors + TTNNcore + ES/logic · Ψ) Author forecast: Exploding interest to expert systems and "traditional"logical/Bayesian AI for Ψ
15. Tensor Decompositions in Statistical Estimation and Modelling Thanks! [1] Ivan Oseledets, Eugene Tyrtyshnikov. TT-cross approximation for multidimensional arrays. Linear Algebra and its Applications 432 (2010) 70–88. URL: www.mat.uniroma2.it/~tvmsscho/papers/Tyrtyshnikov5.pdf [2] Evrim Acar, Daniel M. Dunlavy, Tamara G. Kolda, Morten Mørup. Scalable Tensor Factorizations with Missing Data. 2010. URL: http://www.cs.sandia.gov/~dmdunla/publications/AcDuKoMo10.pdf [3] Tensor Decompositions: Applications and Efficient Algorithms at SIAM CSE17. URL: http://perso.ens-lyon.fr/bora.ucar/tensors-cse17/index.html [4] К.В.Анохин. Мозг, Творчество, Сознание. Конф. “Философия творчества”“, 8-9 апреля 2015, Институт философии РАН, Москва. URL: https://iphras.ru/uplfile/root/biblio/Phil_tvorch_2015.pdf [5] IV Oseledets. Tensor-Train Decomposition. SIAM Journal on Scientific Computing, 2011, 33, 2295-2317. URL: http://epubs.siam.org/doi/abs/10.1137/090752286