Optimal and Diffusion Transports in Machine Learning
Slides for a general introduction to the use of Optimal Transport methods in learning, with an emphasis on diffusion models, flow matching, training 2 layers neural networks and deep transformers.
d'un chercheur CNRS qui présente l’IA générative devant un large auditoire. Le professeur Jacques Leclair, célèbre directeur de l’Institut de Mathématiques du CNRS, monte sur scène avec l’enthousiasme d’un enfant qui vient de découvrir que son yaourt préféré a un nouveau goût. Devant lui, un large auditoire aux regards curieux. “Aujourd’hui, mes amis,” commence-t-il avec un clin d’œil complice, “je vais vous prouver que les mathématiques peuvent être aussi cool qu’un chaton jouant avec une pelote de laine… et je parle bien sûr de l’IA générative!”. Représente un chercheur en mathématiques en train de présenter l’IA générative devant un large auditoire. DALL·E 2
d'un chercheur CNRS qui présente l’IA générative devant un large auditoire. Le professeur Jacques Leclair, célèbre directeur de l’Institut de Mathématiques du CNRS, monte sur scène avec l’enthousiasme d’un enfant qui vient de découvrir que son yaourt préféré a un nouveau goût. Devant lui, un large auditoire aux regards curieux. “Aujourd’hui, mes amis,” commence-t-il avec un clin d’œil complice, “je vais vous prouver que les mathématiques peuvent être aussi cool qu’un chaton jouant avec une pelote de laine… et je parle bien sûr de l’IA générative!”. Représente un chercheur en mathématiques en train de présenter l’IA générative devant un large auditoire. DALL·E 2 Le professeur Jacques Leclair, célèbre directeur de l’Institut de Mathématiques du CNRS monte Pre-training: next token prediction. Generation: auto-regressive. Pre-training: denoising. Generation: dynamic transport.
α)2 + f(α) Implicit Euler step: Wasserstein Gradient Flows Felix Otto David Kinderlehrer Richard Jordan αt αt+τ αt+2τ Optimization over distributions: min α f(α)
e⟨Qxi ,Kxℓ ⟩ Vxj Transformers and attention mechanism … + <latexit sha1_base64="aTL0Qvb1dLhAur6wfZM9PGylzLY=">AAA9iHictVttcxu3EYbTt1h9c9qPnelcq7iTdFyNpHqadjKaifViSbFiySYlOwltDV9O1NknHs0jJdkM/0e/tn+kv6P/oP3Uv9DdBXDAkbhbQHV1IwkH4nl2sQcsdoFjZ5gm+Xh19Z+3Pvje93/wwx99eHvpxz/56c9+fuejX5zk2WTUjY+7WZqNnnfaeZwmg/h4nIzT+PlwFLcvOmn8rPN6Cz9/dhmP8iQbNMdvh/GLi3Z/kJwl3fYYql62iGE6inuz69O10zvLqyur9BMtFtZUYVmon6Pso1+vi5boiUx0xURciFgMxBjKqWiLHK5vxZpYFUOoeyGmUDeCUkKfx2ImlgA7gVYxtGhD7Wv424e7b1XtAO6RMyd0F6Sk8DsCZCTuAiaDdiMoo7SIPp8QM9ZWcU+JE3V7C/87iusCasfiHGo5nG7pi8O+jMWZ+DP1IYE+DakGe9dVLBOyCmoeWb0aA8MQ6rDcg89HUO4SUts5IkxOfUfbtunzf1FLrMX7rmo7Ef8mLe/CFYmG6n1WMLTFJfFH9DQn8JnUJwXJfWCIVR+xdEW2vqDeD6D9FOofwzWjkrZJB64p1c5qkVtwuZBbLHIXLhdyl0UewOVCHrDII7hcyCOFROyIbO7GN+By4Rus5CdwuZBPWORTuFzIpyzyBC4X8oRFfgOXC/kNi3wIlwv5kEU+gsuFfMQim3C5kE0WeQyXC3nMInfgciF3FLJ6po7gyognYWblAyiXZaCnSKHmAavfJnlHF3bTY053K7D8rN6G/27stodN4wrsjse4O6vA8iNvF3ykG8v7oj1aTVzYPRa7DyPAjd1nsV+KVxXYLz1m2usKLD/XDqCdG8t736/gzo39isU+hpIby69Rh1Djxh56rBjDCuwRi30i3lRgfbz+qALL+/0G+BU3ll+nmtDejfXxppMKLO9PTyCCcWP51eoZ1Lqxz1jsc3FdgX3OYr8G7+7Gfu2xwr6rwOo1dolWkD7FIzHM2Dq2djErsTQEtjYjPy3WlpRi4w7Uc5h+gekT5oJF7BaIXU/EQYE48NYrL/xoTvEuL6VRIBqeiE6xNmFpzLbvFe2xlHogtgvE9hyiLiLFZ637cknRha7hkONi5cKST5+ywn9jKVbjod7zasRhCSHH9jmN/HuULWEGhZaqYzsv1niJjOi+DnFF2ZvupZbB48aFV7BR1yyq40B1WNRbB+oti5o4UBMWdelAXbIoM/NtXMtjBBj747OY0p0cATJGrr4iiAoewKqzB3M0gvFzBFHgU6o5hP8Nyr25q04zzOZxncRdjhclTzyC0lQsQ73JCrcpv05phsWgmWx5qHJ8vMO9jamac9ILz4qVPCp2TPx5EtKnX/BgtBjRfArjeUQ1M4ruZCkMv1fMe10Kw++QxWcUxctSGH6stB/fQPemwjZvgG3AbBoq65tyKIfcf5EcurxEqy56XHyqF2rMIN91IP++ejL7N3guW1SS9jHlMI7c6l9e6l8Ih7Fzbtk5jAWjJxn16lIU3JOByntNOVSHjFbRgdLD3IU+GWzTU09Gl8M4jiDi2qKce2qVQ0fvsOiNKYdxnAi57zmjSF6Xwzj6dC/tYcphHLjb0lZ5vimHena0gMydTTnUqw9oFxj3gOSYlzUmKhpRnDRRbAnFB/W7NXbMv7iO4Z7NyyJHqGcysW01T6dYy+o10vFCDF5tHKgHxhcTKwYrc0zFOptfSR3GpfV9kces8Wj5A7BiBLNfngFwe+YpaKj3JNB7p8C4xmZd5Z5p3DqLw1FyNodqqdoxGy0auXLXqFx3SrVcXmZ6a+zYIn+d09gbUkx4QJbl7HBQ+YSrGDkLHZQsxPOF2O6dmq9l66+yuOEcYliMtC6dCMmTtPo81WX1hmXju+qUZwyXPPMx4xd3m8+Ut8GcJyNfhLrUybTb6X0kuw7X1XvC7HHLzyJ6ouivLslrJHQilbNZqN4tltH4lO4N9zGdyaEMydGF5xgplqGQp2a4i4776RF5VNvfcrLRXnqHTpZz8rraH9ej+xa670CH5zhbsGI8hlITcoZjuGt6ZDlLha0ysvhI/KE4Hc3oCdZn9GnJQ2oO6W/ikoesy7LPSyxXgMbRILN0f455Ho1vLTDxWb9LH5O7lj3/XTq51efbbRrj1aO5eiemR1LXSWpEs0ae6sq7eQlSg6nzk3WKX+t7ifJCJKIP5aS+tCRLuwzoxD+mDHZIkXFKs42bHeXW9v7U/Cda0pHQZ+d4mp2Rh4zI/0WwPmU0JiP6td8d0Cfo0iOk5CN9/E5SRDeuWCdhx5iJ4xIh32ow4y0mXzYh+ZrXnl05jUWZMch1YDY3trVNDigWjEnqSHl3M7frVx9Emvck7FEiGc1Y+YTkf0p/9a8eJ8sLIwItjE8gV77O9TwyylnQRm1a5et9kG5ra/lxocNLpbVZ/4xOH5c026aMC/XB1boHkrt0L2XhKBmR3vlCG7mO1u3mIvNwzo7Y2zPK4qXf76sVGPW+R6vkMs25Fo2SPoyCcZFF6LbcLvK83HpZZXY/7vz/wm5sXbYaMkbC7OBKC3H7+zFla7aWKYxqOX5f02xyW30016pezoDG4oU1l7+D2t/AX623vvfj6ZS8wiaNAclg7oxFZE200MJP1mZJlh6ZmsvcG3lmTOpWds1N8mvp3UyOfRnMckSj5lrtWujyTTheWRyvPG3YpLNGY0Vdrz3RKZtbNNVppa+8EGnNAOYJy8xHZBqVeGhp51J+rD2Wlc/xNeody7XKcrVhttqnAfac90G65/r87P6uWN0j8ZBimy5FYDJ/6dEsTSjm0rX1mZpkQMn3lX+1Z3+LalB6hzwoMsv3OHHGyFOnLl2zQtPfqZUtIz9vPIJ+b+lKtdE+tkXlPy4gL2hO5DQvNeI+tYiV/rYe0ZxHWrFijoh2/tsUU8m4oz5ntlubZxKV4gmTb8pZZWTJTGFA9ud23vYXstd9K3+NKCecqOi6A1zhTxgZJEbvJLgjy5yeEK5y8iRBRrQd8p+Lfkqe4g0sjVZI66nY8PAxMus1Y90eW7rHum+/h5ZodfPUXS14eam3RE7eTU702rSqXagYdTp3fzOutlrlyvd1dpjMyTX2mFAbO7MwWV4Z0xKfe0uRGoVJkRgfKWG9CNE/TPMQneXplC+zbq2ZyzsN0secU77EvQeKCFd094kzmvuU6Udnga9DWJtN1nBMuBuXqf0B29PirtTthXVI1t6uXY1SayWqWik0u71aGP8tPWRM3i8V3J6NbG3r3iplKfwujGToCvlGb1V+aHN+Dhf+jYQrO9QSffYOGxDfPhBbYuc9vA3xRpXljmZENegLenO5d1v1s9yi3kZvLHab30eCv4wEbM1pn9BKGqq7ZOY1t9n9+a/IC4xEzGpvWob3wZbC92RRUkh/EvJsfG8Sob+LE9oXLcGnJ2Up/nLkuQbXizOhv9MU1gfNzvegLCFEhn6Pwe+Zm9bhsmxJ9fZalOIrQ64C+sRF4/DkrzpXMe18PNTIeiLvXwJ6h7Madr1a/K/90HKMpHBZvtJy+q7ZK4+nLtvFakcW4+HwOWOk+Yzmaon+MrOidyZacsuTcV8U9KQyqzfvnx/jUTMGtKypkPugvHYSb48io68vC54LuHTIxH/EP27x30Z4U3BU6RHCpM8pqtl0C55Nf+PS1Tv9mY9OhqdKpzKbySMa9EbsltgXD+F3q4gAQ98Old+llP8R6/7+bA9qz8h76F10uXPQorqYdj/MKVqP7tUe4+md5bX5byEvFk7WV9b+tHL/yf3lLzbVN5Q/FL8Sv4W8ZE18Jr4Qe9DfYzpX+Kv4m/j7xtLG6sZnG3+RTT+4pTC/FKWfjc3/Ajzs2dE=</latexit> x1 <latexit sha1_base64="7Z/IumRXp79HdyogVfnC2DA+LeM=">AAA9iHictVttcxu3EYbTt1h9c9qPnelcq7iTdFyNpHqadjKaifViSbFiySYlOwltDV9O1NknHs0jJdkM/0e/tn+kv6P/oP3Uv9DdBXDAkbhbQHV1IwkH4nl2sQcsdoFjZ5gm+Xh19Z+3Pvje93/wwx99eHvpxz/56c9+fuejX5zk2WTUjY+7WZqNnnfaeZwmg/h4nIzT+PlwFLcvOmn8rPN6Cz9/dhmP8iQbNMdvh/GLi3Z/kJwl3fYYql62iGE6inuz69P10zvLqyur9BMtFtZUYVmon6Pso1+vi5boiUx0xURciFgMxBjKqWiLHK5vxZpYFUOoeyGmUDeCUkKfx2ImlgA7gVYxtGhD7Wv424e7b1XtAO6RMyd0F6Sk8DsCZCTuAiaDdiMoo7SIPp8QM9ZWcU+JE3V7C/87iusCasfiHGo5nG7pi8O+jMWZ+DP1IYE+DakGe9dVLBOyCmoeWb0aA8MQ6rDcg89HUO4SUts5IkxOfUfbtunzf1FLrMX7rmo7Ef8mLe/CFYmG6n1WMLTFJfFH9DQn8JnUJwXJfWCIVR+xdEW2vqDeD6D9FOofwzWjkrZJB64p1c5qkVtwuZBbLHIXLhdyl0UewOVCHrDII7hcyCOFROyIbO7GN+By4Rus5CdwuZBPWORTuFzIpyzyBC4X8oRFfgOXC/kNi3wIlwv5kEU+gsuFfMQim3C5kE0WeQyXC3nMInfgciF3FLJ6po7gyognYWblAyiXZaCnSKHmAavfJnlHF3bTY053K7D8rN6G/27stodN4wrsjse4O6vA8iNvF3ykG8v7oj1aTVzYPRa7DyPAjd1nsV+KVxXYLz1m2usKLD/XDqCdG8t736/gzo39isU+hpIby69Rh1Djxh56rBjDCuwRi30i3lRgfbz+qALL+/0G+BU3ll+nmtDejfXxppMKLO9PTyCCcWP51eoZ1Lqxz1jsc3FdgX3OYr8G7+7Gfu2xwr6rwOo1dolWkD7FIzHM2Dq2djErsTQEtjYjPy3WlpRi4w7Uc5h+gekT5oJF7BaIXU/EQYE48NYrL/xoTvEuL6VRIBqeiE6xNmFpzLbvFe2xlHogtgvE9hyiLiLFZ637cknRha7hkONi5cKST5+ywn9jKVbjod7zasRhCSHH9jmN/HuULWEGhZaqYzsv1niJjOi+DnFF2ZvupZbB48aFV7BR1yyq40B1WNRbB+oti5o4UBMWdelAXbIoM/NtXMtjBBj747OY0p0cATJGrr4iiAoewKqzB3M0gvFzBFHgU6o5hP8Nyr25q04zzOZxncRdjhclTzyC0lQsQ73JCrcpv05phsWgmWx5qHJ8vMO9jamac9ILz4qVPCp2TPx5EtKnX/BgtBjRfArjeUQ1M4ruZCkMv1fMe10Kw++QxWcUxctSGH6stB/fQPemwjZvgG3AbBoq65tyKIfcf5EcurxEqy56XHyqF2rMIN91IP++ejL7N3guW1SS9jHlMI7c6l9e6l8Ih7Fzbtk5jAWjJxn16lIU3JOByntNOVSHjFbRgdLD3IU+GWzTU09Gl8M4jiDi2qKce2qVQ0fvsOiNKYdxnAi57zmjSF6Xwzj6dC/tYcphHLjb0lZ5vimHena0gMydTTnUqw9oFxj3gOSYlzUmKhpRnDRRbAnFB/W7NXbMv7iO4Z7NyyJHqGcysW01T6dYy+o10vFCDF5tHKgHxhcTKwYrc0zFOptfSR3GpfV9kces8Wj5A7BiBLNfngFwe+YpaKj3JNB7p8C4xmZd5Z5p3DqLw1FyNodqqdoxGy0auXLXqFx3SrVcXmZ6a+zYIn+d09gbUkx4QJbl7HBQ+YSrGDkLHZQsxPOF2O6dmq9l66+yuOEcYliMtC6dCMmTtPo81WX1hmXju+qUZwyXPPMx4xd3m8+Ut8GcJyNfhLrUybTb6X0kuw7X1XvC7HHLzyJ6ouivLslrJHQilbNZqN4tltH4lO4N9zGdyaEMydGF5xgplqGQp2a4i4776RF5VNvfcrLRXnqHTpZz8rraH9ej+xa670CH5zhbsGI8hlITcoZjuGt6ZDlLha0ysvhI/KE4Hc3oCdZn9GnJQ2oO6W/ikoesy7LPSyxXgMbRILN0f455Ho1vLTDxWb9LH5O7lj3/XTq51efbbRrj1aO5eiemR1LXSWpEs0ae6sq7eQlSg6nzk3WKX+t7ifJCJKIP5aS+tCRLuwzoxD+mDHZIkXFKs42bHeXW9v7U/Cda0pHQZ+d4mp2Rh4zI/0WwPmU0JiP6td8d0Cfo0iOk5CN9/E5SRDeuWCdhx5iJ4xIh32ow4y0mXzYh+ZrXnl05jUWZMch1YDY3trVNDigWjEnqSHl3M7frVx9Emvck7FEiGc1Y+YTkf0p/9a8eJ8sLIwItjE8gV77O9TwyylnQRm1a5et9kG5ra/lxocNLpbVZ/4xOH5c026aMC/XB1boHkrt0L2XhKBmR3vlCG7mO1u3mIvNwzo7Y2zPK4qXf76sVGPW+R6vkMs25Fo2SPoyCcZFF6LbcLvK83HpZZXY/7vz/wm5sXbYaMkbC7OBKC3H7+zFla7aWKYxqOX5f02xyW30016pezoDG4oU1l7+D2t/AX623vvfj6ZS8wiaNAclg7oxFZE200MJP1mZJlh6ZmsvcG3lmTOpWds1N8mvp3UyOfRnMckSj5lrtWujyTTheWRyvPG3YpLNGY0Vdrz3RKZtbNNVppa+8EGnNAOYJy8xHZBqVeGhp51J+rD2Wlc/xNeody7XKcrVhttqnAfac90G65/r87P6uWN0j8ZBimy5FYDJ/6dEsTSjm0rX1mZpkQMn3lX+1Z3+LalB6hzwoMsv3OHHGyFOnLl2zQtPfqZUtIz9vPIJ+b+lKtdE+tkXlPy4gL2hO5DQvNeI+tYiV/rYe0ZxHWrFijoh2/tsUU8m4oz5ntlubZxKV4gmTb8pZZWTJTGFA9ud23vYXstd9K3+NKCecqOi6A1zhTxgZJEbvJLgjy5yeEK5y8iRBRrQd8p+Lfkqe4g0sjVZI66nY8PAxMus1Y90eW7rHum+/h5ZodfPUXS14eam3RE7eTU702rSqXagYdTp3fzOutlrlyvd1dpjMyTX2mFAbO7MwWV4Z0xKfe0uRGoVJkRgfKWG9CNE/TPMQneXplC+zbq2ZyzsN0secU77EvQeKCFd094kzmvuU6Udnga9DWJtN1nBMuBuXqf0B29PirtTthXVI1t6uXY1SayWqWik0u71aGP8tPWRM3i8V3J6NbG3r3iplKfwujGToCvlGb1V+aHN+Dhf+jYQrO9QSffYOGxDfPhBbYuc9vA3xRpXljmZENegLenO5d1v1s9yi3kZvLHab30eCv4wEbM1pn9BKGqq7ZOY1t9n9+a/IC4xEzGpvWob3wZbC92RRUkh/EvJsfG8Sob+LE9oXLcGnJ2Up/nLkuQbXizOhv9MU1gfNzvegLCFEhn6Pwe+Zm9bhsmxJ9fZalOIrQ64C+sRF4/DkrzpXMe18PNTIeiLvXwJ6h7Madr1a/K/90HKMpHBZvtJy+q7ZK4+nLtvFakcW4+HwOWOk+Yzmaon+MrOidyZacsuTcV8U9KQyqzfvnx/jUTMGtKypkPugvHYSb48io68vC54LuHTIxH/EP27x30Z4U3BU6RHCpM8pqtl0C55Nf+PS1Tv9mY9OhqdKpzKbySMa9EbsltgXD+F3q4gAQ98Old+llP8R6/7+bA9qz8h76F10uXPQorqYdj/MKVqP7tUe4+md5bX5byEvFk7WV9b+tHL/yf3lLzbVN5Q/FL8Sv4W8ZE18Jr4Qe9DfYzpX+Kv4m/j7xtLG6sZnG3+RTT+4pTC/FKWfjc3/AnoD2dI=</latexit> x2 <latexit sha1_base64="Fgw+vWgPriclgLxpVoHDEicBDLw=">AAA9h3ictVv9chu3EYfTr9j9cto/O9O5VnYnybiqpHrSzGQ0E1uSJcWKJZuU7CS0Nfw40WcfeTSPlGQzfI7+2z5Jn6Nv0P7VV+juAjjgSNwtoLrCSMKB+O0uFsB+AMfOKE3yydraP6998IMf/ujHP/nw+o2f/uznv/jlzY9+dZJn03E3Pu5maTZ+1mnncZoM4+NJMknjZ6Nx3B500vhp5/UWfv70PB7nSTZsTt6O4ueDdn+YnCXd9gSant9qzS5Pk9b8dJbMb53eXFlbXaOfaLmyriorQv0cZR/9dkO0RE9koiumYiBiMRQTqKeiLXIo34l1sSZG0PZczKBtDLWEPo/FXNwA7BR6xdCjDa2v4W8fnr5TrUN4Rpo5obvAJYXfMSAjcRswGfQbQx25RfT5lChjaxXtGdFE2d7C/46iNYDWiXgJrRxO9/TF4Vgm4kx8TmNIYEwjasHRdRWVKWkFJY+sUU2AwgjasN6Dz8dQ7xJS6zkiTE5jR9226fN/UU9sxeeu6jsV/yYpb0OJREONPisotMU50Y9oNqfwmZQnBc59oBCrMWLtgnQ9oNEPof8M2h9BmVNN66QDZUat81rkFhQXcotF7kJxIXdZ5AEUF/KARR5BcSGPFBKxY9K5G9+A4sI3WM6PobiQj1nkEygu5BMWeQLFhTxhkd9CcSG/ZZEPoLiQD1jkQygu5EMW2YTiQjZZ5DEUF/KYRe5AcSF3FLJ6p46hZEQnYXblPaiXeaClSKHlHivffbKOLux9jz3drcDyu3ob/rux2x46jSuwOx7r7qwCy6+8XbCRbixvi/bIm7iweyx2H1aAG7vPYr8SryqwX3nstNcVWH6vHUA/N5a3vl/Dkxv7NYt9BDU3lvdRh9Dixh56eIxRBfaIxT4WbyqwPlZ/XIHl7X4D7Ioby/upJvR3Y32s6bQCy9vTE4hg3FjeWz2FVjf2KYt9Ji4rsM9Y7Ddg3d3Ybzw87LsKrPaxN8iD9CkeiWHH1lFrF7sSayOg1mb4p4VvSSk27kA7h+kXmD5hBixit0DseiIOCsSBt1x5YUdzind5Lo0C0fBEdArfhLUJ279X9Mda6oHYLhDbC4i6iBTnWo/lnKIL3cIhJ4XnwprPmLLCfmMtVuuh3vJqxGEJIdf2S1r5dyhbwgwKNVVH7WXh4yUyouc6xAVlb3qUmgePmxRWwUZdsqiOA9VhUW8dqLcsaupATVnUuQN1zqLMzrdxLY8VYPSPczGjJ7kCZIxcXSKICu6B19mDPRrB+jmCKPAJtRzC/wbl3lypkwyzefSTeMrxvGSJx1CbiRVoN1nhNuXXKe2wGCSTPQ9Vjo9PeLYxU3tOWuF54cmj4sTEn05C8vQLOhgtRrSfwug8pJY5RXeyFobfK/a9roXhd0jjc4riZS0MP1HST64ge1Nhm1fANmA3jZT2TT2Uhjx/kTR0/QZ5XbS4OKsDtWaQ3mUg/X01M/tXmJctqkn9mHoYjdwaX14aXwgNo+fc0nMYFYyeZNSra1HwSIYq7zX1UBky8qJDJYd5Cp0Z7NNTM6PrYTSOIOLaopx7ZtVDV++oGI2ph9E4EfLcc06RvK6H0ejTs9SHqYfRwNOWtsrzTT3UsqMGZO5s6qFWfUinwHgGJNe8bDFR0ZjipKmillB8UH9aY8f8y34Mz2xeFDlCPSUT21bT6RS+rF4iHS/EYNUmgXJgfDG1YrAyjZnYYPMrKcOk5N+X6Rgfj5o/AC1GsPvlHQB3Zp6ChPpMAq13ChTX2ayrPDKN22BxuErOFlAt1Tpho0XDV54aldtOqZXLy8xojR5bZK9zWnsjigkPSLOcHg4qZ7iKIqehg5KGeHohunun9mtZ+2ssbrSAGBUrrUs3QvImrT5PdWm9Yen4trrlmUCRdz5m/eJp85myNpjzZGSLUJY6nnY/fY5kt6FfvSPMGbf8LKIZRXt1TlYjoRupnM1C9WmxjMZn9GxoH9OdHPKQNLowj5GiMhLy1gxP0fE8PSKLattbjjfqS5/QyXpOVlfb43p030L3HejwHGcLPMYjqDUhZziGp6ZHlnOj0FVGGh+LPxa3oxnNYH1Gn5YspKYh7U1cspB1WfbLEpULQONqkFm6P41FOhrfWqLEZ/0ueUzuWrb8t+nmVt9vt2mNV6/m6pOYHnHdIK4R7Rp5qyufFjlICWbOTzYofq0fJfIL4Yg2lOP6wuIs9TKkG/+YMtgRRcYp7TZud5R72+dTi59oTkdC353jbXZGFjIi+xeBf8poTUb0a787oG/QpUVIyUb62J2kiG5csU7CrjETxyVCvtVg1ltMtmxK/DVde3fltBZlxiD9wHxhbWudHFAsGBPXsbLuZm/Xex9Emvck7FUiKZq18jHx/4T+6l+9TlaWVgRqGGcgV7bONR8Z5SyoozZ5+XobpPvaUt4qZHihpDb+z8h0qyTZNmVcKA966x5w7tKz5IWrZExy50t9pB+tO81FyqMFPeJozyiLl3a/rzwwyn2HvOQK7bkWrZI+rIJJkUXovtwp8iLfel5l6n608/8LdaPrstaQYiTMCa7UEHe+H1O2ZkuZwqqW6/c17Sa31scLver5DGktDqy9/D20/g7+arn1sx+dTskq3Kc1ICmYJ6MR2RIt9fDjdb/ES69MTcs8G35mTepedstV8mtp3UyOfR5M5YhWzaU6tdD1q9B4ZdF45anDJt01Gi3qdm2JTtncoqluK335hXBrBlCespT5iEyjEg8p7VzKj2qPpcrn+Br1jqW1xtJqw261bwPsPe+DdO/1xd39feHdI/GAYpsuRWAyf+nRLk0o5tKt9ZmapICc7yr7au/+FrUg9w5ZUKQs3+PEHSNvnbpU5oWkf1CeLSM7byyCfm/pQvXRNrZF9T8vIQe0J3Lalxpxl3rESn5bjmjBIq1aMUdEJ/9tiqlk3FGfM9u9zZxEpXjC5JtyVxleMlMYkv65k7f9pex138pfI8oJpyq67gCt8BlGChKjTxLckWVOM4ReTt4kyIi2Q/Zz2U7JW7yhJdEqST0Tmx42Rma9Zq3ba0uPWI/tU+iJWjez7urB80u9OXL8rnKj1yavNlAx6mzh+Wq02srLlZ/r9DBd4Gv0MaU+dmZhsrwypiW+8OYiJQrjIjE+XMJGESJ/mOQhMsvbKV/KuremXD5pkDbmJeVL3HugiHBFdx87o7lPmHF0luh1CGtTky0cJTyNy9T5gG1p8VTq+pIfkq3Xa71RanmiKk+hqdvewthvaSFjsn6p4M5sZG9b9lYpS+FPYSSFrpBv9FblhzbNL6Dg30i4skPN0efssAHx7T2xJXbew9sQb1RdnmhG1IK2oLeQe7fVOMs96nX0xqJu0/fh4M8jAV1z0ifkSUNll5R5yW3q/vQvyAqMRcxKb3qGj8Hmwo9kmVPIeBKybPxoEqG/ixM6Fs3BZyRlLv585L0GN4ozob/TFDYGTZ0fQZlDCA/9HoPfnJve4bxsTvX6Wubiy0N6AX3jonF481edq5h+PhZqbM3I++eA1uGshrr2Fv/rODQfwymcly+3nL5r9spj1mW/WJ3IYjwcvmcMN5/VXM3Rn2dWjM5ES25+Mu6LgmYqs0bz/uljPGrWgOY1E/IclJdO4u1VZOT1pYL3Ai4ZMvEf8Y9r/LcR3hQ0quQIoaTvKaqp6R48Nf2NS9fo9Gc+Mhk6VTKVqZk8okFvxG6JffEAfreKCDD07VD5XUr5H7Hu78/2oPWMrIc+RZcnBy1qi+n0w9yi9ehZnTGe3lxZX/wW8nLlZGN1/bPVu483Vr68r76h/KH4jfg95CXr4i/iS7EH4z0mTf1V/E38ffP65p82P9v8XHb94JrC/FqUfjbv/Rd5M9kH</latexit> {xi }i Points cloud Positional encoding Token encoding Tokenize Le groupe thématique SMAI- S I G M A ( S i g n a l - I m a g e - Géométrie-Modélisation- Approximation) est issu de l ’A s s o c i a t i o n F r a n ç a i s e d ’A p p r o x i m a t i o n ( A FA ) , association créée en 1989 et intégrée en tant que groupe au sein de la SMAI en 2000. Le groupe thématique SMAI- S I G M A ( S i g n a l - I m a g e - G é o m é t r i e - M o d é l i s a t i o n - Approximation) est issu de l ’ A s s o c i a t i o n F r a n ç a i s e d ’ A p p ro x i m a t i o n ( A FA ) , association créée en 1989 et intégrée en tant que groupe au sein de la SMAI en 2000. xi xj (Unmasked) Attention layer … next token probabilities Attention Norm MLP Classif N × …
e⟨Qxi ,Kxℓ ⟩ Vxj Transformers and attention mechanism … + <latexit sha1_base64="aTL0Qvb1dLhAur6wfZM9PGylzLY=">AAA9iHictVttcxu3EYbTt1h9c9qPnelcq7iTdFyNpHqadjKaifViSbFiySYlOwltDV9O1NknHs0jJdkM/0e/tn+kv6P/oP3Uv9DdBXDAkbhbQHV1IwkH4nl2sQcsdoFjZ5gm+Xh19Z+3Pvje93/wwx99eHvpxz/56c9+fuejX5zk2WTUjY+7WZqNnnfaeZwmg/h4nIzT+PlwFLcvOmn8rPN6Cz9/dhmP8iQbNMdvh/GLi3Z/kJwl3fYYql62iGE6inuz69O10zvLqyur9BMtFtZUYVmon6Pso1+vi5boiUx0xURciFgMxBjKqWiLHK5vxZpYFUOoeyGmUDeCUkKfx2ImlgA7gVYxtGhD7Wv424e7b1XtAO6RMyd0F6Sk8DsCZCTuAiaDdiMoo7SIPp8QM9ZWcU+JE3V7C/87iusCasfiHGo5nG7pi8O+jMWZ+DP1IYE+DakGe9dVLBOyCmoeWb0aA8MQ6rDcg89HUO4SUts5IkxOfUfbtunzf1FLrMX7rmo7Ef8mLe/CFYmG6n1WMLTFJfFH9DQn8JnUJwXJfWCIVR+xdEW2vqDeD6D9FOofwzWjkrZJB64p1c5qkVtwuZBbLHIXLhdyl0UewOVCHrDII7hcyCOFROyIbO7GN+By4Rus5CdwuZBPWORTuFzIpyzyBC4X8oRFfgOXC/kNi3wIlwv5kEU+gsuFfMQim3C5kE0WeQyXC3nMInfgciF3FLJ6po7gyognYWblAyiXZaCnSKHmAavfJnlHF3bTY053K7D8rN6G/27stodN4wrsjse4O6vA8iNvF3ykG8v7oj1aTVzYPRa7DyPAjd1nsV+KVxXYLz1m2usKLD/XDqCdG8t736/gzo39isU+hpIby69Rh1Djxh56rBjDCuwRi30i3lRgfbz+qALL+/0G+BU3ll+nmtDejfXxppMKLO9PTyCCcWP51eoZ1Lqxz1jsc3FdgX3OYr8G7+7Gfu2xwr6rwOo1dolWkD7FIzHM2Dq2djErsTQEtjYjPy3WlpRi4w7Uc5h+gekT5oJF7BaIXU/EQYE48NYrL/xoTvEuL6VRIBqeiE6xNmFpzLbvFe2xlHogtgvE9hyiLiLFZ637cknRha7hkONi5cKST5+ywn9jKVbjod7zasRhCSHH9jmN/HuULWEGhZaqYzsv1niJjOi+DnFF2ZvupZbB48aFV7BR1yyq40B1WNRbB+oti5o4UBMWdelAXbIoM/NtXMtjBBj747OY0p0cATJGrr4iiAoewKqzB3M0gvFzBFHgU6o5hP8Nyr25q04zzOZxncRdjhclTzyC0lQsQ73JCrcpv05phsWgmWx5qHJ8vMO9jamac9ILz4qVPCp2TPx5EtKnX/BgtBjRfArjeUQ1M4ruZCkMv1fMe10Kw++QxWcUxctSGH6stB/fQPemwjZvgG3AbBoq65tyKIfcf5EcurxEqy56XHyqF2rMIN91IP++ejL7N3guW1SS9jHlMI7c6l9e6l8Ih7Fzbtk5jAWjJxn16lIU3JOByntNOVSHjFbRgdLD3IU+GWzTU09Gl8M4jiDi2qKce2qVQ0fvsOiNKYdxnAi57zmjSF6Xwzj6dC/tYcphHLjb0lZ5vimHena0gMydTTnUqw9oFxj3gOSYlzUmKhpRnDRRbAnFB/W7NXbMv7iO4Z7NyyJHqGcysW01T6dYy+o10vFCDF5tHKgHxhcTKwYrc0zFOptfSR3GpfV9kces8Wj5A7BiBLNfngFwe+YpaKj3JNB7p8C4xmZd5Z5p3DqLw1FyNodqqdoxGy0auXLXqFx3SrVcXmZ6a+zYIn+d09gbUkx4QJbl7HBQ+YSrGDkLHZQsxPOF2O6dmq9l66+yuOEcYliMtC6dCMmTtPo81WX1hmXju+qUZwyXPPMx4xd3m8+Ut8GcJyNfhLrUybTb6X0kuw7X1XvC7HHLzyJ6ouivLslrJHQilbNZqN4tltH4lO4N9zGdyaEMydGF5xgplqGQp2a4i4776RF5VNvfcrLRXnqHTpZz8rraH9ej+xa670CH5zhbsGI8hlITcoZjuGt6ZDlLha0ysvhI/KE4Hc3oCdZn9GnJQ2oO6W/ikoesy7LPSyxXgMbRILN0f455Ho1vLTDxWb9LH5O7lj3/XTq51efbbRrj1aO5eiemR1LXSWpEs0ae6sq7eQlSg6nzk3WKX+t7ifJCJKIP5aS+tCRLuwzoxD+mDHZIkXFKs42bHeXW9v7U/Cda0pHQZ+d4mp2Rh4zI/0WwPmU0JiP6td8d0Cfo0iOk5CN9/E5SRDeuWCdhx5iJ4xIh32ow4y0mXzYh+ZrXnl05jUWZMch1YDY3trVNDigWjEnqSHl3M7frVx9Emvck7FEiGc1Y+YTkf0p/9a8eJ8sLIwItjE8gV77O9TwyylnQRm1a5et9kG5ra/lxocNLpbVZ/4xOH5c026aMC/XB1boHkrt0L2XhKBmR3vlCG7mO1u3mIvNwzo7Y2zPK4qXf76sVGPW+R6vkMs25Fo2SPoyCcZFF6LbcLvK83HpZZXY/7vz/wm5sXbYaMkbC7OBKC3H7+zFla7aWKYxqOX5f02xyW30016pezoDG4oU1l7+D2t/AX623vvfj6ZS8wiaNAclg7oxFZE200MJP1mZJlh6ZmsvcG3lmTOpWds1N8mvp3UyOfRnMckSj5lrtWujyTTheWRyvPG3YpLNGY0Vdrz3RKZtbNNVppa+8EGnNAOYJy8xHZBqVeGhp51J+rD2Wlc/xNeody7XKcrVhttqnAfac90G65/r87P6uWN0j8ZBimy5FYDJ/6dEsTSjm0rX1mZpkQMn3lX+1Z3+LalB6hzwoMsv3OHHGyFOnLl2zQtPfqZUtIz9vPIJ+b+lKtdE+tkXlPy4gL2hO5DQvNeI+tYiV/rYe0ZxHWrFijoh2/tsUU8m4oz5ntlubZxKV4gmTb8pZZWTJTGFA9ud23vYXstd9K3+NKCecqOi6A1zhTxgZJEbvJLgjy5yeEK5y8iRBRrQd8p+Lfkqe4g0sjVZI66nY8PAxMus1Y90eW7rHum+/h5ZodfPUXS14eam3RE7eTU702rSqXagYdTp3fzOutlrlyvd1dpjMyTX2mFAbO7MwWV4Z0xKfe0uRGoVJkRgfKWG9CNE/TPMQneXplC+zbq2ZyzsN0secU77EvQeKCFd094kzmvuU6Udnga9DWJtN1nBMuBuXqf0B29PirtTthXVI1t6uXY1SayWqWik0u71aGP8tPWRM3i8V3J6NbG3r3iplKfwujGToCvlGb1V+aHN+Dhf+jYQrO9QSffYOGxDfPhBbYuc9vA3xRpXljmZENegLenO5d1v1s9yi3kZvLHab30eCv4wEbM1pn9BKGqq7ZOY1t9n9+a/IC4xEzGpvWob3wZbC92RRUkh/EvJsfG8Sob+LE9oXLcGnJ2Up/nLkuQbXizOhv9MU1gfNzvegLCFEhn6Pwe+Zm9bhsmxJ9fZalOIrQ64C+sRF4/DkrzpXMe18PNTIeiLvXwJ6h7Madr1a/K/90HKMpHBZvtJy+q7ZK4+nLtvFakcW4+HwOWOk+Yzmaon+MrOidyZacsuTcV8U9KQyqzfvnx/jUTMGtKypkPugvHYSb48io68vC54LuHTIxH/EP27x30Z4U3BU6RHCpM8pqtl0C55Nf+PS1Tv9mY9OhqdKpzKbySMa9EbsltgXD+F3q4gAQ98Old+llP8R6/7+bA9qz8h76F10uXPQorqYdj/MKVqP7tUe4+md5bX5byEvFk7WV9b+tHL/yf3lLzbVN5Q/FL8Sv4W8ZE18Jr4Qe9DfYzpX+Kv4m/j7xtLG6sZnG3+RTT+4pTC/FKWfjc3/Ajzs2dE=</latexit> x1 <latexit sha1_base64="7Z/IumRXp79HdyogVfnC2DA+LeM=">AAA9iHictVttcxu3EYbTt1h9c9qPnelcq7iTdFyNpHqadjKaifViSbFiySYlOwltDV9O1NknHs0jJdkM/0e/tn+kv6P/oP3Uv9DdBXDAkbhbQHV1IwkH4nl2sQcsdoFjZ5gm+Xh19Z+3Pvje93/wwx99eHvpxz/56c9+fuejX5zk2WTUjY+7WZqNnnfaeZwmg/h4nIzT+PlwFLcvOmn8rPN6Cz9/dhmP8iQbNMdvh/GLi3Z/kJwl3fYYql62iGE6inuz69P10zvLqyur9BMtFtZUYVmon6Pso1+vi5boiUx0xURciFgMxBjKqWiLHK5vxZpYFUOoeyGmUDeCUkKfx2ImlgA7gVYxtGhD7Wv424e7b1XtAO6RMyd0F6Sk8DsCZCTuAiaDdiMoo7SIPp8QM9ZWcU+JE3V7C/87iusCasfiHGo5nG7pi8O+jMWZ+DP1IYE+DakGe9dVLBOyCmoeWb0aA8MQ6rDcg89HUO4SUts5IkxOfUfbtunzf1FLrMX7rmo7Ef8mLe/CFYmG6n1WMLTFJfFH9DQn8JnUJwXJfWCIVR+xdEW2vqDeD6D9FOofwzWjkrZJB64p1c5qkVtwuZBbLHIXLhdyl0UewOVCHrDII7hcyCOFROyIbO7GN+By4Rus5CdwuZBPWORTuFzIpyzyBC4X8oRFfgOXC/kNi3wIlwv5kEU+gsuFfMQim3C5kE0WeQyXC3nMInfgciF3FLJ6po7gyognYWblAyiXZaCnSKHmAavfJnlHF3bTY053K7D8rN6G/27stodN4wrsjse4O6vA8iNvF3ykG8v7oj1aTVzYPRa7DyPAjd1nsV+KVxXYLz1m2usKLD/XDqCdG8t736/gzo39isU+hpIby69Rh1Djxh56rBjDCuwRi30i3lRgfbz+qALL+/0G+BU3ll+nmtDejfXxppMKLO9PTyCCcWP51eoZ1Lqxz1jsc3FdgX3OYr8G7+7Gfu2xwr6rwOo1dolWkD7FIzHM2Dq2djErsTQEtjYjPy3WlpRi4w7Uc5h+gekT5oJF7BaIXU/EQYE48NYrL/xoTvEuL6VRIBqeiE6xNmFpzLbvFe2xlHogtgvE9hyiLiLFZ637cknRha7hkONi5cKST5+ywn9jKVbjod7zasRhCSHH9jmN/HuULWEGhZaqYzsv1niJjOi+DnFF2ZvupZbB48aFV7BR1yyq40B1WNRbB+oti5o4UBMWdelAXbIoM/NtXMtjBBj747OY0p0cATJGrr4iiAoewKqzB3M0gvFzBFHgU6o5hP8Nyr25q04zzOZxncRdjhclTzyC0lQsQ73JCrcpv05phsWgmWx5qHJ8vMO9jamac9ILz4qVPCp2TPx5EtKnX/BgtBjRfArjeUQ1M4ruZCkMv1fMe10Kw++QxWcUxctSGH6stB/fQPemwjZvgG3AbBoq65tyKIfcf5EcurxEqy56XHyqF2rMIN91IP++ejL7N3guW1SS9jHlMI7c6l9e6l8Ih7Fzbtk5jAWjJxn16lIU3JOByntNOVSHjFbRgdLD3IU+GWzTU09Gl8M4jiDi2qKce2qVQ0fvsOiNKYdxnAi57zmjSF6Xwzj6dC/tYcphHLjb0lZ5vimHena0gMydTTnUqw9oFxj3gOSYlzUmKhpRnDRRbAnFB/W7NXbMv7iO4Z7NyyJHqGcysW01T6dYy+o10vFCDF5tHKgHxhcTKwYrc0zFOptfSR3GpfV9kces8Wj5A7BiBLNfngFwe+YpaKj3JNB7p8C4xmZd5Z5p3DqLw1FyNodqqdoxGy0auXLXqFx3SrVcXmZ6a+zYIn+d09gbUkx4QJbl7HBQ+YSrGDkLHZQsxPOF2O6dmq9l66+yuOEcYliMtC6dCMmTtPo81WX1hmXju+qUZwyXPPMx4xd3m8+Ut8GcJyNfhLrUybTb6X0kuw7X1XvC7HHLzyJ6ouivLslrJHQilbNZqN4tltH4lO4N9zGdyaEMydGF5xgplqGQp2a4i4776RF5VNvfcrLRXnqHTpZz8rraH9ej+xa670CH5zhbsGI8hlITcoZjuGt6ZDlLha0ysvhI/KE4Hc3oCdZn9GnJQ2oO6W/ikoesy7LPSyxXgMbRILN0f455Ho1vLTDxWb9LH5O7lj3/XTq51efbbRrj1aO5eiemR1LXSWpEs0ae6sq7eQlSg6nzk3WKX+t7ifJCJKIP5aS+tCRLuwzoxD+mDHZIkXFKs42bHeXW9v7U/Cda0pHQZ+d4mp2Rh4zI/0WwPmU0JiP6td8d0Cfo0iOk5CN9/E5SRDeuWCdhx5iJ4xIh32ow4y0mXzYh+ZrXnl05jUWZMch1YDY3trVNDigWjEnqSHl3M7frVx9Emvck7FEiGc1Y+YTkf0p/9a8eJ8sLIwItjE8gV77O9TwyylnQRm1a5et9kG5ra/lxocNLpbVZ/4xOH5c026aMC/XB1boHkrt0L2XhKBmR3vlCG7mO1u3mIvNwzo7Y2zPK4qXf76sVGPW+R6vkMs25Fo2SPoyCcZFF6LbcLvK83HpZZXY/7vz/wm5sXbYaMkbC7OBKC3H7+zFla7aWKYxqOX5f02xyW30016pezoDG4oU1l7+D2t/AX623vvfj6ZS8wiaNAclg7oxFZE200MJP1mZJlh6ZmsvcG3lmTOpWds1N8mvp3UyOfRnMckSj5lrtWujyTTheWRyvPG3YpLNGY0Vdrz3RKZtbNNVppa+8EGnNAOYJy8xHZBqVeGhp51J+rD2Wlc/xNeody7XKcrVhttqnAfac90G65/r87P6uWN0j8ZBimy5FYDJ/6dEsTSjm0rX1mZpkQMn3lX+1Z3+LalB6hzwoMsv3OHHGyFOnLl2zQtPfqZUtIz9vPIJ+b+lKtdE+tkXlPy4gL2hO5DQvNeI+tYiV/rYe0ZxHWrFijoh2/tsUU8m4oz5ntlubZxKV4gmTb8pZZWTJTGFA9ud23vYXstd9K3+NKCecqOi6A1zhTxgZJEbvJLgjy5yeEK5y8iRBRrQd8p+Lfkqe4g0sjVZI66nY8PAxMus1Y90eW7rHum+/h5ZodfPUXS14eam3RE7eTU702rSqXagYdTp3fzOutlrlyvd1dpjMyTX2mFAbO7MwWV4Z0xKfe0uRGoVJkRgfKWG9CNE/TPMQneXplC+zbq2ZyzsN0secU77EvQeKCFd094kzmvuU6Udnga9DWJtN1nBMuBuXqf0B29PirtTthXVI1t6uXY1SayWqWik0u71aGP8tPWRM3i8V3J6NbG3r3iplKfwujGToCvlGb1V+aHN+Dhf+jYQrO9QSffYOGxDfPhBbYuc9vA3xRpXljmZENegLenO5d1v1s9yi3kZvLHab30eCv4wEbM1pn9BKGqq7ZOY1t9n9+a/IC4xEzGpvWob3wZbC92RRUkh/EvJsfG8Sob+LE9oXLcGnJ2Up/nLkuQbXizOhv9MU1gfNzvegLCFEhn6Pwe+Zm9bhsmxJ9fZalOIrQ64C+sRF4/DkrzpXMe18PNTIeiLvXwJ6h7Madr1a/K/90HKMpHBZvtJy+q7ZK4+nLtvFakcW4+HwOWOk+Yzmaon+MrOidyZacsuTcV8U9KQyqzfvnx/jUTMGtKypkPugvHYSb48io68vC54LuHTIxH/EP27x30Z4U3BU6RHCpM8pqtl0C55Nf+PS1Tv9mY9OhqdKpzKbySMa9EbsltgXD+F3q4gAQ98Old+llP8R6/7+bA9qz8h76F10uXPQorqYdj/MKVqP7tUe4+md5bX5byEvFk7WV9b+tHL/yf3lLzbVN5Q/FL8Sv4W8ZE18Jr4Qe9DfYzpX+Kv4m/j7xtLG6sZnG3+RTT+4pTC/FKWfjc3/AnoD2dI=</latexit> x2 <latexit sha1_base64="Fgw+vWgPriclgLxpVoHDEicBDLw=">AAA9h3ictVv9chu3EYfTr9j9cto/O9O5VnYnybiqpHrSzGQ0E1uSJcWKJZuU7CS0Nfw40WcfeTSPlGQzfI7+2z5Jn6Nv0P7VV+juAjjgSNwtoLrCSMKB+O0uFsB+AMfOKE3yydraP6998IMf/ujHP/nw+o2f/uznv/jlzY9+dZJn03E3Pu5maTZ+1mnncZoM4+NJMknjZ6Nx3B500vhp5/UWfv70PB7nSTZsTt6O4ueDdn+YnCXd9gSant9qzS5Pk9b8dJbMb53eXFlbXaOfaLmyriorQv0cZR/9dkO0RE9koiumYiBiMRQTqKeiLXIo34l1sSZG0PZczKBtDLWEPo/FXNwA7BR6xdCjDa2v4W8fnr5TrUN4Rpo5obvAJYXfMSAjcRswGfQbQx25RfT5lChjaxXtGdFE2d7C/46iNYDWiXgJrRxO9/TF4Vgm4kx8TmNIYEwjasHRdRWVKWkFJY+sUU2AwgjasN6Dz8dQ7xJS6zkiTE5jR9226fN/UU9sxeeu6jsV/yYpb0OJREONPisotMU50Y9oNqfwmZQnBc59oBCrMWLtgnQ9oNEPof8M2h9BmVNN66QDZUat81rkFhQXcotF7kJxIXdZ5AEUF/KARR5BcSGPFBKxY9K5G9+A4sI3WM6PobiQj1nkEygu5BMWeQLFhTxhkd9CcSG/ZZEPoLiQD1jkQygu5EMW2YTiQjZZ5DEUF/KYRe5AcSF3FLJ6p46hZEQnYXblPaiXeaClSKHlHivffbKOLux9jz3drcDyu3ob/rux2x46jSuwOx7r7qwCy6+8XbCRbixvi/bIm7iweyx2H1aAG7vPYr8SryqwX3nstNcVWH6vHUA/N5a3vl/Dkxv7NYt9BDU3lvdRh9Dixh56eIxRBfaIxT4WbyqwPlZ/XIHl7X4D7Ioby/upJvR3Y32s6bQCy9vTE4hg3FjeWz2FVjf2KYt9Ji4rsM9Y7Ddg3d3Ybzw87LsKrPaxN8iD9CkeiWHH1lFrF7sSayOg1mb4p4VvSSk27kA7h+kXmD5hBixit0DseiIOCsSBt1x5YUdzind5Lo0C0fBEdArfhLUJ279X9Mda6oHYLhDbC4i6iBTnWo/lnKIL3cIhJ4XnwprPmLLCfmMtVuuh3vJqxGEJIdf2S1r5dyhbwgwKNVVH7WXh4yUyouc6xAVlb3qUmgePmxRWwUZdsqiOA9VhUW8dqLcsaupATVnUuQN1zqLMzrdxLY8VYPSPczGjJ7kCZIxcXSKICu6B19mDPRrB+jmCKPAJtRzC/wbl3lypkwyzefSTeMrxvGSJx1CbiRVoN1nhNuXXKe2wGCSTPQ9Vjo9PeLYxU3tOWuF54cmj4sTEn05C8vQLOhgtRrSfwug8pJY5RXeyFobfK/a9roXhd0jjc4riZS0MP1HST64ge1Nhm1fANmA3jZT2TT2Uhjx/kTR0/QZ5XbS4OKsDtWaQ3mUg/X01M/tXmJctqkn9mHoYjdwaX14aXwgNo+fc0nMYFYyeZNSra1HwSIYq7zX1UBky8qJDJYd5Cp0Z7NNTM6PrYTSOIOLaopx7ZtVDV++oGI2ph9E4EfLcc06RvK6H0ejTs9SHqYfRwNOWtsrzTT3UsqMGZO5s6qFWfUinwHgGJNe8bDFR0ZjipKmillB8UH9aY8f8y34Mz2xeFDlCPSUT21bT6RS+rF4iHS/EYNUmgXJgfDG1YrAyjZnYYPMrKcOk5N+X6Rgfj5o/AC1GsPvlHQB3Zp6ChPpMAq13ChTX2ayrPDKN22BxuErOFlAt1Tpho0XDV54aldtOqZXLy8xojR5bZK9zWnsjigkPSLOcHg4qZ7iKIqehg5KGeHohunun9mtZ+2ssbrSAGBUrrUs3QvImrT5PdWm9Yen4trrlmUCRdz5m/eJp85myNpjzZGSLUJY6nnY/fY5kt6FfvSPMGbf8LKIZRXt1TlYjoRupnM1C9WmxjMZn9GxoH9OdHPKQNLowj5GiMhLy1gxP0fE8PSKLattbjjfqS5/QyXpOVlfb43p030L3HejwHGcLPMYjqDUhZziGp6ZHlnOj0FVGGh+LPxa3oxnNYH1Gn5YspKYh7U1cspB1WfbLEpULQONqkFm6P41FOhrfWqLEZ/0ueUzuWrb8t+nmVt9vt2mNV6/m6pOYHnHdIK4R7Rp5qyufFjlICWbOTzYofq0fJfIL4Yg2lOP6wuIs9TKkG/+YMtgRRcYp7TZud5R72+dTi59oTkdC353jbXZGFjIi+xeBf8poTUb0a787oG/QpUVIyUb62J2kiG5csU7CrjETxyVCvtVg1ltMtmxK/DVde3fltBZlxiD9wHxhbWudHFAsGBPXsbLuZm/Xex9Emvck7FUiKZq18jHx/4T+6l+9TlaWVgRqGGcgV7bONR8Z5SyoozZ5+XobpPvaUt4qZHihpDb+z8h0qyTZNmVcKA966x5w7tKz5IWrZExy50t9pB+tO81FyqMFPeJozyiLl3a/rzwwyn2HvOQK7bkWrZI+rIJJkUXovtwp8iLfel5l6n608/8LdaPrstaQYiTMCa7UEHe+H1O2ZkuZwqqW6/c17Sa31scLver5DGktDqy9/D20/g7+arn1sx+dTskq3Kc1ICmYJ6MR2RIt9fDjdb/ES69MTcs8G35mTepedstV8mtp3UyOfR5M5YhWzaU6tdD1q9B4ZdF45anDJt01Gi3qdm2JTtncoqluK335hXBrBlCespT5iEyjEg8p7VzKj2qPpcrn+Br1jqW1xtJqw261bwPsPe+DdO/1xd39feHdI/GAYpsuRWAyf+nRLk0o5tKt9ZmapICc7yr7au/+FrUg9w5ZUKQs3+PEHSNvnbpU5oWkf1CeLSM7byyCfm/pQvXRNrZF9T8vIQe0J3Lalxpxl3rESn5bjmjBIq1aMUdEJ/9tiqlk3FGfM9u9zZxEpXjC5JtyVxleMlMYkv65k7f9pex138pfI8oJpyq67gCt8BlGChKjTxLckWVOM4ReTt4kyIi2Q/Zz2U7JW7yhJdEqST0Tmx42Rma9Zq3ba0uPWI/tU+iJWjez7urB80u9OXL8rnKj1yavNlAx6mzh+Wq02srLlZ/r9DBd4Gv0MaU+dmZhsrwypiW+8OYiJQrjIjE+XMJGESJ/mOQhMsvbKV/KuremXD5pkDbmJeVL3HugiHBFdx87o7lPmHF0luh1CGtTky0cJTyNy9T5gG1p8VTq+pIfkq3Xa71RanmiKk+hqdvewthvaSFjsn6p4M5sZG9b9lYpS+FPYSSFrpBv9FblhzbNL6Dg30i4skPN0efssAHx7T2xJXbew9sQb1RdnmhG1IK2oLeQe7fVOMs96nX0xqJu0/fh4M8jAV1z0ifkSUNll5R5yW3q/vQvyAqMRcxKb3qGj8Hmwo9kmVPIeBKybPxoEqG/ixM6Fs3BZyRlLv585L0GN4ozob/TFDYGTZ0fQZlDCA/9HoPfnJve4bxsTvX6Wubiy0N6AX3jonF481edq5h+PhZqbM3I++eA1uGshrr2Fv/rODQfwymcly+3nL5r9spj1mW/WJ3IYjwcvmcMN5/VXM3Rn2dWjM5ES25+Mu6LgmYqs0bz/uljPGrWgOY1E/IclJdO4u1VZOT1pYL3Ai4ZMvEf8Y9r/LcR3hQ0quQIoaTvKaqp6R48Nf2NS9fo9Gc+Mhk6VTKVqZk8okFvxG6JffEAfreKCDD07VD5XUr5H7Hu78/2oPWMrIc+RZcnBy1qi+n0w9yi9ehZnTGe3lxZX/wW8nLlZGN1/bPVu483Vr68r76h/KH4jfg95CXr4i/iS7EH4z0mTf1V/E38ffP65p82P9v8XHb94JrC/FqUfjbv/Rd5M9kH</latexit> {xi }i Points cloud Positional encoding Token encoding Tokenize Le groupe thématique SMAI- S I G M A ( S i g n a l - I m a g e - Géométrie-Modélisation- Approximation) est issu de l ’A s s o c i a t i o n F r a n ç a i s e d ’A p p r o x i m a t i o n ( A FA ) , association créée en 1989 et intégrée en tant que groupe au sein de la SMAI en 2000. Le groupe thématique SMAI- S I G M A ( S i g n a l - I m a g e - G é o m é t r i e - M o d é l i s a t i o n - Approximation) est issu de l ’ A s s o c i a t i o n F r a n ç a i s e d ’ A p p ro x i m a t i o n ( A FA ) , association créée en 1989 et intégrée en tant que groupe au sein de la SMAI en 2000. xi xj (Unmasked) Attention layer Arbitrary number of tokens Arbitrary number of layers Expressivity Understanding … next token probabilities Attention Norm MLP Classif N × …
0 Theorem: If , then μ(0) = 𝒩 (m(0), Σ(0)) · m = V(Id+ΣQ⊤K)m · Σ = VΣQ⊤KΣ + ΣK⊤QΣV⊤ Γθ [μ](x) := ∫ e⟨Qx,Ky⟩ ∫ e⟨Qx,Ky′  ⟩dμ(y′  ) Vy dμ(y) θ(t) = (Q(t), K(t), V(t)) μ(s) = 𝒩 (m(s), Σ(s)) Theorem [Valérie Castin]: If and symmetric, stationary points of have rank less than V(t) = Id K(t)⊤Q(t) Σ(t) d/2. Conjecture: low-rank stationary covariances for any . K, Q, V … t μ(0) μ(∞) [Geshkovski, Letrouit, Polyanskiy, Rigollet 2023] The attention matrix converges to low-rank. → Clustering of for un-normalized attention. → μ t μ(0) μ(t) Valérie Castin
Early efforts to cartography cell identity relied on microscopy1. Recent initiatives me molecular profile of s in the human body, ly different functions. Early efforts to cartography cell identity relied on microscopy1. Recent initiatives measure th molecular profile of the cell e human body, erent functions. Early efforts to cartography cell identity relied on microscopy1. Recent initiatives measure the molecular profile of the cell2. ~ cells in the human body, with vastly different functions. 1013 Early efforts to cartography cell identity relied on microscopy. [Ramón y Cajal, 1899] Recent initiatives measure the molecular pro fi le of the cell. [Regev et al., eLife, 2017] Unraveling cell diversity
for each type. Applications: cancer mutations, dynamic of adaptation, development, … <latexit sha1_base64="JmboHyrN9W6kgItyZjvFMpB0lgs=">AABBxXictVzbchu5EYU3t7Vz8yaPeZlE65Q35Tiy4kpStZWqlSVZ1pq2aZOSvbu0XbyMaNpDDs0h6QtXlU/Ia/I1+Y78QfKUX0hfgAGGxExjFMcoSRgQp7vRAzS6G6B702SUzbe3/3nho+9893vf/8HHFy/98Ec//slPL3/ys5MsXcz68XE/TdLZk143i5PRJD6ej+ZJ/GQ6i7vjXhI/7r3aw88fL+NZNkon7fm7afx03B1ORqejfncOTcf9OEmy55e3tq9v079os3JDV7aU/tdMP4mOVUcNVKr6aqHGKlYTNYd6oroqg/KNuqG21RTanqoVtM2gNqLPY3WmLgF2Ab1i6NGF1lfwewhP3+jWCTwjzYzQfeCSwM8MkJG6ApgU+s2gjtwi+nxBlLG1jPaKaKJs7+BvT9MaQ+tcvYBWCWd6huJwLHN1qv5EYxjBmKbUgqPrayoL0gpKHjmjmgOFKbRhfQCfz6DeJ6TRc0SYjMaOuu3S5/+intiKz33dd6H+TVJegRKplh59mlPoqiXRj+htLuAzlicBzkOgEOsxYu0N6XpMo59A/xW034dyRjWjkx6UFbWeVSL3oPiQeyLyEIoPeSgiG1B8yIaIbELxIZsaidgZ6dyPb0Hx4Vsi54dQfMiHIvIRFB/ykYg8geJDnojIr6H4kF+LyNtQfMjbIvIuFB/yrohsQ/Eh2yLyGIoPeSwiD6D4kAcaWb5SZ1BSojMSVuUu1Is80FIk0LIryneLrKMPeytgTfdLsPKq3oe/fux+gE7jEuxBwLw7LcHKM+8QbKQfK9uiO7Sb+LB3ROwRzAA/9kjEfqlelmC/DFhpr0qw8lprQD8/Vra+9+DJj70nYu9DzY+V96gH0OLHPgjYMaYl2KaIfahel2BDrP6sBCvb/RbYFT9W3qfa0N+PDbGmixKsbE9PwIPxY+Xd6jG0+rGPRewT9bYE+0TEfgXW3Y/9KmCHfV+CNXvsJdpBhuSPxLBiq6h181WJtSlQ6wr8k3xvScg37kG7hBnmmCFhxiLiMEccBiIaOaIRLFeW29GM/F2ZSytHtAIRvXxvwtpc7D/I+2MtCUDs54j9NUSVR4rv2oxlSd6FaZGQ83znwlrImNLcfmMt1vOh2vIaxIMCguf2C5r51yhawggKNVVF7UW+xzMyoucqxBuK3swoDQ8ZN8+tgot6K6J6HlRPRL3zoN6JqIUHtRBRSw9qKaLsyndxnYAZYPWP72JFTzwD2EcuLxF4Bbuw69yBNRrB/GmCF/iIWh7A3xbF3lKpkgyjedwnMcvxtGCJZ1BbqS1ot1HhPsXXCa2wGCTjng90jI9PmNtY6TXHVvgs38mjPGMSTmdE8gxzOugtRrSe6tG5Sy1n5N1xrR7+Tr7uTa0e/oA0fkZePNfq4eda+vk5ZG9rbPsc2BaspqnWvq3XpcH5F6Zh6pdo10WLi291rOcM0ntbk/6RfjNH53gve1Rj/dh6PRqZM76sML46NKyeM0fP9aig98Rer6lFtUcy0XGvrdeVIaVddKLlsE913wz2Geg3Y+r1aDTB49qjmHvl1OvO3mk+GluvR+NEcd7zjDx5U69HY0jPrA9br0cDsy1dHefbel3Ljhrg2NnW61r1CWWBMQfEc55brFc0Iz9poamNyD+ozta4Pv/mPoY5m2d5jFBNyfq25XR6+V5WLZHxF2KwavOacqB/sXB8sCKNldoR4yuWYV7Y3zfp2D0eNd8ALUaw+vkMQMqZJyChyUmg9U6A4g0x6iqOzOB2RBzOktM1VEe3zkVv0fLlrFGx7Tm1SnGZHa3VY4fsdUZzb0o+YYM0K+mhUfqGyyhKGmoUNCTTq6O793q9FrW/LeKma4hpPtP6dCLEJ2nVcapP6y1Hx1f0Kc8cCp/52PmL2eZTbW0w5knJFqEsVTzdfiaP5LbhvnpN2Rw3fxbRG0V7tSSrMaITqUyMQk22mL3xFT1b2sd0Joc8mEYf3mOkqUwVn5phFh3z6RFZVNfeSrxRXyZDx/WMrK6xx9XooYMeetD1Y5w92DHuQ60NMcMxPLUDopxLua5S0vhM/TY/HU3pDVZH9EnBQhoabG/igoWsirJfFKi8ATTOBo7Sw2ms0zH4zgYlOer3yWNj16Llv0Int+Z8u0tzvHw2l2diBsR1h7hGtGr4VJef1jmwBCvvJzvkv1aPEvnV4Yg2VOL6zOHMepnQiX9MEeyUPOOEVpu0Ooq93fzU+ieGU1OZs3M8zU7JQkZk/yLYn1KakxH9uHcHzAk6W4SEbGSI3Rnl3o3P1xmJc8z6cSPFtxrsfIvJli2Iv6Hrrq6M5iJHDLwPnK3NbaOTBvmCMXGdaetu13b17oNIe0/CnSVM0c6Vq8T/M/ptfsw82dqYEahhfAOZtnW+95FSzII66tIuX22DTF9Xyk9zGZ5pqe3+Z2X6tCDZPkVcKA/u1gPg3Kdn5oWzZEZyZxt9eB+tyuYi5emaHnG0pxTFs90f6h0Y5b5Gu+QWrbkOzZIhzIJ5HkWYvlIWeZ1vNa8i9TDa2f+FutV1UWtIMVI2g8sakvL7MUVrrpQJzGqev69oNfm1PlvrVc1nQnNx7Kzlb6H1l/DbyG2ew+j0ClbhFs0BpmCfrEa4JdroEcbrVoGXmZmGln22/OycNL3clvPE12zdbIy9rE2lSbPmrc5amPp5aLx0aLwM1GGbzhqtFk27sUTPxdiirU8rQ/nV4dauQXkhUpY9MoMaBUjpxlJhVAciVTnGN6j3Iq1tkVYXVqt7GuCu+RCkf62vr+5v8909UrfJt+mTB8bxy4BW6Yh8LtNaHakxBeR8U9tXd/V3qAW598iCImW+x4krhk+d+lTOckl/rXe2lOy8tQjm3tIb3cfY2A7Vf7+BHNOayGhdGsRN6hFr+V05ojWLdN3xOSLK/HfJp2K/ozpmdnvbdxIV/Akbb/Kqsrw4UpiQ/qXM29FG9HrkxK8RxYQL7V33gFb9N4wUGGMyCX7PMqM3hLscnySwR9sj+7lpp/gUb+JIdJ2kXqk/B9gYjnrtXHfnlhmxGdtvoCdq3b51Xw+ZXxLMUeJ3nhO9Lu1qY+2jrtaez0erq3e54nOVHhZrfK0+FtTHjSxslFfEdNTnwVxYonpcGBPCpd4o6shfT/I6MvPpVChl09tQLmYa2Ma8oHhJugeKCJ93d9XrzX0mjKO3Qa9HWJcat0iUMBuX6vyAa2kxK3VxYx/i1ouVu1Hi7ERlO4Wh7u4W1n6zhYzJ+iVKytlwb1f2TiFKkbMwTKGv+EZvWXzo0vwcCv6OlC86NBxDcoct8G931Z46+AC3IV7rOmc0I2pBWzBYi727epzFHtU6eu1Qd+mHcAjnMQJdS9KPaCetKztTliV3qYfTf0NWYKZiUXrbs/4YXC7ySDY51RnPiCybPJqRMt/FqTsWwyFkJEUu4Xz4XEMaxaky32mqNwZDXR5BkUMdHuYeQ9g7t73r83I5Vetrk0soD94FzImLweHJX3msYvuFWKiZ80Y+PAe0DqcV1M1u8b+Ow/CxnOrzCuWW0XfNXga8de4X64ws+sP114zlFjKbyzmG80zz0Vlvyc+P/b6o1ptKndF8eProj9o5YHitFOdBZekY784iK28oFTwX8MmQqv+of1yQv43wOqdRJkcdSuacopya6SFTM9+49I3OfBYik6VTJlORmo0jWnQjdk8dqdvws5d7gHVvh/J3KfkvYv3fnx1A6ylZD5NF58xBh9piyn7YU7QBPdv7s2US411evtvbhhY8C29QK97zvU/98a5vuzC28m+Q8Fq/p1I1KEQk66d7dl31YATFkzfOAZnv+UZ0l56zWHzzbBxwtmjuT61LtKJP5JsFvVJ8z5GyT3N1qs/q8eQAb9h38/xQpH5HbV1t53HPlTg3Szk31zhnpJ0ih7fOZ9V3s8q47DlcBnnubKn7pRRn2/O86tzofikXvoNejR9W4IeOlC3S/iuKhGeqOpu3qKC50DK5J6wTZTKRrAeMM7v5+66ObJcVvJYB479bir7rSHoIsvQo/x3RCduM6CVaNwckPd90rM6k3qmQVn+P8vnlrRvr/5fBZuVk5/qNP1y/+XBn64tb+v85+Fj9Qv1KXYU1/kf1BVBrqmM6d/+r+pv6++7h7nh3vrvkrh9d0Jifq8K/3b/8F2JKomY=</latexit> cells <latexit sha1_base64="6LS8OfUSD/ef5J7/bEX8BJJLTug=">AABDBXictVxLkxPJES7WrwW/WPvoS9uzOFgHiwdM2I7YcMQyD4ZZBAwjzQC7AkKPlhC01EItDQ/tnP0DfLV/gm8OX/d3+B/YJx/8B5yPqq5qqbqzeozpmJnqUn2ZWVlVWZlZJbrTZJTNNzf/ce6Db337O9/93ofnL3z/Bz/80Y8vfvST4yxdzHrxUS9N0tmjbieLk9EkPpqP5kn8aDqLO+NuEj/svtzGzx+exLNslE5a87fT+Mm4M5yMBqNeZw5Vj9ujSdQ+PHzaf3ZxY/PqJv2L1gvXdGFD6X8H6UfRf1Rb9VWqemqhxipWEzWHcqI6KoPnK3VNbaop1D1RS6ibQWlEn8fqVF0A7AJaxdCiA7Uv4fcQ3r7StRN4R5oZoXvAJYGfGSAjdQkwKbSbQRm5RfT5gihjbRntJdFE2d7C366mNYbauXoOtRLOtAzFYV/maqB+T30YQZ+mVIO962kqC9IKSh45vZoDhSnUYbkPn8+g3COk0XNEmIz6jrrt0Of/pJZYi+893Xah/kVSXoInUk3d+zSn0FEnRD+i0VzAZyxPApyHQCHWfcTSa9L1mHo/gfZLqL8HzymVjE668Cyp9rQSuQ2PD7ktIvfg8SH3RGQDHh+yISIP4PEhDzQSsTPSuR/fhMeHb4qcH8DjQz4QkYfw+JCHIvIYHh/yWER+CY8P+aWIvAWPD3lLRN6Bx4e8IyJb8PiQLRF5BI8PeSQid+HxIXc1snylzuBJic5IWJU3oVzkgZYigZqbonxbZB192K2ANd0rwcqregf++rE7ATqNS7C7AfNuUIKVZ94e2Eg/VrZFt2k38WFvi9h9mAF+7L6I/UK9KMF+EbDSXpZg5bXWgHZ+rGx978KbH3tXxN6Dkh8r71H3ocaPvR+wY0xLsAci9oF6VYINsfqzEqxs95tgV/xYeZ9qQXs/NsSaLkqwsj09Bg/Gj5V3q4dQ68c+FLGP1JsS7CMR+xisux/7OGCHfVeCNXvsBdpBhuSPxLBiq6h18lWJpSlQ6wj8k3xvScg37kK9hBnmmCFhxiJiL0fsBSIaOaIRLFeW29GM/F2ZSzNHNAMR3XxvwtJcbN/P22MpCUDs5IidFUSVR4pjbfpyQt6FqZGQ83znwlJIn9LcfmMp1vOh2vIaxP0Cguf2c5r5VyhawggKNVVF7Xm+xzMyovcqxGuK3kwvDQ8ZN8+tgot6I6K6HlRXRL31oN6KqIUHtRBRJx7UiYiyK9/FtQNmgNU/jsWS3ngGsI9c/kTgFdyEXec2rNEI5s8BeIGHVHMf/jYp9paeKskwmsd9ErMcTwqWeAalpdqAehsV7lB8ndAKi0Eybnlfx/j4hrmNpV5zbIVP8508yjMm4XRGJM8wp4PeYkTrqR6dO1RzSt4dl+rhb+fr3pTq4XdJ46fkxXOpHn6upZ+fQfaWxrbOgG3Cappq7dtyXRqcf2EapnyBdl20uDiqYz1nkN6bmvT39cjsn2FctqnE+rHlejQyp39ZoX91aFg9Z46e61FB74m9XlOKavdkouNeW64rQ0q76ETLYd/qjgy26euRMeV6NA7A49qmmHvplOvO3mneG1uuR+NYcd7zlDx5U65HY0jvrA9brkcDsy0dHefbcl3Ljhrg2NmW61r1CWWBMQfEc55rrFc0Iz9poamNyD+ozta4Pv/6PoY5m6d5jFBNyfq25XS6+V5WLZHxF2KwavOacqB/sXB8sCKNpbouxlcsw7ywv6/TsXs8ar4BWoxg9fMZgJQzT0BCk5NA650AxWti1FXsmcFdF3E4SwYrqLaunYveouXLWaNi3TOqleIy21urxzbZ64zm3pR8wgZpVtJDo3SEyyhKGmoUNCTTq6O7d3q9FrW/KeKmK4hpPtN6dCLEJ2nVcapP601Hx5f0Kc8cHj7zsfMXs80DbW0w5knJFqEsVTzddiaP5NbhvnpF2Rw3fxbRiKK9OiGrMaITqUyMQk22mL3xJb1b2kd0Joc8mEYPxjHSVKaKT80wi4759IgsqmtvJd6oL5Oh43JGVtfY42r00EEPPej6Mc427Bj3oNSCmOEI3loBUc6FXFcpaXymPs1PR1MaweqIPilYSEOD7U1csJBVUfbzApXXgMbZwFF6OI1VOgbfXqMkR/0+eWzsWrT8l+jk1pxvd2iOl8/m8kxMn7heJ64RrRo+1eW3VQ4swdL7yXXyX6t7ifzqcEQbKnF96nBmvUzoxD+mCHZKnnFCq01aHcXWbn5q9RPD6UCZs3M8zU7JQkZk/yLYn1KakxH9uHcHzAk6W4SEbGSI3Rnl3o3P1xmJc8z6cSPFtxrsfIvJli2Iv6Hrrq6M5iJHDLwPnK7MbaOTBvmCMXGdaetu13b17oNIe0/CnSVM0c6Vy8T/E/ptfsw82VibEahhHIFM2zrfeKQUs6COOrTLV9sg09aV8uNchqdaarv/WZk+Lki2QxEXyoO7dR849+ideeEsmZHc2Vob3kersrlIebqiR+ztgKJ4tvtDvQOj3Fdol9ygNdemWTKEWTDPowjTVsoir/Kt5lWkHkY7+79Qt7ouag0pRspmcFlDUn4/pmjNlTKBWc3z9yWtJr/WZyutqvlMaC6OnbX8NdT+HH4buc17GJ1uwSps0RxgCvbNaoRrorUWYby2CrzMzDS07LvlZ+ekaeXWnCW+ZutmY+yT2lQOaNa80VkLUz4LjRcOjReBOmzRWaPVoqk3luiZGFu09GllKL863Fo1KC9EyrJHZlCjACndWCqMal+kKsf4BvVOpLUp0urAanVPA9w1H4L0r/XV1f11vrtH6hb5Nj3ywDh+6dMqHZHPZWqrIzWmgJxvaPvqrv421SD3LllQpMz3OHHF8KlTj57TXNJf6p0tJTtvLYK5t/RatzE2tk3l36whx7QmMlqXBnGDWsRafleOaMUiXXV8jogy/x3yqdjvqI6Z3dZ2TKKCP2HjTV5VlhdHChPSv5R521+LXved+DWimHChvesu0Ko/wkiBMSaT4PcsMxoh3OX4JIE92i7Zz3U7xad4E0eiqyT1Uv0hwMZw1Gvnuju3TI9N334FLVHrdtR9LWR+STBHid9ZTvQ6tKuNtY+6XHk/G62O3uWK71V6WKzwtfpYUBs3srBRXhHTVp8Fc2GJ6nFhTAiXer2oI389yevIzKdToZRNa0O5mGlgG/Oc4iXpHigifN7dZa8394nQj+4avS5hXWpcI1HCbFyq8wOupcWs1Pm1fYhrz1fuRomzE5XtFIa6u1tY+80WMibrlygpZ8OtXdnbhShFzsIwhZ7iG71l8aFL8zN48HekfNGh4RiSO2yCf3tTbavd93Ab4pUuc0Yzohq0Bf2V2Luj+1lsUa2jVw51l34Ih3AeI9C1JP2IdtK6sjNlWXKXejj912QFZioWpbct6/fB5SL3ZJ1Tnf6MyLLJvRkp812cun0xHEJ6UuQSzofPNaReDJT5TlO9Phjqcg+KHOrwMPcYwsbctq7Py+VUra91LqE8eBcwJy4Ghyd/5bGKbRdioWbOiLx/DmgdBhXUzW7xv/bD8LGc6vMK5ZbRd81eBIw6t4t1Rhb94fprxnILmc3lHMN5pnnvrLfk58d+X1RrpFKnN++fPvqjdg4YXkvFeVBZOsa7s8jKG0oFzwV8MqTq3+qbc/K3EV7lNMrkqEPJnFOUUzMtZGrmG5e+3pnPQmSydMpkKlKzcUSTbsRuq311C362cw+w7u1Q/i4l/0Ws//uzfagdkPUwWXTOHLSpLqbshz1F69O7vT9bJjHe5eW7vS2owbPwBtXiPd971B7v+rYKfSv/Bgmv9bsqVf1CRLJ6umfXVRd6UDx54xyQ+Z5vRHfpOYvFN8/GAWeL5v7UqkRL+kS+WdAtxXcdKXs0V6f6rB5PDvCGfSfPD0Xq11TX0XYe91yJ80Ep54MVzhlpp8jhjfNZ9d2sMi7bDpd+njs70e1SirPteV51bnSnlAvfQa/GDyvwQ0fKJmn/JUXCM1WdzVtU0FxomdwT1okymUjWA8aZnXy8qyPbkwpeJwH9v1OKvuNIugeydCn/HdEJ24zoJVo3uyQ933SszqTerpDWfI+Sadq7jnYemFuL1Vn6RM87zlyY/41gSesBd1NzM1HKnsQldLp0IzAmSnxHsprSQJBnIFIYipLomfrs4sa11f8HYr1wfP3qtd9evfHgxsbnW/r/iPhQ/Uz9Ql0G+/g79TmMxIE6Ak5j9Sf1Z/WXrT9u/XXrb1t/56YfnNOYn6rCv61v/guvM+Qz</latexit> 2 Rd Tissue Dissociation isolation RNA amplification sequencing … <latexit sha1_base64="6L8H4Gt+fXCCPCipj/akDBzsUqw=">AABDDXictVzbchu5EYU3t7VzWW/ymJdJZKe8KUeRFFeSqq1UrXWxrLVsyyYle3dpu3gZ0rSHHJpDyheuviEfkNfkE/KWSh7zDfmD5CkP+YH0BRhgSMw0RnE8ZQkD4nQ3GkCjuwGqM0mG2Wxj4x8XPvjGN7/17e98ePHSd7/3/R98dPnjH55k6XzajY+7aZJOH3faWZwMx/HxbDhL4seTadwedZL4UeflDn7+6DSeZsN03Jy9ncRPRu3BeNgfdtszqHp2+aMrvaiVDUfR5sbTxdbZlWeX1zbWN+hftFrY1IU1pf8dpR9H/1Et1VOp6qq5GqlYjdUMyolqqwyer9Sm2lATqHuiFlA3hdKQPo/VmboE2Dm0iqFFG2pfws8BvH2la8fwjjQzQneBSwL/p4CM1FXApNBuCmXkFtHnc6KMtWW0F0QTZXsLvzua1ghqZ+o51Eo40zIUh32Zqb76LfVhCH2aUA32rqupzEkrKHnk9GoGFCZQh+UefD6FcpeQRs8RYTLqO+q2TZ//k1piLb53ddu5+hdJeRWeSDV079OcQludEv2IRnMOn7E8CXAeAIVY9xFLr0nXI+r9GNovoP4ePGdUMjrpwLOg2rNK5A48PuSOiNyHx4fcF5GH8PiQhyLyCB4f8kgjETslnfvxDXh8+IbI+QE8PuQDEfkQHh/yoYg8gceHPBGRX8LjQ34pIm/B40PeEpF34PEh74jIJjw+ZFNEHsPjQx6LyD14fMg9jSxfqVN4UqIzFFblTSgXeaClSKDmpijfNllHH3Y7YE13S7Dyqt6F337sboBO4xLsXsC865dg5Zm3DzbSj5Vt0W3aTXzY2yL2AGaAH3sgYj9XL0qwnwestJclWHmtHUI7P1a2vnfhzY+9K2LvQcmPlfeo+1Djx94P2DEmJdgjEftAvSrBhlj9aQlWtvsNsCt+rLxPNaG9HxtiTeclWNmenoAH48fKu9UjqPVjH4nYx+pNCfaxiP0CrLsf+0XADvuuBGv22Eu0gwzIH4lhxVZRa+erEksToNYW+Cf53pKQb9yBegkzyDEDwoxExH6O2A9EHOaIw2C5styOZuTvylwaOaIRiOjkexOWZmL7Xt4eS0kAYjdH7C4hqjxSHGvTl1PyLkyNhJzlOxeWQvqU5vYbS7GeD9WW1yDuFxA8t5/TzL9O0RJGUKipKmrP8z2ekRG9VyFeU/Rmeml4yLhZbhVc1BsR1fGgOiLqrQf1VkTNPai5iDr1oE5FlF35Lq4VMAOs/nEsFvTGM4B95PInAq/gJuw6t2GNRjB/jsALfEg19+F3g2Jv6amSDKN53Ccxy/GkYImnUFqoNai3UeEuxdcJrbAYJOOW93WMj2+Y21joNcdW+CzfyaM8YxJOZ0jyDHI66C1GtJ7q0blDNWfk3XGpHv52vu5NqR5+jzR+Rl48l+rhZ1r62Tlkb2ps8xzYBqymida+LdelwfkXpmHKl2jXRYuLozrScwbpvalJ/0CPzME5xmWHSqwfW65HI3P6lxX6V4eG1XPm6LkeFfSe2Os1pah2T8Y67rXlujKktIuOtRz2re7IYJueHhlTrkfjCDyuHYq5F0657uyd5L2x5Xo0ThTnPc/IkzflejQG9M76sOV6NDDb0tZxvi3XteyoAY6dbbmuVR9TFhhzQDznucZ6RVPyk+aa2pD8g+psjevzr+5jmLN5mscI1ZSsb1tOp5PvZdUSGX8hBqs2qykH+hdzxwcr0lioLTG+Yhlmhf19lY7d41Hzh6DFCFY/nwFIOfMEJDQ5CbTeCVDcFKOuYs8MbkvE4SzpL6FaunYmeouWL2eNinXPqFaKy2xvrR5bZK8zmnsT8gkPSbOSHg5LR7iMoqShw4KGZHp1dPdOr9ei9jdE3GQJMclnWpdOhPgkrTpO9Wm94ej4qj7lmcHDZz52/mK2ua+tDcY8KdkilKWKp9vO5JHcOtxXryub4+bPIhpRtFenZDWGdCKViVGoyRazN76gd0v7mM7kkAfT6MI4RprKRPGpGWbRMZ8ekUV17a3EG/VlMnRczsjqGntcjR446IEHXT/G2YEd4x6UmhAzHMNbMyDKuZTrKiWNT9Uv8tPRlEawOqJPChbS0GB7ExcsZFWU/bxA5TWgcTZwlB5OY5mOwbdWKMlRv08eG7sWLf9VOrk159ttmuPls7k8E9MjrlvENaJVw6e6/LbMgSVYeD/ZIv+1upfIrw5HtKES16cOZ9bLmE78Y4pgJ+QZJ7TapNVRbO3mp5Y/MZyOlDk7x9PslCxkRPYvgv0ppTkZ0X/37oA5QWeLkJCNDLE7w9y78fk6Q3GOWT9uqPhWg51vMdmyOfE3dN3VldFc5IiB94GzpbltdHJIvmBMXKfautu1Xb37INLek3BnCVO0c+Ua8f+Efpr/Zp6srcwI1DCOQKZtnW88UopZUEdt2uWrbZBp60p5JZfhqZba7n9WpisFyXYp4kJ5cLfuAecuvTMvnCVTkjtbacP7aFU2FylPlvSIve1TFM92f6B3YJT7Ou2Sa7TmWjRLBjALZnkUYdpKWeRlvtW8itTDaGf/F+pW10WtIcVI2Qwua0jK78cUrblSJjCref6+pNXk1/p0qVU1nzHNxZGzlr+G2p/ATyO3eQ+j0ylYhW2aA0zBvlmNcE200iKM13aBl5mZhpZ9t/zsnDSt3JrzxNds3WyMfVqbyhHNmjc6a2HK56HxwqHxIlCHTTprtFo09cYSPRNji6Y+rQzlV4dbswbluUhZ9sgMahggpRtLhVHtiVTlGN+g3om0NkRabVit7mmAu+ZDkP61vry6v85390jdIt+mSx4Yxy89WqVD8rlMbXWkxhSQ8w1tX93V36Ia5N4hC4qU+R4nrhg+derSc5ZL+jO9s6Vk561FMPeWXus2xsa2qPyrFeSI1kRG69IgblCLWMvvyhEtWaR1x+eIKPPfJp+K/Y7qmNltbcckKvgTNt7kVWV5caQwJv1LmbeDlej1wIlfI4oJ59q77gCt+iOMFBhjMgl+zzKjEcJdjk8S2KPtkP1ctVN8ijd2JFonqRfqdwE2hqNeO9fduWV6bPr2c2iJWrej7msh80uCOUr8znOi16ZdbaR91MXS+/lotfUuV3yv0sN8ia/Vx5zauJGFjfKKmJb6NJgLS1SPC2NCuNTrRR3560leR2Y+nQqlbFobysVMA9uY5xQvSfdAEeHz7q55vblPhH50Vuh1COtS4xqJEmbjUp0fcC0tZqUuruxDXHuxcjdKnJ2obKcw1N3dwtpvtpAxWb9ESTkbbu3K3ipEKXIWhil0Fd/oLYsPXZqfwoM/I+WLDg3HkNxhA/zbm2pH7b2H2xCvdJkzmhHVoC3oLcXebd3PYotqHb1yqLv0QziE8xiCriXph7ST1pWdKcuSu9TD6b8mKzBVsSi9bVm/Dy4XuSernOr0Z0iWTe7NUJnv4tTti+EQ0pMil3A+fK4h9aKvzHea6vXBUJd7UORQh4e5xxA25rZ1fV4up2p9rXIJ5cG7gDlxMTg8+SuPVWy7EAs1dUbk/XNA69CvoG52i/+1H4aP5VSfVyi3jL5r9iJg1LldrDOy6A/XXzOWW8hsLucYzjPNe2e9JT8/9vuiWiOVOr15//TRH7VzwPBaKM6DytIx3p1FVt5QKngu4JMhVf9Wf7sgfxvhVU6jTI46lMw5RTk100KmZr5x6eud+SxEJkunTKYiNRtHNOhG7I46ULfg/07uAda9HcrfpeTfiPV/f7YHtX2yHiaLzpmDFtXFlP2wp2g9erf3Z8skxru8fLe3CTV4Fn5ItXjP9x61x7u+zULfyr9Bwmv9rkpVrxCRLJ/u2XXVgR4UT944B2S+5xvRXXrOYvHNs1HA2aK5P7Us0YI+kW8WdErxHUfKLs3ViT6rx5MDvGHfzvNDkfol1bW1ncc9V+J8VMr5aIlzRtopcnjjfFZ9N6uMy47DpZfnzk51u5TibHueV50b3S3lwnfQq/GDCvzAkbJB2n9JkfBUVWfz5hU051om94R1rEwmkvWAcWY7H+/qyPa0gtdpQP/vlKLvOJLugywdyn9HdMI2JXqJ1s0eSc83HaszqbcrpDXfo2Sa9q6jnQfm1mJ1lj7R844zF+avESxoPeBuam4mStmTuIROh24ExkSJ70hWU+oL8vRFCgNREj1Tn11e21z+OxCrhZOt9c1fr994sLX22bb+GxEfqh+rn6prYB9/oz6DkThSxwr/VsQf1B/Vn7Z/v/3n7b9s/5WbfnBBY36kCv+2//5fZBjmAw==</latexit> d ⇠ 102 Multi-omics integration: next frontier … Accessability Gene Expression A B C A B C A (n) A (n) A (n) A (n) A (n) Protein A (n) RNA Chromatin Protein Abundance A D E Gene Expression A B C D E F G RNA DNA Protein Transcription Translation A (n) A (n) A (n) A (n) A (n) Protein A (n) RNA Chromatin Protein Abundance A D E Gene Expression A B C D E F G ATAC-seq Different omics spaces: RNA-seq CITE-seq <latexit sha1_base64="v4HVUJlrkr1NIl1MUAlx55L31yI=">AABDDXictVzbchu5EYU3t7VzWW/ymJdJZKe8KUeRFFeSqq1UrXWxrLVsyyYle3dpu3gZ0rSHHJpDyheuviEfkNfkE/KWSh7zDfmD5CkP+YH0BRhgSMw0RnE8ZQkD4nQ3GkCjuwGqM0mG2Wxj4x8XPvjGN7/17e98ePHSd7/3/R98dPnjH55k6XzajY+7aZJOH3faWZwMx/HxbDhL4seTadwedZL4UeflDn7+6DSeZsN03Jy9ncRPRu3BeNgfdtszqHp2+aMrvaiVDUfR5sbTxY2zK88ur22sb9C/aLWwqQtrSv87Sj+O/qNaqqdS1VVzNVKxGqsZlBPVVhk8X6lNtaEmUPdELaBuCqUhfR6rM3UJsHNoFUOLNtS+hJ8DePtK147hHWlmhO4ClwT+TwEZqauASaHdFMrILaLP50QZa8toL4gmyvYWfnc0rRHUztRzqJVwpmUoDvsyU331W+rDEPo0oRrsXVdTmZNWUPLI6dUMKEygDss9+HwK5S4hjZ4jwmTUd9Rtmz7/J7XEWnzv6rZz9S+S8io8kWro3qc5hbY6JfoRjeYcPmN5EuA8AAqx7iOWXpOuR9T7MbRfQP09eM6oZHTSgWdBtWeVyB14fMgdEbkPjw+5LyIP4fEhD0XkETw+5JFGInZKOvfjG/D48A2R8wN4fMgHIvIhPD7kQxF5Ao8PeSIiv4THh/xSRN6Cx4e8JSLvwOND3hGRTXh8yKaIPIbHhzwWkXvw+JB7Glm+UqfwpERnKKzKm1Au8kBLkUDNTVG+bbKOPux2wJrulmDlVb0Lv/3Y3QCdxiXYvYB51y/ByjNvH2ykHyvbotu0m/iwt0XsAcwAP/ZAxH6uXpRgPw9YaS9LsPJaO4R2fqxsfe/Cmx97V8Teg5IfK+9R96HGj70fsGNMSrBHIvaBelWCDbH60xKsbPcbYFf8WHmfakJ7PzbEms5LsLI9PQEPxo+Vd6tHUOvHPhKxj9WbEuxjEfsFWHc/9ouAHfZdCdbssZdoBxmQPxLDiq2i1s5XJZYmQK0t8E/yvSUh37gD9RJmkGMGhBmJiP0csR+IOMwRh8FyZbkdzcjflbk0ckQjENHJ9yYszcT2vbw9lpIAxG6O2F1CVHmkONamL6fkXZgaCTnLdy4shfQpze03lmI9H6otr0HcLyB4bj+nmX+doiWMoFBTVdSe53s8IyN6r0K8pujN9NLwkHGz3Cq4qDciquNBdUTUWw/qrYiae1BzEXXqQZ2KKLvyXVwrYAZY/eNYLOiNZwD7yOVPBF7BTdh1bsMajWD+HIEX+JBq7sPvBsXe0lMlGUbzuE9iluNJwRJPobRQa1Bvo8Jdiq8TWmExSMYt7+sYH98wt7HQa46t8Fm+k0d5xiSczpDkGeR00FuMaD3Vo3OHas7Iu+NSPfztfN2bUj38Hmn8jLx4LtXDz7T0s3PI3tTY5jmwDVhNE619W65Lg/MvTMOUL9GuixYXR3Wk5wzSe1OT/oEemYNzjMsOlVg/tlyPRub0Lyv0rw4Nq+fM0XM9Kug9sddrSlHtnox13GvLdWVIaRcdaznsW92RwTY9PTKmXI/GEXhcOxRzL5xy3dk7yXtjy/VonCjOe56RJ2/K9WgM6J31Ycv1aGC2pa3jfFuua9lRAxw723Jdqz6mLDDmgHjOc431iqbkJ801tSH5B9XZGtfnX93HMGfzNI8RqilZ37acTiffy6olMv5CDFZtVlMO9C/mjg9WpLFQW2J8xTLMCvv7Kh27x6PmD0GLEax+PgOQcuYJSGhyEmi9E6C4KUZdxZ4Z3JaIw1nSX0K1dO1M9BYtX84aFeueUa0Ul9neWj22yF5nNPcm5BMekmYlPRyWjnAZRUlDhwUNyfTq6O6dXq9F7W+IuMkSYpLPtC6dCPFJWnWc6tN6w9HxVX3KM4OHz3zs/MVsc19bG4x5UrJFKEsVT7edySO5dbivXlc2x82fRTSiaK9OyWoM6UQqE6NQky1mb3xB75b2MZ3JIQ+m0YVxjDSVieJTM8yiYz49Iovq2luJN+rLZOi4nJHVNfa4Gj1w0AMPun6MswM7xj0oNSFmOIa3ZkCUcynXVUoan6pf5KejKY1gdUSfFCykocH2Ji5YyKoo+3mBymtA42zgKD2cxjIdg2+tUJKjfp88NnYtWv6rdHJrzrfbNMfLZ3N5JqZHXLeIa0Srhk91+W2ZA0uw8H6yRf5rdS+RXx2OaEMlrk8dzqyXMZ34xxTBTsgzTmi1Sauj2NrNTy1/YjgdKXN2jqfZKVnIiOxfBPtTSnMyov/u3QFzgs4WISEbGWJ3hrl34/N1huIcs37cUPGtBjvfYrJlc+Jv6LqrK6O5yBED7wNnS3Pb6OSQfMGYuE61dbdru3r3QaS9J+HOEqZo58o14v8J/TT/zTxZW5kRqGEcgUzbOt94pBSzoI7atMtX2yDT1pXySi7DUy213f+sTFcKku1SxIXy4G7dA85demdeOEumJHe20ob30apsLlKeLOkRe9unKJ7t/kDvwCj3ddol12jNtWiWDGAWzPIowrSVssjLfKt5FamH0c7+L9StrotaQ4qRshlc1pCU348pWnOlTGBW8/x9SavJr/XpUqtqPmOaiyNnLX8NtT+Bn0Zu8x5Gp1OwCts0B5iCfbMa4ZpopUUYr+0CLzMzDS37bvnZOWlauTXnia/ZutkY+7Q2lSOaNW901sKUz0PjhUPjRaAOm3TWaLVo6o0leibGFk19WhnKrw63Zg3Kc5Gy7JEZ1DBASjeWCqPaE6nKMb5BvRNpbYi02rBa3dMAd82HIP1rfXl1f53v7pG6Rb5Nlzwwjl96tEqH5HOZ2upIjSkg5xvavrqrv0U1yL1DFhQp8z1OXDF86tSl5yyX9Gd6Z0vJzluLYO4tvdZtjI1tUflXK8gRrYmM1qVB3KAWsZbflSNaskjrjs8RUea/TT4V+x3VMbPb2o5JVPAnbLzJq8ry4khhTPqXMm8HK9HrgRO/RhQTzrV33QFa9UcYKTDGZBL8nmVGI4S7HJ8ksEfbIfu5aqf4FG/sSLROUi/U7wJsDEe9dq67c8v02PTt59AStW5H3ddC5pcEc5T4nedEr0272kj7qIul9/PRautdrvhepYf5El+rjzm1cSMLG+UVMS31aTAXlqgeF8aEcKnXizry15O8jsx8OhVK2bQ2lIuZBrYxzyleku6BIsLn3V3zenOfCP3orNDrENalxjUSJczGpTo/4FpazEpdXNmHuPZi5W6UODtR2U5hqLu7hbXfbCFjsn6JknI23NqVvVWIUuQsDFPoKr7RWxYfujQ/hQd/RsoXHRqOIbnDBvi3N9WO2nsPtyFe6TJnNCOqQVvQW4q927qfxRbVOnrlUHfph3AI5zEEXUvSD2knrSs7U5Yld6mH039NVmCqYlF627J+H1wuck9WOdXpz5Asm9yboTLfxanbF8MhpCdFLuF8+FxD6kVfme801euDoS73oMihDg9zjyFszG3r+rxcTtX6WuUSyoN3AXPiYnB48lceq9h2IRZq6ozI++eA1qFfQd3sFv9rPwwfy6k+r1BuGX3X7EXAqHO7WGdk0R+uv2Yst5DZXM4xnGea9856S35+7PdFtUYqdXrz/umjP2rngOG1UJwHlaVjvDuLrLyhVPBcwCdDqv6t/nZB/jbCq5xGmRx1KJlzinJqpoVMzXzj0tc781mITJZOmUxFajaOaNCN2B11oG7B/53cA6x7O5S/S8m/Eev//mwPavtkPUwWnTMHLaqLKfthT9F69G7vz5ZJjHd5+W5vE2rwLPyQavGe7z1qj3d9m4W+lX+DhNf6XZWqXiEiWT7ds+uqAz0onrxxDsh8zzeiu/ScxeKbZ6OAs0Vzf2pZogV9It8s6JTiO46UXZqrE31WjycHeMO+neeHIvVLqmtrO497rsT5qJTz0RLnjLRT5PDG+az6blYZlx2HSy/PnZ3qdinF2fY8rzo3ulvKhe+gV+MHFfiBI2WDtP+SIuGpqs7mzStozrVM7gnrWJlMJOsB48x2Pt7Vke1pBa/TgP7fKUXfcSTdB1k6lP+O6IRtSvQSrZs9kp5vOlZnUm9XSGu+R8k07V1HOw/MrcXqLH2i5x1nLsxfI1jQesDd1NxMlLIncQmdDt0IjIkS35GsptQX5OmLFAaiJHqmPru8trn8dyBWCydb65u/Xr/xYGvts239NyI+VD9WP1XXwD7+Rn0GI3GkjhX+rYg/qD+qP23/fvvP23/Z/is3/eCCxvxIFf5t//2/6U7mBQ==</latexit> d ⇠ 104 <latexit sha1_base64="HcQbgOHgcxyfyhjs9lnWFC1z8NU=">AABDDXictVzbchu5EYU3l107l/Umj3mZRHbKm/I6kuJcqrZStdbFsta0LZuU7N2l7eJlRNMecmgOKV+4+oZ8QF6TT8hbKnnMN+QPkqc85AfSF2CAITHTGMXrKUsYEKe70QAa3Q1Q3UkyzGbr6/889963vv2d777/wfkL3/v+D3744cWPfnSUpfNpLz7spUk6fdTtZHEyHMeHs+EsiR9NpnFn1E3ih90X2/j5w5N4mg3TcWv2ZhI/HnUG4+HxsNeZQdXTix9e6kftbDiKNtafLH59eunpxbX1a+v0L1otbOjCmtL/DtKPov+qtuqrVPXUXI1UrMZqBuVEdVQGz1dqQ62rCdQ9Vguom0JpSJ/H6lRdAOwcWsXQogO1L+DnAN6+0rVjeEeaGaF7wCWB/1NARuoyYFJoN4Uycovo8zlRxtoy2guiibK9gd9dTWsEtTP1DGolnGkZisO+zNSx+h31YQh9mlAN9q6nqcxJKyh55PRqBhQmUIflPnw+hXKPkEbPEWEy6jvqtkOf/4taYi2+93Tbufo3SXkZnkg1de/TnEJHnRD9iEZzDp+xPAlwHgCFWPcRS69I1yPq/RjaL6D+LjynVDI66cKzoNrTSuQ2PD7ktojcg8eH3BORDXh8yIaIPIDHhzzQSMROSed+fBMeH74pcr4Pjw95X0Q+gMeHfCAij+DxIY9E5Jfw+JBfisib8PiQN0XkbXh8yNsisgWPD9kSkYfw+JCHInIXHh9yVyPLV+oUnpToDIVVeQPKRR5oKRKouSHKt0XW0YfdCljTvRKsvKp34LcfuxOg07gEuxsw745LsPLM2wMb6cfKtugW7SY+7C0Ruw8zwI/dF7Gfq+cl2M8DVtqLEqy81hrQzo+Vre8dePNj74jYu1DyY+U96h7U+LH3AnaMSQn2QMTeVy9LsCFWf1qCle1+E+yKHyvvUy1o78eGWNN5CVa2p0fgwfix8m71EGr92Ici9pF6XYJ9JGK/AOvux34RsMO+LcGaPfYC7SAD8kdiWLFV1Dr5qsTSBKh1BP5Jvrck5Bt3oV7CDHLMgDAjEbGXI/YCEY0c0QiWK8vtaEb+rsylmSOagYhuvjdhaSa27+ftsZQEIHZyxM4SosojxbE2fTkh78LUSMhZvnNhKaRPaW6/sRTr+VBteQ3iXgHBc/sZzfyrFC1hBIWaqqL2LN/jGRnRexXiFUVvppeGh4yb5VbBRb0WUV0Pqiui3nhQb0TU3IOai6gTD+pERNmV7+LaATPA6h/HYkFvPAPYRy5/IvAKbsCucwvWaATz5wC8wAdUcw9+Nyn2lp4qyTCax30SsxyPC5Z4CqWFWoN6GxXuUHyd0AqLQTJueU/H+PiGuY2FXnNshU/znTzKMybhdIYkzyCng95iROupHp3bVHNK3h2X6uFv5evelOrhd0njp+TFc6kefqaln51B9pbGts6AbcJqmmjt23JdGpx/YRqmfIF2XbS4OKojPWeQ3uua9Pf1yOyfYVy2qcT6seV6NDKnf1mhf3VoWD1njp7rUUHvib1eU4pq92Ss415britDSrvoWMth3+qODLbp65Ex5Xo0DsDj2qaYe+GU687eSd4bW65H40hx3vOUPHlTrkdjQO+sD1uuRwOzLR0d59tyXcuOGuDY2ZbrWvUxZYExB8RznmusVzQlP2muqQ3JP6jO1rg+/+o+hjmbJ3mMUE3J+rbldLr5XlYtkfEXYrBqs5pyoH8xd3ywIo2F2hTjK5ZhVtjfV+nYPR413wAtRrD6+QxAypknIKHJSaD1ToDihhh1FXtmcJsiDmfJ8RKqrWtnordo+XLWqFj3lGqluMz21uqxTfY6o7k3IZ+wQZqV9NAoHeEyipKGGgUNyfTq6O6tXq9F7a+LuMkSYpLPtB6dCPFJWnWc6tN609HxZX3KM4OHz3zs/MVs87G2NhjzpGSLUJYqnm47k0dy63Bfvapsjps/i2hE0V6dkNUY0olUJkahJlvM3viC3i3tQzqTQx5MowfjGGkqE8WnZphFx3x6RBbVtbcSb9SXydBxOSOra+xxNXrgoAcedP0YZxt2jLtQakHMcAhvrYAo50Kuq5Q0PlWf5KejKY1gdUSfFCykocH2Ji5YyKoo+1mByitA42zgKD2cxjIdg2+vUJKjfp88NnYtWv7LdHJrzrc7NMfLZ3N5JqZPXDeJa0Srhk91+W2ZA0uw8H6ySf5rdS+RXx2OaEMlrk8czqyXMZ34xxTBTsgzTmi1Sauj2NrNTy1/YjgdKHN2jqfZKVnIiOxfBPtTSnMyov/u3QFzgs4WISEbGWJ3hrl34/N1huIcs37cUPGtBjvfYrJlc+Jv6LqrK6O5yBED7wOnS3Pb6KRBvmBMXKfautu1Xb37INLek3BnCVO0c+UK8f+Yfpr/Zp6srcwI1DCOQKZtnW88UopZUEcd2uWrbZBp60p5KZfhiZba7n9WpksFyXYo4kJ5cLfuA+cevTMvnCVTkjtbacP7aFU2FylPlvSIvT2mKJ7t/kDvwCj3Vdol12jNtWmWDGAWzPIowrSVssjLfKt5FamH0c6+EepW10WtIcVI2Qwua0jK78cUrblSJjCref6+oNXk1/p0qVU1nzHNxZGzlr+G2p/CTyO3eQ+j0y1YhS2aA0zBvlmNcE200iKM11aBl5mZhpZ9t/zsnDSt3JqzxNds3WyMfVKbygHNmtc6a2HKZ6Hx3KHxPFCHLTprtFo09cYSPRVji5Y+rQzlV4dbqwbluUhZ9sgMahggpRtLhVHti1TlGN+g3oq01kVaHVit7mmAu+ZDkP61vry6v85390jdJN+mRx4Yxy99WqVD8rlMbXWkxhSQ83VtX93V36Ya5N4lC4qU+R4nrhg+derRc5pL+nO9s6Vk561FMPeWXuk2xsa2qfyrFeSI1kRG69IgrlOLWMvvyhEtWaRrjs8RUea/Qz4V+x3VMbPb2o5JVPAnbLzJq8ry4khhTPqXMm/7K9HrvhO/RhQTzrV33QVa9UcYKTDGZBL8nmVGI4S7HJ8ksEfbJfu5aqf4FG/sSHSNpF6o3wfYGI567Vx355bpsenbL6Alat2Ouq+FzC8J5ijxO8uJXod2tZH2URdL72ej1dG7XPG9Sg/zJb5WH3Nq40YWNsorYtrq02AuLFE9LowJ4VKvF3Xkryd5HZn5dCqUsmltKBczDWxjnlG8JN0DRYTPu7vi9eY+FvrRXaHXJaxLjWskSpiNS3V+wLW0mJU6v7IPce35yt0ocXaisp3CUHd3C2u/2ULGZP0SJeVsuLUre7sQpchZGKbQU3yjtyw+dGl+Cg/+jJQvOjQcQ3KHTfBvb6httfsObkO81GXOaEZUg7agvxR7d3Q/iy2qdfTSoe7SD+EQzmMIupakH9JOWld2pixL7lIPp/+KrMBUxaL0tmX9Prhc5J6scqrTnyFZNrk3Q2W+i1O3L4ZDSE+KXML58LmG1ItjZb7TVK8PhrrcgyKHOjzMPYawMbet6/NyOVXra5VLKA/eBcyJi8HhyV95rGLbhVioqTMi754DWofjCupmt/h/+2H4WE71eYVyy+i7Zs8DRp3bxToji/5w/TVjuYXM5nKO4TzTvHfWW/LzY78vqjVSqdObd08f/VE7BwyvheI8qCwd491ZZOUNpYLnAj4ZUvUf9fdz8rcRXuY0yuSoQ8mcU5RTMy1kauYbl77emc9CZLJ0ymQqUrNxRJNuxG6rfXUT/m/nHmDd26H8XUr+jVj/92f7UHtM1sNk0Tlz0Ka6mLIf9hStT+/2/myZxHiXl+/2tqAGz8IbVIv3fO9Se7zr2yr0rfwbJLzW76hU9QsRyfLpnl1XXehB8eSNc0Dme74R3aXnLBbfPBsFnC2a+1PLEi3oE/lmQbcU33Wk7NFcneizejw5wBv2nTw/FKlfUl1H23nccyXOB6WcD5Y4Z6SdIofXzmfVd7PKuGw7XPp57uxEt0spzrbnedW50Z1SLnwHvRo/qMAPHCmbpP0XFAlPVXU2b15Bc65lck9Yx8pkIlkPGGd28vGujmxPKnidBPT/din6tiPpHsjSpfx3RCdsU6KXaN3skvR807E6k3qrQlrzPUqmae862nlgbi1WZ+kTPe84c2H+GsGC1gPupuZmopQ9iUvodOlGYEyU+I5kNaVjQZ5jkcJAlETP1KcX1zaW/w7EauFo89rGb65dv7+59tmW/hsRH6ifqJ+pK2Aff6s+g5E4UIcK/1bEH9Wf1J+3/rD1l62/bv2Nm753TmN+rAr/tv7xPyv45gY=</latexit> d ⇠ 105
to identify a new ubpopulation of CD8 TEM cells8. Spatial transcriptomics profiled across time have allowed to study development at unprecedented resolution9. CD103+ CD8+ TEM CD8+ TEM Zebrafish embryos along development ing of gene expression and ns enabled to identify a new of CD8 TEM cells8. Spatial transcriptomics profiled across time have allowed to study development at unprecedented resolution9. CD103+ CD8+ TEM Zebrafish embryos along development The joint pro fi ling of gene expression and surface proteins enabled to identify a new subpopulation of CD8 TEM cells. [Hao et al., Cell, 2021] Spatial transcriptomics pro fi led across time have allowed to study development at unprecedented resolution. [Liu et al., Developmental Cell, 2022] Examples of recent biological discoveries
find … Straight flow easier to discretize! Optimizing: going deeper with ResNet Still a Wasserstein flow! Global convergence open … Raphaël Barboni Conclusion
find … Straight flow easier to discretize! Optimizing: going deeper with ResNet Still a Wasserstein flow! Global convergence open … Raphaël Barboni Transformers: understanding in-context learning Transformers are universal! Nothing else is known … Takashi Furuya Conclusion