Slide 3
Slide 3 text
Feature learning of NN 3
Benefit of feature learning with
optimization guarantee.
β’ [Computation] Suzuki, Wu, Nitanda: βConvergence of mean-field Langevin dynamics: Time and space
discretization, stochastic gradient, and variance reduction.β NeurIPS2023.
β’ [Generalization]
β’ Suzuki, Wu, Oko, Nitanda: βFeature learning via mean-field Langevin dynamics: Classifying sparse parities
and beyond.β NeurIPS2023.
β’ Nitanda, Oko, Suzuki, Wu: βAnisotropy helps: improved statistical and computational complexity of the
mean-field Langevin dynamics under structured data.β ICLR2024.
Especially, we compare the generalization error between
neural networks and kernel methods.
Trade-off: Statistical complexity vs computational complexity
Feature learning Optimization
Neural network β Non-convex
Kernel method Γ Convex