 |
Universität Augsburg
|
|
|
Universität Augsburg
|
|
Professor Dr. François-Xavier Vialard
spricht am
Donnerstag, 3. Juli 2025
um
15:45 Uhr
im
TUM, Boltzmann-3, Garching, Raum 03.08.011, Etage 3
über das Thema:
| Abstract: |
| In this talk, we study the training of a key architecture of neural networks called, residual neural networks. This architecture has been introduced ten years ago and is now adopted in most of all deep learning architecture since it solved the problem of vanishing gradient. We will show that, in the infinite depth limit, called neural ODE, and the infinite width limit (know as mean field limit), Residual Networks enjoy a relatively nice optimization landscape, in particular in the linear parametrization setting. In the general setting, we use conditional optimal transport to understand the training of these ResNets, showing a local convergence result, based on a local Polyak-Lojasiewicz condition. The second part of the presentation presents the optimization of single-hidden layer in the context of mean-field regime under the so-called VarPro optimization setting. This optimization setting can be seen as the limit of the two-timescale regime where the outer layer is optimized at a faster pace. We show the connection of this VarPro regime with the ultra-fast diffusion PDE equation. |
| Hierzu ergeht herzliche Einladung. |
| Prof. Gero Friesecke |