ProjectMPOL – Mehrphasige probabilistische Optimierer für tiefe Lernprobleme

Basic data

Acronym:
MPOL
Title:
Mehrphasige probabilistische Optimierer für tiefe Lernprobleme
Duration:
7/1/2021 to 6/30/2024
Abstract / short description:
This proposal to SPP 2298/1 proposes to investigate a novel paradigm for the training of deep neural networks. The peculiarities of deep models, in particular strong stochasticity (SNR<1), preclude the use of classic optimization algorithms. And contemporary alternatives, of which there are already many, are wasteful with resources. Rather than add yet another optimization rule to the long and growing list of such methods, this project aims to make substantial conceptual progress by investigating two key ideas: First, leveraging the entire probability distribution of gradients across the dataset (empirically estimated at runtime) to identify algorithmic parameters that the user would otherwise have to manually tune. And second, splitting the optimization process into at least three distinct phases with differing mathematical objectives. The goal is to develop (concretely, as a software library) an optimizer that requires no manual tuning, and achieves good generalization performance without repeated re-starts.
Keywords:
machine learning
Maschinelles Lernen
optimising
Optimierung
Deep Learning
Probabilistik

Involved staff

Managers

Wilhelm Schickard Institute of Computer Science (WSI)
Department of Informatics, Faculty of Science

Other staff

Department of Informatics
Faculty of Science
Wilhelm Schickard Institute of Computer Science (WSI)
Department of Informatics, Faculty of Science

Local organizational units

Department of Informatics
Faculty of Science
University of Tübingen

Funders

Bonn, Nordrhein-Westfalen, Germany
Help

will be deleted permanently. This cannot be undone.