Competition

Feed-forward neural networks (aka multilayer perceptrons) have been widely applied to supervised learning problems since the mid-1980s. Over this time, thousands of different datasets have been used in thousands of different experimental studies, with results reported in the literature. This research has helped to fuel tremendous progress in the field. However, documented reproduction of published experimental results has never been attempted in many cases. The availability of computational resources, software libraries and datasets creates the opportunity to attempt to reproduce and even expand on experiments that previously took a large amount of time. In addition, it is possible to run experiments not just to try and locate single best minimizer of the training loss function, but to collect and explore numerous convergence points on a loss landscape, to better understand the properties of problem instances (e.g. in relation to multimodal optimization and exploratory-driven techniques such as quality-diversity search).

The goal of this competition is to challenge researchers to: (a) attempt to reproduce an existing experimental study and report on their findings (including successes, failures and lessons learned); and/or (b) carry out an experimental study on the loss landscape of a neural network training problem instance to reveal new insights (including finding multiple high-quality solutions and points of attraction for different training algorithms).

https://marcusgal.github.io/reproduce_nnexpt_ijcnn2023.html