Research project
From perceptron to perception: psychoacoustically motivated audio reconstruction using learned components
Abstract.
State-of-the-art methods for the reconstruction of degraded audio signals are successful at their
performance. However, they still suffer from perceptually unpleasant or annoying artifacts
coming from the reconstruction process. Only a few recent approaches involved
psychoacoustics to alleviate this disturbing phenomena. Unfortunately, it turns out that the
incorporation of auditory models into current methods it strongly limited. Their use therein is
prevented by their complexity, non-differentiability and non-convexity. Recent results from the
field of deep learning show that functionals can be trained to distinguish between faithful and
implausible audio. Such discriminators come in the form of a neural network, thus being nonlinear
and non-convex, but, most importantly, differentiable. The project aims at using these
discriminators as universal regularizers in algorithms inspired in convex optimization. This will
not only lead to a general reconstruction framework, but also to significant improvements of
perceptual quality in a wide range of audio inverse problems.
Goals.
1. Develop a new theoretical framework for audio reconstruction, involving perceptually motivated neural networks.
2. Investigate the neural discriminator from the point of view of current knowledge of psychoacoustics.
3. Create and analyze trainable models with the discriminator as a loss function.
Keywords:
Signal processing; audio; signal reconstruction; regularization; deep learning; neural network; discriminator; iterative algorithms; auditory modeling; psychoacoustics
Duration:
2023–2025
The team:
-
Signal Processing Laboratory, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno
prof. Mgr. Pavel Rajmic, Ph.D.
Ing. Pavel Záviška, Ph.D.
Ing. Ondřej Mokrý
Ing. Marie Mangová, Ph.D.
doc. RNDr. Vítězslav Veselý, CSc.
The partners:
Funding