Parameter identifiability of a deep feedforward ReLU neural network

DOI10.1007/S10994-023-06355-4MaRDI QIDQ6188062zbMATH OpenOpenAlexFDO

Authors Joachim Bona-Pellissier, François Bachoc, François Malgouyres

Publication date 11 January 2024

Published in Machine Learning (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2112.12982

zbMATH Keywords

deep learningequivalent parametersparameter recoveryReLU networkssymmetries

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07)

Abstract: The possibility for one to recover the parameters-weights and biases-of a neural network thanks to the knowledge of its function on a subset of the input space can be, depending on the situation, a curse or a blessing. On one hand, recovering the parameters allows for better adversarial attacks and could also disclose sensitive information from the dataset used to construct the network. On the other hand, if the parameters of a network can be recovered, it guarantees the user that the features in the latent spaces can be interpreted. It also provides foundations to obtain formal guarantees on the performances of the network. It is therefore important to characterize the networks whose parameters can be identified and those whose parameters cannot. In this article, we provide a set of conditions on a deep fully-connected feedforward ReLU neural network under which the parameters of the network are uniquely identified-modulo permutation and positive rescaling-from the function it implements on a subset of the input space.

Recommendations

• An embedding of ReLU networks and an analysis of their identifiability
• Neural network identifiability for a family of sigmoidal nonlinearities
• Stable recovery of entangled weights: towards robust identification of deep neural networks from minimal samples
• Affine symmetries and neural network identifiability
• Robust and resource-efficient identification of two hidden layer neural networks

Cites work

• An embedding of ReLU networks and an analysis of their identifiability
• Cryptanalytic extraction of neural network models
• Deep learning and multivariate time series for cheat detection in video games
• Guaranteed Recovery of One-Hidden-Layer Neural Networks via Cross Entropy
• scientific article; zbMATH DE number 710482 (Why is no real title available?)
• Lossless compression of deep neural networks
• Multilinear compressive sensing and an application to convolutional linear networks
• Protect privacy of deep classification networks by exploiting their generative power
• Reconstructing a neural net from its output
• Topological properties of the set of functions generated by neural networks of fixed size
• TRU-NET: a deep learning approach to high resolution prediction of rainfall

Cited in

(3)

• Stable recovery of entangled weights: towards robust identification of deep neural networks from minimal samples
• An embedding of ReLU networks and an analysis of their identifiability
• Polynomial time cryptanalytic extraction of neural network models

This page was built for publication: Parameter identifiability of a deep feedforward ReLU neural network

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6188062)