Novel relationship between computational topology and neural networks

Explainable neural networks through a topological perspective

Previous results

We did a first approach to the effects of datasets to neural networks in [1]. There, the size of datasets is reduced to speed up the training process but keeping the performance of the neural network. Furthermore, recently, we took part in a European project call FET-OPEN [2] devoted to the development of an explainable artificial intelligence inspired in the way the human brain explain its memories and were one of the research lines leaded by our team is precisely knowledge representation and explainability of neural networks.

Objectives

Recently, the extraction of knowledge from artificial neural networks has received a lot of attention from the research community and the problem is attacked from different angles. In this research line, we plan to deploy a new topology-based framework to establish a relationship between the internal knowledge representation and “knowledge primitives”. Following this aim, the main challenges we want to address are to preserve ontological knowledge representation and axiomatization, to provide enoughlevels of knowledge/information abstractionand to ground acquired knowledge to both real-world concepts and physical or geometrical properties. Specific goals are listed below:

1. Develop a new knowledge representation based on simplicial complexes.
2. Study the internal learning from a neural network from a topology point of view.
3. Establish a relationship between the internal knowledge representation and the knowledge primitives.

Simplicial neural networks

Previous results

In [3] we provided an effective proof of the universal approximation theorem through a constructive method to find the weights of a two-hidden-layer feed-forward network which approximates a given continuous function between two triangulable metric spaces. The method only depends on the desired level of approximation to the given function. Our approach is based on the classical result from algebraic topology named simplicial approximation theorem. Roughly speaking, our result is based on two observations: Firstly, triangulable spaces can be “modelled” using simplicial complexes, and a continuous function between two triangulable spaces can be approximated by a simplicial map between simplicial complexes. Secondly, a simplicial map between simplicial complexes can be “modelled” as a two-hidden-layer feed-forward network.

Objectives

In this research line we plan to develop a complete framework of neural networks entirely based on simplicial complexes. To do so, there are two main blocks to study and develop:

1. The definition of a simplicial neural network and the theoretical proof of its properties. Its motivation comes from the classic universal approximation theorem. It ensures the existence of a single hidden layer neural network that can approximate any function as close as desired. However, it is not constructive. Relating this result with the simplicial approximation theorem, it is possible to define an architecture such that we can provide a constructive version of the universal approximation theorem.
2. Application and implementation of this theoretical artillery to extend and facilitate its used by other researchers,as well as test the architecture.

1. Proof the robustness of simplicial neural network architecture to adversarial attacks.
2. Provide a competitive and easy to use implementation of the architecture.
3. Apply the simplicial neural network architecture to real-world problems.

References

1. R Gonzalez-Diaz, E Paluzo-Hidalgo, MA Gutiérrez-Naranjo. Representative datasets for neural networks, Electronic Notes in Discrete Mathematics 68: 89-94 (2019)
2. R Gonzalez-Diaz, MA Gutiérrez-Naranjo, E Paluzo-Hidalgo, et al.Cognition-based introspective topological trustable explainable artificial intelligence. Call: H2020-FETOPEN-2018-2020. Type of action: RIA. Proposal number: 964835. Submitted.
3. R Gonzalez-Diaz, MA Gutiérrez-Naranjo, E Paluzo-Hidalgo. Two-hidden-layer Feedforward Neural Networks are Universal Approximators: A Constructive Approach. Neural Networks. Accepted for publication