Supervised Topological Maps (2008.06395v3)

Abstract: Controlling the internal representation space of a neural network is a desirable feature because it allows to generate new data in a supervised manner. In this paper we will show how this can be achieved while building a low-dimensional mapping of the input stream, by deriving a generalized algorithm starting from Self Organizing Maps (SOMs). SOMs are a kind of neural network which can be trained with unsupervised learning to produce a low-dimensional discretized mapping of the input space. They can be used for the generation of new data through backward propagation of interpolations made from the mapping grid. Unfortunately the final topology of the mapping space of a SOM is not known before learning, so interpolating new data in a supervised way is not an easy task. Here we will show a variation from the SOM algorithm consisting in constraining the update of prototypes so that it is also a function of the distance of its prototypes from extrinsically given targets in the mapping space. We will demonstrate how such variants, that we will call Supervised Topological Maps (STMs), allow for a supervised mapping where the position of internal representations in the mapping space is determined by the experimenter. Controlling the internal representation space in STMs reveals to be an easier task than what is currently done using other algorithms such as variational or adversarial autoencoders.