Spatio temporal Memories Theory and Algorithms

Abstract

This thesis is primarily focused on building a systematic theory for spatio-temporal memories based on unsupervised learning and exploring its applications. Also, contributions are made in the area of supervised learning techniques. The self-organizing map (SOM) is a biologically inspired unsupervised learning paradigm which finds numerous applications in learning, clustering and recalling spatial input patterns. SOM is also used as a software tool for visualizing high-dimensional data space by projecting it onto a lower dimensional space, typically two-dimensions and can be considered as a vector quantization (VQ) technique. Since the time SOM was originally published, many variants of it have appeared and used extensively in engineering applications. A notable variant is the neural gas (NGAS) algorithm. Though the SOM and its variants work well in practice, the learning rules in the basic SOM are heuristic. Also, the techniques are well suited for clustering and recall of spatial patterns. However, if the data is spatio-temporal, these techniques fail to learn the temporal dynamics along with the spatial proximities in the input data space. The traditional approach for learning spatio-temporal patterns is to incorporate time on the output space of a SOM along with heuristic update rules that work well in practice. Inspired by the pioneering work of Alan Turing, who used reaction-diffusion equations to explain spatial pattern formation, we develop an analogous theoretical model for building a spatio-temporal memory to learn and recall temporal patterns. Using coupled reaction-diffusion equations, we develop a mathematical theory from first principles for constructing a spatio-temporal SOM (STSOM) and derive an update rule for learning based on the gradient of a potential function. The temporal plasticity effect observed during recall in response to the input dynamics is mathematically quantifield....