Developing Training Algorithms for Convolutional Neural Networks

During the last decades several visual recognition problems have been investigated. Image processing permits for example face detection, face recognition, facial expression analysis, car detection, optical character recognition, or hand written digit recognition. Neural networks (NN) have been found almost unavoidable in pattern recognition. In fact recognition systems are more efficient when they focus on learning techniques.

LeCun proposed convolutional neural networks (CNN) which are NN with three key architectural ideas: local receptive fields, weight sharing, and sub-sampling in the spatial domain. Networks are designed for the recognition of two-dimensional visual patterns. CNN have many strengths. Firstly, feature extraction and classification are integrated into one structure and are fully adaptive. Secondly, the network extracts two-dimensional image features at increasing dyadic scales. Thirdly, it is relatively invariant to geometric, local distortions in the image.

This project aims to develop fast training algorithms for CNN. It is based on Phung and Bouzerdoum’s CNN library and develops two algorithms: the improved resilient back-propagation and the algorithm for pattern recognition. They are compared on a face versus non-face classification with the two existing algorithms (resilient back-propagation and gradient descent). The programming is done in MATLAB.