Imperial College London


Faculty of EngineeringDepartment of Electrical and Electronic Engineering

Professor of Signal Processing



+44 (0)20 7594 6271d.mandic Website




Miss Vanessa Rodriguez-Gonzalez +44 (0)20 7594 6267




813Electrical EngineeringSouth Kensington Campus






BibTex format

author = {Mandic, DP and Chambers, JA},
pages = {7--12},
title = {A nonlinear adaptive predictor realised via recurrent neural networks with annealing},
year = {1999}

RIS format (EndNote, RefMan)

AB - A Minimum Mean Square Error (MMSE) nonlinear predictor based on the Nonlinear Autoregressive Moving Average (NARMA) model is developed for nonlinear and nonstationary signals. This is achieved through modular, nested Recurrent Neural Networks (RNN)s. A Pipelined Recurrent Neural Network (PRNN), which consists of a number of simple small-scale RNN modules with low computational complexity is introduced, which offers an improved nonlinear processing capability within the MMSE prediction framework. Since modules of the PRNN perform simultaneously in a pipelined parallel manner, this leads to a significant improvement in the total computational efficiency of such a NARMA predictor. However, some difficulties encountered with training these networks with the Real Time Recurrent Learning (RTRL) algorithm in that context may be attributed to the fact that the learning rate is maintained constant throughout the computation. To overcome this difficulty, we introduce the learning-rate annealing schedule for the PRNN. This search-then-converge scheme combines the desirable features of the standard RTRL algorithm and traditional stochastic approximation algorithms. Simulation results for nonlinear prediction of speech, which is a nonlinear and nonstationary signal, support our approach.
AU - Mandic,DP
AU - Chambers,JA
EP - 12
PY - 1999///
SN - 0963-3308
SP - 7
TI - A nonlinear adaptive predictor realised via recurrent neural networks with annealing
ER -