On training with slope adaptation for feedforward NNs - Mandic,DP; Krcmar,IR (2000)
Relationships between the learning rate η and the slopes β in the tanh activation function for a feedforward neural network (NN) are provided. The analysis establishes the equivalence in the static and dynamic sense between a referent and an arbitrary feedforward NN which helps to reduce the number of degrees of freedom in learning algorithms for NNs.