Imperial College London
Portrait Picture

ProfessorPier LuigiDragotti

Faculty of Engineering, Department of Electrical and Electronic Engineering

Professor of Signal Processing
 
 
 
//

Contact

 

+44 (0)20 7594 6192p.dragotti

 
 
//

Location

 

814Electrical EngineeringSouth Kensington Campus

//

Summary

 

Publications

Citation

BibTex format

@inproceedings{Perez-Nieves:2019:10.32470/ccn.2019.1173-0,
author = {Perez-Nieves, N and Leung, VCH and Dragotti, PL and Goodman, DFM},
doi = {10.32470/ccn.2019.1173-0},
publisher = {Cognitive Computational Neuroscience},
title = {Advantages of heterogeneity of parameters in spiking neural network training},
url = {http://dx.doi.org/10.32470/ccn.2019.1173-0},
year = {2019}
}
Download

RIS format (EndNote, RefMan)

TY  - CPAPER
AB  - It is very common in studies of the learning capabilities of spiking neural networks (SNNs) to use homogeneous neural and synaptic parameters (time constants, thresholds, etc.). Even in studies in which these parameters are distributed heterogeneously, the advantages or disadvantages of the heterogeneity have rarely been studied in depth. By contrast, in the brain, neurons and synapses are highly diverse, leading naturally to the hypothesis that this heterogeneity may be advantageous for learning. Starting from two state-of-the-art methods for training spiking neural networks (Nicola & Clopath, 2017, Shrestha & Orchard 2018}, we found that adding parameter heterogeneity reduced errors when the network had to learn more complex patterns, increased robustness to hyperparameter mistuning, and reduced the number of training iterations required. We propose that neural heterogeneity may be an important principle for brains to learn robustly in real world environments with highly complex structure, and where task-specific hyperparameter tuning may be impossible. Consequently, heterogeneity may also be a good candidate design principle for artificial neural networks, to reduce the need for expensive hyperparameter tuning as well as for reducing training time.
AU  - Perez-Nieves,N
AU  - Leung,VCH
AU  - Dragotti,PL
AU  - Goodman,DFM
DO  - 10.32470/ccn.2019.1173-0
PB  - Cognitive Computational Neuroscience
PY  - 2019///
TI  - Advantages of heterogeneity of parameters in spiking neural network training
UR  - http://dx.doi.org/10.32470/ccn.2019.1173-0
UR  - https://ccneuro.org/2019/Papers/ViewPapers.asp?PaperNum=1173
UR  - http://hdl.handle.net/10044/1/78173
ER  -
Download