When edge meets learning: adaptive control for resource-constrained distributed machine learning - Leung,KK; Wang,S; Tuor,T; Salonidis,T; Makaya,C; He,T; Chan,K (IEEE, 2018)
Emerging technologies and applications includingInternet of Things (IoT), social networking, and crowd-sourcinggenerate large amounts of data at the network edge. Machinelearning models are often built from the collected data, to enablethe detection, classification, and prediction of future events.Due to bandwidth, storage, and privacy concerns, it is oftenimpractical to send all the data to a centralized location. In thispaper, we consider the problem of learning model parametersfrom data distributed across multiple edge nodes, without sendingraw data to a centralized place. Our focus is on a generic classof machine learning models that are trained using gradient-descent based approaches. We analyze the convergence rate ofdistributed gradient descent from a theoretical point of view,based on which we propose a control algorithm that determinesthe best trade-off between local update and global parameteraggregation to minimize the loss function under a given resourcebudget. The performance of the proposed algorithm is evaluatedvia extensive experiments with real datasets, both on a networkedprototype system and in a larger-scale simulated environment.The experimentation results show that our proposed approachperforms near to the optimum with various machine learningmodels and different data distributions.