Imperial College London
Portrait Picture

ProfessorDenizGunduz

Faculty of Engineering, Department of Electrical and Electronic Engineering

Professor in Information Processing
 
 
 
//

Contact

 

+44 (0)20 7594 6218d.gunduz Website

 
 
//

Assistant

 

Ms Joan O'Brien +44 (0)20 7594 6316

 
//

Location

 

1016Electrical EngineeringSouth Kensington Campus

//

Summary

 

Publications

Citation

BibTex format

@article{Somuyiwa:2018:10.1109/JSAC.2018.2844985,
author = {Somuyiwa, S and Gunduz, D and Gyorgy, A},
doi = {10.1109/JSAC.2018.2844985},
journal = {IEEE Journal on Selected Areas in Communications},
pages = {1331--1344},
title = {A reinforcement-learning approach to proactive caching in wireless networks.},
url = {http://dx.doi.org/10.1109/JSAC.2018.2844985},
volume = {36},
year = {2018}
}
Download

RIS format (EndNote, RefMan)

TY  - JOUR
AB  - We consider a mobile user accessing contents in a dynamic environment, where new contents are generated over time (by the user’s contacts), and remain relevant to the user for random lifetimes. The user, equipped with a finite-capacity cache memory, randomly accesses the system, and requests all the relevant contents at the time of access. The system incurs an energy cost associated with the number of contents downloaded and the channel quality at that time. Assuming causal knowledge of the channel quality, the content profile, and the user-access behavior, we model the proactive caching problem as a Markov decision process with the goal of minimizing the long-term average energy cost. We first prove the optimality of a threshold-based proactive caching scheme, which dynamically caches or removes appropriate contents from the memory, prior to being requested by the user, depending on the channel state. The optimal threshold values depend on the system state, and hence, are computationally intractable. Therefore, we propose parametric representations for the threshold values, and use reinforcement-learning algorithms to find near-optimal parametrizations. We demonstrate through simulations that the proposed schemes significantly outperform classical reactive downloading, and perform very close to a genieaided lower bound.Index Terms—Markov decision process, proactive content caching, policy gradient methods, reinforcement learning.
AU  - Somuyiwa,S
AU  - Gunduz,D
AU  - Gyorgy,A
DO  - 10.1109/JSAC.2018.2844985
EP  - 1344
PY  - 2018///
SN  - 0733-8716
SP  - 1331
TI  - A reinforcement-learning approach to proactive caching in wireless networks.
T2  - IEEE Journal on Selected Areas in Communications
UR  - http://dx.doi.org/10.1109/JSAC.2018.2844985
UR  - https://ieeexplore.ieee.org/document/8374824
UR  - http://hdl.handle.net/10044/1/59220
VL  - 36
ER  -
Download