Deep reinforcement learning (RL) – learning optimal behaviors from interaction data, using deep neural networks – is often seen as one of the next frontiers in artificial intelligence. While current RL algorithms do not escape the relentless pursuit of larger models, bigger data and more computation demands, we posit real-world impacts of RL will also stem from algorithms that are relevant in the small data regime, on reasonable computing architectures. RL is at a crossroads where one wishes to retain the versatility and representational abilities of deep neural networks, while coping with limited data and resources. Under such real world limitations, understanding how to preserve algorithmic convergence properties, robustness to uncertainties, worst case guarantees, transferable features, or behavior explanation elements is an open field. Hence, we endeavor to put RL on a diet, in order to reach a better understanding of frugal, life-long RL: its theoretical foundations, the many ways one can compensate for limited data, the sound algorithms one can design, and the practical impacts it can have on the many real world applications where, intrinsically, data is costly and resources are limited, ranging from autonomous robotics to personalized medicine.
0 commit comments