A Dantzig Selector Approach to Temporal Difference Learning - CentraleSupélec Access content directly
Conference Papers Year : 2012

A Dantzig Selector Approach to Temporal Difference Learning

Bruno Scherrer
Alessandro Lazaric
Mohammad Ghavamzadeh
  • Function : Author
  • PersonId : 868946


LSTD is one of the most popular reinforcement learning algorithms for value function approximation. Whenever the number of samples is larger than the number of features, LSTD must be paired with some form of regularization. In particular, L1-regularization methods tends to perform feature selection by promoting sparsity and thus they are particularly suited in high-dimensional problems. Nonetheless, since LSTD is not a simple regression algorithm but it solves a fixed-point problem, the integration with L1-regularization is not straightforward and it might come with some drawbacks (see e.g., the P-matrix assumption for LASSO-TD). In this paper we introduce a novel algorithm obtained by integrating LSTD with the Dantzig Selector. In particular, we investigate the performance of the algorithm and its relationship with existing regularized approaches, showing how it overcomes some of the drawbacks of existing solutions.
No file

Dates and versions

hal-00749480 , version 1 (07-11-2012)


  • HAL Id : hal-00749480 , version 1


Matthieu Geist, Bruno Scherrer, Alessandro Lazaric, Mohammad Ghavamzadeh. A Dantzig Selector Approach to Temporal Difference Learning. ICML-12, Jun 2012, Edinburgh, United Kingdom. pp.1399-1406. ⟨hal-00749480⟩
338 View
0 Download


Gmail Facebook Twitter LinkedIn More