This paper addresses the problem of efficient video streaming to mobile users. A cross-layer optimization of various parameters of the coding and transmission chain (coding parameters, buffer management, AC-layer management) is performed to account for the time-varying nature of the characteristics of the transmitted contents and of the wireless channel. The problem is cast in the framework of Markov Decision Processes (MDP). This formalism provides efficient tools to compute a foresighted control policy maximizing some long term discounted sum of rewards linked to the video quality received by the user. Experimental results illustrate the benefits in terms of average PSNR of this approach compared to a short-term (myopic) policy. The robustness of the proposed control policy to variations of the transmitted contents is also illustrated.