Spoken Dialogue Systems are man-machine interfaces which use spoken language as the medium of interaction. In recent years, dialogue optimization using reinforcement learning has evolved to be a state of the art technique. The primary focus of research in the dialogue optimization domain is to learn some optimal policy with regard to the task description (reward function) and the user simulation being employed. However in case human-human interaction, the parties involved in the dialogue conversation mutually evolve over the period of interaction. This very ability of humans to co-adapt attributes largely towards increasing the naturalness of the dialogue. This paper outlines a novel framework for co-adaptation in spoken dialogue systems, where the dialogue manager and user simulation evolve over a period of time; they incrementally and mutually optimize their respective behaviors.