A Decentralized control of Autonomous Delay Tolerant Networks: Multi Agents Markov Decision Processes Framework
Résumé
We consider a mobile delay tolerant networks (MDTNs) with energy harvesting and wireless energy transfer capabilities. In order to determine energy management policies that will improve network capacity, packet delivery ratio and maximize the system throughput. We consider that a source node seeks to send packets to a destination node. The optimal policy for the source is varies according to its system state, which will guarantee a maximum delivery probability rate. Each mobile source node transmits wirelessly a portion of its energy as a reward to relay mode. Our problem is modeled by decision theory; as a start, we are interested in the MDP, which are used to model and solve such sequential decision problems. In this paper, for each node, we try to optimize a utility depending on a random environment and decisions made by a node. As the MDP formalism reaches its limits when it is necessary to take into account the interactions between the different several nodes, that's why we chose to use the Multi agents Markov Decision Processes (MMDP) which is a MDP with a large space of states and actions. The set of agents are then considered as a single agents whose goal are to compute an optimal attached policy for MDP. To make a realistic analysis of our model, we assume that the policy of the MMDP is applied in a decentralized way, which makes finding optimal control intractable; thus, we will develop several approximations and evaluate their effectiveness.