Power allocation to satisfy user demands, in the presence of large number of interferers (in a multicellular network), is a challenging task. Further, the power to be allocated depends upon the system architecture, for example upon components like coding, modulation, transmit precoder, rate allocation algorithms, available knowledge of the interfering channels, etc. This calls for an algorithm via which each base sta- tion in the network can simultaneously allocate power to their respective users so as to meet their demands (whenever they are within the achievable limits), using whatever information is available of the other users. The goal of our research is to propose one such algorithm which in fact is universal: the proposed algorithm works from a fully co-operative setting to almost no co-operation and or for any configuration of modulation, rate allocation, etc. schemes. The algorithm asymptotically satisfies the user demands, running simultaneously and independently within a given total power budget at each base station. Further, it requires minimal information to achieve this: every base station needs to know its own users demands, its total power constraint and the transmission rates allocated to its users in every time slot. We formulate the power allocation problem in a system specific game theoretic setting, define system specific capacity region and analyze the proposed algorithm using ordinary differential equation (ODE) framework. Simulations further confirm the effectiveness of the proposed algorithm. We also demonstrate the tracking abilities of the algorithm. In heterogeneous networks, it is hard to expect the various agents to update their algorithms in a synchronous manner. Using two time scale stochastic approximation analysis we study the proposed algorithm operating in a simple example scenario, wherein the heterogeneous agents update (their power profiles) at different speeds. Further, backed by numerical examples (for various generic example scenarios), we show that the algorithm converges to the same power profile, as long as the demands remain same, irrespective of the dis- parities in the operating speeds at different agents.