This paper addresses the distributed computation of exact, non-asymptotic confidence regions for the parameter estimation of a linear model from observations at different nodes of a network of sensors. If a central unit gathers all the data, the sign perturbed sums (SPS) method proposed by Csáji et al. can be used to define guaranteed confidence regions with prescribed confidence levels from a finite number of measurements. SPS requires only mild assumptions on the measurement noise. This work proposes distributed solutions, based on SPS and suited to a wide variety of sensor networks, for distributed in-node evaluation of non-asymptotic confidence regions as defined by SPS. More specifically, a Tagged and Aggregated Sum information diffusion algorithm is introduced, which exploits the specificities of SPS to avoid flooding the network with all measurements provided by the sensors. The performance of the proposed solutions is evaluated in terms of required traffic load, both analytically and experimentally on different network topologies. The best information diffusion strategy among nodes depends on how structured the network is.