This paper considers source localization in a Wireless Sensor Network (WSN) from Received Signal Strength (RSS) measurements. Its aim is to characterize confidence regions (CR) of the search space to which the source parameters (location, reference power, path loss exponent) are guaranteed to belong with a pre-specified probability level. The Leave-out Sign-dominant Correlation Regions (LSCR) method is adapted to this source localization context. LSCR defines CR considering very mild assumptions on the measurement noise corrupting the RSS readings. The confidence level may be arbitrarily chosen and the LSCR approach is valid even when very few measurements are available (non-asymptotic regime). The CR, as defined by LSCR may be non-convex or even non-connected sets. Their characterization is performed via interval analysis, which provides inner and outer approximations of CR. The CR obtained via LSCR are compared to set estimates obtained using (robust) bounded-error estimation techniques, as well as to more classical confidence ellipsoids derived from Cramér-Rao lower bounds.