Incipient fault detection is growing as a challenging and hot topic in industrial and academic areas. It is essential to avoid slight unpermitted changes of a system state that can be aggravated and lead to severe security issues. The main challenge of this problem lies in the fact that tiny changes in the early stage can be blurred with noise and create confusion leading to poor detection performance of typical fault detection methods. To detect subtle deviations buried in noise and cope with the non-Gaussian distributed data condition while keeping with the time series information, a sensitive fault detection methodology combining a specifically tuned Local Mahalanobis Distance (LMD) algorithm and an Empirical Probability Density (EPD) estimation technique is proposed. More specifically, first, a healthy domain estimation is proposed to compute the local Mahalanobis distance with optimally tuned characteristics. To approximate a healthy domain, this work proposes a down-sampling algorithm for anchors generation and a parameter estimation method optimally tuned and based on Generalized Extreme Value distribution (GEV) for the domain margin selection. Subsequently, the EPD cumulative sum technique is applied to the LMD result for improving the detection sensitivity further. The performance analysis based on simulation data shows that our proposal is effective to non-Gaussian data and sensitive for incipient fault detection. A case study based on the Continuous-flow Stirred Tank Reactor (CSTR) further validates the effectiveness of our proposal and highlights its benefit by comparing it with state-of-the-art-based solutions in terms of detection delay, detection probability, false alarm probability, and area under the receiver operating characteristic curve (AUC).