We address Gaussian process regression of frequency or parametric responses, where the system is either given as a state space model or a partial differential equation in the frequency domain. In both cases, we approximate the dependency of a single output quantity on the frequency variable (or another parameter) from data. We adopt a complex-valued setting, which requires the specfication of a dedicated covariance and pseudo-covariance kernel. Inspired from work in Bayesian system identification, these kernels are of rational type and constructed by taking the Fourier transform of fast-decaying non-stationary kernels on the positive real line, which enables the approximation of smooth functions. We compare the kernel methods against other available rational and polynomial approximation methods and relate the setting to a recently introduced Gaussian process regression framework for parametric differential equations. We also outline the possibility to adaptively select new sampling points and to include low-order rational terms in the mean function of the Gaussian process. The complex-valued regression algorithm is realized in a non-intrusive way, which allows to employ existing real-valued Gaussian-process regression tools. The efficiency of the algorithm will be illustrated with several numerical results. In particular, we consider discrete circuit models, simple analytical functions and Helmholtz-type problems, which are used to simulate wave propagation.