In this paper, we present and analyze parallel substructuring methods based on conjugate gradient method, a iterative Krylov method, for solving sparse linear systems on GPUs. Numerical experiments performed on a set of matrices coming from the finite element analysis of large scale engineering problems, show the efficiency and robustness of substructuring methods based on iterative Krylov method for solving sparse linear systems in a context of a hybrid multi-core-GPU. © 2016 IEEE.