We consider the problem of inferring connectivity from time-series data under the presence of time-dependent common input originating from non-measured variables. We analyze a simple method to filter out the influence of such confounding variables in multivariate auto-regressive models (MVAR). The method learns the parameters of an extended MVAR model with latent variables. Using synthetic MVAR models we characterize where connectivity reconstruction is possible and useful and show that regularization is convenient when the common input has strong influence. We also illustrate how the method can be used to correct partial directed coherence, a causality measure used often in the neuroscience community.