Most graph-based semi-supervised learning methods model the structure of a dataset as a single k-NN graph. Although graph construction is an important task, many existing graph-based methods build a graph from a dataset directly and naively. While the resulting k-NN graph provides relatively a good representation of the dataset,it generally produces inappropriate shortcuts on cluster boundaries. In this paper, we propose a novel approach for modeling and combining multiple graphs with different edge weights to avoid such undesirable behavior. Using the combination of those graphs, we can systematically reduce the effect of noise in conceptually similar fashion to an ensemble approach. Experimental results demonstrate that our approach improves classification accuracy on both benchmark and artificial datasets.