We present a novel graph-based resource allocation and interference coordination framework to mitigate downlink interference in small cell networks. In our scheme, two types of clusters from the original network are generated, namely cell-cluster and user-cluster. We first divide small cells into cell-clusters according to their neighboring relationships in the network and examine the network morphology under different neighborhood bounds. Then we develop another graph clustering scheme to group the users in each cell-cluster into user-clusters with minimum intra-cluster interference. Finally, we employ a proportional fair scheduling scheme to assign subchannels among the user-clusters and allocate power using water-filling method. The simulation results show that our method achieves a significant improvement in spectral efficiency compared to previous works under both co-channel and orthogonal deployments.