In this paper, we propose a multi-atlas-based framework for labeling hippocampus regions in the MR images. Our work aims at extending the random forests techniques for better performance, which contains two novel contributions: First, we design a novel strategy for training forests, to ensure that each forest is specialized in labeling the certain sub-region of the hippocampus in the images. In the testing stage, a novel approach is also presented for automatically finding the forests relevant to the corresponding sub-regions of the test image. Second, we present a novel localized registration strategy, which further reduces the shape variations of the hippocampus region in each atlas. This can provide better support for the proposed sub-region random forest approach. We validate the proposed framework on the ADNI dataset, in which atlases from NC, MCI and AD subjects are randomly selected for the experiments. The estimations demonstrated the validity of the proposed framework, showing that it yields better performances than the conventional random forests techniques.