In this paper, an adaptive framework based on histogram separation and mapping for image contrast enhancement is presented. In this framework, the histogram is separated by binary tree structure with the proposed adaptive histogram separation strategy. Generally, histogram equalization (HE) is an effective technique for contrast enhancement. However, the conventional HE usually gives the processed image with unnatural look and artifacts by excessive enhancement. For overcoming this shortage, the adaptive histogram separation unit (AHSU) is proposed to convert the global enhancement problem into local. And for mapping the histogram partitions into more optimal ranges, the exact histogram separation is discussed. Finally, an adaptive histogram separation and mapping framework (AHSMF) for contrast enhancement is presented, and the experimental results show better effectiveness than other histogram based methods.