Image contrast distortion is a common type of distortion in digital images. However, there is almost no research on the no-reference image quality assessment (NR-IQA) algorithm for image contrast. Therefore, we propose a histogram-based NR-IQA algorithm for contrast distorted images. Firstly, we analyze the image sequence with gradually changing contrast, and the image features are extracted from two aspects: objective statistical attribute, subjective perception attribute. And then we propose three statistical features, including image local contrast, histogram shape and image brightness, which can describe the image quality more simply and intuitively. Furthermore, we introduce the Just noticeable difference (JND) model, which makes the proposed algorithm have a higher matching degree between the human vision system (HVS) and the objective features in the algorithm. Finally, the support vector regression (SVR) is utilized to obtain the mapping relationship between the quantified features and the subjective scores to predict the quality of contrast distorted images. The outstanding performance of the proposed algorithm have been proved on CSIQ, TID2013 and CCID2014 databases.