Quantitative human evaluations give a much finer description while qualitative human evaluations are more stable, consistent and can be much easier to be obtained. Quantitative assessments have been widely explored, while the interaction between qualitative and quantitative evaluations has barely been exploited. A deep convolutional neural network with multi-task learning framework was utilized to perform quantitative evaluations and qualitative evaluations at the same time. The supervision of qualitative evaluations could help the model overcome the inconsistency existed in quantitative evaluations. Further, multi-task learning gives more information to facilitate the learning of discriminative features to describe image quality. As shown in the experiments, referring to qualitative evaluations has boosted the performance of quantitative assessments and the state of art performance has been achieved.