Psoriasis is a chronic skin disease which occurs to 2%–3% of the world’s entire population. If treated properly, patients can still maintain a relatively high quality of life. Otherwise, Psoriasis could cause severe complications or even threat to life. Therefore, continuous tracking of severity degree is critical in Psoriasis treatment. However, due to the shortage of dermatologists, it’s hard for patients to receive regular severity evaluation. Furthermore, evaluating the severity degree of Psoriasis is both time-consuming and error-prone which poses a heavy burden for dermatologists. To address this problem, we propose an automatic rating model which measures the severity degree quantitatively based on skin lesion pictures. The proposed rating model applies coarse to fine grained neural networks to evaluate skin lesions from multiple perspectives. According to experimental results, the proposed model outperforms experienced dermatologists.