Hardhat is a class of indispensable equipment for workers to enter construction sites. Considering that many accidents occurred at the construction sites are related to the violations of rules by workers, detection of workers whether wearing hardhats is particularly significant for production safety. However, due to the complex environment of the construction sites, it is a challenging issue to accurately detect whether workers are wearing hardhats. In this paper, a practical detection model with high detection accuracy is proposed. Firstly, after revising and supplementing the existing hardhat- wearing dataset, a large Hardhat-Head dataset is constructed, which consists of 11172 images, including 23766 head instances wearing hardhats, annotated as hat class, and 124928 head instances not wearing hardhats, annotated as person class. Secondly, in contrast to the commonly multiple-stage methods based on pedestrian detection or face detection, this paper adopts a higher accuracy and faster one-stage method to perform hardhats wearing detection. Finally, by training and testing four models modified based on the Cascade RCNN algorithm on our constructed Hardhat-Head dataset, the four trained models achieve the highest average precision (AP) value of 92% in the hat class and 94% in the person class, the highest mean AP value reaches 92.9%.