We introduce PBG-Net, an object detection system based on an elaborately designed multi-feature deep CNN which works without proposal algorithms. Firstly, PBG-Net aggregates hierarchical features into multi-feature maps and discretizes the output of Conv5 feature map into a set of predicting boxes, namely Predicting Boxes Generation (PBG). Then, PBG-Net crops multi-feature maps via mapping the predicting boxes and handles the outcome into multi-feature concatenation. Finally, we exploit an iterative regression localization model based on a novel overlap loss function and online hard boxes selection. PBG-Net with around 100 boxes and an end-to-end joint training can achieve 74.2% and 71.1% mAP on the detection of PASCAL VOC 2007 and PASCAL VOC 2012 correspondingly at 12 fps on a NVIDIA GTX 1070p GPU, better than the Faster R-CNN counterparts.