Mammogram plays an essential role in reducing breast cancer deaths by detecting cancer early. In this study, Convolutional Neural Networks (CNN) based Faster R-CNN model was applied to detect mass and calcification in breast cancer effectively. Mammography dataset contains public dataset including DDSM, INbreast, and BCD, as well as private dataset, which was obtained from Shenzhen People’s Hospital, China. Final detection result for public dataset was 0.804 in Average Precision (AP), and 0.975 in recall for mass detection, and 0.686 AP and 0.925 recall in calcification detection. Result for private dataset was higher with 0.902 AP and 0.978 recall for mass detection, and 0.605 AP and 0.834 recall for calcification detection.