Background Clinicians vary markedly in their ability to detect murmurs during cardiac auscultation and identify the underlying pathological features. Deep learning approaches have shown promise in medicine by transforming collected data into clinically significant information. The objective of this research is to assess the performance of a deep learning algorithm to detect murmurs and clinically significant valvular heart disease using recordings from a commercial digital stethoscope platform. Methods and Results Using >34 hours of previously acquired and annotated heart sound recordings, we trained a deep neural network to detect murmurs. To test the algorithm, we enrolled 962 patients in a clinical study and collected recordings at the 4 primary auscultation locations. Ground truth was established using patient echocardiograms and annotations by 3 expert cardiologists. Algorithm performance for detecting murmurs has sensitivity and specificity of 76.3% and 91.4%, respectively. By omitting softer murmurs, those with grade 1 intensity, sensitivity increased to 90.0%. Application of the algorithm at the appropriate anatomic auscultation location detected moderate-to-severe or greater aortic stenosis, with sensitivity of 93.2% and specificity of 86.0%, and moderate-to-severe or greater mitral regurgitation, with sensitivity of 66.2% and specificity of 94.6%. Conclusions The deep learning algorithm's ability to detect murmurs and clinically significant aortic stenosis and mitral regurgitation is comparable to expert cardiologists based on the annotated subset of our database. The findings suggest that such algorithms would have utility as front-line clinical support tools to aid clinicians in screening for cardiac murmurs caused by valvular heart disease. Registration URL: https://clinicaltrials.gov; Unique Identifier: NCT03458806.