Recently, deep learning (DL) demonstrated capable to identify atrial fibrillation (AF) from electrocardiograms (ECGs) with significant performance. Nevertheless, these models may present an exaggerated self-confidence in their predictions and showing poor calibration in their output probabilities. In addition, such models cannot quantify the uncertainty of the predictions: a fundamental property in the clinical practice. In this study, we compared two DL models with the same architecture, but the second one had the first and last layers trained using a Bayesian approach, i.e., variational inference (VI), allowing the estimate of uncertainty of the predictions. We then compared the models in terms of predictive uncertainty H and Expected Calibration Error (ECE). Our experiments showed that the both models performed very well on the MIT-BIH Atrial Fibrillation dataset (sensitivity and specificity> 96%). The first model proved being i) more confident than the second one (H: 0.006 vs 0.090); and ii) more poorly calibrated (ECE: 0.360 vs 0.028). Despite the computational demand required for using the Bayesian approach in DL, our study demonstrated the importance of quantifying uncertainty of DL-based predictions for AF detection from ECG signals.