Jun 7, 2023 · We use dropouts at the inference phase in order to measure the uncertainty of these transformer models. This approach, commonly known as Monte ...

We use dropouts at the inference phase in order to measure the uncertainty of these transformer models. This approach, commonly known as Monte Carlo Dropout ( ...

This study approaches the unexplored issue of uncertainty estimation among three popular and effective transformer models employed in computer vision, ...

We investigate the applicability of uncer- tainty estimates based on dropout usage at the inference stage (Monte Carlo dropout). The series of experiments on ...

Methods such as deep-ensemble and MC-Dropout introduce a heavy computational overhead, especially when applied to LLMs (Malinin and Gales, 2021; ...

This work investigates the applicability of uncertainty estimates based on dropout usage at the inference stage (Monte Carlo dropout) for Transformer-based ...

Missing: transformers analysis

Oct 21, 2024 · Out-of-Distribution Detection (OOD): Use Monte Carlo Dropout to identify when the model encounters inputs outside of its training distribution.

Mar 7, 2024 · Our study includes a sensitivity analysis of crucial hyperparameters like Dropout probability and layer con- figuration, impacting uncertainty ...

Oct 8, 2023 · Monte Carlo Dropout is an effective technique used to estimate uncertainty in deep learning models. Unlike traditional dropout, which is ...