Unsupervised anomaly detection and localization via bidirectional knowledge distillation

X Wang, Y Wang, Z Pan, G Wang - Neural Computing and Applications, 2024 - Springer
X Wang, Y Wang, Z Pan, G Wang
Neural Computing and Applications, 2024Springer
Abstract Knowledge distillation has demonstrated significant potential in addressing the
challenge of unsupervised anomaly detection (AD). The representation discrepancy of
anomalies in the teacher–student (TS) model provides evidence for anomaly detection and
localization. However, the teacher model is pretrained for classification, while the anomaly
scores in the distillation-based anomaly detection method are indirectly derived from the
classification scores. The mismatch between the two tasks can hinder the optimization of the …
Abstract
Knowledge distillation has demonstrated significant potential in addressing the challenge of unsupervised anomaly detection (AD). The representation discrepancy of anomalies in the teacher–student (T-S) model provides evidence for anomaly detection and localization. However, the teacher model is pretrained for classification, while the anomaly scores in the distillation-based anomaly detection method are indirectly derived from the classification scores. The mismatch between the two tasks can hinder the optimization of the model. To tackle this issue, we propose an innovative bidirectional knowledge distillation model. In this approach, forward knowledge distillation is pivotal in bolstering the model’s capacity for generalization. Simultaneously, backward knowledge distillation promotes diversity in representing anomalies. This reciprocal knowledge exchange effectively wards off potential performance declines due to target inconsistency. Through bidirectional knowledge distillation, we establish a more encompassing and resilient framework for knowledge transfer. Additionally, we introduce a novel data augmentation strategy to simulate anomalies and effectively eliminate unnecessary noise. In experiments on the MVTec AD, the proposed model achieves competitive results compared to state-of-the-art methods, 97.47% on image-level AUC, 98.23% on pixel-level AUC, and 94.77% on instance-level PRO. These results demonstrate the practicality of our approach in anomaly detection and localization.
Springer
Showing the best result for this search. See all results