A major challenge for both theoretical treatment and practical application of unsupervised learning tasks such as clustering, anomaly detection or generative modeling, is the inherent lack of quantifiable objectives. Choosing methods and evaluating outcomes is then often a matter of ad-hoc heuristics or personal taste. Anomaly detection is often employed as a preprocessing step to other learning tasks, and unsound decisions for this task may thus have long reaching consequences. In this work, we propose an axiomatic framework for analyzing behaviours of anomaly detection methods. We propose a basic set of desirable properties (or axioms) for distance-based anomaly detection methods and identify dependencies and (in-)consistencies between subsets of these. We then demonstrate the benefits of this axiomatic perspective on behaviors of anomaly detection methods by illustrating empirically how some commonly employed algorithms violate, perhaps unexpectedly, a basic desirable property.