Recommender systems are popularly used to deal with an information overload issue. Existing systems mainly focus on user–item interactions and semantic information derived from metadata of users and items to improve recommendation accuracy. Item images provide useful information to infer users’ individual preferences, especially for those domains where visual factors are influential such as fashion items. However, this type of information has been ignored by most previous work. To bridge this gap and meet the requirements of performance from the aspects of Accuracy, Scalability, and Explainability evaluation metrics, this paper proposes a scalable and explainable visually-aware recommender system framework called SEV-RS. This framework contains a visually-augmented heterogeneous information network, a scalable meta-path feature extraction method for multi-hop relations, and a shallow explainable meta-path based Collaborative Filtering recommendation approach. We compared SEV-RS with the state-of-the-art models such as the deep learning model using Graph Attention Network on two real-world datasets and one synthetic dataset. The results show that SEV-RS produced more accurate and more explainable recommendations. Also, SEV-RS has substantially less computational time than the compared deep learning models.