Deep learning-driven diagnostic methods have shown significant ability to diagnose certain types of insulation defects. However, for these methods to achieve excellent results, it is necessary for the clients to obtain a large amount of data, which is not realistic for a typical client. For this reason, there is an incentive for different clients to collaboratively develop effective diagnostic models, but due to conflicts of interest and privacy protection issues, data sharing has become a challenge. In addition, due to domain shift, it is difficult to apply the model developed by some clients to others. In response to this, we propose a novel federated transfer learning (FTL) framework. Specifically, we developed federated adversarial learning that achieves domain adaptation while protecting data privacy, and we introduced a federated minimax (FedMM) algorithm for global model aggregation, which not only solves the problem of gradient drift caused by sample imbalance but also improves the accuracy of the global model. We verified that our method can achieve high-precision and robust diagnosis of gas-insulated switchgear(GIS) insulation defects with experiments composed of laboratory and field clients. The superior performance for unbalanced small samples on-site shows that the application of the proposed FTL is promising.