The consumer unmanned aerial vehicle (UAV) market has grown significantly over the past few years. Despite its huge potential in spurring economic growth by supporting various applications, the increase of consumer UAVs poses potential risks to public security and personal privacy. To minimize the risks, efficiently detecting and identifying invading UAVs is in urgent need for both invasion detection and forensics purposes. Aiming to complement the existing physical detection mechanisms, we propose a machine learning-based framework for fast UAV identification over encrypted Wi-Fi traffic. It is motivated by the observation that many consumer UAVs use Wi-Fi links for control and video streaming. The proposed framework extracts features derived only from packet size and inter-arrival time of encrypted Wi-Fi traffic, and can efficiently detect UAVs and identify their operation modes. In order to reduce the online identification time, our framework adopts a re-weighted ℓ ₁ -norm regularization, which considers the number of samples and computation cost of different features. This framework jointly optimizes feature selection and prediction performance in a unified objective function. To tackle the packet inter-arrival time uncertainty when optimizing the trade-off between the detection accuracy and delay, we utilize maximum likelihood estimation (MLE) method to estimate the packet inter-arrival time. We collect a large number of real-world Wi-Fi data traffic of eight types of consumer UAVs and conduct extensive evaluation on the performance of our proposed method. Evaluation results show that our proposed method can detect and identify tested UAVs within 0.15-0.35s with high accuracy of 85.7-95.2%. The UAV detection range is within the physical sensing range of 70m and 40m in the line-of-sight (LoS) and non-line-of-sight (NLoS) scenarios, respectively. The operation mode of UAVs can be identified with high accuracy of 88.5-98.2%.