With rapid development of digital imaging and communication technologies, image set based face recognition (ISFR) is becoming increasingly important and popular. On one hand, easy capture of large number of samples for each subject in training and testing makes us have more information for possible utilization. On the other hand, this large size of data will eventually increase training and classification time and possibly reduce the recognition rate if they are not used appropriately. In this paper, a new face recognition approach is proposed based on the K-SVD dictionary learning to solve this large sample problem by using joint sparse representation. The core idea of this proposed approach is to learn variation dictionaries from gallery and probe face images separately, and then we propose an improved joint sparse representation, which employs the information learned from both gallery and probe samples effectively. Finally, the proposed method is compared with some related methods on several popular face databases, including YaleB, AR, CMU-PIE, Georgia and LFW databases. The experimental results show that the proposed method outperforms several related face recognition methods.