Most existing feature extraction algorithms aim at best preserving information in the original data or at improving the separability of data, but fail to consider the possibility of further reducing the number of used features. In this paper, we propose a parsimonious feature extraction algorithm. Its motivation is using as few features as possible to achieve the same or even better classification performance. It searches for the optimal features using a genetic algorithm and evaluates the features referring to Support Vector Machines. We tested the proposed algorithm by face recognition on the Yale and FERET databases. The experimental results proved its effectiveness and demonstrated that parsimoniousness should be a significant factor in developing efficient feature extraction algorithms.