A Gabor filters based face recognition algorithm named POMA-Gabor is proposed in this paper. The algorithm uses particular Gabor wavelets in the feature extraction on specific areas of the face image and a particle swarm optimization (PSO) based memetic algorithm (POMA), which combines comprehensive learning particle swarm optimizer (CLPSO) global search and self-adaptive intelligent single particle optimizer (AdpISPO) local search, is introduced to select the Gabor filter parameters. The experimental results demonstrate that POMA obtains better performance than other comparative PSO algorithms. Employing POMA for Gabor filter design, POMA-Gabor is capable of obtaining more representative information and higher recognition rate with less computational time.