Emotional recognition from videos or images requires large amount of data to obtain high performance and classification accuracy. However, large datasets are not always easily available. A good solution to this problem is to augment the data and extrapolate it to create a bigger dataset for training the classifier. In this paper, we evaluate the impact of different geometric data augmentation (GDA) techniques on emotion recognition accuracy using facial image data. The GDA techniques that were implemented were horizontal reflection, cropping, rotation separately and combined. In addition to this, our system was further evaluated with four different classifiers (Convolutional Neural Network (CNN), Linear Discriminant Analysis (LDA), K-Nearest Neighbor (kNN) and Decision Tree (DT)) to determine which of the four classifiers achieves the best results. In the proposed system, we used augmented data from a dataset (SAVEE) to perform training, and testing was carried out by the original data. A combination of GDA techniques using the CNN classifier was found to give the best performance of approximately 97.8%. Our system with GDA augmentation was shown to outperform previous approaches where only the original dataset was used for classifier training.