Training time is an important consideration in classification practices. Various algorithms like SVM usually suffer from high computational cost during the training process. In this paper, we proposed a hybrid classification algorithm, by combining RNN and SVM together. RNN is a rapid neural network with randomly generated input weights, which performs as a fast, light weighted data selector. Selected data by RNN are then used to train an alternative SVM. The definition of crucial data and corresponding selection method is defined for RNN in this paper. We then proposed an intuitive parameter selecting method, so that to use only one parameter in the training process to determine the crucial data margin. Experimental results conducted on artificially generated databases and several public benchmarks show that the proposed method does retrieve equivalent dataset, with significant reduction of the sample number.