Most of the leading speech recognition technologies are based on frequency domain analysis. Most of them gives good accuracy. We generally take an assumption in frequency domain analysis that speech is a periodic signal and so we can use the Fourier transform for analysis purposes. But in reality speech is not a periodic signal; to be exact speech is a quasi-periodic signal. So using the Fourier transform for analysis purposes in itself introduces some approximation errors, which results into some inherent noise in the system. We think that this is the reason why the existing recognition techniques are getting saturated and accuracy is not improving. In spite of these limitations we cannot deny the importance of the frequency domain analysis in speech recognition area as it provides a very good mechanism to extract very relevant features form the speech. So in this work we have tried to use efficiency of frequency domain analysis without using Fourier transform along with time domain analysis. To represent frequency domain we have selected Pitch for analysis purposes and to incorporate time domain we have retained its temporal information. In our previous work we suggested a method for speech tokenization using pitch (Fundamental Frequency), energy and detected unvoiced regions. In this work we will be showing that it is possible to use this tokenization scheme to make a speech recognition system. Results are encouraging and we are hoping that this frame work in conjunction with the existing techniques will result into better recognition systems.