This paper proposes a novel approach for background extraction of a scene captured by a moving camera. Proposed method uses a codebook, which is a compression technique used to store data from a long sequence of video frames. This technique has been used to construct a model which can segment out the foreground and that with using only few initial video frames as a training sequence. It is a dynamic model which keeps on learning from new video frames throughout its lifetime and simultaneously produces the output. It uses a pixel-wise approach, and the codebooks for each pixel are made independently. Special emphasis has been laid on the intensity of an image as the human eye is more sensitive to intensity variations. A two layer modelling has been performed where codebooks are passed from the cache to the background model after satisfying the frequency and negative run length conditions. Experimental results show the efficacy of the proposed method.