We present a real time system for detecting people and interpreting some of their postures. The system runs reliably on different people and under different lighting conditions. It uses an inexpensive PC desktop camera to acquire color monocular video sequences. Two fundamental problems of such a CCD camera for background-foreground segmentation are analyzed and a solution to them has been proposed. The system locates humans by extracting their profile using a physics based approach, which is robust in the presence of noise. Next it uses a pattern recognition algorithm to detect and interpret the posture of a human. The system can be used as a peripheral user interface, in pervasive computing applications, surveillance, monitoring people activities in an indoor environment, visual computer games, smart screen savers etc. We have already used this system for several real-world interactive video game applications.