Video has become an indispensable carrier of information and requires extremely computation expensive analysis to understand high level semantics. The computational difficulties in extracting embedded information and bridging the semantic gap present the major challenges in interoperability, scalability and real-time response of video analytic systems. In this paper, we propose a distributed scalable infrastructure VAP (Video Analytic Platform) for supporting real-time video stream analysis. In VAP, the application requirements are represented as a Directed Acrylic Graph (DAG), where nodes stand for video analysis computation modules and links show data flow and dependencies between nodes. VAP leverages UIMA (Unstructured Information Management Architecture) framework as the data flow control engine and multiple commodity databases as the storage and computation resources. The actual executions of video analysis computation modules have been pushed down into database engines to minimize the data movement cost. We initially choose video surveillance in retail store as a representative application domain. As a case study, a prototype system has been developed to achieve fundamental functions of real-time video surveillance, including video capture/store, human detection/tracking and customer shopping trajectory analysis.