Now a days many Interactive Theater research is appeared [1][2][3][4]. We have developed an emotion-based interactive theater system. The characteristic is the use of Interactive Computer Graphics technology and the generation of three-dimensional imagery to create autonomous actors and a cyberspace in which all participants obtain a feeling of immersion. We have developed a system that includes multi person participation, emotion recognition and gesture recognition through. In this system, participants can feel they are actually contributing to the development of the story in cyberspace. This is done a system that shows avatars on thescreen as the alter ego of the participants. We have also produced an interactive story based on this system.