A graph based approach to speaker retrieval in talk show videos with transcript-based supervision
This paper proposes a graph based strategy to retrieve frames containing the queried
speakers in talk show videos. Based on who is speaking and when information from the
audio transcript, an initial audio-based step, that restricts the queried person to frames
corresponding to when he/she is speaking, with a second step that analyzes visual features
of shots is combined. Specifically, based on the production property of talk show video,(1)
Shot based graph is constructed first. Then the densest sub-graph is returned as the final …
speakers in talk show videos. Based on who is speaking and when information from the
audio transcript, an initial audio-based step, that restricts the queried person to frames
corresponding to when he/she is speaking, with a second step that analyzes visual features
of shots is combined. Specifically, based on the production property of talk show video,(1)
Shot based graph is constructed first. Then the densest sub-graph is returned as the final …
Abstract
This paper proposes a graph based strategy to retrieve frames containing the queried speakers in talk show videos. Based on who is speaking and when information from the audio transcript, an initial audio-based step, that restricts the queried person to frames corresponding to when he/she is speaking, with a second step that analyzes visual features of shots is combined. Specifically, based on the production property of talk show video, (1) Shot based graph is constructed first. Then the densest sub-graph is returned as the final result. But instead of direct search (DS) of the densest part, (2) We model the intra node connection and inter node connection by a frame layer degree map to take into account the duration information within each shot node; (3)A graph partition strategy without restriction on the shape and the number of sub-graphs is proposed, in which shots containing the same person are more similar to each other. Experiments on one episode of the French talk show “Le Grand Echiquier” show more than 10% improvement to audio only method and more than 7.5% improvement to DS method on average.
Springer
Showing the best result for this search. See all results