With the deployment of heterogeneous networks, mobile users are expecting ubiquitous connectivity when using applications. For bandwidth-intensive applications such as Internet Protocol Television (IPTV), multimedia contents are typically transmitted using a multicast delivery method due to its bandwidth efficiency. However, not all networks support multicasting. Multicasting alone could lead to service disruption when the users move from a multicast-capable network to a non-multicast network. In this paper, we propose a handover scheme called application layer seamless switching (ALSS) to provide smooth real-time multimedia delivery across unicast and multicast networks. ALSS adopts a soft handover to achieve seamless playback during the handover period. A real-time streaming testbed is implemented to investigate the overall handover performance, especially the overlapping period where both network interfaces are receiving audio and video packets. Both the quality of service (QoS) and objective-mapped quality of experience (QoE) metrics are measured. Experimental results show that the overlapping period takes a minimum of 56 and 4 ms for multicast-to-unicast (M2U) and unicast-to-multicast (U2M) handover, respectively. The measured peak signal-to-noise ratio (PSNR) confirms that the frame-by-frame quality of the streamed video during the handover is at least 33 dB, which is categorized as good based on ITU-T recommendations. The estimated mean opinion score (MOS) in terms of video playback smoothness is also at a satisfactory level.