Heterogeneous integration of InP and GaSb on Si substrates holds a huge potential interest in near-infrared and mid-infrared optoelectronic devices. In this study, 2-inch 180-nm-thick InP and 185-nm-thick GaSb thin layers were successfully transferred onto the Si substrates to form high-quality and ultra-smooth InP/Si and GaSb/Si templates using molecular beam epitaxy (MBE) and the ion-slicing technique together with selective chemical etching. The relocation of the implantation-introduced damage in the sacrificial layer enables the transfer of relatively defect-free InP and GaSb thin films. The sacrificial layers were completely etched off by selective chemical etching, leaving ultra-smooth epitaxial surfaces with a roughness of 0.2 nm for the InP/Si template and 0.9 nm for the GaSb/Si template, respectively. Thus, the chemical mechanical polishing (CMP) process was not required to smooth the surface which usually introduces particles and chemical contaminations on the transferred templates. Furthermore, the donor substrate is not consumed and can be recycled to reduce the cost, which provides a paradigm for the sustainable and economic development of the Si integration platform.