Recently, the performance of RGB-D salient object detection (SOD) has been significantly improved with the development of deep-learning techniques. However, most of them depend on complex structures with a large amount of parameters and multi-add operations, which need huge computational resources and are not applicable to real-world applications, especially on mobile devices. To handle this problem, we propose a lightweight network, namely LCH, for RGB-D SOD. Specifically, we first design a novel lightweight ghost in ghost (GiG) module to enable a more powerful multi-scale feature extraction, which not only efficiently tackles the scale variations of salient objects but also guarantees a high inference speed. Then, the proposed GiG module works in conjunction with ghost bottlenecks (G-bneck) and, therefore, forms a new lightweight U-Net structure network for fast CPU RGB-D SOD. Furthermore, to enhance the feature discriminability of our LCH, we also design a novel hybrid knowledge distillation approach (HKD), for which both semantic structure information and pixel-wise similarity information are transferred from a powerful state-of-the-art RGB-D SOD network to the proposed LCH model, so as to generate more satisfactory detection results. Compared with previous work, LCH owns 5.6x fewer parameters and achieves 5.4x faster inference speed on the CPU devices, but obtains comparable SOD performance. Extensive experiments are carried out on commonly-used RGB-D SOD datasets. Both qualitative and quantitative results prove the effectiveness of our proposed method.