Infrared and visible image fusion is a hot research direction in the fields of computer vision and image processing, and it is a common multimodal image fusion. An effective image fusion algorithm via convolutional sparse representation (CSR) and guided filtering is proposed for fusing infrared and visible images in this paper. First, a series of dictionary filters are trained by the CSR strategy, and the smooth image component and detailed image component are obtained by classifying those filters into high-pass filters and low-pass filters. Then two rules are designed to fuse the smooth image component and detailed image component, respectively. For the detailed image component, a weight construction method based on guided filtering is designed to get the weight maps, and the smooth image component is fused by applying the “choose-max” strategy to the corresponding sparse coefficients. Finally, the fused image is obtained by combining the fused smooth image component and detailed image component. Experimental results show that the proposed algorithm achieves good fusion results and exhibits advantages over comparison image fusion methods.