The image reference approach is very popular in industrial inspection due to its generality for different inspection tasks. Unfortunately, this approach is sensitive to illumination variations. A novel illumination compensation algorithm is proposed in this paper for correcting smooth intensity variations due to illumination changes. By using the proposed algorithm as a preprocessing step in the image reference based inspection or localization, we can make the image inspection or localization algorithm robust against spatially smooth illumination changes. This technique is very useful to achieve a reliable automated visual inspection system under different illumination conditions. The proposed illumination compensation algorithm is based on the assumption that the underlining image reflectance function is approximately piecewise constant and the image irradiance function is spatially smooth. Reliable gradient constraints on the smooth irradiance function are computed and selected from the image brightness function by using a local uniformity test. Two surface fitting algorithms are presented to recover the smooth image irradiance function from the selected reliable gradient constraints. One is a polynomial surface fitting algorithm and the other is a spline surface fitting algorithm. The spline surface fitting formulation leads to solving a large linear system, which is accomplished by an efficient preconditioned conjugate gradient algorithm. Once the image irradiance function is estimated, the spatial intensity inhomogeneities can be easily compensated. Some experimental results are shown to demonstrate the usefulness of the proposed algorithm. © 2001 SPIE and IS&T.