ISPRS Commission V Symposium 'Image Engineering and Vision Metrology'

CALIBRATION FOR INCREASED ACCURACY OF

THE RANGE IMAGING CAMERA SWISSRANGERTM

T. Kahlmann a, F. Remondino b, H. Ingensand a

a
Geodetic Metrology and Engineering Geodesy (geomETH) - (kahlmann, ingensand)@geod.baug.ethz.ch
http://www.geometh.ethz.ch
b
Photogrammetry and Remote Sensing - [email protected]
http://www.photogrammetry.ethz.ch
Institute of Geodesy and Photogrammetry, ETH Zurich, 8093 Zurich, Switzerland

KEY WORDS: Camera, Range Imaging, Measurement System, Flash Ladar, Calibration, Accuracy, LIDAR, Laser scanning

ABSTRACT:
Range imaging is a new suitable choice for measurement and modeling in many different applications. But due to the technology’s
relatively new appearance on the market with a few different realizations, the knowledge of its capabilities is very low. In most
applications, like robotics and measurement systems, the accuracy wanted, lies at some millimeters. The raw data of range imaging
cameras do not reach this level. Therefore, the calibration of the sensors output is needed. In this paper some of the parameters
which influence the behavior and performance of the range imaging camera SwissRangerTM (provided by the Swiss Center for
Electronics and Microtechnology - CSEM) are described. Because of the highly systematic structure and correlations between
parameters and output data, a parameter based calibration approach is presented. This includes a photogrammetric camera
calibration and a distance system calibration with respect to the reflectivity and the distance itself.

1. INTRODUCTION The sensor itself is an Avalanche Photo Diode (APD) with an

external clock.
Range Imaging (RIM) is a fusion of two different technologies.
According to the terminology, it integrates distance
measurement as well as imaging aspects. It is a brand new
measurement technique, not yet well known and investigated.
Therefore both parts are shortly described in the following.

1.1 Electro-Optical Distance Measurement

The distance measurement principle of range imaging cameras Figure 1. Time-of-flight distance measurement principle. Pulse Runtime
is dominated by the time-of-flight principle. The time the light (left) and Phase Shift (right).
needs to travel from one point to another is directly proportional
to the distance the light has traveled: On the other hand, the phase shift measurement principle avoids
t high precision clocks and uses more complex and integrative
d= ⋅c (1) sensor design. The sensor investigated in this paper is based on
2 this principle. The emitted (incoherent) light is modulated in
where d = distance between sensor and object amplitude with a sinusoidal modulation (Figure 2) whereas
c = speed of light other methods are FMCW, pseudo-noise or polarization
t = time between emitting and receiving modulation. The reflected, sinusoidal modulated light is
demodulated by means of four sampling points that are
Figure 1 points out two different distance measurement triggered to the emitted wave. Out of the four measurements
methods. A modulated radiation (e.g. light) is emitted, reflected c(τ0), c(τ1), c(τ2) and c(τ3) the phase shift φ, the offset B and
by an object and partially mapped onto the receiver. Therefore, the amplitude A can be calculated:
the distance sensor-object is half of the traveled distance of the
radiation. Mainly, two different types of the time-of-flight c (τ 0 ) + c (τ 1) + c (τ 2 ) + c (τ 3)
principle are known: B= (2)
4
1. Pulse Runtime Method ( c (τ 0 ) − c (τ 2 )) 2 + ( c (τ 1) − c (τ 3)) 2
2. Phase Shift Determination A= (3)
2
In the first case the runtime of a single pulse is directly ⎛ c (τ 0 ) − c (τ 2 ) ⎞
measured. In order to reach a distance accuracy of a few ϕ = arctan ⎜⎜ ⎟⎟ (4)
millimeters, the clock accuracy has to be as low as a few ⎝ c (τ 1) − c (τ 3) ⎠
picoseconds. Thus very good clock circuits are unavoidable.
The EPF Lausanne (ETH Lausanne) has integrated such The distance between the sensor and the object then can be
distance sensors in CMOS technology (Niclass et al., 2005). calculated as follows:

136
 IAPRS Volume XXXVI, Part 5, Dresden 25-27 September 2006

λ mod ϕ those of imaging arrays. Simplified, it just enables each pixel to

d= ⋅ (5) measure the distance towards the corresponding object point.
2 2π
This is regarded as an array of range finders. Figure 4 presents
where d = distance between sensor and object the schematic principle of range imaging. A single signal
λmod = modulation wavelength emitter sends out modulated light. This light is reflected by an
φ = phase shift object partially back to the sensor and is mapped onto a custom-
build solid state sensor by means of optics (not shown in the
As (Oggier et al., 2003) and (Seitz, 2005) have shown, the figure) and therefore onto the smart pixels which are able to
distance accuracy limitation, given by the nature of light for the measure the distance. The measured distances in connection
investigated range imaging camera in this paper, is around 5 to with the geometrical camera relationships can be afterwards
10 mm. used to compute the 3D coordinates which represent a
reconstruction of the imaged object.

Figure 2. Phase shift distance measurement principle.

Figure 4. Range imaging principle.
Because of the light’s quantum structure the amplitude of light
cannot be measured at a specific point of time. A time window
∆t is necessary. Two opposed aspects define the size of the One possible realization, similar but not equal to the one used in
window. The quantum structure of the light is dominated by the the investigated SwissRangerTM camera, is shown in Figure 5.
light statistics at a higher ∆t. But the higher ∆t is, the lower the By means of two photogates, the photon generated electrons are
demodulation contrast is. According to (Lange, 2000) the separated with respect to the modulation frequency. Two
sampling process is usually done with ∆t at about half the photogates allow only a one-tap or two-tap measurement, which
modulation wavelength which results in a demodulation depth means that only one resp. two of the four required intensities
of about 64% (Figure 3). c(τi) are acquired. Therefore, this measurement has to be
repeated four resp. two times.

Figure 3. Sampling process and demodulation depth (Lange, 2000).

Irregularities in the sinusoidal continuous wave modulation

with an uneven degree result in circular deviations of the
linearity within the distance measurement (Lange, 2000). These
deviations will be of importance in section 3.2. Figure 5. Lateral section of a RIM sensor from PMDTec
(Möller et al., 2005).
1.2 Range Imaging Technology with CMOS sensor
The investigated SwissRangerTM SR-2 is shown in Figure 6.
The distance measurement principle outlined in section 1.1 is The emitting system consists of 48 LEDs. They are controlled
primarily able to measure distances in one direction at a time. simultaneously in order to reach a sufficient optical output
Implemented in a scanning device (laser scanner, geodetic total power. The incoming radiation is mapped onto the sensor array,
station), a 3D capturing of the environment is possible. But the which is produced in a specialized mixed CCD and CMOS
serial mode of operation still remains. Range imaging combines process.
the distance measurement technology with the advantages of

137
 ISPRS Commission V Symposium 'Image Engineering and Vision Metrology'

it is recommended to use the sensor in some kind of master

mode, where it acquires continuously and stabilizes in
temperature. But due to the fact that small gaps remain between
the single frames, a cooling process takes place and small
effects still remain as can be seen in Figures 8 and 9. The
SR-2’s concept does not provide an independent continuous
mode. But nevertheless the effect remains regular and can
therefore be factored in the calibration.

Figure 6. SwissRangerTM SR-2.

Table 7 outlines some of the SwissRangerTM’s characteristics.

This sensor has a non-ambiguity distance of 7.5 m. Remarkable
is the high number of pixels (nearly 20’000). The large pixel
size corresponds to the high degree of integration of electronics.

SwissRangerTM SR-2
development CSEM
features
number of pixel 160 124
pixel size 39.2 μm 54.8 μm
sensor field 6.27 mm 6.8 mm
mod. frequency 20 MHz
carrier wavelength 870 nm
non-ambiguity distance 7.5 m
Interface USB 2.0 Figure 8: Distance offset drift (fixed target) caused by self-induced
heating of the sensor.
Table 7. Characteristics of the investigated range imaging camera
SwissRangerTM SR-2
2.1.2 External Temperature
The sensor can be run with different integration times. The The second important influence of temperature on the distance
integration time indicates how often/long the intensities c(τi) measurement comes from the external temperature. To
were collected and thus integrated. For a higher integration investigate this effect, the SwissRangerTM was placed in front of
time, the electrons (also the photon generated ones) are a fixed target. It has to be remarked that the sensor was not run
collected for a higher number of cycles. The calculations are in continuous mode, so far (separate thread in the mastering
done afterwards. computer). Therefore, the trend of the single graphs in Figure 9
Currently a new version of the sensor has been produced is valid for exactly the computer and program setup they were
(SwissRanger SR-3000, http://www.swissranger.ch). acquired with. But the main systematic behaviour can be seen.
With a higher temperature the measured distance increases. It
also can be pointed out that the effect is systematic and thus
2. SYSTEM PROPERTIES removable with a calibration procedure, as well. The
determined drift lies around 8mm/°C.
At the laboratories of the Institute of Geodesy and
Influence of Integration-Time and Temperature on Measured Distance by SwissRanger
Photogrammetry (IGP), ETH Zurich, many different properties 2.5

of the SwissRangerTM were investigated. The impact of the

39.7 °C
28.8 °C
19.2 °C
internal and external temperature influences on the distance 2.4
9.2 °C
5.2 °C
measurements are reported in the next sections. 0.0 °C
2.3 -5.6 °C
-10.0 °C
Measured Distance [m]

2.1 Influence of the Temperature onto the distance 2.2

measurement
2.1

A problem very well known in semiconductor technology is the

materials high respond to different temperatures and changes in 2

the temperature. Especially in CCD and CMOS photo sensors

an increased temperature causes a higher rate of thermal 1.9

generated electrons. These electrons do not contain any usable

1.8
information; they only fill the storage sites of the sensor equally 0 20 40 60 80 100
Integration-Time [ms]
120 140 160 180

and thus decrease and limit the usable optical capability for the
actual signal to be derived. The saturation is reached sooner. In Figure 9: Relation between Integration-Time and Measured Distance
case of the SR-2, two different effects were observed. with respect to different external temperatures

2.1.1 Internal Temperature

The first effect observed relates to an internal heating of the 2.2 Distance Accuracy
sensor. As seen in Figure 8, in the first minutes after the sensor
Figure 10 shows the standard deviation for the distance
starts working, the measured distance towards a fixed target
measurements in every pixel. As it can be clearly seen, the
increases. After a few minutes the process stabilizes. Therefore

138
 IAPRS Volume XXXVI, Part 5, Dresden 25-27 September 2006

expected best accuracy of 5 to 10 mm was only achieved in the

center of the sensor, where the illumination of the used object
(white wall at approx. 2.453 m) was at its highest value
(compare (Kahlmann, Ingensand, 2005)).

Figure 11: Different testfields used to calibrate the range imaging

camera. None of them was usable for the calibration of the SR-2.

The calibration network of the TOF camera included 35

stations, well distributed, in particularly in depth direction as
Figure 10: Standard deviation for a single distance measurement (of a we used a planar testfield. After the self-calibrating bundle
white wall) measured and calculated for the whole sensor array. It was adjustment, the final standard deviations of the computed object
calculated out of 200 distance measurements in every pixel. The points resulted σx = 1.4 mm, σy =1.3 mm, and σz = 1.8 mm,
integration time was approx. 0.1 s. which represent a relative accuracy of approximately 1: 270 in
the testfield plane and 1:120 in the depth direction.
Between the additional parameters, only the first two terms of
3. SENSOR CALIBRATION the radial distortions turned out to be statistically significant.
The calibration network is shown in Figure 12.
Due to the systematic effects, the sensor data shows that a
calibration of the system is needed and possible. Different
calibration approaches were followed up.

3.1 Photogrammetric Calibration

Accurate camera calibration and orientation procedures are

necessary prerequisites for the extraction of precise and reliable
3D metric information from images. A camera is considered as
calibrated if the focal lenght, principal point location and lens
distortion parameters are known. In many applications,
especially in computer vision, only the focal length is recovered
while for precise photogrammetric measurements all the
calibration parameters are generally employed. The choice of
the camera model is often related to the final application and
required accuracy. Projective or perspective camera model can
be used but generally a precise sensor orientation is performed
by means of a least squares bundle adjustment with additional
parameters.

3.1.1 Calibration procedure

For the TOF camera under investigation, different tests were
performed. Due to the low resolution of the sensor, the selection Figure 12: The final planar object with 25 infrared LED used to
of a suitable test field was crucial (Figure 11). A large test field calibrate the TOF camera (above). The sensor connected via USB to a
would not allow the precise measurements of its targets due to laptop is also visible in the foreground. The recovered camera poses
the small sensor format. Large rounded targets occupy too during the calibration procedure (below).
many pixels. Squared targets have no well defined corners.
Retro-reflective targets generated noise effects in the images.
Moreover, the lightning of the room influences a lot the data 3.2 Distance Calibration
acquisition. For all these reasons, a planar testfield whose The distance measurement accuracy of a single pixel in the
targets are represented by NIR LEDs was build. These active center of the sensor has been investigated. In order to reach a
targets can be automatically extracted from the intensity image high reliability of the acquired data, the calibration has been
provided by the camera due to their high intensity in the image. done on the high accuracy distance measurement track line
They are automatically extracted by means of a threshold and a available at the laboratories of the IGP. It is supported by an
centroid operator and used as input measures for the calibration interferometer and the target carrier (trolley) runs fully
procedure. automatically. The relative accuracy of the distance

139
 ISPRS Commission V Symposium 'Image Engineering and Vision Metrology'

measurement is around a few microns. The absolute accuracy of distance calibration cannot be done for all pixels. A simple
0.1 mm is not reached due to the not standardized mounting of strategy to reduce an offset is to acquire a Fixed Pattern Noise
the SR-2. Therefore the absolute accuracy is about 1mm. matrix. This matrix contains an individual offset for every
pixel. To gain this matrix, an accurate procedure was
developed. The SR-2 was mounted above a total station. The
8

In te g ra t io n T im e 1 6

7
In te
In te
g
g
ra
ra
t io
t io
n
n
T
T
im
im
e
e
2
4
0
0 total station is able to measure coordinates of an object in front
In te g ra t io n T im e 6 0
In te
In te
g
g
ra
ra
t io
t io
n
n
T
T
im
im
e
e
1
1
00
20
of the setup. In this case a white wall was chosen. By means of
comparison between the nominal distances (calculated out of
6 In te g ra t io n T im e 1 40

5 the nominal coordinates measured by the total station) and the

data acquired by the camera, the FPN matrix can be determined.
Measured Distance [m]

-1
0 1 2 3 4 5 6 7 8
N o m in a l D is ta n c e [m ]

Figure 13: Filtered distance calibration data. The circular differences

to the expected linearity can be clearly seen.

Two different parameters were varied for the calibration

measurements. First all measurements were done with different
integration times. Second, five targets with a different
reflectivity were used (white, light grey, middle grey, dark grey
and black).
The SR-2 has been placed along the track line. The targets were
placed from 1.25 m to 7.5 m with steps of 0.05 m. At each
position 10 distance measurement of one pixel (row:80, col:62) Figure 15: FPN matrix for the SR-2. Integration time was about 0.1 s,
were made. These measurements were then median filtered. nominal distance 2.452 m. Damaged pixel can be depicted, too.
The trolley ran forth and back three times. The measurements
were averaged to eliminate linear drift terms. At last the As Figure 15 depicts, the variation of the offset is not negligible
measurements were filtered over the distance range by means of and has to be considered in the calibration of the sensor. The
an ideal low-pass-filter with a Hamming window. range goes from about -0.3 m to about +0.15 m. A systematic
tenor can be outlined. In this case this FPN matrix is only valid
-0 .1

In te g ra tio n T im e 1 6
for an integration time of about 0.1 s. For every integration
time, respectively every controlling computer/program setup
In te g ra tio n T im e 2 0
In te g ra tio n T im e 4 0
In te g ra tio n T im e 6 0
-0 .1 5 In te g ra tio n T im e 1 0 0
In
In
te
te
g
g
ra
ra
tio
tio
n
n
T
T
im
im
e
e
1
1
2 0
4 0 such FPN is specific and has to be determined.
-0 .2

In Figure 16 and 17 the 3D comparisons between the data

-0 .2 5
recorded before and after the calibration procedure are reported.
Offset [m]

-0 .3

-0 .3 5

-0 .4

-0 .4 5
0 1 2 3 4 5 6 7 8
N o m in a l D is ta n c e [m ]

Figure 14: Final Look Up Table (LUT) data for different integration
times. The reflectivity of the targets in comparison to the other
influences is negligible.

Figure 13 shows the filtered data. The predicted circular effect

can be clearly seen. Figure 14 shows the Look Up Table (LUT)
data. This data is used for the correction of the distance data.
Experiments with a modeling of the offset data with linear and
cosine functions were done. But these did not brought better
results. Therefore, a LUT is fully adequate.

3.3 Fixed Pattern Noise (FPN)

Figure 16: Deviations towards a flat wall before the calibration of the
The distance calibration for a single pixel is not sufficient for sensor. The highest deviation is around -30 cm (dark blue area).
the whole sensor array. Due to the high demand for time, a

140
 IAPRS Volume XXXVI, Part 5, Dresden 25-27 September 2006

In geodesy, RIM will promote the further integration of sensors

into surveying instruments. After the integration of GPS and
total station in Leica’s Smart station, a combination of RIM,
inclination sensor and north-finding GPS will enable new
scenarios in data capture. As in most cases GPS can operate
very closely to the object. The missing connection can be
carried out by RIM technology. Analogous to the T-probe
approach (Leica) in industrial metrology a so called Geo-Probe
configuration allows for efficient data capturing if GPS
generates additionally the orientation of the system (Figure 18;
patent pending).

5. CONCLUSIONS

With the high-resolution range imaging camera SwissRangerTM,

a powerful new tool for many applications with a need for
metrology is available. Many influencing parameters have an
Figure 17: Deviations towards a flat wall after calibration of the sensor. impact on the accuracy of the system. But due to their
The deviations are reduced from about -30 to +10 cm down to about ±1 systematic appearance, they can be identified and removed
cm after the calibration. Some systematic behaviour still remains. within a calibration procedure.
In this paper, many different aspects of such a calibration were
shown. It can be stated that a calibration is needed and possible.
4. APPLICATION AREAS OF RANGE IMAGING The achievable accuracy with respect to the calibrations
SENSORS depicted is around 1 cm. The recovered relative accuracy shows
the potentiality of the TOF sensor for middle-accuracy
It is probable that RIM will replace close range scanners which applications, like autonomous navigation or real-time modeling.
work sequentially and cannot be used for kinematic approaches. Further studies should be done in order to fully use the sensors
Compared to existing scanner solutions, an additional abilities and exploit its potentialities.
advantage of RIM is the speed of the 3D data capture which is
at the present technical state in the range of 20 Hz. This enables
new application areas which are covered now by REFERENCES
videogrammetry in the field of human or machine motion
analysis. Additionally, range imaging sensors are very cheap in CSEM - Centre Suisse d'Electronique et de Microtechnique SA:
comparison to the existing technologies. http://www.csem.ch
It is obvious that RIM can be advantageous for fast 3D face
recognition, as stereoscopic image capture needs a wider field Kahlmann, T., Ingensand, H., 2005. Calibration and
of view. Other applications can be envisaged in man-robot improvements of the high-resolution range-imaging camera
interaction environment to enable cooperative collaboration of SwissRangerTM, Proc. SPIE Vol. 5665, pp. 144-155.
man and machine. RIM will be used in the area of automotive
sensors as so called precrash sensors to prevent the contact of Lange, R., 2000. 3D Time-of-flight distance measurement with
pedestrians with the car engine or other solid parts. As a custom solid-state image sensors in CMOS/CCD-technology.
solution the hood will be lifted previously in order to damp the PhD-Thesis at the University of Siegen, Siegen, Germany.
collision of person and car. Another potential application of the
Möller, T., Kraft, H., Frey, J., Albrecht, M., Lange, R., 2005.
RIM is to control the airbag systems where RIM will determine
Robust 3D Measurement with PMD Sensors. In: Proceedings of
the position of the passengers.
the 1st Range Imaging Research Day at ETH Zurich, Zurich,
Switzerland, pp. “Supplement to the Proceedings”.

Niclass, C., Besse, P.-A., Charbon, E., 2005. Arrays of Single

Photon Avalanche Diodes in CMOS Technology: Picosecond
Timing Resolution for Range Imaging. In: Proceedings of the
1st Range Imaging Research Day at ETH Zurich, Zurich,
Switzerland, pp. “Supplement to the Proceedings”.

Oggier, T., Lehmann, M., Kaufmann R., Schweizer, M.,

Richter, M., Metzler, P., Lang, G., Lustenberger, F., Blanc, N.,
2003. An all-solid-state optical range camera for 3D-real-time
imaging with sub-centimeter depth-resolution (SwissRanger),
Proc. SPIE Vol. 5249, pp. 634-545.

Seitz, P., 2003. Unified analysis of the performance and

physical limitations of optical range-imaging techniques. In:
Proceedings of the 1st Range Imaging Research Day at ETH
Zurich, Zurich, Switzerland, pp. 9-19.

Figure 18: Geo-Probe application for RIM sensors.

141