The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018

FOSS4G 2018 – Academic Track, 29–31 August 2018, Dar es Salaam, Tanzania

THE USE OF THE UAV IMAGES FOR THE BUILDING 3D MODEL GENERATION

G. Vacca a, G. Furfaro b, A. Dessì a

a Dept. of Civil Engineering, Environmental and Architecture, University of Cagliari (CA), Italy – (vaccag, andessi)@unica.it
b STF land &sea survey Brindisi, Italy - [email protected]

Commission IV, WG IV/4

KEY WORDS: UAV; photogrammetry; oblique images; SfM, open source, VisualSfM

ABSTRACT:

The growing interest in recent years in Unmanned Aerial Vehicles (UAVs) by the scientific community, software developers, and
geomatics professionals, has led these systems to be used more and more widely, in different fields of engineering and architecture.
This is thanks, above all, to their flexibility of use and low cost compared to traditional photogrammetric flights using expensive metric
digital cameras or LiDAR sensors. In recent years, UAVs have also been used in the field of monitoring and inspection of public or
private buildings that are remarkable in terms of size and architecture. This is mainly due to the focus a sustainability and resource
efficiency in the building and infrastructure sector, which aims to extend their lifetimes. Through the use of remote checking using
UAVs, the monitoring and inspection of buildings can be brought to a new level of quality and saving.
This paper focuses on the processing and study of 3D models obtained from images captured by an UAV. In particular, the authors
wanted to study the accuracy gains achieved in the building 3D model obtained with both nadir and oblique UAV flights. The images
from the flights were processed using Structure-for Motion-based approach for point cloud generation using dense image-matching
algorithms implemented in an open source software. We used the open source software VisualSfM, developed by Chanchang Wu in
collaboration with the University of Washington and Google. The dense matching plug-in integrated in its interface, PMVS/CMVS,
made by Yasutaka Furukawa, was employed to generate the dense cloud. The achieved results were compare with those gained by
Photoscan software by Agisoft and with 3D model from the Terrestrial Laser Scanner (TLS) survey.

1. INTRODUCTION creation of 3D City Models or GIS 3D (Deidda et al., 2013,

Deidda et al., 2015) of our cities, allowing in this way a better
The growing interest in recent years in Unmanned Aerial knowledge and management of the building (Vacca et al., 2018).
Vehicles (UAVs) by the scientific community, software Finalized to the full achievement of these last objectives, and in
developers, and geomatics professionals, has led these systems to particular for a complete geometric reconstruction of buildings
be used more and more widely, in different fields of engineering with high accuracy, from several years the nadiral and oblique
and architecture. This is thanks, above all, to their flexibility of UAV flights are used (Vacca et al.,2017).
use and low cost compared to traditional geomatic survey as The use of oblique images allows the reconstruction of 3D mod-
photogrammetric flights with expensive metric digital cameras or els of buildings including façades. Furthermore, this technique
LiDAR sensors (Ezequiel et al., 2014). Other advantages are the permits the measurement and survey of the higher parts of the
reduced image acquisition time and the ease of images processing objects, which are not easily achievable by the common survey
thanks to the presence on the market or web of low-cost or techniques, such as terrestrial laser scanning (TLS) or terrestrial
free/open source softwares. photogrammetry (Lingua et al., 2017).
UAV systems were used first for military purposes and later for This paper focuses on the processing and study of 3D models
civilian ones, such as agricultural and forest management obtained from images captured by UAV. In particular, the authors
(precision farming, assessment of woodlots, fire surveillance, wanted to study the accuracy gains achieved in the surveying and
etc.) (Remondino et al., 2011; Aicardi et al., 2016), archaeology the measurement of the dimensions of the buildings in the 3D
and cultural heritage surveying (3D documentation and mapping) models or their health state obtained with nadir/oblique UAV
(Berni et al., 2009; Chiabrando et al.; 2011), for 3D flights. These types of flights are particularly suitable for the 3D
reconstruction (Chiabrando et al., 2012.) and others. Specifically, modelling of cities or urban agglomerations, where it is important
in recent years, UAVs have also been used in the field of to achieve a complete building reconstruction, including façades
monitoring and inspection of public or private buildings that are and footprints of buildings. For this purpose, some UAV surveys
remarkable in terms of size and architecture. This is mainly due with both nadir and oblique axes were performed on the
to the focus a sustainability and resource efficiency in the buildings.
building and infrastructure sector, which aims to extend their The images from the flights were processed using Structure-for
lifetimes. This type of monitoring and inspection has always Motion-based approach for point cloud generation using dense
required sophisticated and expensive methods to quickly and image-matching algorithms implemented in an open source
safely identify possible damage that could jeopardize the stability software. In particular we used the open source software
and safety of buildings and people. Through the use of remote VisualSfM, developed by Chanchang Wu in collaboration with
checking using UAVs, the monitoring and inspection of buildings the University of Washington and Google, was used for the
can be brought to a new level of quality and saving. images alignment by means of the SfM technique (Wu, 2011;
The buildings 3D model, 3D, particularly those of historical- Wu, 2007). The dense matching plug-in integrated in its interface,
architectural importance, become of strategic importance also for PMVS/CMVS, made by Yasutaka Furukawa, was employed to
Historical Building Information Modeling (HBIM) or for the generate the dense cloud (Furukawa, 2010). The achieved results

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-4-W8-217-2018 | © Authors 2018. CC BY 4.0 License. 217
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018
FOSS4G 2018 – Academic Track, 29–31 August 2018, Dar es Salaam, Tanzania

were compare with those gained by Photoscan software by namely that the 3D structure can be resolved from a series of
Agisoft and with 3D model from the Terrestrial Laser Scanner overlapping images. However, they fundamentally differ because
(TLS) survey. in SfM, the geometry of the scene, camera positions, and
2. METHODOLOGY orientation is solved automatically without points known. The
points are solved simultaneously using a highly redundant,
2.1 Instruments iterative bundle adjustment procedure, based on a database of
features automatically extracted from a set of multiple images
The purpose of this research is to study the accuracy of 3D with a high degree of overlap. The approach is most suited to sets
models obtained from nadiral and oblique imagery acquired by of images with a high degree of overlap that capture the full three-
UAV systems and processed with the software free VisualSfM. In dimensional structure of the scene viewed from a wide array of
order to achieve these goals, we performed an UAV flight with positions, or as the name suggests, images derived from a moving
nadir and oblique images on a single building. sensor (Szeliski, 2010; Westoby, 2012).
The UAV system used is the FlyNovex UAV by FlyTop S.p.A. The building 3D model obtained from the UAV flight was
(Figure 1). It’s a hexacopter weighing 6 kg with a 5 km range and validated by comparison with the 3D model from a terrestrial
20 minutes of autonomy. The system contains an IMU (inertial Laser Scanner (TLS) survey. The instrument used was a Faro
measurement unit), a GPS receiver and a camera mounted on the Focus 3D. This is a compact scanner characterized by an
gimbal. The UAV has a wireless connection with a computer operative range that varies between 0.6 m and 120 m, with a
working as the ground control station. The camera is a Sony Alfa linear distance error of ±2 mm for scanner-object distances
6000 model with a resolution of 6000 pixel x 4000 pixel, a focal comprised between 10 m and 25 m. It has a vertical visual field
length of 16 mm, sensor size of 23.5 mm x 15.6 mm and a pixel of 305° and a horizontal one of 360°. The vertical and horizontal
size of 3.92 micron. The camera’s calibration parameters are resolution is 0.009°.
reported in Table 1. The processing of the scans was done using the JRC
Reconstructor Software v. 3.1.0 (335) by Gexcel Ltd. (Spin Off
of the University of Brescia, Bergamo, Italy). Reconstructor is a
software that enables all the processing operations of the point
cloud before the 3D model of the structure or building scanned is
obtained.
The comparison was done both comparing the measurements of
distances taken on the point clouds obtained from UAV 3D model
and TLS surveys, and comparing the point clouds themselves
using the CloudCompare software (Open Source Software).
Specifically, the comparison was done by calculating the minimal
distance between every point of the models using the nearest
neighbor algorithm. Furthermore, the software allows the
calculation of statistical values, such as the minimal distance,
maximal distance, average distance, and standard deviation.
The images were processed also with a similar well-known
proprietary commercial software, Agisoft Photoscan,
Figure 1: FlyNovex UAV system (www.agisoft.com) which reaches very high performances and
accuracies, as a number of studies proves (Remondino, 2014).
Focal lengh 16 mm The VisualSfM and Photoscan work in the same way and they are
Pixel size x 0.004 mm following these steps: images import, image alignment,
generation of the sparse cloud, georeference and dense image
Pixel size y 0.004 mm
matching.
fx 4074.9533 pixel
fy 4074.4493 pixel
2.2 The nadir/oblique UAV flight
cx 2996.9340
cy 1936.5894 For this study we planned and executed a flight over a single
skew 0.934748 building and we validated the results with a TLS survey done
k1 -0.019653 directly by us. The chosen building was an abandoned
K2 0.013821 construction with a regular shape, about 12 m tall. The flight was
K3 -0.006089 characterized by images acquired with nadiral axis and with an
K4 0.008760 axis inclined by 45° with respect to the nadiral direction.
p1 -0.001868 Nowadays, considering the case of oblique images, the question
p2 -0.001065 on the modalities through which flight configuration allows
Date 14/10/2016 covering the entire building or object is still open (Lingua, 2017).
In the recent years, several configurations have been tested and
evaluated (Rossi et al., 2017): the Maltese Cross with a nadir
Table 1: Calibration parameters of the camera
camera and four oblique ones (45°) pointing in the cardinal
The images were processed using VisualSfM, a GUI (Graphical directions, and the Fan configuration that increases the swath
User Interface) application for 3D reconstruction, devoloped by width along the track to cover more area. In our case, the figure
Chanchang Wu in collaboration with the University of 2 shows our oblique flight that was done with a 45° inclination
Washington and Google, was used for the images alignment by of the optical axis. Table 2 shows the parameters of the flight plan
means of the SfM technique. for the nadir and oblique flights.
The SfM is a low-cost photogrammetric method for high- The images from the UAV flight were processed using
resolution topographic reconstructions. The SfM operates under VirtualSfM and Photoscan. The dense point cloud was calculated
the same basic tenets of the stereoscopic photogrammetry, with Photoscan setting the Quality parameter to “Medium” and

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-4-W8-217-2018 | © Authors 2018. CC BY 4.0 License. 218
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018
FOSS4G 2018 – Academic Track, 29–31 August 2018, Dar es Salaam, Tanzania

the Depth Filtering to “Moderate”. Georeferencing point clouds

were made using four GCPs on the terrain and five on the façade
(Figure 3), surveyed in the ETRF2000 datum (Barbarella, 2014).
2
The r.m.s of the georeferenced image processing have been
attested to a few centimetres.
It is important to point out that the nadir flight was realized in
fully automatic mode, while the oblique flight was in manual 3
mode. The pilot, in this case, with the help of the remote viewing 1
camera, made the shots attempting to cover the required 80%
overlap. From the pre-processing of the images, however, it has
been verified that the overlap of 80% was guaranteed.
4
Nadir flight Flight altitude 90 m
GSD 2.20 cm Figure 4: TLS station points
Forward overlay 80 %
Side overlay 75%
Number of images 25 Table 3 reports the number of points recorded in the different
Oblique flight Flight altitude 50 m point clouds representing only the building.
GSD 1.2 cm
Forward overlay About 80 % Flight/Survey and Software N. of points
Side overlay About 80 % UAV Flight - Photoscan 1.207.609
Number of images 119 UAV Flight - VisualSfM 1.042.332
Axis inclination 45° TLS Faro Focus 3D 5.203.483
Table 2: Flights parameter Table 3: Number of points in the 3D model

As can be seen from table 3, the number of points processed by

Visual SfM is comparable with those of the point cloud processed
by Photoscan, while both are much lower than that of the TLS
survey.
The point clouds obtained from the processing were validated in
two different ways. The first one consists in taking measurements
L1, L2 and H (figure 5) on the building through the different point
clouds; the other one, by calculating the minimal distance
between every point of the 3D model obtained from each flight
and the 3D model produced by the TLS survey. This calculation
used the Nearest Neighbor algorithm of the Cloud Compare
program.

Figure 2: Oblique flight

The TLS survey was done using a Faro Focus 3D laser scanner,
with four exterior scans (figure 4) with a resolution of one point
per 7.67 mm at 10 m (Vacca et al., 2016; Deidda et al., 2012).
The TLS point clouds were georeferenced on the same GCPs
used for point clouds obtained by UAV flight, with r.m.s. of 3.5 L2  L1 
cm. The final 3D model is composed by 5.203.483 points and the
r.m.s. of the registration is 2 mm.

H 

Figure 5: The measurements taken on the building 

Figure 3: GCPs used

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-4-W8-217-2018 | © Authors 2018. CC BY 4.0 License. 219
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018
FOSS4G 2018 – Academic Track, 29–31 August 2018, Dar es Salaam, Tanzania

The table 4 reports the comparisons between each quantity

measured on the 3D model.

Measure TLS VSfM PS TLS- TLS-

VSfM PS
H (m) 11,93 12,03 11,34 -0.1 0,63
L1 (m) 8,72 9,06 8,53 -0,34 0,50
L2 (m) 3,20 3,31 3,02 -0.11 0,24

Table 4: Comparison between the measurements

The analysis of table 4 shows that the measurements made on the

point cloud generated by VisualSfM differ from those coming
from the TLS of a few tens of centimeters, while those obtained
from Photoscan are slightly worse. Figure 7: UAV Photoscan processing
The second validation comported a direct comparison between
the point clouds from the UAV flight and from TLS survey,
through the calculation of the minimal distance between every
point. For practical reasons, this comparison did not interest the
whole building but only the northern façade (figure 6), which did
not contain any eaves or obstructions, thus ensuring that the
statistical results depend only on the processing and not on the
geometry or the contour conditions of the examined element.
Table 5 reports the number of points of the portion of point cloud
representing the north façade for each configuration.

Flight/Survey and software N. points of the 3D model

VisualSfM 123.430
Photoscan 187.441
TLS Faro Focus 3D 515.775
Table 5: Number of points North facade
Also from table 5 it is confirmed that the result on dot density is
better in processing with Photoscan respect to VisualSfM. Figure 8: UAV VisualSfm processing
The calculation of the minimal distance between every point of
the point clouds obtained from the different software packages
and the single cloud obtained from the TLS survey was done In figure 6, the North façade from TLS, in the figures 7 the North
using the CloudCompare software, which compares point clouds façade from UAV processed with Photoscan and in the figure 8
representing the same object but acquired in different times the North façade from UAV processed with VisualSfM.
and/or with different instruments. CloudCompare also calculates Table 6 reports the results of the comparisons between the point
the following statistics: minimal distance, maximal distance, clouds obtained with TLS and the point clouds from VisualSfM
average distance and standard deviation. and Photoscan processed.

Software VisualSfM Photoscan

Min (m) 0 0
Max (m) 0.363 0.397
Mean (m) 0.064 0.008
Dev. Stand (m) 0.073 0.031
Table 6: Statistical values of the comparisons between the TLS
data and the point clouds

Table 6 shows that statistics on point clouds from VisualSfM and

Photoscan are comparable. Only the VisualSfM standard
deviation is double that of Photoscan but always below 10 cm.
Figures 9, 10, 11 and 12 show the discrepancies maps and
histogram between the TLS point clouds and the ones obtained
from the UAV surveys.

Figure 6: TLS point cloud.

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-4-W8-217-2018 | © Authors 2018. CC BY 4.0 License. 220
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018
FOSS4G 2018 – Academic Track, 29–31 August 2018, Dar es Salaam, Tanzania

Figure 9: Discrepancy map between the TLS cloud and UAV

Photoscan processing.

Figure 12: Discrepancy histogram between the TLS cloud and

UAV VisualSfM processing.

Table 7 shows comparisons between VisualSfM and Photoscan

processing.

Software
Min (m) 0
Max (m) 0.597
Mean (m) 0.070
Dev. Stand (m) 0.089

Table 7: Statistical values of the comparisons between the

Photoscan data and the VisualSfM data

Figure 10: Discrepancy histogram between the TLS cloud and

UAV Photoscan processing.

Figure 13: Discrepancy map between the UAV Photoscan

processing and UAV VisualSfM processing.

Figure 11: Discrepancy map between the TLS cloud and UAV
VisualSfM processing.

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-4-W8-217-2018 | © Authors 2018. CC BY 4.0 License. 221
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018
FOSS4G 2018 – Academic Track, 29–31 August 2018, Dar es Salaam, Tanzania

Berni J.A.J.; Zarco-Tejada P.J.; Suárez L.; González-Dugo V.;

Fereres E., 2009. Remote sensing of vegetation from UAV
platforms using lightweight multispectral and thermal imaging
sensors. Int. Archives of Photogrammetry, Remote Sensing and
Spatial Information Sciences, Hannover, Germany, 38 (1-4-
7/W5)

Chiabrando, F.; Nex, F.; Piatti, D.; Rinaudo F., 2011. UAV and
RPV systems for photogrammetric surveys in archeological
areas: two tests in the Piedmont region (ITALY). Journal of
Archaeological Science (3):697–710.
doi:10.1016/j.jas.2010.10.022, ISSN: 0305–4403

Chiabrando, F.; Lingua, A.; Rinaudo, F.; Spanò, A., 2012.

Archaeological site monitoring: UAV photogrammetry could be
an answer. Int. Archives of Photogrammetry, Remote Sensing
and Spatial Information Sciences, 39(5)

Figure 14: Discrepancy histogram between the UAV Photoscan Deidda, M.; Vacca, G., 2012. Tecniche di rilievo Laser Scanner a
processing and UAV VisualSfM processing. supporto del progetto di restauro conservativo dei beni culturali.
L’esempio del Castello di Siviller e del campanile di Mores. Boll.
3. CONCLUSION SIFET, 4, 23–39.
The paper describes a UAV methodology and instrumentation for Deidda, M.; Pala, A.; Vacca, G., 2013. An example of a tourist
3D surveying of buildings through the acquisition of nadiral and location-based service (LBS) with open-source software. Applied
oblique images. In particular we wanted to test the accuracy and Geomatic, 5, pp.73–86.
functionality of an open source software and compare it to those
already extensively tested by Photoscan, as well as to the 3D Deidda, M.; Musa, C.; Vacca, G., 2015. A GIS of Sardinia’s
model coming from a TLS survey. Coastal defense system (XVI—XVIII century). Int. Arch.
The use of VisualSfM was easy and immediate, with faster Photogramm. Remote Sens. Spat. Inf. Sci., 40–4/W7, pp.17–22.
processing times especially compared to those of Photoscan. It
would be useful, however, to be able to customize, via the input Ezequiel, C. A. F.; Cua, M.; Libatique, N. C., 2014. UAV Aerial
of parameters, the management of the elaborations from the Imaging Applications for Post-Disaster Assessment
creation of the sparse cloud to the dense one, through the use of Environmental Management and Infrastructure Development. In
PMVS/CMVS. Proceedings of the International Conference on Unmanned Aerial
Another problem encountered was the georeferencing, which Systems, pp. 274-283
was a long and not very automated process. Overall, however, we
can say that the open source software VisualSfM is really useful Furukawa, Y., Ponce, J., 2010. Accurate, dense, and robust mul-
in all those contexts where one needs a 3D reconstruction of tiview stereopsis. IEEE transactions on pattern analysis and
buildings quickly and accurately, aimed at the geometric machine intelligence, 32(8), pp. 1362-1376.
knowledge of the building, at the analysis of his health or as the
basis for a possible conservative restoration project. Lingua, A.; Noardo, F.; Spanò, A.; Sanna, S.; Matrone, F., 2017.
The research will continue by applying the same methodology to 3D Model generation using oblique images acquired by UAV. Int.
buildings of historical-architectural importance to verify the Archives of Photogrammetry, Remote Sensing and Spatial
correspondence of the processing of VisualSfM software to Information Sciences, Volume XLII-4/W2 pp. 107-115
architectural details whose modeling could be more complex.
Furthermore, the 3D models generated can be a good basis for Remondino, F.; Barazzetti, L.; Nex, F. ; Scaioni, M.; Sarazzi, D.,
the creation of 3D City Models or 3D GIS. 2011. UAV Photogrammetry for mapping and 3D modeling -
current status and future perspectives-. The International
4. ACKNOWLEDGEMENTS Archives of the Photogrammetry, Remote Sensing and Spatial
Information Sciences, Volume XXXVIII-1/C22, pp. 25-31
ISPRS Zurich 2011 Workshop, 14-16 September 2011, Zurich,
This study was also supported by the project "Healthy Cities and
Switzerland
Smart Territories", founded by the Foundation of Sardinia and
Autonomous Region of Sardinia (Fondazione di Sardegna – Remondino, F.; Nex, F.; Menna, F.; Nocerino, E.; Spra, M.G.,
Convenzione triennale tra la Fondazione di Sardegna e gli Atenei 2014. State of the art in High Density image matching. The
Sardi Regione Sardegna 2016). Photogrammetric Record 29(146): pp. 144-166.
5. REFERENCES Szeliski, R., 2010. Computer vision: algorithms and applications.
Springer
Aicardi, I.; Dabove, P.; Lingua A.; Piras M., 2016. Integration
between TLS and UAV photogrammetry techniques for foresty Rossi, P.; Mancini, F.; Dubbini, M.; Mazzone, F.; Capra, A.,
applications. iForest, pp. e1-e7 doi:10.3832/ifor1780-009 2017. Combining nadir and oblique UAV imagery to reconstruct
quarry topography: methodology and feasibility analysis.
Barbarella, M., 2014. Digital technology and geodetic
European Journal of Remote Sensing, 50:1, 211-221, DOI:
infrastructures in Italian cartography. Città e Storia, vol 9, p.91-
10.1080/22797254.2017.1313097.
110 ISSN 1828-6364.

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-4-W8-217-2018 | © Authors 2018. CC BY 4.0 License. 222
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4/W8, 2018
FOSS4G 2018 – Academic Track, 29–31 August 2018, Dar es Salaam, Tanzania

Vacca, G.; Dessì, A.; Sacco, A., 2017. The Use of Nadir and
Oblique UAV Images for Building Knowledge. ISPRS Int. J.
Geo-Inf., 6, 393

Vacca, G.; Mistretta, F.; Stochino, F.; Dessi, A., 2016. Terrestrial
laser scanner for monitoring the deformations and the damages
of buildings. In Proceedings of the International Archives of
Photogrammetry, Remote Sensing and Spatial Information
Sciences, pp. 453–460.

Vacca, G.; Fiorino, D.R.; Pili, D., 2018. A Spatial Information

System (SIS) for the Architectural and Cultural Heritage of
Sardinia (Italy). ISPRS Int. J. Geo-Inf., 7, 49.

Westoby, M.J.; Brasington, J.; Glasser N.F.; Hambrey, M.J.;

Reynolds, J.M., 2012. ‘Structure-from-Motion’
photogrammetry: a low-cost, effective tool for geoscience
applications. Geomorphology, 179, pp. 300–314.

Wu, C., 2011. VisualSFM: A visual structure from motion

system. http://ccwu.me/vsfm (accessed 24 March 2018)

Wu, C., 2007. SiftGPU: A GPU implementation of scale invari-

ant feature transform (SIFT).

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-4-W8-217-2018 | © Authors 2018. CC BY 4.0 License. 223