Ham 2013

Accepted Manuscript
Title: EPAR: Energy Performance Augmented Reality Models

for Identification of Building Energy Performance Deviations
between Actual Measurements and Simulation Results
Authors: Youngjib Ham, Mani Golparvar-Fard
PII: S0378-7788(13)00148-5
DOI: http://dx.doi.org/doi:10.1016/j.enbuild.2013.02.054
Reference: ENB 4179
To appear in: ENB
Received date: 9-9-2012

Revised date: 5-1-2013
Accepted date: 19-2-2013
Please cite this article as: Y. Ham, M. Golparvar-Fard, EPAR: Energy Performance
Augmented Reality Models for Identification of Building Energy Performance
Deviations between Actual Measurements and Simulation Results, Energy and
Buildings (2013), http://dx.doi.org/10.1016/j.enbuild.2013.02.054
This is a PDF file of an unedited manuscript that has been accepted for publication.
As a service to our customers we are providing this early version of the manuscript.
The manuscript will undergo copyediting, typesetting, and review of the resulting proof
before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that
apply to the journal pertain.
Elsevier Energy and Buildings
1 EPAR: Energy Performance Augmented Reality Models for Identification of Building
2 Energy Performance Deviations between Actual Measurements and Simulation Results
3 Youngjib Ham1 and Mani Golparvar-Fard2
t
4 Abstract
ip
5 Building energy performance simulation tools such as EnergyPlus, Ecotect, and eQuest are
cr
6 widely used to model energy performance of existing buildings and assess retrofit alternatives.
us
7 Nevertheless, predictions from simulations typically deviate from actual measurements.
8 Monitoring actual performance and measuring deviations from simulated data in 3D can help
an
9 improve simulation accuracy through model calibrations, and in turn facilitate identification of
10 energy performance problem. To do that, this paper presents Energy Performance Augmented
M
11 Reality (EPAR) modeling that leverages collections of unordered digital and thermal imagery, in
12 addition to computational fluid dynamics (CFD) models. First, users collect large numbers of
d
13 digital and thermal imagery from the building under inspection using a single thermal camera.
te
14 Through an image-based reconstruction pipeline, actual 3D spatio-thermal models are

p
15 automatically generated and are superimposed with expected building energy performance
ce
16 models generated using CFD analysis through a user-driven process. The outcomes are EPAR
17 models which visualize actual and expected models in a common 3D environment. Within the
Ac
18 EPAR models, actual measurements and simulated results can be systematically compared and
19 analyzed. The method is validated on typical residential and instructional buildings. The results
1
Vecellio Pre-doctoral Fellow, Vecellio Construction Engineering and Management Group, Via Department of Civil
and Environmental Engineering, and Myers-Lawson School of Construction, Virginia Tech, Blacksburg, VA; PH
(540) 235-6532; FAX (540) 231-7532; email: [email protected]
2
Assistant Professor, Vecellio Construction Engineering and Management Group, Via Department of Civil and
Environmental Engineering, and Myers-Lawson School of Construction, Virginia Tech, Blacksburg, VA; PH (540)
231-7255; FAX (540) 231-7532; email: [email protected]
1
Page 1 of 56
1 demonstrate that EPAR models facilitate calibration of building energy performance models and
2 support detection and analysis of building performance deviations.
3 Keywords: Thermogaphy, Image-based 3D Reconstruction, Computational Fluid Dynamics
4 (CFD), Energy Audits
t
ip
5
cr
6 1. Introduction
us
7 The building sector is an increasingly important area for retrofit and renovations. Currently,
8 about 75 percent of all electricity and 54 percent of all natural gas produced in the U.S. are
10 an
consumed in buildings [1]. According to the latest forecast on energy consumptions by the U.S.
Energy Information Administration (EIA) [2], between 2010 and 2030, the increase in the energy
M
11 consumption of the building sector (2098 billion kWh) is anticipated to be much more than those
12 of the industrial and transportation sectors (1187 and 984 billion kWh respectively). This
d
13 increase in energy use can have significant destructive impacts as the majority of this energy is
te
14 primarily produced from burning fossil fuels (e.g., coal, petroleum, and gas). This in turn makes
p
15 the building sector the largest emitter of Green House Gases (GHG) and the single primary
ce
16 contributor to anthropogenic climate change [3]. In order to reduce the excessive energy
17 consumptions of the building sector and minimize their environmental impacts, much attention is
Ac
18 paid to retrofitting and renovation of existing buildings. Consequently, over the next 30 years
19 roughly half of the entire existing buildings in the U.S. (i.e., about 14 billion square meters of the
20 existing buildings) require renovations to meet higher energy efficiency standards [4]. The issue
21 of poor energy performance is not only limited to the existing buildings. About a quarter of the
22 newly certified buildings do not also save as much energy as their primary predictions [5].
2
Page 2 of 56
1 Finding energy performance problems is an important step in improving energy
2 efficiency of existing buildings [8]. The most dominant method for modeling energy
3 performance of existing buildings and assessing various retrofit alternatives is the use of building
4 energy performance simulation tools such as EnergyPlus, Ecotect, and eQuest. Nevertheless, due
t
ip
5 to many assumptions and simplifications that are typically made during the modeling process,
6 predictions obtained from these simulation tools deviate from the actual measurements [5-7].
cr
7 Hence, oftentimes energy simulation models need to be calibrated to represent the actual energy
us
8 performance levels. Without a proper baseline model which contains the detailed and accurate
9 energy performance at the building, system, and component levels, accurately capturing all major
10
11 an
performance problems can be challenging [8]. Any deviations between actual measurements and
simulated results of building energy performance can be attributed to the following reasons:
M
12 (1) The difficulty in modeling occupancy patterns over the lifecycle of a building: The
d
13 challenges associated with modeling dynamic patterns of building occupancy may

te
14 result in misrepresentation of the actual energy performance of a building over its
15 lifecycle.
p
ce
16
17 (2) Building performance problems [8, 9]: Typically during the modeling process, all
Ac
18 building elements are assumed to have similar material conditions. As a result, faulty
19 behaviors in buildings such as poor insulation and construction defects - which can
20 account for up to 10 percent of the total building energy consumption [10] and can
21 influence the charge in HVAC systems - are not accurately modeled [11]. This is
22 primarily for two reasons:
23 a. The actual building conditions are not known at the simulation stage,
3
Page 3 of 56
1 hence all material conditions and compositions of the building elements
2 are considered to be the identical;
3 b. The current simulation tools typically do not enable modeling varying
4 conditions of the building elements. Thus, in contrast to the site
t
ip
5 measurements, simulated data may not reflect the actual building
6 performance problems.
cr
7
us
8 Systematic comparisons between actual performance measurements and simulated results
9 can support identification of the abnormal building performance issues. Thus, in order to
10
11 an
proactively rectify building performance issues and improve energy efficiency, there is a need for
robust methods that can assist with detection and measurement of performance deviations. These
M
12 methods need to be rapid, non-destructive, and inexpensive so they can be widely applied to all
d
13 buildings.
te
14 One of the interesting developments and rather contemporary practices for qualitative
15 assessment of building energy performance is the infrared thermography [12-15]. Infrared

p
ce
16 thermography as a reality capturing method measures building surface temperatures, allowing
17 thermal anomalies caused by structural problems to be detected both from inside and outside
Ac
18 buildings. Compared to spot radiometers which measure thermal radiation one-spot at a time or
19 thermal line scanners that show radiant temperature value along a line, thermal cameras can
20 capture the wide distribution of the surface temperatures at a high level of detail [16]. In contrast
21 to contact thermometers, thermography provides non-contact and continuous measurements.
22 Nevertheless, current thermographic practices and studies on capturing actual building energy
23 performances have several limitations:
4
Page 4 of 56
1 (1) The current practice of thermography primarily focuses on direct application of
2 2D thermal imagery which makes the process of data collection and analysis time-
3 consuming. Due to the low spatial resolution of consumer-level thermal cameras
4 (typically 160 120 or 320 240 pixels), the building areas which can be represented
t
ip
5 in a single thermal image are relatively small compared to their digital counterparts.
6 Thus, modeling the entire building energy performance requires a large number of
cr
7 thermal images;
us
8 (2) Since these large numbers of unordered thermal images are typically not geo-
9 tagged, it is difficult for auditors to figure out where these images were captured from
10
11 an
at a later stage, or what areas and elements they are representing. Although a current
practice [17] suggests registration of pairs of thermal and digital images (picture-in-
M
12 picture overlay), yet analyzing a large number of these overlaid 2D images for the
d
13 purpose of whole building energy performance assessment can be time-consuming.

te
14
15 In order to (1) minimize the challenges associated with the current practice of
p
ce
16 thermography and (2) make better use of the continuous thermal information captured by thermal
17 imagery for performance comparison, there is a need for modeling and exploration of both actual
Ac
18 and expected energy performance in a 3D environment. To do that, this paper presents a new
19 Energy Performance Augmented Reality (EPAR) modeling method which jointly models and
20 visualizes actual and expected energy performance of buildings in form of spatio-thermal models
21 within a common 3D environment. The proposed method leverages the state-of-the-art computer
22 vision techniques for image-based 3D reconstruction, a consumer-level single thermal camera
23 which has a built-in digital camera, and computational fluid dynamics (CFD) simulation. In the
5
Page 5 of 56
1 proposed method, the digital and thermal imagery are capturing the as-is building conditions and
2 the actual energy performance respectively. In the meantime, the CFD models represent the
3 expected building energy performance. Within the resulting EPAR environments wherein both
4 thermal and digital imagery are automatically 3D-registered, the actual measurements acquired
t
ip
5 from a consumer-level thermal camera and the simulation results from CFD analysis can be
6 systematically compared in 3D. Such comparisons can help calibrate simulation models and
cr
7 identify potential retrofit areas. Figure 1 shows the overall concept of generating actual and
us
8 expected energy performance models and integrating those models in augmented reality
9 environments.
10
an
<Insert Figure 1>
M
11
12 In the following sections, first the challenges of the current methods for 3D actual
d
13 spatio-thermal modeling and identification of building performance deviations using CFD

te
14 analysis are reviewed. Then, the underlying modeling methodology for generating the EPAR
p
15 environments is discussed in detail. Finally, the experimental results on three case studies are
ce
16 discussed, and the potential benefits and challenges of the proposed method are presented in
17 detail. Video demos along with additional supplementary materials including image data can be
Ac
18 found at www.raamac.cee.vt.edu/epar.
19
20 2. Background and Related Works
21 2.1 3D spatio-thermal modeling of buildings
22 Over the past few years, several research groups have focused on overcoming the current
6
Page 6 of 56
1 challenges associated with the application of a large number of unordered and non-geo-registered
2 thermal imagery. These efforts are primarily focusing on improving the quality of measurements
3 and how building thermal performance data can be interpreted. The common focus among these
4 works is the generation and use of 3D thermal models as opposed to conventional 2D image
t
ip
5 representations. A 3D building model containing thermal values allows users to easily and
6 quickly recognize how temperature values are spatially distributed in a given space. One recent
cr
7 example is the work of Lagüela et al. [18] which presented a methodology for combining 3D
us
8 geometrical information acquired from a laser scanner and temperature data from a thermal
9 camera. Despite the effectiveness of this method for exterior surveys and the accuracy in
10
11 an
measurements, it may not be easily applicable for interior surveying purposes. In their more
recent work [19], Lagüela et al. discussed the difficulties associated with the application of
M
12 fusing thermography with laser scanning point clouds for confined indoor environments or time-
d
13 limited surveys. To overcome these challenges, they introduced an image-based method for 3D
te
14 thermal modeling of buildings. To do that, thermal image mosaics and the corresponding digital
15 images are initially fused. By using Photomodeler Scanner [20] and through a semi-automated
p
16 calibration process, the fused imagery are then used for 3D thermal modeling purposes. This
ce
17 works validates the application of image-based 3D building thermal models; nonetheless, this
Ac
18 process involves several semi-automatic image registration and mosaicking steps which can be
19 time-consuming and labor-intensive. For example, in order to implement the area-based
20 registration algorithm based on cross-correlation technique, the user needs to manually choose
21 the common regions between consecutive thermal image pairs. Moreover, manual verifications
22 are required to eliminate the mismatches during feature matching step. Cho and Wang [21]
23 proposed 3D thermal modeling of building envelops using a hybrid LIDAR system which
7
Page 7 of 56
1 consists of a laser scanner to generate the building geometry and an thermal camera to collect
2 temperature data of building surfaces. This system generates the 3D building point cloud with the
3 corresponding temperature values at the point level. The proposed method has been tested on
4 residential building façades, and promising results are presented. More recently, Im et al. [22]
t
ip
5 proposed a new method for visualizing 3D thermal models of building envelops on web-based
6 geospatial systems such as Google Earth which can be very useful when modeling of multiple
cr
7 buildings are considered. Another recent work is Nüchter [23] which presented a 3D thermal
us
8 modeling method for building indoor environments using a robot equipped with a laser scanner
9 and a thermal camera. However, no validation results are presented on actual confined and
10
an
complex indoor environments where the robot movement can be challenged.
M
11 Despite all benefits of using laser scanners, their direct application can suffer from the
12 mixed pixel phenomena within the built environments [24, 25]. This in particular requires the
d
13 user to manually remove the associated noise in a post-processing stage. Also, the outcomes of
te
14 laser scanning buildings are typically Cartesian point clouds. These models lack building
15 semantics such as surface and material characteristics as well as spatial relationships between
p
ce
16 elements. Hence, users need to perform additional tasks to identify what building elements are
17 associated with the specific thermal performance. The cost of high precision laser scanners is
Ac
18 also still relatively high (~$70-100K). Although their cost is decreasing, yet it can still limit their
19 application for small-size engineering works such as energy auditing of a single-family housing.
20 Overall, laser scanners can be very useful for the purpose of large-scale 3D thermal modeling,
21 yet they have several practical challenges that may make their application difficult to easily adapt
22 to building interior surveys.
23
8
Page 8 of 56
1 2.2 Building energy simulation using CFD
2 Today, computational fluid dynamics (CFD) is the most sophisticated type of airflow modeling
3 used for the building sector. CFD analysis can predict the detailed spatial distributions of
4 temperature and velocity by solving the partial differential governing equations for mass,
t
ip
5 momentum, and energy conservation within each zone of buildings [26, 27]. CFD has been
cr
6 effectively applied to many indoor built environment analyses, and the results are typically more
7 accurate compared to other airflow modeling methods. In contrast to other methods which model
us
8 space-averaged indoor environmental conditions, the results of CFD analysis provide the most
9 detailed information on building energy performance. To address the discomfort problem during
10
an
winter and identify optimized ventilation opening geometry, Carrilho da Graca et al. [28] utilized
M
11 CFD analysis to simulate the indoor airflow. To find the optimized configurations, several
12 experiments were conducted on different building envelop permeability levels. Walker et al. [29]
d
13 simulated indoor airflow distributions using CFD analysis to understand buoyancy-driven natural
te
14 ventilation. They compared the outcomes with the actual data measured by thermocouples in a
15 reduced-scale building model to determine the temperature variations. The CFD analysis was
p
ce
16 recently used by Li et al. [30] to analyze the indoor thermal environment of a train station
17 building for different types of air-condition systems. They also compared their expected results
Ac
18 with the actual measurement data sensed by thermocouples. Van Hooff et al. [31] used the wind
19 tunnel experiments and CFD analysis to compare and explore the air flow conditions in a venturi
20 shaped roof for natural ventilation of buildings. All of these research studies demonstrate the
21 effectiveness of the CFD analysis for indoor energy performance modeling and the comparison
22 with actual measurements. Nonetheless, there is still a need for more detailed comparisons
23 between actual and simulated energy performance in a given space. This is particularly important
9
Page 9 of 56
1 because in almost all of these cases, a few individual measurements from several designated
2 locations were utilized for performance comparison purposes.
3 In the meantime, Malkawi and Srinivasan [32] and Lakaemper and Malkawi [33]
4 presented a CFD-based augmented reality environment for immersive visualization of the
t
ip
5 expected indoor energy performance. Despite the advantages in visualizing building energy
cr
6 performance, the proposed augmented reality environments do not contain the actual energy
7 performance data. Rather, they only show the simulated energy performance overlaid on the
us
8 surrounding physical environment. If the simulated energy performance is visually overlaid on
9 the actual energy performance of the real-world in 3D, it can significantly facilitate the detailed
10
an
comparison of the two results and can help automate measurement of the performance deviations.
M
11 More research needs to be done to automatically generate 3D actual energy performance models
12 and integrate those with CFD models for the purpose of inspecting and analyzing performance
d
13 deviations in 3D.
te
14
p
15 3. Overview of the EPAR Modeling

ce
16 Given a collection of unordered yet paired digital and thermal imagery, as well as a CFD models,
17 our goal is to create a new Energy Performance Augmented Reality (EPAR) modeling method.
Ac
18 The method enables identification and analysis of deviations between actual and simulated
19 building energy performances in a 3D environment. The data and process in the proposed
20 method are illustrated in Figure 2.
21 <Insert Figure 2>
22
10
Page 10 of 56
1 Our approach mainly consists of three parts: (1) First, a large number of thermal and
2 digital images of the building under inspection are casually collected using a single thermal
3 camera. Here, a thermal camera which has a built-in digital camera is used to simultaneously
4 capture pairs of thermal and digital images. Next, a 3D dense point cloud of the building is
t
ip
5 automatically created using an image-based 3D reconstruction pipeline which consists of
6 Graphic Processing Unit (GPU)-based Structure-from-Motion (SfM) and Multi-View Stereo
cr
7 (MVS) algorithms. Subsequently, a 3D dense thermal point cloud of the building is reconstructed
us
8 using a new image-based 3D thermal modeling algorithm involving a one-time thermal camera
9 calibration and the estimation of the relative pose between thermal and digital lenses. The
10
11 an
resulting 3D thermal point clouds are then automatically superimposed with the 3D reconstructed
scenes, generating an actual 3D spatio-thermal model which stores thermal values at 3D point
M
12 level. In the meantime, (2) to form an expected 3D spatio-thermal model and provide detailed
d
13 information on the dynamic thermal distribution, the building energy performance is simulated
te
14 through CFD analysis. The geometrical information extracted from the 3D building point clouds
15 and the environmental boundary conditions from EnergyPlus simulation using the local weather
p
16 data are used to initialize the CFD analysis. Finally, (3) the actual and expected 3D spatio-
ce
17 thermal models are superimposed within a common 3D environment. The outcomes are EPAR
Ac
18 models which jointly visualize the 3D actual and expected spatio-thermal models and
19 automatically 3D-register thermal and digital imagery within the same environment. The
20 following explains each of these steps in more detail.
21
22 4. Actual 3D Spatio-Thermal Modeling
23 4.1 Image-based 3D reconstruction of as-built representation
11
Page 11 of 56
1 To estimate the geometry of the scene and recover the camera configuration, we use a computer
2 vision technique known as Structure from Motion (SfM). The goal of SfM is to find and produce
3 the 3D structure of a scene from the motion of a camera. Throughout the process of SfM, the 3D
4 scene geometry as well as the relative locations and orientations of the camera are semi-
t
ip
5 automatically or automatically recovered. In the proposed SfM process, the structure and motion
6 are estimated and recovered from uncalibrated image sequences. Calibration of image sequences
cr
7 refers to recovering the focal length, distortion, and other intrinsic parameters of a camera
us
8 through a supervised or unsupervised process. Our proposed SfM process which leverages
9 collections of uncalibrated imagery consists of the following four steps: (1) a group of feature
10
11 an
points are independently detected in each of the input images, and a descriptor is learned for each
feature point. In this step, we use Scale-Invariant Feature Transform (SIFT) algorithm [34] to
M
12 automatically find distinctive 2D visual features. These features with a high probability can be
d
13 detected from images taken from various camera locations, orientations, and scales. Once these
te
14 feature points are detected, a descriptor vector is learned for each. The descriptor captures a
15 probabilistic representation of intensity changes in all directions around that feature point. To cut
p
16 down the overall computational time of the process, we use a GPU-based implementation of
ce
17 SIFT [35]; (2) the descriptor vectors of the detected feature points are matched between each pair
Ac
18 of images using a nearest neighborhood matching algorithm. The Epipolar geometries between
19 each digital image pair are then formed by estimating the Fundamental matrix (F) within a
20 RANdom SAmple Consensus (RANSAC) model fitting algorithm [36]. Fitting F-matrix in the
21 RANSAC loop helps remove erroneous matches. In the context of reconstructing built
22 environments, many visual features are typically repeated. Hence, this process can eliminate
23 those false matches that are not consistent with the dominant rotation and translation between
12
Page 12 of 56
1 pairing visual features; (3) the 3D sparse reconstruction process is initialized using an image pair
2 with (a) maximum percentage of inliers after fitting F-matrix between matching features in the
3 RANSAC loop and (b) minimum percentage of inliers among matching features after fitting a
4 Homography matrix in the RANSAC loop. We set the minimum number of matching feature
t
ip
5 points to be 100, and then search for the pair that has the lowest percentage of inliers to the
6 recovered Homography matrix in the RANSAC loop. Such selection heuristics helps identify an
cr
7 image pair that has a significant overlap, yet images are taken from two different locations with a
us
8 reasonably wide baseline. (4) Starting from the initial 3D reconstruction, the additional locations
9 and orientations of the 3D points are incrementally calculated. The intrinsic and extrinsic
10
11 an
parameters of the added cameras are also estimated using the Direct Linear Transformation
(DLT) in the RANSAC loop; (5) finally, the camera parameters and the locations of the 3D
M
12 points are optimized using bundle adjustment algorithm [37]. Here, we use the GPU-based
d
13 sparse bundle adjustment library [38] to reduce the computational cost. The readers are
te
14 encouraged to look into [39, 40] for more details on the SfM procedure. The outcomes of the
15 GPU-based SfM process are sparse 3D point clouds along with the intrinsic and extrinsic digital
p
16 camera parameters which will be used for the dense 3D reconstruction step.
ce
17 For dense reconstruction, a robust Multi-View Stereo (MVS) algorithm [41, 42] is used.
Ac
18 The MVS algorithm requires all cameras to be internally and externally calibrated. Here, the
19 camera calibration information from the SfM algorithm is used as the input. The following data
20 structures are fed into the pipeline: (1) a set of camera projection matrices calculated from the
21 GPU-based SfM algorithm; (2) a collection of undistorted digital images which contains the as-
22 built geometrical conditions. By using the above datasets, features detected by Harris and
23 Difference-of-Gaussians (DoG) operators are first matched for all image sets, returning sparse
13
Page 13 of 56
1 2D patches possibly containing several false positives. Next, the expansion and filtering
2 processes are iteratively implemented to make the patches denser and filter away false matches
3 using visibility constraints. The outcomes of this reconstruction process are dense 3D point cloud
4 models which represent the as-built conditions of the under inspection buildings.
t
ip
5
cr
6 4.2 3D thermal modeling of building interiors
us
7 Our method for thermal modeling of building interiors builds upon our recent work on
8 reconstructing 3D building thermal performance models from collections of unordered thermal
an
9 images [11]. Our initial approach for 3D thermal modeling was to use the SfM algorithm [42]
10 which is recently used for 3D building modeling purposes. We considered two alternatives: (1)
M
11 using the thermal images within a SfM process; (2) using the digital images within a SfM
12 process and finding correspondences between unordered digital and thermal imagery. To
d
13 automatically implement the GPU-based SfM process for the unordered thermal images, we
te
14 applied the state-of-the-art feature detecting and matching algorithms such as SIFT, Affine-SIFT
p
15 (ASIFT), and Speeded Up Robust Feature (SURF). However, none of these feature detecting and
ce
16 matching algorithms worked, or they performed poorly for the entirety of our experiments (for
17 both pairs of thermal-thermal and thermal-digital imagery). More importantly, even before fitting
Ac
18 the F-matrix within the RANSAC loop, the majority of the matching features in image pairs
19 were incorrect. Figure 3 shows examples on the limitation of the state-of-the-art feature
20 detecting and matching algorithm between (1) a pair of thermal images; and (2) a pair of digital
21 and thermal images. Subfigure (b) and (d) show 232 and 210 SIFT keypoints detected in
22 subfigure (a) and (c) respectively while (e) shows 24 keypoints that were matched between the
23 pair of thermal images. Subfigure (g) and (i) show 10,992 and 232 SIFT keypoints detected in
14
Page 14 of 56
1 subfigure (f) and (h) respectively while (j) shows only 19 keypoints that were matched between
2 the pair of digital and thermal images. The number of matched points is not enough for SfM
3 process, and more importantly most matches are incorrect in the both cases.
4 <Insert Figure 3>
t
ip
5
cr
6 Such poor performance can be attributed to the inherent low resolution of thermal images
us
7 (160 120 or 320 240 pixels) which itself is due to the current technical limitations in the
8 manufacturing process of thermal lens. Another more important reason is that thermal images
10 an
typically lack distinct visual features that could be easily detected by the state-of-the-art feature
detection techniques. For thermal imaging purposes, gradient color-coding is commonly used to
M
11 represent a range of temperature values. As a result, thermal imagery only show sharp intensity
12 changes when there is an actual difference in the temperature. Thus, distinct features detected in
d
te
13 digital imagery due to geometrical configurations are not typically captured within thermal
14 imagery. As a result of these limitations, it is difficult to use the SfM algorithm for calculation of
p
15 the camera parameters from unordered thermal imagery. To overcome this challenge, a new 3D
ce
16 thermal modeling method is introduced in this work.

Ac
17 First, a new camera calibration process is formalized and implemented to estimate the
18 intrinsic parameters of the thermal camera. These parameters include focal length ( f T ) and
1 2
19 radial distortion parameters ( k T and k T ). For calibrating digital images, the current practice
20 involves application of calibration rigs [43]. However, the corners and lines on these traditional
21 boards cannot be distinctly differentiated in thermal imagery. This is because the thermal camera
22 can only detect the changes in infrared energy and hence cannot locate areas which have similar
15
Page 15 of 56
1 temperature values. Thus, user-selection of the intersecting corners of the calibration rig from
2 thermal images is still a major challenge. To overcome this issue, a thermal calibration rig
3 (550 700mm) with 42 small LED lights located on the grid intersections (δ=10cm) was designed
4 (Figure 4). This rig was subsequently used to facilitate the extraction of the grid corners from the
t
ip
5 thermal imagery. The LED lights generate enough heat to easily distinct themselves from their
cr
6 surrounding environment. This facilitates the user-centric process of selecting grid corners for
7 calibration purposes.
us
8 <Insert Figure 4>
an
9
10 Thermal cameras are typically equipped with wide-angle lens. These lenses facilitate the
M
11 data collection process by increasing a camera’s field-of-view. Nonetheless, they can also create
12 significant radial distortions. Hence, in this step, the radial distortion of each thermal image is
d
te
13 removed using the distortion parameters calculated from the calibration process. Assuming a
14 pinhole camera model, the thermal images are undistorted using the following equations:
p
pundistorted = f T × r ( p distorted ) × p distorted (1)

ce
r ( p ) =1.0 + kT1 × p + kT2 × p (2)

2 4
Ac
15
16 where, p = [u , v ,1]T is the local pixel coordinates, and r(p) is a function which calculates a scale
17 factor to undo the distortion. Before undistorting thermal images, the different temperature scales
18 of thermal images are normalized to a fixed color spectrum so that the RGB color values of each
19 point in thermal imagery correspond to an absolute temperature value.
16
Page 16 of 56
1 Next, we calculate the relative pose of the thermal lens with respect to its digital
2 counterpart. This process helps estimate the extrinsic parameters of the thermal camera (i.e.,
3 location and orientation). We use the extrinsic parameters of the digital camera acquired from the
4 GPU-based SfM algorithm, as a baseline for our calculations. We assume that the digital camera
t
ip
5 is located at the origin of the coordinate system and has an identity rotation. The Nistér’s five-
6 point pose estimation algorithm is used next to calculate the relative pose between the digital and
cr
7 thermal lenses [44]. First, the Essential Matrix is estimated by forming the Epipolar geometry
us
8 between the digital and thermal camera using five corresponding feature points. Then, the four
9 candidates of the relative rotation and translation metrics of the thermal camera with respect to
10
11 an
the digital camera are recovered from the Singular Value Decomposition (SVD) of the Essential
Matrix. In order to determine the true configuration among the four candidates, the Cheirality
M
12 constraint is imposed [36]. The selected corresponding feature points are triangulated using the
d
13 DLT algorithm based on the digital camera (assuming [I | 0]) and the four candidates (Rrel and
te
14 Trel). The results of the triangulations are points Q in 3D. The spatial configuration of these
15 points is verified using the following constraint:

p
If c1 ≡ Q3Q4 > 0 and c2 ≡ (Pi Q )3 Q4 > 0 , then return Pi and Q

ce
(3)
16
Ac
17 where Q3 and Q4 are the third and fourth components of the Homogenous coordinates of the 3D
18 point Q respectively, and Pi =[Rrel | Trel] is the relative pose metrics of the thermal camera with
19 respect to the digital camera. Using the extrinsic parameters of the digital camera (RD and TD)
20 and the transformation matrix between the thermal and digital lens ([Rrel | Trel]), the extrinsic
21 parameters of the thermal camera ([RT | TT]) are calculated through Eq. 4.
17
Page 17 of 56
[ RT | TT ] = [ RD Rrel | RDTrel + TD ] (4)
2 Finally, once the intrinsic and extrinsic parameters of the thermal camera are extracted,
t
3 the 3D dense thermal point cloud model is reconstructed through the same dense 3D
ip
4 reconstruction algorithm presented in the previous section. The undistorted thermal images along
cr
5 with thermal camera parameters are fed into this MVS algorithm. All steps in this process are
us
6 fully automated.
8 4.3 Aligning the 3D building and thermal models

an
M
9 The final step of the process is to superimpose the resulting building and thermal point cloud
10 models within a common 3D environment and form 3D actual spatio-thermal models. In the
d
11 proposed method, the 3D building and thermal models share the same coordinate system. This is
te
12 because for 3D thermal modeling purposes, we used the relative location and orientation of the
p
13 thermal camera with respect to the digital camera to derive the extrinsic camera parameters.
ce
14 Therefore, both 3D building and thermal models can be automatically superimposed by carrying
15 the two point cloud models into a single environment. Figures 5(a) and (b) show several
Ac
16 experimental results of the 3D spatio-thermal modeling on both outdoor and indoor built
17 environments.
19
20 5. Expected 3D Spatio-Thermal Simulation
18
Page 18 of 56
1 5.1 Extracting 3D geometrical information and initializing environmental conditions
2 In order to generate 3D geometrical models of enclosed spaces for CFD analysis, the boundary
3 points are manually extracted from the generated 3D dense building point cloud models (Figure
4 6(a)). Using the 3D coordinates of the selected boundary points (e.g., corners of windows and a
t
ip
5 ceiling), a 3D virtual wireframe model is created (Figure 6(b)). The 3D wireframe models
cr
6 extracted from the 3D building point clouds are not in metric coordinate system. This is because
7 the 3D image-based point cloud models are up-to-scale (i.e., their metric scale is unknown). In
us
8 order to model the real as-built environments, the unit and scale factors of X, Y, and Z
9 coordinates are calibrated based on the site coordinate system. The resulting calibrated domain is
10
an
divided into a series of small discrete volumes using a mesh (Figure 6(c)), and then the energy
M
11 conservation equations are imposed at each mesh element for iterative simulation process.
12 Subsequently, the boundary types are specified according to each building component (Figure
d
13 6(d)). In this step, the fluid is modeled as an ideal air, and the velocity inlet and outflow
te
14 boundary types which represent HVAC systems are also defined for the solution domain.
p

ce
16
17 The primary environmental boundary conditions for initializing CFD analysis can be
Ac
18 acquired from diverse energy building energy simulation tools. Among them, in this research as a
19 proof of concept, we used EnergyPlus simulation tool. EnergyPlus and CFD simulation provide
20 complementary information required for evaluation of both energy consumption and thermal
21 performances. Integrating these two tools by exchanging the complementary data can eliminate
22 the principal modeling assumptions employed in the separate applications, and thus can provide
23 more accurate predictions on building performance [45]. Given the large size of complementary
19
Page 19 of 56
1 data between EnergyPlus and CFD simulation, there are several available coupling methods.
2 According to Zhai and Chen’s research [26, 46], transferring indoor surface temperatures from
3 simulation tool to CFD and returning indoor air temperature gradient and convective heat
4 transfer coefficient from CFD back to the simulation tool is the most reliable and efficient
t
ip
5 coupling methods. This reliability and efficiency is in forms of the convergence condition,
6 computational time, and the accuracy of the simulation process. Hence, for more accurate CFD
cr
7 analysis, the interior surface temperatures of walls and windows are acquired from EnergyPlus
us
8 simulation. Next, this information is fed into the CFD analysis as environmental boundary
9 conditions.
10
an
M
11 5.2 CFD analysis
12 By using the boundary conditions obtained from the previous section, the expected building
d
13 energy performance in form of a 3D spatio-thermal model is simulated using CFD analysis. The
te
14 simulation specific parameters (e.g., the numeric convergence error tolerances) are manually set
p
15 by the user. Gambit 2.2.30 and Fluent 6.2.16 are used to set up the geometrical model and run the
ce
16 CFD analysis respectively. The air flow in a given space is assumed to be 3-dimentional.
17 Although this requires more computational cost for solution convergence compared to 2-
Ac
18 dimentional modeling, yet the results are more accurate since more realistic airflow
19 characteristics such as airflow separation over building sharp edges can be considered [47].
20 Airflow in buildings is mainly considered to be turbulent; thus the turbulent model is reasonably
21 selected for solving the energy equations. For the turbulence modeling, the renormalization
22 group (RNG) k − ε model [48] is used in this research. The RNG k − ε model was experimentally
23 demonstrated as a robust turbulent model that can provide the most reliable results which closely
20
Page 20 of 56
1 match with the experimental measurements for assessment of indoor thermal environment [49-
2 52]. Also, the segregated solver is used, which the energy equations are sequentially solved or
3 segregated from each other. Based on the above condition, the thermal distribution of each zone
4 is governed by the conservation laws of energy, mass, and momentum. By using the finite
t
ip
5 volume method, the simulation is iterated until the results are converged. Throughout this
6 process, the convergences turbulence parameters (e.g., momentum and continuity of the viscous
cr
7 model) are monitored until the overall convergence conditions are met. In this research, the error
us
8 tolerances for continuity, momentum, and turbulence dissipation rate are assumed to be 10-3
9 respectively. The error acceptable for turbulent kinetic energy is set to 10-6. After converging to a
10
an
solution, the CFD analysis provides detailed information on the thermal distribution of the space.
M
11
12 6. EPAR Modeling for Integrated Visualization of Actual/ Expected Energy Performances

d
13 In order to analyze and visualize the deviations between actual and expected building energy
te
14 performances, the simulated performance model needs to be overlaid on the actual performance
p
15 model. To do that, the graphics window displays of the resulting CFD model is first converted to
ce
16 Virtual Reality Modeling Language (VRML) format. The VRML is a graphics interchange
17 format which enables export of 3D geometrical entities of the simulation results.

Ac
18 As discussed in Section 5.1, the integrated visualization step also involves transforming
19 the 3D spatio-thermal models into the site coordinate system. To find the absolute transformation,
20 we use a closed-form solution based on calculating the unit quaternion [53, 54]. This
21 transformation has 7 degrees of freedom, and hence requires a minimum of three points to be
22 paired between the model and the site coordinate system. These points can either be pre-located
23 during the data collection or could be easily extracted from known dimensions of the built
21
Page 21 of 56
1 environments (e.g., height and width of building elements). In this study, the points are denoted
i i
2 by rModel and rSite respectively, where i is the number of corresponding points, and Model and
3 Site represent the 3D spatio-thermal model and site coordinate systems respectively. The
4 transformation between each corresponding pair (i.e., from one system into another) can be
t
ip
5 i
formulated using Eq. 5. Where s is the scaling factor, R rModel ( )
is the rotation of the 3D spatio-
cr
6 thermal model, and T is the translation factor.
us
i
rSite = sR ( rModel
i
) +T (5)
an
7
8 From the n corresponding pairs, the sum of squared residual errors can be minimized
M
9 through Eq. 6. This formulation enables to transform 3D spatio-thermal model into the site
10 coordinate systems and measure the registration accuracy.

d
− sR ( rModel ) −T
n n
te
2
∑e
1
i 2
= ∑ rSite
1
i i
(6)
p
11
ce
12 Aligning the actual and expected performance models requires geometric transformations
13 between the two models (e.g., scaling, rotation, and translation) to be known. Since both actual
Ac
14 3D spatio-thermal model and the 3D geometry for CFD analysis were calibrated based on the
15 site coordinate system, two models have the same coordinate systems. Consequently, the two
16 models can be automatically superimposed. The digital and thermal images are also
17 automatically 3D-registered on the EPAR models by using the digital camera location and
18 orientation calculated from the SfM step and the relative pose of the thermal camera with respect
19 to the digital camera respectively. The final outcomes are EPAR models which jointly visualize
22
Page 22 of 56
1 the 3D actual and expected spatio-thermal models in a common 3D environment, along with the
2 3D-tagged thermal and digital imagery.
3 In the formed EPAR models, the actual measurement and simulation results of building
4 energy performance can be extracted, and the deviations on 3D indoor surface thermal
t
ip
5 distributions can be analyzed. Here, the actual and simulated temperature values are calculated
cr
6 from the RGB color values of each 3D point clouds based on the individual normalized
7 temperature scale. For the integrated visualization of both the actual and expected building
us
8 thermal performance in addition to the 3D-registration of digital and thermal imagery in a 3D
9 environment, a new visualization environment is also created and prototyped. Within the virtual
10
an
environment, each camera is rendered in form of a pyramid-shape frustum. Once a camera is
M
11 visited in the reconstructed 3D scene, it is automatically texture-mapped with the corresponding
12 digital and thermal images. The user can select to view the digital or thermal image captured
d
13 from the particular camera location and change the viewpoint to jointly study the 3D point
te
14 clouds with the corresponding 3D-registered imagery. Ultimately, the user can navigate the
15 actual and simulated 3D thermal distribution in a given space and analyze performance
p
ce
16 deviations at the level of 3D points.
17
Ac
18 7. Experimental Setup and Validation
19 7.1. Data collection
20 In order to validate the proposed method, three experiments were conducted: an office room of
21 an instructional facility during winter (Case 1); a bedroom of a residential building during
22 summer (Case 2); and a different office room of the same instructional facility during winter
23
Page 23 of 56
1 (Case 3). The digital and thermal images were captured using an FLIR E60 thermal camera
2 which has a built-in digital camera. Table 1 presents the detailed technical specifications of the
3 thermal camera used.
4 <Insert Table 1>
t
ip
5
cr
6 The building component of the first office room in the instructional facility (Case1) was
us
7 modeled as follows: the walls were modeled as concrete; the floor and ceiling were modeled as
8 concrete masonry units and gypsum boards respectively. Then, the bedroom in the residential
an
9 building (Case 2) was modeled as follows: the floor was modeled as concrete; and the ceiling
10 and walls were modeled as wood. Finally, the second office room in the instructional facility
M
11 (Case 3) was modeled as follows: the floor and ceiling were modeled as concrete; and the walls
12 were modeled as concrete and gypsum board. All windows and doors in the rooms are modeled
d
13 as closed conditions in the all experiments. The material properties are assumed to be constant
te
14 and are shown in Table 2.

p
15 <Insert Table 2>

ce
16
Ac
17 Since CFD models are typically implemented in a steady state, the airflow from the
18 HVAC system is assumed to be uniformly distributed on all supply openings with a constant
19 temperature and velocity. The detailed experimental conditions of simulation are summarized in
20 Table 3.
21 <Insert Table 3>
22
24
Page 24 of 56
1 7.2. Performance evaluation measures
2 For proper validation, the 3D-registration accuracy of the 3D actual spatio-thermal model into
3 the site coordinate system was calculated using Eq. 6. This accuracy gives an indication of how
4 precisely the performance comparison can be conducted in 3D. We also evaluated the image-
t
ip
5 based 3D reconstruction process based on model completeness (e.g., the point clouds density and
cr
6 the percentage of successfully registered images), the required computational time for modeling,
7 and the simulation accuracy.
us
8
an
9 7.3. Experimental results
10 In this following section, we first present the experimental results from applications of our
M
11 proposed method. In the subsequent sections, we present various aspects of the method and
12 discuss the open research problems. Table 4 presents the detailed results on the image-based 3D
d
13 reconstruction using digital and thermal imagery. As observed, a relatively high number of
te
14 images were collected for each of these case studies; yet the computational time benchmarked on
p
15 a typical engineering workstation still seems quite reasonable. The success ratio of 0.85 and 0.81
ce
16 in the case 2 and 3 respectively show that several images did not have enough overlap with other
17 imagery. The accuracy of registering the actual 3D spatio-thermal models with the site
Ac
18 coordinate systems was measured by calculating the registration error (Eq. 6). The results are
19 presented in Table 4. As observed from the results, although our method was tested on different
20 cases with varying number of registration points (more than 3 points which is the minimum
21 numbers of the required points for solving 7 DOF), the registration accuracy is within a
22 reasonable practical range. This level of accuracy holds promise that high precision comparison
23 tasks between actual and simulated thermal performances can be conducted within the 3D EPAR
25
Page 25 of 56
1 environments.
2 <Insert Table 4>
t
4 Figure 7, 8, and 9 show actual and simulated 3D spatio-thermal models and the 3D-
ip
5 registered imagery for the experiments. Subfigure (a), (b), and (c) of each figure show the 3D
cr
6 building, 3D thermal, and 3D spatio-thermal models wherein (c) illustrates the superimposition
us
7 of the outcomes from (a) and (b). Subfigure (d) shows the expected 3D spatio-thermal model.
8 This model is the outcome of the CFD analysis for which the required 3D geometry was
an
9 extracted from the 3D building point cloud (Subfigure (a)). The user can navigate the 3D
10 environment, view the scene from any given camera viewpoint, and study the thermal
M
11 performance in relation to the underlying building conditions. Subfigure (f), (g), and (h) present
12 the building and thermal point clouds along with the automatically 3D-registered digital and
d
13 thermal imagery. As presented in the section 4.1, the locations and orientations of each camera
te
14 registered in the 3D virtual environment (Subfigure (e)) are automatically calculated without
p
15 help from dedicated location tracking technologies such as GPS, Ultra Wideband, or RFID tags.
ce
16 In fact, the camera extrinsic parameters required for 3D-registration are calculated through the
17 SfM and digital/thermal camera pose estimation algorithms. In these figures, the camera location
Ac
18 is preserved, and the field of view is slightly tilted to provide better 2D/3D contrasts. With the
19 availability of the 3D-registered imagery on the reconstructed 3D scene, the actual thermal
20 performance and geometrical conditions of the buildings can be interactively interpreted.
26
Page 26 of 56
1 <Insert Figure 9>
3 The EPAR models are visually illustrated in Figure 10, 11, and 12 wherein the actual
t
4 thermal performance models are registered with the simulated models. Subfigure (b), (c), and (d)
ip
5 of each figure, presents the actual and simulated thermal distribution of indoor surface from the
cr
6 same viewpoint. More visual results in form of videos can be found at
7 www.raamac.cee.vt.edu/epar. Each 3D point in the EPAR model acts as a temperature sensor
us
8 which is documenting the temperature at the time the thermal images were captured. In the
an
9 current version of the EPAR visualizer, each 3D point within both 3D spatio-thermal models and
10 CFD models has its own unique RGB color value. The RGB color values are converted into
M
11 numerical temperature data based on the individual temperature scales, and here each color
12 indicates an absolute temperature value. By comparing the temperature values in the EPAR
d
13 models, the actual and expected thermal performance of various areas in a given space can be
te
14 easily compared, the deviations can be easily measured.

p

ce

Ac
18
19 As a proof of concept, we selected five surface regions in these models and compared
20 their actual and simulated thermal performances: an area in the corner of two intersecting walls,
21 three flat regions on the surface of interior walls, and a flat area around a window. The locations
22 of the selected areas are shown in Figure 10(d), 11(d), and 12(d). The temperatures for these
27
Page 27 of 56
1 surfaces were measured by averaging the temperature values from all the 3D reconstructed
2 points within the selected regions.
3 <Insert Table 5>
t
4
ip
5 Table 5 shows the results of comparing the measured wall surface temperature against the
cr
6 predicted simulation results. The performance deviations for each area are reported in forms of
us
7 average temperature (µ) and its standard deviation (σ). The deviations between the actual and
8 simulated values are reported with 96% confidence in measurement accuracy (i.e., µ±2.05σ). In
an
9 case 1, a reasonable agreement can be found between the measured and simulated results. For the
10 measurements in the areas 2 to 5, the average temperature deviation was 0.52-0.96°C. Due to
M
11 typical inaccuracies of measurement and simulation, it is expected to have minor temperature
12 deviations. Hence, the outcomes of the CFD analysis for the areas 2 to 5 were acceptable.
d
13 However, the performance deviation for the area 1 was relatively higher compared to the other
te
14 regions. For case 2, for most areas except for area 1, the wall temperature distributions from the
p
15 estimations of the CFD analysis and the field measurements were approximately consistent. As
ce
16 shown in Table 5, the deviations for the area 1 was more significant compared to the others. For
17 case 3, the average surface temperature deviation was 0.79-1.61°C. Overall, the performance of
Ac
18 the CFD model on predicting the wall surface temperature was reasonable in our experiments.
19 Although simulated temperatures were slightly overestimated or under-predicted for most areas,
20 nonetheless several areas in both cases 1 and 2 have critical deviations. These deviations indicate
21 the need for more detailed performance analysis. Considering the degradation level of these two
22 facilities where built in 1929 and beginning of the 1980s respectively, such critical discrepancies
23 may be caused by the building performance problems. This is primarily due to construction
28
Page 28 of 56
1 defects or insulation voids since the trends of the simulated and measured temperature profiles of
2 the other areas comply well. Additional diagnostics such as blower door tests and partial
3 destructions of these areas are required to find the possible source of heat loss.
t
ip
5 8. Discussion on the Proposed Method and Research Challenges
cr
6 In order to systematically identify and quantify the deviation between the actual and simulated
us
7 building energy performance, the EPAR modeling methodology using the state-of-the-art
8 computer vision techniques and CFD analysis is presented. The proposed method presents a new
an
9 approach for enabling the actual measurement and simulation results on 3D indoor thermal
10 distribution of buildings to be sensed and holistically compared. Through a systematic and non-
M
11 intrusive comparison of measured and simulated surface temperatures of a building, potential
12 building energy performance problems (e.g., heating and cooling losses and building envelope
d
13 water leakages) can be quickly and reliably identified. Thermal performance deviations above
te
14 the pre-defined threshold can provide feedbacks to building auditors on what areas require
p
15 additional detailed diagnostics and in turn facilitate building retrofit since these discrepancies
ce
16 may be caused by building or system degradations. Moreover, during model calibration process,
17 the calculated performance deviations can be used as a point of reference to build actual building
Ac
18 energy performance model within an acceptable tolerance in accuracy. Thus, exploring these
19 performance deviations can support rapid identification of potential energy performance
20 problems in building as well as validating building energy performance simulation results.
21 Although the proposed EPAR model shows the potentials of the systematic comparison between
22 the actual and simulated building energy performance, several research problems remain open
23 which include the followings:
29
Page 29 of 56
1 • Automated extraction of 3D geometry from 3D reconstructed scenes. For the
2 purpose of energy simulation, the proposed method involves a manual process to generate
3 the 3D geometry from the underlying 3D building point cloud model. Compared to the
4 current practice which requires detailed surveying and modeling of the built environment,
t
ip
5 the underlying 3D building point cloud model significantly expedites the data collection
6 and analysis process. Nevertheless, to reduce the time for modeling, more work needs to
cr
7 be done on creating robust algorithms that can automatically extract boundary conditions
us
8 (points and lines) from the underlying 3D building point clouds.
9 • Completed modeling of indoor thermal performance. The proposed method using a
10
11 an
single thermal camera showed potentials on generating 3D thermal models of confined
and complex indoor environments. In the conducted experiment, surface temperatures

M
12 were measured along the walls, nevertheless the models are not yet complete, and as a
d
13 result several zones were still missing. More experiments needs to be conducted to
te
14 establish data collection guidelines and provide proper selection heuristics on image pair
15 for reconstructing the entirety of closed-models. Particularly, these models should include
p
16 reflective surfaces (e.g., windows), ceilings, and floors which are essential for modeling
ce
17 purposes.
Ac
18 • Sensitivity analysis of simulation input parameters. Accuracy and reliability of CFD
19 analysis is sensitive to a wide range of computational parameters such as grid resolution,
20 type of turbulence model, and the level of iterative convergence which are typically set
21 by the user. Thus, detailed sensitivity analyses are needed for more accurate simulation.
22 • Dynamic coupling of EnergyPlus and CFD simulation. According to Zhai and Chen’s
23 research [46], dynamic coupling method can increase the accuracy of simulating energy
30
Page 30 of 56
1 performance for building interiors. For this purpose, EnergyPlus and CFD simulations
2 need to be continuously coupled at each time step of the simulation. Nonetheless, such
3 coupling methods require high computational costs for large-scale facilities. More
4 experiments needs to be conducted to study the iterative coupling methods for more
t
ip
5 accurate building thermal performance simulations.
cr
7 9. Conclusions
us
8 Detecting building energy performance deviations between actual measurements and simulated
9 results can help validate the simulation models and understand inherent building performances
10
an
problems. To that end, this paper presents a new methodology of using unordered digital and
M
11 thermal imagery as well as CFD models to compare between actual and simulated building
12 energy performance and identify the deviations in 3D environment. We validated the proposed
d
13 method in three interior locations of an instructional facility and a residential building. The
te
14 experimental results show the robustness of the proposed EPAR models for identifying
15 performance deviations in terms of surface temperatures in 3D. The EPAR model can be used to
p
ce
16 facilitate not only the identification of potential areas for building retrofit but also the validation
17 of simulation models for calibration purposes. The method still has several areas for
Ac
18 improvement which are left as open research problems: (1) automatically generating complete
19 3D building geometry from photos; (2) automated computing to reduce modeling and analysis
20 time; (3) more experiments with varying conditions to increase the simulation accuracy; and (4)
21 EPAR modeling of an entire building by combining EPAR models of several adjacent closed
22 spaces in a building. These are now currently being explored as part of ongoing research.
23
31
Page 31 of 56
1 References
2 [1] U.S. DOE, 2010 U.S. DOE buildings energy databook, U.S. Department of Energy, 2010.
3 [2] EIA, Annual Energy Review 2010, U.S. Energy Information Administration, 2010.
4 [3] Architecture2030, Problem: The Building Sector, <http://www.architecture 2030.org/>, 2011,
t
ip
5 (accessed 2012).
cr
6 [4] K. Gould, L. Hosey, Ecology & Design: Ecological Literacy in Architecture Education,
7 American Institute of Architects (AIA), 2006.
us
8 [5] C. Turner, M. Frankel, Energy Performance of LEED for New Construction Buildings, New
9 Buildings Institute, Vancouver, WA, 2008.
10
an
[6] J. Yudelson, Greening Existing Buildings (McGraw-Hill's Greensource), McGraw-Hill, New
M
11 York, NY, 2010.
12 [7] E. Azar, C. Menassa, Agent-Based Modeling of Occupants' Impact on Energy Use in

d
13 Commercial Buildings, Journal of Computing in Civil Engineering, (2011),

te
14 doi:http://dx.doi.org/10.1061/(ASCE)CP.1943-5487.0000158.
15 [8] T. Maile, Comparing measured and simulated building energy performance data, Stanford
p
ce
16 University, 2010.
17 [9] J. Kunz, T. Maile, V. Bazjanac, Summary of the energy analysis of the first year of the
Ac
18 Stanford Jerry Yang & Akiko Yamazaki Environment & Energy (Y2E2) Building, in,
19 Stanford, CA: Center for Integrated Facility Engineering, Stanford University., 2009.
20 [10] K. Roth, D. Westphalen, M. Feng, P. Llana, L. Quartararo, The Energy Impact of
21 Commercial Building Controls and Performance Diagnostics: Market Characterization,
22 Energy Impact of Building Faults and Energy Savings Potential, TIAX LCC Technical
23 Report Cambridge, MA, 2005.
32
Page 32 of 56
1 [11] Y. Ham, M. Golparvar-Fard, Rapid 3D Energy Performance Modeling of Existing Buildings
2 using Thermal and Digital Imagery, 2012 Construction Research Congress, West Lafayette,
3 IN, 2012, pp. 991-1000.
4 [12] F. Cerdeira, M.E. Vázquez, J. Collazo, E. Granada, Applicability of infrared thermography
t
ip
5 to the study of the behaviour of stone panels as building envelopes, Energy and Buildings,
6 43 (8) (2011) 1845-1851.
cr
7 [13] S. Martı́n Ocaña, I. Cañas Guerrero, I. González Requena, Thermographic survey of two
us
8 rural buildings in Spain, Energy and Buildings, 36 (6) (2004) 515-523.
9 [14] A. Molin, P. Rohdin, B. Moshfegh, Investigation of energy performance of newly built low-
10
11 an
energy buildings in Sweden, Energy and Buildings, 43 (10) (2011) 2822-2831.
[15] C.A. Balaras, A.A. Argiriou, Infrared thermography for building diagnostics, Energy and
M
12 Buildings, 34 (2) (2002) 171-183.
d
13 [16] U.S. DOE, Thermographic Inspection, U.S. Department of Energy, < http://www.
te
14 energysavers.gov/your_home/energy_audits/index.cfm/mytopic=11200>, 2012, (accessed
15 2012).
p
16 [17] HomeInspex, Thermal Imaging to Identify Air Leaks, Sample Report, HomeInspex, 2011.
ce
17 [18] S. Lagüela, J. Martínez, J. Armesto, P. Arias, Energy efficiency studies through 3D laser
Ac
18 scanning and thermographic technologies, Energy and Buildings, 43 (6) (2011) 1216-1221.
19 [19] S. Lagüela, J. Armesto, P. Arias, J. Herráez, Automation of thermographic 3D modelling
20 through image fusion and image matching techniques, Automation in Construction, 27 (0)
21 (2012) 24-31.
22 [20] PhotoModeler, PhotoModeler Scanner, <http://www.photomodeler.com/products/pm-
23 scanner.htm.>, 2012, (accessed 2012)
33
Page 33 of 56
1 [21] Y. Cho, C. Wang, 3D Thermal Modeling for Existing Buildings Using Hybrid LIDAR
2 System, Proceedings of the 2011 ASCE International Workshop on Computing in Civil
3 Engineering ASCE, Miami, Florida, 2011, pp. 68-68.
4 [22] H.J. Im, M. Gai, C. Wang, Y. Cho, Hybrid Approach to Visualize Building Energy
t
ip
5 Information Model in Geospatial Application Programs, 2012 Construction Research
6 Congress, 2012, pp. 1262-1270.
cr
7 [23] A. Nüchter, Project ThermalMapper, 2012, <http://www.faculty.jacobs-university.de/
us
8 anuechter/thermalmapper.html>, 2012, (accessed 2012).
9 [24] M. Golparvar-Fard, J. Bohn, J. Teizer, S. Savarese, F. Peña-Mora, Evaluation of image-
10
11 an
based modeling and laser scanning accuracy for emerging automated performance
monitoring techniques, Automation in Construction, 20 (8) (2011) 1143-1155.

M
12 [25] S. Kiziltas, B. Akinci, E. Ergen, P. Tang, Technological assessment and process implications
d
13 of field data capture technologies for construction and facility/infrastructure management,

te
14 ITcon, 13 (Special Issue Sensors in Construction and Infrastructure Management) (2008)
15 134-154.
p
16 [26] Z. Zhai, Q. Chen, Solution characters of iterative coupling between energy simulation and
ce
17 CFD programs, Energy and Buildings, 35 (5) (2003) 493-505.

Ac
18 [27] S. Somarathne, M. Seymour, M. Kolokotroni, Dynamic thermal CFD simulation of a typical
19 office by efficient transient solution methods, Building and Environment, 40 (7) (2005)
20 887-896.
21 [28] G. Carrilho da Graça, N.R. Martins, C.S. Horta, Thermal and airflow simulation of a
22 naturally ventilated shopping mall, Energy and Buildings, 50 (0) (2012) 177-188.
23 [29] C. Walker, G. Tan, L. Glicksman, Reduced-scale building model and numerical
34
Page 34 of 56
1 investigations to buoyancy-driven natural ventilation, Energy and Buildings, 43 (9) (2011)
2 2404-2413.
3 [30] Q. Li, H. Yoshino, A. Mochida, B. Lei, Q. Meng, L. Zhao, Y. Lun, CFD study of the thermal
4 environment in an air-conditioned train station building, Building and Environment, 44 (7)
t
ip
5 (2009) 1452-1465.
6 [31] T. van Hooff, B. Blocken, L. Aanen, B. Bronsema, A venturi-shaped roof for wind-induced
cr
7 natural ventilation of buildings: Wind tunnel and CFD evaluation of different design
us
8 configurations, Building and Environment, 46 (9) (2011) 1797-1807.
9 [32] A.M. Malkawi, R.S. Srinivasan, A new paradigm for Human-Building Interaction: the use of
10
11 an
CFD and Augmented Reality, Automation in Construction, 14 (1) (2005) 71-84.
[33] R. Lakaemper, A.M. Malkawi, Integrating Robot Mapping and Augmented Building
M
12 Simulation, Journal of Computing in Civil Engineering, 23 (6) (2009) 384-390.
d
13 [34] D. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of
te
14 Computer Vision, 60 (2) (2004) 91-110.
15 [35] W. Changchang, SiftGPU: A GPU implementation of Scale Invaraint Feature Transform

p
16 (SIFT), <http://cs.unc.edu/~ccwu/siftgpu>, 2007, (accessed 2012).

ce
17 [36] R. Hartley, A. Zisserman, Multiple view geometry, 2 ed., Cambridge University Press, 2004.
Ac
18 [37] B. Triggs, P. McLauchlan, R. Hartley, A. Fitzgibbon, Bundle adjustment - a modern
19 synthesis, International Workshop on Vision Algorithms, Corfu, Greece, 1999, pp. 153-177.
20 [38] W. Changchang, S. Agarwal, B. Curless, S.M. Seitz, Multicore bundle adjustment,
21 Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, 2011, pp.
22 3057-3064.
23 [39] M. Golparvar-Fard, F. Peña-Mora, S. Savarese, Application of D4AR- A 4-Dimensional
35
Page 35 of 56
1 augmented reality model for automating construction progress monitoring data collection,
2 processing and communication, ITcon 14 (Special Issue Next Generation Construction IT:
3 Technology Foresight, Future Studies, Roadmapping, and Scenario Planning ) (2009) 129-
4 153.
t
ip
5 [40] N. Snavely, S. Seitz, R. Szeliski, Modeling the World from Internet Photo Collections,
6 International Journal of Computer Vision, 80 (2) (2008) 189-210.
cr
7 [41] Y. Furukawa, J. Ponce, Accurate, Dense, and Robust Multiview Stereopsis, Pattern Analysis
us
8 and Machine Intelligence, IEEE Transactions on, 32 (8) (2008) 1362-1376.
9 [42] M. Golparvar-Fard, F. Peña-Mora, S. Savarese, Automated model-based progress
10
11 an
monitoring using unordered daily construction photographs and IFC as-planned models,
Journal of Computing in Civil Engineering, (2012) doi:http://dx.doi.org/10.1061/(ASCE)

M
12 CP.1943-5487.0000205.
d
13 [43] J.-Y. Bouguet, Camera Clibration Toolbox, <http://wiki.aia.org/wiki%20pages

te
14 /energy%20modeling.aspx>, 2010, (accessed 2012).
15 [44] D. Nister, An efficient solution to the five-point relative pose problem, Pattern Analysis and
p
16 Machine Intelligence, IEEE Transactions on, 26 (6) (2004) 756-770.

ce
17 [45] Z. Zhai, Q. Chen, P. Haves, J.H. Klems, On approaches to couple energy simulation and
Ac
18 computational fluid dynamics programs, Building and Environment, 37 (8–9) (2002) 857-
19 864.
20 [46] Z.J. Zhai, Q.Y. Chen, Performance of coupled building energy and CFD simulations, Energy
21 and Buildings, 37 (4) (2005) 333-344.
22 [47] O.S. Asfour, M.B. Gadi, A comparison between CFD and Network models for predicting
23 wind-driven ventilation in buildings, Building and Environment, 42 (12) (2007) 4079-4085.
36
Page 36 of 56
1 [48] V. Yakhot, S.A. Orszag, Renormalization group analysis of turbulence. I. Basic theory,
2 Journal of Scientific Computing, 1 (1) (1986) 3-51.
3 [49] P. Rohdin, B. Moshfegh, Numerical predictions of indoor climate in large industrial
4 premises. A comparison between different k–ε models supported by field measurements,
t
ip
5 Building and Environment, 42 (11) (2007) 3872-3882.
6 [50] J.D. Spengler, Q. Chen, Indoor Air Quality Factors in Designing A Healthy Building,
cr
7 Annual Review of Energy and the Environment, 25 (1) (2000) 567-600.
us
8 [51] Q. Chen, Comparison of Different k-ε Models for Indoor Air Flow Computations,
9 Numerical Heat Transfer, Part B: Fundamentals, 28 (3) (1995) 353-369.
10
11 an
[52] G. Evola, V. Popov, Computational analysis of wind driven natural ventilation in buildings,
Energy and Buildings, 38 (5) (2006) 491-501.

M
12 [53] B. Horn, Closed-form Solution of Absolute Orientation using Unit Quaternions, Journal of
d
13 the Optical Society, 4 (4) (1987) 629-642.

te
14 [54] M. Golparvar-Fard, F. Pena-Mora, S. Savarese, Integrated Sequential As-Built and As-
15 Planned Representation with D4AR Tools in Support of Decision-Making Tasks in the

p
16 AEC/FM Industry, Journal of Construction Engineering and Management, 137 (12) (2011)
ce
17 1099-1116.
Ac
18
19
20
37
Page 37 of 56
t
ip
3D Actual 3D Simulated
EPAR models
Spatio-Thermal Model Spatio-Thermal Model
2
cr
3 Figure 1: The concept of the proposed EPAR modeling
us
4
5
6
an
7
8
9
10
M
11
12
13
14
d
15
te
16
17
18
p
19
20
ce
21
22
23
24
Ac
25
26
27
28
29
30
31
32
33
34
35
36
38
Page 38 of 56
Data Process
Camera Calibration Multi-View Stereo

3D-Registration
Intrinsic Extrinsic
Thermal Camera Thermal Camera
Thermal Parameters Parameters Dense
imagery 3D Thermal
Point Cloud 3D Actual Spatio-
Thermal Model
Undistorted Euclidean
Thermal Imagery Transformation
t
ip
Extrinsic
GPU-Based Feature Digital Camera Multi-View Stereo
Thermal Camera Detection & Matching Parameters
cr
Initial Intrinsic
Digital Dense
Digital Camera
imagery Reconstruction Parameters
3D Building
Point Cloud
Multi-Core Bundle
us
Adjustment Registered
Digital Images
SfM
Superimposition
Geometric Data Building
an
Extraction Geometry EPAR
Computational Fluid 3D Simulated Models
Dynamics (CFD) Spatio-Thermal
Simulation Model Geo-registered
Initial
EnergyPlus Thermal and
Boundary
simulation Digital Images
1 Conditions
M
2 Figure 2: Overview of the data and process in our proposed EPAR modeling methodology
3
d
4
5
te
6
7
p
8
9
ce
10
11
12
13
Ac
14
15
16
17
18
19
20
21
22
23
24
25
39
Page 39 of 56
(a) (b) (e)
(c) (d)
t
ip
(f) (g) (j)
cr
us
(h) (i)
an
1
2 Figure 3: Examples of the technical challenge of SIFT feature detecting and matching algorithms
3 between thermal images (top) and digital and thermal imagery (bottom). Figure is best seen in
M
4 color.
5
6
d
7
8
te
9
10
11
p
12
ce
13
14
15
16
Ac
17
40
Page 40 of 56
(b) (c)
t
ip
cr
(a) (d) (e)
1
2 Figure 4: Thermal camera calibration using the thermal calibration rig.
us
3
4
5
an
6
7
8
9
M
10
11
12
d
13
14
te
15
16
17
p
18
19
ce
20
21
22
Ac
23
24
25
26
27
28
29
30
31
32
33
34
41
Page 41 of 56
(a-1) (a-2) (a-3) 0°C 23°C (a-4) 0°C 23°C
t
ip
cr
us
(b-1) (b-2) (b-3) 11°C 22°C
(b-4) 11°C 22°C
1
2 Figure 5: 3D spatio-thermal modeling (a: a façade of an instructional facility, b: an office room
an
3 of the same building). From left to right columns (1-4): unordered digital and thermal imagery,
4 the building point cloud, the thermal point cloud, the integrated visualization of two models.
5 Figure is best seen in color.
M
6
7
8
9
d
10
11
te
12
13
14
p
15
ce
16
17
18
19
Ac
20
21
22
23
24
25
26
27
28
29
30
31
42
Page 42 of 56
Velocity inlet Windows

Outflow
(c) External walls

(a) (b) (d) Internal walls
1
t
2 Figure 6: Extracting 3D geometry from 3D building point clouds, meshing, and specifying
ip
3 boundary types: From the point cloud in (a), the mesh models in (b) and (c) are generated and the
4 boundary conditions are specified in (d).
cr
5
6
7
us
8
9
10
11
an
12
13
14
M
15
16
17
18
d
19
20
te
21
22
23
p
24
25
ce
26
27
28
Ac
43
Page 43 of 56
Wall 3
Wall 2
Wall 1
Wall 4
11°C 22°C 10°C 27°C
(a) (b) (c) (d)
t
ip
cr
Camera
Location
(e) (f) (g) (h)
1
us
2 Figure 7: Actual and simulated 3D spatio-thermal models of Case #1 with 3D-registered imagery.
4
an
5
6
M
7
8
9
d
10
te
11
12
p
13
ce
14
15
16
Ac
17
18
19
20
21
22
23
24
44
Page 44 of 56
19°C 30°C 22°C 29°C

Wall 1
Wall 2
Wall 3
Wall 4
(a) (b) (c) (d)
t
Camera
ip
Location
cr
(e) (f) (g) (h)
1
us
4
an
5
6
M
7
8
9
d
10
te
11
12
p
13
ce
14
15
16
Ac
17
18
19
20
21
22
23
24
45
Page 45 of 56
18°C 29°C 17°C 30°C

Wall 4
Wall 2
Wall 1
Wall 3
(a) (b) (c) (d)
t
ip
cr
Camera
Location
(e) (f) (g) (h)

1
us
4
an
5
6
7
M
8
9
10
11
d
12
13
te
14
15
p
16
17
ce
18
19
20
21
Ac
22
23
24
25
26
27
46
Page 46 of 56
11°C 22°C
(b) (c)
t
ip
#4 #3 #1
#5 #2
cr
(a) (d) 10°C 27°C (e)
1
us
2 Figure 10: (a): the EPAR model of Case #1, (b): integrated indoor visualization of actual thermal
3 and CFD models of Case #1, (c): actual 3D thermal model from the same view point shown in
4 (b), (d): CFD model from the same view point, and (e): CFD model with 3D-registered thermal
an
5 image. Figure is best seen in color.
6
7
M
8
9
d
10
11
te
12
13
p
14
ce
15
16
Ac
17
18
19
20
21
22
23
24
25
47
Page 47 of 56
(b) (c) 19°C 30°C
t
#1
ip
#5 #4
#2
cr
#3
(a) (d) 22°C 29°C (e)
1
us
an
6
7
M
8
9
d
10
11
te
12
13
p
14
ce
15
16
Ac
17
18
19
20
21
22
23
24
25
48
Page 48 of 56
(b) (c) 18°C 29°C
t
#1
ip
#3 #2
#5
#4
cr
(a) (d) 17°C 30°C (e)
1
us
an
6
M
d
p te
ce
Ac
49
Page 49 of 56
1 Highlights
2 • A vision-based method for modeling actual energy performance in 3D is presented.

3 • Actual and simulated building energy performances are jointly visualized in EPAR model
4 s.
5 • In EPAR models, actual and simulated building energy performances can be compared in
6 3D.
t
7 • EPAR models can be used for retrofit analysis and validation of simulation models.
ip
8
9
cr
us
an
M
d
p te
ce
Ac
50
Page 50 of 56
1 List of Figures
2 Figure 1. The concept of the proposed EPAR modeling

3 Figure 2. Overview of the data and process in our proposed EPAR modeling methodology
4 Figure 3. Figure 3: Examples of the technical challenge of SIFT feature detecting and matching
5 algorithms between thermal images (top) and digital and thermal imagery (bottom).
t
6 (b) and (d) show 232 and 210 SIFT keypoints detected in (a) and (c) images
ip
7 respectively while (e) shows 24 keypoints that were matched between two images; (g)
8 and (i) show 10,992 and 232 SIFT keypoints detected in (f) and (h) images
9 respectively while (j) shows only 19 keypoints that were matched between two images.
cr
10 Most matches are even incorrect in the both cases. Figure is best seen in color.
11 Figure 4. Thermal camera calibration using the thermal calibration rig.
us
12 Figure 5. 3D spatio-thermal modeling (a: a façade of an instructional facility, b: an office room
13 of the same building). From left to right columns (1-4): unordered digital and thermal
14 imagery, the building point cloud, the thermal point cloud, the integrated visualization
an
15 of two models. Figure is best seen in color.
16 Figure 6. Extracting 3D geometry from 3D building point clouds, meshing, and specifying
17 boundary types: From the point cloud in (a), the mesh models in (b) and (c) are
18 generated and the boundary conditions are specified in (d).
M
19 Figure 7. Actual and simulated 3D spatio-thermal models of Case #1 with 3D-registered imagery.
d
te
p
25 Figure 10. (a): the EPAR model of Case #1, (b): integrated indoor visualization of actual thermal
26 and CFD models of Case #1, (c): actual 3D thermal model from the same view point
ce
27 shown in (b), (d): CFD model from the same view point, and (e): CFD model with 3D-
28 registered thermal image. Figure is best seen in color.
Ac
37
38
51
Page 51 of 56
1 Table 1: Camera Technical Data
Technical Items Values

Built-in digital camera resolution 2048 × 1536 pixels
Thermal camera resolution 320 × 240 pixels
Digital camera Field-of-View 53° × 41°
Thermal camera Field-of-View 25° × 19°
Thermal sensitivity < 0.05°C
t
Measurement accuracy ±2°C or ±2% of reading
ip
2
3
4
cr
5
6
7
us
8
9
10
11
an
12
13
14
15
16
M
17
18
19
20
d
21
22
te
23
24
25
26
p
27
28
ce
29
30
31
32
Ac
33
34
35
36
37
38
39
40
41
42
43
44
45
52
Page 52 of 56
1 Table 2: Material properties used in the case studies
Thermal Density Specific

Materials conductivity (kg/m3) heat
(W/m-k) (J/kg-k)
Concrete 0.7 2370 750
Glass 0.96 2580 840
Gypsum board 0.17 10.2 1090
t
Wood 0.13 10.4 2500
ip
Air 0.024 1.225 1006
2
cr
3
4
5
6
us
7
8
9
10
an
11
12
13
14
M
15
16
17
18
19
d
20
21
te
22
23
24
p
25
26
27
ce
28
29
30
31
Ac
32
33
34
35
36
37
38
39
40
41
42
43
44
45
53
Page 53 of 56
1 Table 3: Experimental conditions of CFD simulation
Technical Items Case 1 Case 2 Case 3

Turbulence model RNG k − ε model RNG k − ε model RNG k − ε model
Number of vertices in mesh 2,778,117 2,830,660 4,621,160
Time of the experiment Winter (December) Summer (July) Winter (December)
Fluid (air) motion type Stationary Stationary Stationary
t
Flow rate weighting for each outflow 1.0 1.0 1.0
ip
Operating pressure of air flow 101,325 N/m2 101,325 N/m2 101,325 N/m2
Temperature of inlet air 32°C (Heating) 19°C (Cooling) 35°C (Heating)
Velocity of inlet air 0.7 m/s 0.6 m/s 0.3 m/s
cr
Wall motion type Stationary Stationary Stationary
Wall shear condition No slip No slip No slip
Wall surface temperature, window 11°C 28.3°C 18°C
us
Wall surface temperature, ceiling 13°C 23.5°C 22°C
Wall surface temperature, floor 15°C 24°C 22°C
Wall surface temperature, wall 1¨ 14.2°C 23.5°C 18.5°C
an
Wall surface temperature, wall 3¨ 14°C 25.5°C 22.5°C
Wall roughness height 0 0 0
Wall roughness constant 0.5 0.5 0.5
M
2 ¨
These walls are shown in Figures 7(a), 8(a), and 9(a) respectively.
3
d
4
5
6
te
7
8
9
p
10
11
ce
12
13
14
15
Ac
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
54
Page 54 of 56
1 Table 4: Experimental results of image-based 3D spatio-thermal modeling

Case 1 Case 2 Case 3
# of digital images 440 452 209
# of thermal images 440 452 209
# of the registered digital images 429 382 170
Success ratio on digital image registration 0.98 0.85 0.81
# of points in the building clouds 8,488,888 3,458,541 3,614,375
t
# of points in the thermal clouds 2,838,478 1,977,623 527,715
ip
Δ e mm 9.3 7.09 6.71
Computational time ¨ 2hr 21min 1hr 52min 1hr 12min
cr
2 ¨
Benchmarked on an Intel(R) Core(TM) i7 960 with 24GBs RAM and NVIDIA GeForce GTX 400
us
4
5
6
7
an
8
9
10
11
12
M
13
14
15
16
d
17
18
te
19
20
21
22
p
23
24
ce
25
26
27
28
Ac
29
30
31
32
33
34
35
36
37
38
39
40
41
42
55
Page 55 of 56
1 Table 5: Performance deviations

Area 1 Area 2 Area 3 Area 4 Area 5
Case 1
µ σ µ σ µ σ µ σ µ σ
Actual Measurement (°C) 11.1 0.13 14.6 0.08 14.4 0.05 14.3 0.05 14.5 0.04
Simulated Results (°C) 13.7 0.05 14.8 0.08 15.1 0.07 15.3 0.05 15.4 0.09
+
Deviations (°C) (2.26, 3.01) (0.17, 0.46) (0.49, 0.98) (0.83, 1.25) (0.59, 1.15)
t
Case 2
ip
cr
+
Deviations (°C) (3.00, 3.59) (0.13, 0.53) (0.56, 1.00) (0.21, 0.62) (0.15, 0.48)
Case 3
us
Deviations (°C)+ (0.88, 1.66) (0.47, 1.03) (1.03, 1.95) (0.71, 1.88) (0.87, 1.53)
an
+
2 Δ(µ-2.05σ, µ+2.05σ)i where i is each section
3
4
5
M
6
7
8
9
d
10
11
te
12
13
14
15
p
16
17
ce
18
19
20
21
Ac
22
23
24
25
26
27
28
29
30
31
32
33
34
35
56
Page 56 of 56

Ham 2013

Uploaded by

Copyright:

Available Formats

Ham 2013

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ham 2013

Uploaded by

Copyright:

Available Formats

Accepted Manuscript

Title: EPAR: Energy Performance Augmented Reality Models

Authors: Youngjib Ham, Mani Golparvar-Fard

To appear in: ENB

Received date: 9-9-2012

1 EPAR: Energy Performance Augmented Reality Models for Identification of Building

2 Energy Performance Deviations between Actual Measurements and Simulation Results

3 Youngjib Ham1 and Mani Golparvar-Fard2

14 Through an image-based reconstruction pipeline, actual 3D spatio-thermal models are

2 support detection and analysis of building performance deviations.

3 Keywords: Thermogaphy, Image-based 3D Reconstruction, Computational Fluid Dynamics

4 (CFD), Energy Audits

1 Finding energy performance problems is an important step in improving energy

13 challenges associated with modeling dynamic patterns of building occupancy may

14 result in misrepresentation of the actual energy performance of a building over its

22 primarily for two reasons:

1 hence all material conditions and compositions of the building elements

2 are considered to be the identical;

3 b. The current simulation tools typically do not enable modeling varying

4 conditions of the building elements. Thus, in contrast to the site

15 assessment of building energy performance is the infrared thermography [12-15]. Infrared

16 thermography as a reality capturing method measures building surface temperatures, allowing

21 to contact thermometers, thermography provides non-contact and continuous measurements.

23 performances have several limitations:

1 (1) The current practice of thermography primarily focuses on direct application of

3 consuming. Due to the low spatial resolution of consumer-level thermal cameras

13 purpose of whole building energy performance assessment can be time-consuming.

22 vision techniques for image-based 3D reconstruction, a consumer-level single thermal camera

13 spatio-thermal modeling and identification of building performance deviations using CFD

20 2. Background and Related Works

21 2.1 3D spatio-thermal modeling of buildings

19 time-consuming and labor-intensive. For example, in order to implement the area-based

22 to building interior surveys.

1 2.2 Building energy simulation using CFD

2 locations were utilized for performance comparison purposes.

4 presented a CFD-based augmented reality environment for immersive visualization of the

15 3. Overview of the EPAR Modeling

20 method are illustrated in Figure 2.

21 <Insert Figure 2>

6 Graphic Processing Unit (GPU)-based Structure-from-Motion (SfM) and Multi-View Stereo

20 following explains each of these steps in more detail.

22 4. Actual 3D Spatio-Thermal Modeling

23 4.1 Image-based 3D reconstruction of as-built representation

8 reconstructing 3D building thermal performance models from collections of unordered thermal

4 <Insert Figure 3>

16 thermal modeling method is introduced in this work.

pundistorted = f T × r ( p distorted ) × p distorted (1)

r ( p ) =1.0 + kT1 × p + kT2 × p (2)

19 point in thermal imagery correspond to an absolute temperature value.

15 points is verified using the following constraint:

If c1 ≡ Q3Q4 > 0 and c2 ≡ (Pi Q )3 Q4 > 0 , then return Pi and Q

[ RT | TT ] = [ RD Rrel | RDTrel + TD ] (4)

8 4.3 Aligning the 3D building and thermal models

18 <Insert Figure 5>

20 5. Expected 3D Spatio-Thermal Simulation

1 5.1 Extracting 3D geometrical information and initializing environmental conditions

15 <Insert Figure 6>

12 6. EPAR Modeling for Integrated Visualization of Actual/ Expected Energy Performances

17 format which enables export of 3D geometrical entities of the simulation results.