Tesi

POLITECNICO DI TORINO
Corso di Laurea Magistrale in Ingegneria Meccanica
Master’s degree thesis
Experimental validation and accuracy assessment of

Velocity Prediction Program for a high-performance
sailing boat
Advisor:
Prof. Giovanni Bracco
Co-Advisor:
Prof. Giuliana Mattiazzo
Candidate
Luca Zannotti
APRIL 2020
TABLE OF SYMBOLS
DEFINITIONS:
• δ: Error (of measurement);
• δmodel: Model error;
• δnum: Numerical error;
• δinput: Input error;
• δD: Data
• u: Uncertainty (of measurement);
• E: Comparison error;
• S: Simulation solution value;
• D: Experimental data value;
• T: True value;
• N: Number of Data;
• σ: Standard deviation;
• h: Grid size;
• x, y, z: Axes;
• Vb: Sailboat speed;
• Va: Apparent wind speed;
• Vr: True wind speed;
• Φ: Leeway angle;
• α: Apparent wind angle;
• β: True wind angle;
• CD: Drag coefficient;
• CL: Lift coefficient.
ACRONYMS:
• VPP: Velocity Prediction Program;
• V&V: Verification & Validation effort;
• PST: Polito Sailing Team;
• ASME: American Society of Mechanical Engineers;
• ISO: International Organization for Standardization;
• GCI: Grid Convergence Index;
• CFD: Computational Fluid Dynamics;
• GC: Gravity Centre.
3
CONTENTS
TABLE OF SYMBOLS............................................................................................................... 3
INTRODUCTION ........................................................................................................................ 7
1.1 PROBLEM DEFINITION AND OBJECTIVES ............................................................................ 7
1.2 OVERVIEW OBJECT SAILING YACHT .................................................................................... 8
BACKGROUND OF VERIFICATION AND VALIDATION PROCEDURE........... 10
2.1 PREFACE ........................................................................................................................................ 10
2.2 OBJECTIVES OF V&V ................................................................................................................. 11
2.3 VALIDATION NOMENCLATURE AND APPROACH ............................................................ 16
2.4 CODE VERIFICATION AND SOLUTION VERIFICATION .................................................. 19
2.4.1 CODE VERIFICATION ............................................................................................................................. 20
2.4.2 SOLUTION VERIFICATION .................................................................................................................... 23
2.5 ESTIMATION OF SIMULATION UNCERTAINTY ................................................................. 28
2.5.1 SENSITIVITY COEFFICIENT (LOCAL) METHOD FOR A PARAMETER UNCERTAINTY

PROPAGATION ........................................................................................................................................................ 28
2.6 UNCERTAINITY OF AN EXPERIMENTAL RESULT ............................................................ 29
2.7 EVALUATION OF VALIDATION UNCERTAINITY ............................................................... 30
2.7.1 ESTIMATING VALIDATION UNCERTAINTY WHEN EXPERIMENTAL VALUE OF

VALIDATION VARIABLE IS DIRECTLY MEASURED (CASE 1) ................................................................... 30
2.7.2 ESTIMATING VALIDATION UNCERTAINTY WHEN EXPERIMENTAL VALUE OF VALIDATION

VARIABLE IS DETERMINED FROM DATA REDUCTION EQUATION (CASE 2) ...................................... 31
2.7.3 INTERPRETATION OF VALIDATION RESULTS ..................................................................................... 32
2.7.4 SOME PRATICAL POINTS ............................................................................................................................ 33
DATA ACQUISITION PROCEDURES.............................................................................. 35
3.1 OVERVIEW OF DATA ACQUISITION PREPARATION ....................................................... 35
3.1.1 SENSORS DATASHEET ............................................................................................................................ 49
3.2 GPS EXPERIMENTAL DATA ACQUIRED ............................................................................... 52
3.3 OVERVIEW OF VELOCITY PREDICTION PROGRAM ....................................................... 53
4
3.4 ASSUMPTIONS AND CONSIDERATIONS FOR PROCESSING EXPERIMENTAL DATA
55
3.5 POST PROCESSING OF EXPERIMENTAL DATA .................................................................. 56
3.5.1 OVERVIEW OF DATA CLEANING PROCESS .......................................................................................... 59
RESULTS .................................................................................................................................... 62
4.1 EXPERIMENTAL DATA RESULTS ........................................................................................... 62
4.1.1 COMPARISON BETWEEN REAL-BOAT SPEED RESULTS AND THOSE PROVIDED BY THE VPP
62
4.1.2 COMPARISON WITH THE VPP OF THE ESTIMATE DISTANCE BETWEEN SAILORS AND THE
CENTRE OF GRAVITY OF THE BOAT ............................................................................................................... 68
4.2 PLOTS EXPERIMENTAL SAILBOAT SPEED VALUE .......................................................... 72
4.2.1 POLAR PLOT OF EXPERIMENTAL SAILBOAT SPEED. .................................................................. 72
4.2.2 VELOCITY PLOT OF EXPERIMENTAL SAILBOAT SPEED DURING TIME ............................... 75
4.2.3 COMPARISON EXPERIMENTAL – VPP’S PREDICTED SAILBOAT SPEED ................................ 81
4.2.4 PLOT MEASURED DISTANCE BETWEEN SAILORS AND SAILBOAT CENTER OF GRAVITY
90
4.2.5 ERROR TREND AS THE NUMBER OF AVAILABLE POPULATION CHANGES ........................ 101
4.2.5 OVERVIEW OF THE ASSESSMENTS ABOUT THE RESULTS ....................................................... 105
CONCLUSIONS AND RECOMMENDATIONS............................................................107
5.1 CONCLUSIONS ............................................................................................................................ 107
REFERENCES.........................................................................................................................113
SUMMARY...............................................................................................................................116
ACKNOWLEDGEMENTS...................................................................................................118
5
1
INTRODUCTION
1.1 PROBLEM DEFINITION AND OBJECTIVES

Sailing Yacht technology has been changing in recent decades in accordance with the
great possibilities of calculation offered by computer processing systems. The
development of mathematical models for the prediction of the performance of sailing
yachts is now a fundamental requirement for any sailing team or any sailboat
manufacturer. It is the ability to accurately predict the performance of a sailing yacht that
has led to the development of better and more efficient hull shapes, appendages and
windage. Due to the obvious unstationary and highly unpredictable behavior of a small
sailing yacht in real sea conditions the accurate performance prediction is very hard to
obtain and even harder to find in literature. It is much easier to find velocity prediction
programs for large yachts, cargo ships or commercial foiling sailboats.
The sailing team of Politecnico di Torino designs and builds R3 class skiffs, characterized
by a very light and small hull, large sail area and a high aspect ratio sail. A performance
prediction tool was necessary to optimize the design or modifications of the sailing yacht
in order to achieve the yacht performance under specific wind and sea conditions. These
tools are called the Velocity Prediction Programs (VPPs).
VPP’s are extremely important design tools: first of all, they are useful to understand how
each component affects the performance of the boat, secondly the output of a VPP is a
performance diagram (polar plot) that states the boats optimal target speed through the
water as a function of the sailing conditions at best possible trimming. From these
performance diagrams it is possible to evaluate the best sailing condition for a given
design and it is also possible to give the sailor an advantage in knowing the strength and
weaknesses of the boat.
In this study, the consistency of a static Velocity Prediction Program for a small, high
performance sailing Yacht is shown in detail.
For this purpose, chapter 2 describes the methodologies indicated by the guides to perform
a code verification and experimental validation effort, also defined as V&V [1].
Validation must be preceded by code verification and solution verification. The objective
of verification is to establish numerical accuracy of the model, it describes each force
7
acting on the entire boat as a function of certain parameters. Especially, code verification
establishes that the code accurately solves the mathematical model incorporated, this is
obviously taken for granted as the equations and their resolution have been verified in the
design phase of the VPP. Validation procedure consists of an assessment of the errors,
associated with physical model’s assumptions, input data and numerical solutions. These
errors will be compared with the experimental results and with possible errors associated.
The goal is to determine how close is the simulation output with the real system output,
difference between the two models will confirm or not the consistency of the theoretical
VPP’s model. The estimation of a range within which the simulation modelling error lies
is a primary objective of the validation process and is accomplished by comparing a
simulation result with an appropriate experimental result for specified validation variables
at a specified set of conditions. Next step is trying to acquire enough sensor data through
experimental tests in real navigation conditions, to allow a comparison between the polar
obtained by numerically solution and the one obtained with measured data. There can be
no validation without experimental data with which to compare the result of the
simulation. [2]
To this end, Chapter 3 describes the procedures and instruments used to carry out the
experimental data acquisition phase. The data are processed by calculation software to
obtain outputs comparable with the values simulated by the VPP, so as not to generate a
comparison error, which would have no meaning. Therefore it is necessary to carry out a
data cleaning phase, which allows you to eliminate the data that deviate excessively from
the average values of the population in question and to eliminate the values that were
acquired in conditions other than those foreseen in the VPP's design phase, i.e. boat at top
speed regime or with constant wind. This fact guarantees the comparability between the
simulation and the experimental computational program.
In chapter 4 the results of the experimental model are shown and compared with the VPP,
and the error that is generated is evaluated. In addition, the uncertainty of the results is
expressed as a standard deviation, with the aim of establishing whether the latter is
variable or if it is fixed. Establishing if the error entities are constant would be useful to
know the possible error offset and to be able to correct it.
1.2 OVERVIEW OBJECT SAILING YACHT

The reference case study for this paper will be Atka, the second R3 class skiff designed
and built entirely by Polito Sailing Team. From this model we derive all the physical data
requested by the VPP.
Atka was built in 2015/2016, designed ad hoc for calm and medium wind. According to
the 1001 Vela Cup rules, 75% of the structure needs to be recyclable. The challenge was
to provide a good structural behaviour and lightness at the same time, using mostly
recyclable materials. The hull was built thanks to the vacuum sandwich technique, in
8
particular the core is made by balsa wood, while glass fibre and a tiny layer of basalt fibre
compose the skin. The deck was made by marine plywood, and in all the boat epoxy resin
was used as a sealant. The overall length is 4.6m, the hull beam is 1.5m while the overall
beam is 2.1m. The total sail surface is 33 square meters. Atka weighs 110kg in total.
During the computation is necessary to consider the sailor’s weighs and their position with
respect to the yacht’s centre of mass.
The following figure shows in detail the characteristics of the aforementioned sailboat
Atka.
Atka
Overall Overall Total sail

Hull beam Weight
length beam surface
4.6 m 1.5 m 2.1 m 33 m2 110 kg
Table 1.1 Atka geometrical data. (Source: [3])
Figure 1.1 Atka sailing yacht CAD. (Source: [3])
9
2
BACKGROUND OF VERIFICATION
AND VALIDATION PROCEDURE
2.1 PREFACE
Verification and Validation is one of the software-engineering disciplines that help build
quality into program. V&V is a collection of analysis and testing activities to
complements the efforts of other quality engineering functions. V&V comprehensively
analyses and tests program to determine that it performs its intended functions correctly,
to ensure that it performs no unintended functions, and to measure its quality and
reliability. Verification involves evaluating software during the design phase to ensure that
it meets the requirements set. Validation involves testing software or its specification at
the end of the development effort to ensure that it meets its requirements (that it does what
it is supposed to). While Verification and Validation have separate definitions, you can
derive the maximum benefit by using them synergistically and treating ‘V&V’ as an
integrated definition [4]. Ideally, V&V parallels software development and yields several
benefits:
• It uncovers high-risk errors early, giving the designer time to evolve a

comprehensive solution rather than forcing a makeshift fix to accommodate
development deadlines.
• It evaluates the products against system requirements.
• It gives management continuous and comprehensive information about the quality
and progress of the development effort.
Boehm and Wallace studies [5], show that V&V can improve quality, cause more stable
requirements, cause more rigorous development planning, catch errors earlier, promote
better progress monitoring, make project management more aware of interim quality and
progress, and result in better criteria and results for decision making. But V&V has several
negative effects: it adds the development cost, requires additional interfaces between
project groups, can lower developer productivity if programmers and engineers spend time
explaining the system to V&V analysts when trying to resolve invalid anomaly reports,
adds to the documentation requirements if the V&V group is receiving incremental
program and documentation releases, requires the sharing of computing facilities and
10
classified data with the V&V group, and increases the paperwork to provide written
responses to the V&V group’s error reports and other V&V data requirements. As the
Radatz study showed [6], you are more likely to recover V&V costs when you start using
it early in the requirements phase. You should consider the interface activities between
development and V&V groups for documentation, data, and software deliveries an
inherently necessary step to evaluate intermediate development products. This is a
necessary by-product of doing what is right from the outset. The cost of the development
interface is minimal, and sometimes non-existent, when the V&V assessment is
independent of the development group.
2.2 OBJECTIVES OF V&V

The objective is to develop an independent assessment of the software’s quality and to
determine whether the software satisfies critical system requirements [4]. The concern of a
V&V is the specification of a verification and validation approach that quantifies the
degree of accuracy inferred from the comparison of solution and data for a specified
variable at a specified validation point. The approach uses the concepts from experimental
uncertainty analysis to consider the errors and uncertainties in both the solution and the
data. The scope is the quantification of the degree of accuracy of simulation of specified
validation variables at a specified validation point for cases in which the conditions of the
actual experiment are simulated [7].
Pertinent definitions to clarify these concepts are as follows:
Verification: ‘The process of determining whether or not the products of a given phase of
the software development cycle fulfils the requirements established during the design
phase’. [8]
Validation: ‘The process of evaluating software at the end of the software development
process to ensure compliance with software requirements and to determine the fitness or
worth of a software product for its operational mission’. [8]
Informally, we might define these terms via the following questions:
• Verification: ‘Am I building the product, right?’ [4]
• Validation: ‘Am I building the right product?’ [4]
The four basic V&V criteria for requirements and design specifications are completeness,
consistency, feasibility, and testability. [4]
• Completeness: ‘A specification is complete to the extent that all of its parts are
present, and each part is fully developed’.
• Consistency: ‘A specification is consistent to the extent that its provisions do not
conflict with each other or with governing specifications and objectives’.
11
• Feasibility: ‘A specification is feasible to the extent that the benefits of the system
specified exceed its costs. Thus, feasibility involves more than verifying that a
system satisfies functional and performance requirements. It also implies
identifying and resolving any high-risk issues before committing resource to
detailed development’.
• Testability: ‘A specification is testable to the extent that one can identify a feasible
technique for determining whether or not the developed program will satisfy the
specification. To be testable, specifications must be unambiguous and quantitative
wherever possible’.
Pertinent definitions from metrology are as follows:
Error (of measurement), δ: “result of a measurement minus a true value of the
measurand”. [9]
Uncertainty (of measurement), u: ‘‘parameter, associated with the result of a
measurement, that characterizes the dispersion of the values that could reasonably be
attributed to the measurand’’. [9]
In measurement of a set, accuracy refers to closeness of the measurements to a specific
value, while precision refers to the closeness of the measurements to each other. The
purpose of measurement is to provide information about a quantity of interest, a
measurand. No measurement is exact. When a quantity is measured, the outcome depends
on the measuring system, the measurement procedure, the skill of the operator, the
environment, and other effects. The dispersion of the measured values would relate to how
well the measurement is performed. Their average would provide an estimate of the true
value of the quantity that generally would be more reliable than an individual measured
value. The dispersion and the number of measured values would provide information
relating to the average value as an estimate of the true value. However, this information
would not generally be adequate. The measuring system may provide measured values
that are not dispersed about the true value, but about some value offset from it. In error
propagation theory, accuracy is the degree of correspondence of the theoretical data,
inferable from a series of measured values (data sample), with the real or reference data,
i.e. the difference between the average sample value and the true or reference. Indicates
the proximity of the value found to the real one. It is a qualitative concept that depends on
both random and systematic errors [9].
Accuracy has two definitions:
• More commonly, it is a ‘description of systematic errors, a measure of statistical

bias; low accuracy causes a difference between a result and a "true" value. ISO
calls this trueness’. [ISO 5725-1].
• Alternatively, ISO defines accuracy as ‘describing a combination of both types of
observational error above (random and systematic), so high accuracy requires both
high precision and high trueness’. [ISO 5725-1].
12
Precision: ‘is a description of random errors, a measure of statistical variability’. [ISO
5725-1].
In simpler terms, given a set of data points from repeated measurements of the same
quantity, the set can be said to be accurate if their average is close to the true value of the
quantity being measured, while the set can be said to be precise if the values are close to
each other. In the first, more common definition of accuracy above, the two concepts are
independent of each other, so a particular set of data can be said to be either accurate, or
precise, or both, and neither.
Figure 2.1 Dataset Gauss Curve refers to true value.
Propagation of uncertainty (or propagation of error) is the effect of variables' uncertainties

(or errors) on the uncertainty of a function based on them [10]. When the variables are the
values of experimental measurements, they have uncertainties due to measurement
limitations (e.g., instrument precision) which propagate due to the combination of
variables in the function. The uncertainty u can be expressed in a number of ways. It may
be defined by the absolute error Δx or can also be defined by the relative error (Δx)/x,
which is usually written as a percentage, the coefficient of variation (C.V). Most
commonly, the uncertainty on a quantity is quantified in terms of the standard deviation,
σ, which is the positive square root of the variance. The value of a quantity and its error
are then expressed as an interval x ± u. If the statistical, probability distribution of the
variable is known or can be assumed, it is possible to derive confidence limits to describe
the region within which the true value of the variable may be found. Following the dictates
of the Gauss curve, for example, the 68% confidence limits for a one-dimensional variable
belonging to a normal distribution are approximately ± one standard deviation σ from the
central value x, which means that the region x ± σ will cover the true value in roughly
13
68% of cases, while the 95% confidence limits are about ± 2σ from the central value x,
which means that the region x ± 2σ will cover the true value in roughly 95% of cases.
In statistics, the 68–95–99.7 rule, also known as the empirical rule, is a shorthand used to
remember the percentage of values that lie within a band around the mean in a normal
distribution with a width of two, four and six standard deviations, respectively; more
accurately, 68.27%, 95.45% and 99.73% of the values lie within one, two and three
standard deviations of the mean, respectively. In mathematical notation, these facts can be
expressed as follows, where Х is an observation from a normally distributed random
variable, μ is the mean of the distribution, and σ is its standard deviation:
Pr(𝜇 − 1𝜎 ≤ Х ≤ 𝜇 + 1𝜎) ≈ 0.6827
Pr(𝜇 − 2𝜎 ≤ Х ≤ 𝜇 + 2𝜎) ≈ 0.9545
Pr(𝜇 − 3𝜎 ≤ Х ≤ 𝜇 + 3𝜎) ≈ 0.9973
In the empirical sciences the so-called three-sigma rule of thumb expresses a conventional
heuristic that nearly all values are taken to lie within three standard deviations of the
mean, and thus it is empirically useful to treat 99.7% probability as near certainty.
Empirical rule is often used to quickly get a rough probability estimate of something,
given its standard deviation, if the population is assumed to be normal. It is also used as a
simple test for outliers if the population is assumed normal, and as a normality test if the
population is potentially not normal. To pass from a sample to a number of standard
deviations, one first computes the deviation, either the error or residual depending on
whether one knows the population mean or only estimates it. The next step is
standardizing, dividing by the population standard deviation, if the population parameters
are known, or use a Student's test, dividing by an estimate of the standard deviation, if the
parameters are unknown and only estimated.
14
Figure 2.2 Confidence limits belonging to a normal distribution refers to mean value.
The above discussion concerns the direct measurement of a quantity, which incidentally
occurs rarely. A measurement model converts a quantity value into the corresponding
value of the measurand. There are many types of measurement in practice and therefore
many models. Correction terms should be included in the measurement model when the
conditions of measurement are not exactly as stipulated. These terms correspond to
systematic errors. Given an estimate of a correction term, the relevant quantity should be
corrected by this estimate. As well as raw data representing measured values, there is
another form of data that is frequently needed in a measurement model. Some such data
relate to quantities representing physical constants, each of which is known imperfectly.
The items required by a measurement model to define a measurand are known as input
quantities in a measurement model. The model is often referred to as a functional
relationship. The output quantity in a measurement model is the measurand.
The American Society of Mechanical Engineers (ASME) has produced a suite of
standards addressing various aspects of measurement uncertainty. The concepts above
were extended to apply to the value of a solution variable from a simulation as well as a
measured value of the variable from an experiment. In that context, then, an error, δ, is a
quantity that has, a particular sign and magnitude, and a specific error, δi is the difference
caused by error source i between a quantity (measured or simulated) and its true value. It
is assumed that each error whose sign and magnitude is known has been removed by
correction. Any remaining error is thus or unknown sign and magnitude, and an
uncertainty u is estimated with the idea that ±u characterizes the range containing δ. In
experimental uncertainty analysis, u is the standard uncertainty and corresponds
conceptually to an estimate of the standard deviation, σ, of the parent distribution from
which σ is a single realization. It is significant to note that no assumption about the form
of the parent distribution is associated with the definition of u [11].
15
2.3 VALIDATION NOMENCLATURE AND APPROACH
In the validation process, a simulation result (solution) is compared with an experimental
result (data) for specified validation variables at a specified set of conditions (validation
point) [12].
Figure 2.3 Schematic showing nomenclature for validation approach. (Source: [13])
As shown in Figure 2.3, we will denote the predicted value from the simulation solution as
S, the value determined from experimental data as D, and the true (but unknown) value as
T.
The validation comparison error E is defined as
E=S–D (2.1)
The error in the solution value S is the difference between S and the true value T,
δS = S – T (2.2)
and similarly, the error in the experimental value D is
δD = D – T (2.3)
16
Using Equations (2.1) though (2.3), E can be expressed as
E = S − D = (T + δS) − (T + δD) = δS − δD (2.4)
The validation comparison error E is thus the combination of all of the errors in the
simulation result and the experimental result, and its sign and magnitude are known once
the validation comparison is made.
All errors in S can be assigned to one of three categories:
• the error δmodel due to modeling assumptions and approximations;

• the error δnum due to the numerical solution of the equations;
• the error δinput in the simulation result due to errors in the simulation input
parameters.
Thus,
δS = δmodel + δnum + δinput (2.5)
As we will discuss, there are ways to estimate the effects of δnum and δinput, but there are no
ways to independently observe or calculate the effects of δmodel. The objective of a
validation exercise is to estimate δmodel to within an uncertainty range.
Combining Eqs. (2.4) and (2.5), the comparison error can then be written as
E = δmodel + δnum + δinput − δD (2.6)
This approach is shown schematically in Figure 6.3, where the sources of error are shown
in the ovals.
Rearranging Eq. (2.6) to isolate δmodel gives:
δmodel = E − (δnum + δinput − δD) (2.7)
17
Figure 2.4 Overview of validation process with sources of error in ovals. (Source:
[13])
Consider the terms on the right-hand side (RHS) of the equation. Once S and D are
determined, the sign and magnitude of E are known from Eq. (2.6). However, the signs
and magnitudes of δnum, δinput, and δD are unknown. The standard uncertainties
corresponding to these errors are unum, uinput, and uD (where unum, for instance, is the
estimate of the standard deviation of the parent distribution from which δnum is a single
realization).
A validation uncertainty uval can be defined as an estimate of the standard deviation of the
parent population of the combination of errors (δnum + δinput − δD).
If the three errors are independent, then
2
2
u𝑣𝑎𝑙 = √𝑢𝑛𝑢𝑚 + 𝑢𝑖𝑛𝑝𝑢𝑡 + 𝑢𝐷2 (2.8)
Considering the relationship shown in Eq. (2.7),
(E ± uval)
then defines an interval within which δmodel falls (with some unspecified degree of
confidence). To obtain an estimate of uval, an estimate of unum must be made; estimates
must be made of the uncertainties in all input parameters that contribute to uinput and
estimates of the uncertainties in the experiment that contribute to uD must be made.
The estimation of uval is at the core of this methodology since knowledge of E and uval
allows determination of an interval within which the modeling error δmodel falls. Two
uncertainty propagation approaches to estimating uval exist, the Taylor Series Method
(TSM) and the Monte Carlo Method (MCM). The first one is a local model using a
18
propagation equation based on a Taylor series expansion, that requires estimates of
simulation solution sensitivity coefficients. The second one is global method that make
direct use of the input parameter standard uncertainties as standard deviations in assumed
parent population error distributions [14].
Note that once D and S have been determined, their values are always different by the
same amount from the true value T. That is, all errors affecting D and S have become
“fossilized” and δD, δinput, δnum, and δmodel are all systematic errors. This means that the
uncertainties to be estimated uinput, unum, and uD are systematic uncertainties.
2.4 CODE VERIFICATION AND SOLUTION

VERIFICATION
The objective of verification is to establish numerical accuracy, independent of the
physical modeling accuracy that is the subject of validation [1]. Code verification is
distinct from solution verification and must be precede it, even though both procedures
utilize grid convergence studies. In general, code verification assesses code correctness
and specifically involves error evaluation for a known solution. Code verification is the
process of ensuring, to the degree possible, that there are no mistakes in a computer code
or inconsistencies in the solution algorithm. By contrast, solution verification involves
error estimation, since the exact solution to the specific problem is unknown. Solution
verification is the process of quantifying the numerical errors that occur in every
numerical simulation.
Code and solution verification are mathematical activities, with no concern whatsoever for
the agreement of the simulation model results with physical data from experiments; that is
the concern of validation. Note, however, that the solution and its error estimation from a
solution verification will be used in the validation process. In this way, code verification,
solution verification, and validation are coupled into an overall process for assessing the
accuracy of the computed solution. The verification methods are specific grid-based
simulations. These include primarily finite difference, finite volume, and finite element
methods in which discrete grid intervals are defined between computational nodes.
Considering the relationship shown in eq. (2.8), an estimate of unum must be made to
obtain and estimate of uval; estimates must be made of the standard uncertainties in all
input parameters that contribute to uinput and of the standard uncertainties in the
experiment that contribute to uD.
Code verification and solution verification is the process of determining that a code is
mathematically correct for the simulations of interest (i.e., it can converge to a correct
continuum solution as the discretization is refined).
19
2.4.1 CODE VERIFICATION
Code verification involves error evaluation from a known benchmark solution. The best
benchmark solution is an exact analytical solution (i.e., a solution expressed in simple
primitive functions like sin, exp, tanh, etc.). Further, it is not sufficient that the analytical
solution be exact; it is also necessary that the solution structure be sufficiently complex
that all terms in the governing equations of the code being tested are exercised.
The recommended approach for code verification is the use of the method of
manufactured solutions (MMS). The MMS assumes a sufficiently complex solution form
(e.g., hyperbolic tangent, tanh, or other transcendental function) so that all of the terms in
the partial differential equations (PDEs) are exercised. The solution is input to the PDEs as
a source term, and grid convergence tests are performed on the code not only to verify that
it converges but also to ascertain at what rate it converges. The magnitude (and sign) of
the error is directly computed from the difference between the numerical solution and the
analytical solution. Whereas grid refinement studies in the context of code verification
provide an evaluation of error, grid refinement studies used in solution verification
provide an estimate of error. The most widely used method to obtain such an error
estimate is classical Richardson extrapolation (RE). Uncertainty estimates at a given
degree of confidence can then be calculated by Roache’s grid convergence index (GCI)
[15]. The GCI is an estimated 95% uncertainty obtained by multiplying the (generalized)
RE error estimate by an empirically determined factor of safety, Fs. The Fs is intended to
convert the best error estimate implicit in the definition of any ordered error estimate (like
RE) into a 95% uncertainty estimate. The GCI, especially the least-squares versions
pioneered by Eça and Hoekstra, is cited as the most robust and tested method available for
the prediction of numerical uncertainty as of this date.
Once a nontrivial exact analytic solution has been generated, by this method of
manufactured solution or perhaps another method, the solution in now used to verify a
code by performing systematic discretization convergence tests (usually, grid convergence
tests) and monitoring the convergence as ℎ → 0, where ℎ is a measure of discretization
(e.g., Δx in space, Δt in time, in a finite difference or finite volume code, and element size
in a finite element code, number of vortices in a discrete vortex method, number of surface
facets in a radiation problem, etc). The main definitions of “order of convergence” is
based on the behaviour of the error of the discrete solution. There are various measures of
discretization error Eh, but in some sense this discussion is always referring to the
difference between the discrete solution f(h) (or a functional of the solution, such as lift
coefficient) and the exact (continuum) solution,
Eh = f(h) – fexact (2.9)
For a p-order method and a well-behaved problem, the error in the solution Eh
asymptotically will be proportional to hp. This terminology applies to the “consistent”
20
methodologies of finite difference method (FDM), finite volume methods (FVM), finite
element methods (FEM), vortex-in-cell, etc., regardless of solution smoothness. Thus,
Eh = f(h) – fexact = C*hp + H.O.T. (2.10)
Where H.O.T. are higher order terms. (For smooth problems, it may be possible in
principle to evaluate the coefficient C and the H.O.T. from the continuum solution, but as
a practical matter, this is not done in the accuracy verification procedure). The
discretization error is then monitored as the grid is systematically refined. However, for a
meaningful assessment of p, grid refinement should not be trivial. The value of the
observed p versus a theoretically expected value of p provides valuable insights to the
numerical erroring the computer code. If the value of the observed p and the theoretical p
vary greatly from each other, then this indicates one of several possible issues:
• grid convergence study has not been carried out to a sufficient level of refinement;
• there are more significant errors being generated in the code than those due to
discretization and thus a detailed review of the code is required;
• boundary conditions may not be appropriate;
• initial conditions may not be appropriate (e.g., exact continuum initial conditions
may not be compatible with solutions to the discretized equations, or are
incompatible with the boundary conditions);
• incomplete iterative convergence and round-off errors.
Finally, when a systematic grid convergence test is verified, (for all point-by-point
values), then the following have been verified:
• any equation transformations used (e.g., nonorthogonal boundary fitted

coordinates);
• the order of the discretization;
• the encoding of the discretization;
• the matrix solution procedure.
As with any nontrivial technique, there are always additional details and subtleties in the
application that a serious user should be aware of. This is true for MMS.
Verification of codes is sometimes approached by code-to-code comparisons. The idea is

to take the solutions of a previously verified code as the benchmark. This can be done at
two levels of applications:
• solutions on a specific grid;

• "grid-free" solutions, i.e. high-resolution solutions that are taken as good
approximations to the exact solutions, such as with Direct Numerical Simulations.
21
The first approach can be useful and economical, but it requires that both codes have
identical discretization: not only at interior points, but also at all boundary points. It also
requires tight iterative convergence tolerance (in essence, close to machine-zero
convergence). In practice, it is effective when the new code to be verified is a new version
of the previously verified code, and the new version does not change any of the
discretization. For example, the new version might contain a new linear solver, or simply
use a new compiler or hardware platform (an important and practical situation). Such
comparisons can be done advantageously even on very coarse grids. However, beyond this
limited though important application, this approach will not give very convincing results
because of the tolerances involved. It can be used economically to develop confidence
during a code development program (even if the benchmark code does not use identical
discretization) but the tolerances involved will usually be too crude or large to enable truly
convincing verification.
The same follows for the second approach. In principle, this would work if the benchmark
code were itself thoroughly verified and if the solutions were indeed "grid-free" or have
resolved all the pertinent length scales of the problem (possibly down to viscous
dissipation) as is the requirement for Direct Numerical Simulations (DNS).
In general, however, small coding errors can be masked by the lack of complete agreement
due to the fuzziness of the benchmark. As with the first approach, it can be used
economically to develop confidence during a code development program, but more
convincing and credible (final) code verification will always be attained by the preferred
approach of MMS. Note that DNS results are often used as being equivalent to "whole-
field experimental data", which then are used to assess predictive performance of Large
Eddy Simulation sub grid scale models. However, this should not be confused with a
formal verification and validation effort as discussed in this chapter, but rather is a
strategy for developing new sub grid scale models. Similar evaluation applies to the
common approach of validation by code-to-code comparisons. In principle, one could
view a previously validated code as a benchmark repository of experimental data
including interpolation algorithms, by solving nonlinear PDEs. The benchmark code must
be accurate to be worthwhile; there is nothing to be gained by comparison with another
code that is merely old.
In historical practice, code-to-code comparisons for code verification and validation have
been notoriously unsatisfying. It is more convincing to perform validation by direct
comparison with experimental data. The methods discussed above do provide valuable
support in the development of computer codes and models. And these are approaches that
should be routinely used to support development and enhancement of codes. However,
these are not appropriate methods for a formal, convincing, and documented verification
and validation effort.
22
2.4.2 SOLUTION VERIFICATION
Prior to performing solution verification, it is assumed that code verification has been
completed and documented. Systematic grid refinement is the cornerstone of verification
processes for either codes or solutions. Whereas grid-refinement studies in the context of
code verification provide an evaluation of error, grid-refinement studies used in solution
verification provide only an estimate of error. The most widely used method to obtain an
error estimate is classical Richardson Extrapolation (RE). [16] There are also single grid
error estimators, notably Zhu-Zienkiewicz estimators, of more specialized application.
Error estimates and uncertainty estimates are related but are not equivalent, and confusion
is common. An error estimate is intended to provide an improvement to the result of a
calculation. For example, if the result of a calculation using a particular grid is f and the
error estimate is ε, then an improved value (closer to the true value ft) is f - ε.
On the other hand, an uncertainty estimates Ux% is intended to provide a statement that the
interval f ± Ux% characterizes a range within which the true (mathematical) value of ft
probably falls, with probability of x%. Quantifying that probability is the goal of
uncertainty estimation. A common uncertainty target (for both experiment and
computation) is more or less 95% (i.e., 20:1 odd, that the true value ft is in fact in the
interval f ± U95%), where U95% is the estimate of the uncertainty at the 95% confidence
level. Note that this target confidence level is compatible with the 2σ range for a Gaussian
distribution, but the concept and the semi-empirical methods presented here do not depend
on the assumption of Gaussian distribution or any other distribution.
Uncertainty estimate can be calculated by Roache’s Grid Convergence Index (GCI). The
GCI is an estimated 95% uncertainty obtained by multiplying the absolute value of the
generalized RE error estimate (or any other ordered error estimator) by an empirically
determined factor of safety, Fs. The Fs is intended to convert an ordered error estimate
into a 95% uncertainty estimate. Since all ordered error estimators for the same quantity
will asymptotically produce the same error estimate, the GCI factor of safety Fs could be
applied to any of these, at least asymptotically; the empirical value of Fs has been
determined from RE estimates. Richardson Extrapolation is based on the assumption that
discrete solutions f, have a power series representation in the grid spacing, h. If the formal
order of accuracy of an algorithm is known, then the method provides an estimate of the
error when using solutions from two different grids. If the normal order of accuracy is not
known, then three different grid solutions are required to determine the observed order of
convergence and the error estimate. Although grid doubling (or halving) is often used with
RE, it is not required, and the ratio of grid spacing, r, may be any real number. Integer grid
refinement is not required; it has an advantage of simplicity (especially for local values
that can be collocated in the grid family) but can cause difficulty. For example, when the
finer grid is just sufficient to resolve scales of interest then a coarse grid with half the
resolution may be insufficient for the problem being simulated. Before any discretization
error estimation is calculated, it must be ensured that iterative convergence is achieved.
23
Iterative methods are always required for nonlinear problems solved by implicit
formulations and may be used as part of an explicit formulation as well. Otherwise, the
incomplete iteration error will pollute the uncertainty estimation. (RE amplifies
incomplete iteration errors). A commonly used but unjustifiable rule of thumb is to require
at least three orders of magnitude decrease in properly normalized residuals for each
equation solved over the entire computational domain. This criterion is used as a default in
some commercial codes but is demonstrably inadequate for many problems even for basic
accuracy, without considering the added requirements of uncertainty estimation. The
preferred approach is to reduce the iterative error to a level negligible compared to the
discretization error. This does not necessarily require iteration to machine zero. Iteration
error and its interaction with discretization error has been thoroughly studied in reference
for one class of problems; there is no reason to assume that other problems are more
benign. If the uncertainty ui contributed by the estimated iteration error is much less than
uh contributed by the ordered discretization error, then we take the numerical uncertainty
unum to be
unum= uh (2.11)
If more care is taken and ui is to be added, it is not adequate to use RMS addition, because
the iteration error affects the results for discretization error (i.e., ui and uh are not
uncorrelated), violating the underlying assumption of RMS addition. Rather, the two must
be combined by less optimistic simple addition.
unum= uh+ui (2.12)
Application of RE and GCI often encounter some difficulties in practical problems. Local
values of predicted variables may not exhibit a smooth, monotonic dependence on grid
resolution, and in a time-dependent calculation, this unsmoothed response will also be a
function of both time and space. The GCI is currently the most robust and tested method
available for the prediction of numerical uncertainty. The errors of these approximations
do not vanish as h→0, and hence are "unordered approximations" or modeling errors
rather than discretization errors. The adequacy of these approximations should be assessed
by sensitivity tests at least on similar problems, but unfortunately in practice these tests are
not often addressed convincingly.
Five-Step Procedure for Uncertainty Estimation is defined below for the application of
the Grid Convergence Index (GCI) method. [1]
Step 1: Define a representative cell, mesh, or grid size, h. For example, for three-
dimensional, structured, geometrically similar grids (not necessarily Cartesian),
h = [(Δxmax)( Δymax)(Δzmax)]1/3 (2.13)
24
For unstructured grids one can define:
h = [(∑𝑛𝑖=1 𝛥𝑉𝑖 )/N]1/3 (2.14)
where:
N= total number of cells used for the computations;
ΔVi = volume of the ith cell;
Step 2: Select three significantly different sets of grid resolutions and run simulations to
determine the values of key variables important to the objective of the simulation study
(e.g., a variable 'ϕ'). There are some advantages to using integer grid refinement, but it is
not necessary. It is desirable that the grid refinement factor, r = hcoarse/hfine, should be
greater than 1.3 for most practical problems. This value of 1.3 is again based on
experience and not on some formal derivation. The grid refinement should, however, be
made systematically; that is, the refinement itself should be structured even if the grid is
unstructured. Geometrically similar cells in the grid sequence are required to avoid noisy
and erroneous observed p. It is highly recommended not to use different grid refinement
factors in different directions (e.g., rx = 1.3 and ry = 1.6), because erroneous observed p
values are produced. (The computational solutions still converge to the correct answers
with rx ≠ ry, but the observed rate of convergence p is affected.)
Step 3: Let h1< h2 < h3 and r21 = h2/h1, r32 = h3/h2 and calculate the order, p
1 𝜀32
𝑝 = [ln(𝑟21)] [ln | 𝜀21 | + 𝑞(𝑝)] (2.15)
p
r
𝑞(𝑝) = ln (𝑟21−𝑠
𝑝 ) (2.16)
−𝑠
32
𝜀32
𝑠 = 𝑠𝑖𝑔𝑛 (𝜀21) (2.17)
Where ε32 = ϕ3 – ϕ2, ε21 = ϕ2 – ϕ1, and ϕk denotes the simulation value of the variable on
the kth grid. Note that q(p) = 0 for r = constant. This set of three equations can be solved
using fixed point iteration with the initial guess equal to the first term (i.e., q=0).
A minimum of four grids is required to demonstrate that the observed order p is constant
for a simulation series. A three-grid solution for the observed order p may be adequate if
some of the values of the variable ϕ predicted on the three grids are in the asymptotic
region f or the simulation series. In fact, it may require more than four grids to
convincingly demonstrate asymptotic response in difficult problems, possibly five or six
grid resolutions in cases where the convergence is noisy. It is all dependent on the initial
grid resolution used and where the predicted value of ϕ lies as a function of grid
resolution. However, to provide a balance between providing both a tractable method and
ensuring a level of accuracy in the predicted observed order p, at least a three-grid study
25
should be performed. If the solution verification error and uncertainty terms δSN and uSN
respectively, are then found to be small compared to the other δi, and ui, terms, three grids
may then be sufficient. If not, then more grids will be required.
Step 4: Calculate the extrapolated values from the equation
𝑝
21 𝑟21 𝜙1 −𝜙2
𝜙𝑒𝑥𝑡 = 𝑝 (2.18)
𝑟21 −1
Step 5: Calculate and report the following error estimates along with the observed order of
the method p.
|𝜙1 −𝜙2 |
𝑒𝑎21 = |𝜙1 |
(2.19)
𝑒𝑎21 = |𝜙1 − 𝜙2 | (2.20)
If 𝜙1 is zero or the user wishes to calculate unum then one should use eq. (2.20).
Estimated extrapolated relative error:
21 |𝜙𝑒𝑥𝑡 −𝜙2 |
𝑒𝑒𝑥𝑡 = |𝜙𝑒𝑥𝑡 |
(2.21)
The fine Grid Convergence Index:

21 𝑒 21
𝐺𝐶𝐼𝑓𝑖𝑛𝑒 = 𝐹𝑠 ∗ 𝑟 𝑝𝑎−1 (2.22)
21
The relative error estimates and the GCI may use normalizing based on values other than
local values; in fact, this is often advantageous for avoiding indeterminacies. Also, the
error estimates and GCI may use dimensional values instead of relative or normalized
values. This is often the natural choice for use with experimental results.
Roache has subsequently recommended a less conservative value for the Factor of Safety,
Fs = 1.25, but only when using at least three grids, solutions and the observed p. He
arrived at this value through empirical studies and suggests that using a value of 1.25
results in a GCI with a 95% confidence interval. The value of Fs = 1.25 has not been
thoroughly evaluated for unstructured refinement. Scatter in observed p is to be expected
because the grid refinement factor r is well defined only for geometrically similar grids.
The accuracy of the GCI will obviously depend on the quality of the unstructured grid
refinement algorithm. Until a sufficient data set is collected and studies are completed for
unstructured refinement, it is generally recommended that the more conservative value of
Fs = 3 be used to obtain a GCI for unstructured grid refinement. If the calculated order of
the method p is less than 1.0, an uncertainty band may also be given by assuming p = 1.0.
26
This is done not to ignore the observed p, but simply to give two calculations, one with the
observed p and one with p = 1.0, as an indicator of the sensitivity of the error band to the
observed value of p. However, the GCI computed with the observed p < 1 is the more
conservative approach. It should also be noted that if the observed value of p is
significantly different from the expected order of the method (for example, the method
might be expected to be third-order for the primary variables but it is observed to be less
than 1), then one should delve into the root cause of this difference. It may suggest a
possible error in the method or its implementation, or that the grid resolutions are not in
the asymptotic region, or that a singularity is present.
The form of the GCI is based on theory, but the use of absolute values for estimated errors
and the factor Fs are based on empiricism involving. The empirical tests involved the
determination of conservatism in 95% of the cases, corresponding to GCI = Unum at 95%
confidence. No assumptions on the form of the error distributions were made nor were
necessary and no assumption of a distribution is required. If the distribution were
Gaussian about the fine grid solution, to reduce normally distributed data to a standard
deviation equivalent, unum would be obtained using an expansion factor k = 2, and the
required term for eq. (2.8) would be:
unum = Unum/k = GCI/2 (2.23)
However, the error distribution about the fine grid solution is roughly Gaussian only for
poorly behaved problems (oscillatory convergence). For well-behaved and highly resolved
problems, the error distribution is roughly Gaussian not about the fine grid solution ϕ1 but
21
rather about the extrapolated solution 𝜙𝑒𝑥𝑡 of eq. (2.18) [i.e., the fine grid solution ϕ1 plus
21
the estimated signed error 𝑒𝑒𝑥𝑡 of eq. (2.21)]. Thus, the error distribution about the fine
grid solution is roughly a shifted Gaussian. Analyses of this situation indicate an
expansion factor k = 1.1 to 1.15 to obtain a conservative value for unum.
unum = Unum/k = GCI/1.15 (2.24)
If the overall uval is later expanded to U95% using k = 2, the numerical contribution will
then be more conservative than 95%.
The five-step procedure presented makes no distinction between steady state computations
or time-dependent computations. The method is independent of temporal resolution in the
sense that Δt does not appear in any of the equations. So, for time dependent
computations, the five-step procedure should be applied at each relevant time step in the
computation at a given node. However, it should be noted that as the spatial grid is refined
during the convergence study, the size of Δt is likely decreasing as well due to numerical
stability issues and thus Δt is implicitly accounted for in the convergence study. The Δt is
treated just like Δx is treated. However, some minor complications arise in the typical case
27
where the numerical methods have different orders of accuracy in space and time, or even
different orders in different spatial directions, as may occur in boundary layer codes.
2.5 ESTIMATION OF SIMULATION UNCERTAINTY

This chapter is concerned with the estimation of simulation uncertainty due to uncertainty
of the simulation input parameters, denoted by uinput in eq. (2.8). The validation uncertainty
has been previously defined as being composed of uncertainty in the numerical
simulations unum, input parameters uinput, and data uD, is given by:
2
u2𝑣𝑎𝑙 = 𝑢𝑛𝑢𝑚
2
+ 𝑢𝑖𝑛𝑝𝑢𝑡 + 𝑢𝐷2 (2.25)
The focus is to estimate uinput, the simulation uncertainty due to uncertainty in simulation
input parameter. Computational simulations usually contain experimentally determined
parameters that have uncertainty associated with them. The model of the system may
range from an algebraic equation to a system of partial differential equations.
Two different approaches for estimating uinput will be presented [7]. The two approaches
depend on whether one takes a local or global view of the uncertainty estimation process.
The local view is concerned with the response of the system in a small (local)
neighbourhood of the nominal parameter vector. In the literature, the local view is known
by a variety of names: sensitivity coefficient method, perturbation method, mean value
method, first order method, and possibly others.
The global view is concerned with the response of the system in a large (global)
neighbourhood of the nominal parameter vector. In the literature, the global view is known
by a variety of names: sampling method, Monte Carlo method, and possibly others. In the
paragraph that follow, only a description of the local uncertainty estimation procedures
will be presented.
2.5.1 SENSITIVITY COEFFICIENT (LOCAL) METHOD FOR A

PARAMETER UNCERTAINTY PROPAGATION
Using a linear Taylor series expansion in parameter space, the input uncertainty
propagation equation for a simulation result S with /I uncorrelated random input
parameters is
∂S 2
2
𝑢𝑖𝑛𝑝𝑢𝑡 = ∑ni=1 (∂X ∗ 𝑢𝑥𝑖 ) (2.26)
i
28
where
S = simulation result;
𝑢𝑥𝑖 = corresponding standard uncertainty in input parameter Xi;
Xi = input parameter;
For situations in which parameters are obtained from a database, the assumption of
uncorrelated errors is a good one. Simulation result S in eq. (2.26) could be a point value
of a simulation variable or an integral quantity such as total drag. The partial derivatives,
∂S
∂Xi
are termed sensitivity coefficients of the result S with respect to input parameter, Xi.
2.6 UNCERTAINITY OF AN EXPERIMENTAL RESULT

This chapter presents the basic concepts from experimental uncertainty analysis that are
used in the determination of the uncertainty of the experimental result, 𝑢𝐷 , in eq. (2.8).
The validation process is dependent upon having an appropriate experimental result that
has a quantified uncertainty estimate, 𝑢𝐷 . ln addition, the experiment will provide many of
the simulation inputs and their associated uncertainties. The experiment will be the reality
of interest that the modeler is trying to simulate. Preliminary simulation results can help in
the design of the experiment and in the proper specification and placement of
instrumentation.
The process used in experimental uncertainty analysis is to calculate the uncertainties of

individual measured variables and then to use these to estimate the uncertainty of the
results determined from these variables. For a measured variable X, the total error is
caused by multiple error sources. The sum of these errors for a measurement is the
difference between the value of the measurement determined in the experiment and the
true value of the measured variable. In experimental programs, corrections to the
measurements are made for those errors that are known, as in the calibration process. For
those errors where the magnitude and sign are unknown, uncertainty estimates are made to
represent the dispersion of possible values for the errors. Use the standard deviation for
each error source to calculate the uncertainty in the measured variable. This standard
deviation quantity is called the standard uncertainty 𝑢 . The uncertainties from error
sources that contribute to the variability of the measurement are classified as random and
the uncertainties from error sources that remain fixed during the measurement process are
classified as systematic.
The systematic standard uncertainty of the measurement of a variable is obtained from the
square root of the sum of the squares of the systematic standard uncertainties for all
independent error sources. For each systematic error source, the experimenter must
estimate a systematic standard uncertainty, bi. Systematic standard uncertainties are
estimated from previous experience, calibration data, analytical models, and the
29
application of sound engineering judgment. The systematic standard uncertainty for
variable Xi is then:
𝑏𝑖 = √𝑏𝑖21 + 𝑏𝑖22 +. . . +𝑏𝑖2𝑛 (2.27)
Estimates of systematic uncertainties are usually made at some confidence level rather
than at the standard deviation level. Typically, these systematic uncertainty estimates are
representative of the 95% limits of the possible values of the systematic error.
2.7 EVALUATION OF VALIDATION UNCERTAINITY

Once an estimate of unum has been made and the uncertainty contributors to uinput and uD
have been made, uval can be obtained by several approaches. The approach illustrated here
is the sensitivity coefficient (local) method. In case 1, the experiment validation variable is
directly measured; in case 2, the experiment validation variable is a result defined by a
data reduction equation that combines variables measured in the experiment. In both these
cases, the values of the variables from the experiment will be inputs to the simulation. The
systematic errors in these inputs are assumed to be uncorrelated for both of them cases.
2.7.1 ESTIMATING VALIDATION UNCERTAINTY WHEN

EXPERIMENTAL VALUE OF VALIDATION VARIABLE IS
DIRECTLY MEASURED (CASE 1)
This case is one in which the experimental value D of the validation variable is directly
measured. A key feature of such cases is that D and S have no shared variables, which
leads to a straightforward evaluation of uinput and uD. The analysis is more complex in case
of D and S have shared variables, as shown in the following section for cases 2.
TSM Approach. Since the experiment validation variable is directly measured, the
experiment and the simulation share no variables and the assumption of effectively
independent errors δinput and δD is reasonable. The expression for uval is, from Eq (2.8),
2
2
u𝑣𝑎𝑙 = √𝑢𝑛𝑢𝑚 + 𝑢𝑖𝑛𝑝𝑢𝑡 + 𝑢𝐷2 (2.28)
with uinput given by the TSM with correlation terms equal to zero,
∂S 2
2
𝑢𝑖𝑛𝑝𝑢𝑡 = ∑ni=1 (∂X ∗ 𝑢𝑥𝑖 ) (2.29)
i
30
Uncertainty exists in the validation condition set point due to uncertainties in the
parameters defining the set point.
2.7.2 ESTIMATING VALIDATION UNCERTAINTY WHEN

EXPERIMENTAL VALUE OF VALIDATION VARIABLE IS
DETERMINED FROM DATA REDUCTION EQUATION (CASE 2)
When the experiment validation variable is not directly measured but is determined from a
data reduction equation using other measured variables, the estimation of uinput and uD (and
subsequently uval) becomes more complex.
Consider the general situation in which the experiment and simulation validation variables
are results determined from data reduction equations each containing some of the N
variables xi where some of the measured variables may share identical error sources. The
general form of the equation for the comparison error is then (where S and D are shown to
be functions of all of the n variables)
E = S(x1, x2, . . . , xN ) − D(x1, x2, . . . , xN ) = δmodel +δnum +δinput −δD (2.30)
In this instance, δinput and δD cannot reasonably be assumed to be independent since S and
D share a dependence on the same measured variables. Application of the TSM approach
to obtain an expression for uval yields
2 2
2
∂S ∂D ∂S ∂D
𝑢𝑣𝑎𝑙 = [( )−( )] 𝑢2𝑥1 + [( ) − ( )] 𝑢2𝑥2 + ⋯
∂x1 ∂x1 ∂x2 ∂x2
∂S ∂D 2 ∂S ∂D
+ [(∂x ) − (∂x )] 𝑢𝑥2𝑛 +2 [(∂x ) − (∂x )]
n n 1 1
∂S ∂D
∗ [(∂x ) − (∂x )] 𝑢2𝑥1𝑥2 + ⋯ + 𝑢2𝑛𝑢𝑚 (2.31)
2 2
where there is a covariance term containing a ux1x2 factor for each pair of x variables that
share identical error sources. When S or D have no dependence on a variable xi, those
2
derivatives will be zero. There is no explicit expression for 𝑢𝑖𝑛𝑝𝑢𝑡 as its components
combine implicitly with components of 𝑢𝐷2 . Equation (2.31) can be expressed in a form
analogous to Eq. (2.8) as
2
u2𝑣𝑎𝑙 = 𝑢𝑛𝑢𝑚
2
+ 𝑢𝑖𝑛𝑝𝑢𝑡+𝐷 (2.32)
31
Where
2 2
2
∂S ∂D ∂S ∂D
𝑢𝑖𝑛𝑝𝑢𝑡+𝐷 = [( )−( )] 𝑢2𝑥1 + [( )−( )] 𝑢2𝑥2 + ⋯
∂x1 ∂x1 ∂x2 ∂x2
∂S ∂D 2 ∂S ∂D
+ [( ) − ( )] 𝑢𝑥2𝑛 +2 [( ) − ( )]
∂xn ∂xn ∂x1 ∂x1
∂S ∂D ∂S ∂D
∗ [(∂x ) − (∂x )] 𝑢2𝑥1𝑥2 + ⋯ + 2 [(∂x ) − (∂xn− )]
2 2 n−1 1
∂S ∂D
∗ [(∂x ) − (∂x )] 𝑢2𝑥𝑛−1𝑥2 (2.31)
n n
2.7.3 INTERPRETATION OF VALIDATION RESULTS
Previous sections have presented a validation methodology based on determining the

validation comparison error, E, and the validation uncertainty, u𝑣𝑎𝑙 , and now discusses
the interpretation of the comparison of these metrics. Note that once a validation effort
reaches the point where the simulation value, S, and the experimental value, D, of a
validation variable have been determined, the sign and magnitude of E = (S – D) are
known.
Recall Equation (2.7):
δmodel = E − (δnum + δinput − δD)
The validation uncertainty 𝑢𝑣𝑎𝑙 is an estimate of the standard uncertainty corresponding to

the standard deviation of the parent population of the combination of all errors (δnum +
δinput − δD) except the modeling error examples for specific cases have been discussed.
Considering the relationship shown in Eq. (2.7), E ± u𝑣𝑎𝑙 , then defines an interval within
whichδmodel falls (with some unspecified degree of confidence). Thus, E is an estimate of
δmodel and u𝑣𝑎𝑙 is the standard uncertainty of that estimate and can properly be designated
uδmodel.
Interpretation with no assumptions made about error distributions: if one has only an
estimate for the standard uncertainty u δ model of δ model and not an estimate of the
probability distribution associated with δnum + δinput − δD, an interval, within which the
value of δmodel falls with a given probability, cannot be estimated without further
assumption. One can make the following statements, however:
32
1. If
|E| > > u𝑣𝑎𝑙 (2.32)
then probably δmodel ≈ E.
2. If
|E| ≤ u𝑣𝑎𝑙 (2.33)
then probably δmodel is of the same order as or less than δnum + δinput − δD.
Interpretation with assumptions made about error distributions: in order to estimate

an interval within which δmodel falls with a given degree of confidence, an assumption
about the probability distribution of the combination of all errors except the modeling
error must be made. This then allows the choice of a coverage factor k such that:
U% = k%u (2.34)
where U is the expanded uncertainty and one can say, for instance, that E ±𝑈95% then
defines an interval within which δmodel falls about 95 times out of 100 (i.e., with 95%
confidence) when the coverage factor has been chosen for a level of confidence of 95%.
2.7.4 SOME PRATICAL POINTS
Ideally, as a V&V program is initiated, the validation variables should be chosen and
defined with care. Each measured variable has an inherent temporal and spatial resolution,
and the experimental result that is determined from these measured variables should be
compared with a predicted result that possesses the same spatial and temporal resolution.
If this is not done, such conceptual errors must be identified and corrected, or estimated in
the initial stages of a V&V effort, or substantial resources can be wasted, and the entire
effort compromised. If uncertainty contributions to u𝑣𝑎𝑙 are considered that examine all of
the error sources in δnum, δinput and δD, then, δmodel includes only errors arising from
modeling assumptions and approximations (“model form” errors). In practice, there are
numerous gradations that can exist in the choices of which error sources are accounted for
in δinput and which error sources are defined as an inherent part of δmodel. The code used
will often have more adjustable parameters or data inputs than the analyst may decide to
use. The decision of which parameters to include in the definition of the computational
model (conceptually separate from the code) is somewhat arbitrary. Some, even all, of the
parameters available may be considered fixed for the simulation. For example, an analyst
may decide to treat parameters as fixed (“hard wired”) and therefore not to be considered
in estimating u𝑖𝑛𝑝𝑢𝑡 , even though these parameters had associated uncertainties. The point
here is that the computational model which is being assessed consists of the code and a
33
selected number of simulation inputs which are considered part of the model. It is crucial
in interpreting the results of a validation effort that which error sources are included in
δmodel and which are accounted for in the estimation of u𝑣𝑎𝑙 be defined precisely and
unambiguously.
34
3
DATA ACQUISITION PROCEDURES
3.1 OVERVIEW OF DATA ACQUISITION

PREPARATION
On October 26, the first data acquisition phase was carried out on the Atka skiff of the
Polito Sailing Team (PST). For the first time on this boat sensors have been installed to
allow data acquisition. The sensors have been developed from the PST sensor area, with
the aim of acquiring data regarding:
• Boat speed;
• Wind speed and direction;
• Roll, pitch and yaw of the hull.
Regarding the speed of the boat, a GPS acquisition system was used, which calculates the
boat's coordinates and its speed. The system was implemented through the Arduino
platform with a sampling frequency (fs) of 1Hz. It is a hardware platform composed of a
series of electronic cards equipped with a microcontroller, combined with a simple
integrated development environment for programming. As you can see from the figures
below, this system was installed inside a waterproof box, positioned near the gravity
centre of the boat, whose position was found using the CAD model. Next to the box there
is also the battery for the acquisition system. Before proceeding with the acquisition, it
was necessary to extend and check the physical condition of the sensor connection cables
with the box. Subsequently, after a verification of the correct installation, we proceeded
with a calibration of the system, to compensate for the lack of accuracy due to systematic
errors present.
35
Figure 3.1 Structure of the Arduino microcontroller, with the relative wiring and two
power supply battery.
The image above shows two batteries, a smaller one, at 5V, to which the Arduino
acquisition system and the GPS and IMU modules are connected. A second, larger one, to
which wind speed and direction transducers are connected, which require a voltage of
12V. To avoid noise and signal interference between the two batteries, the negative has
been shared between both, connecting it in mass.
The last check on the sensors is instead shown below to verify their ignition and correct
operation.
36
Figure 3.2 Programming and calibration of the acquisition system.
The components installed in the data acquisition box are shown below.
37
Figure 3.3 Composition of the box it contains structure of the Arduino
microcontroller, with the relative output wiring and the power supply battery.
Figure 3.4 Zoom on structure of the Arduino microcontroller, with the relative output
wiring and the power supply battery.
38
Figure 3.5 Structure of the Arduino microcontroller connected to Adafruit 9-DOF and
GPS module, with memory SD shield slot installed.
39
Figure 3.6 Waterproof box and connection cables.
From the image above you can understand the type of connection made to connect the
acquisition box with the wind transducers installed in the masthead. The length of the
cables was slightly greater than 8m.
40
Figure 3.7 Ultimate GPS module (Source: [17]).
In order to acquire wind speed and direction, an anemometer and an anemoscope were
installed, respectively. Both positioned masthead with a riveted bracket. From the images
shown below, you can see the support bracket created specifically by the shipbuilding area
of the Polito Sailing Team, in order to be able to install the two sensors in the masthead.
To be able to acquire the data provided, they were connected by a three-pole cable to the
acquisition box located near the centre of gravity of the boat, containing precisely the
aforementioned Arduino, which deals with data acquisition and saving of the same. Before
installation, As can be seen, before assembly, it was verified that the North position of the
anemoscope corresponded to the direction of the boat's axis, so the anemoscope was
calibrated according to the direction of the bow, in order to obtain the wind direction with
respect to the latter.
41
Figure 3.8 Riveted sensor support bracket positioned at the top of the mast.
Figure 3.9 Wind speed sensor (Source: [18]).
42
Figure 3.10 Wind direction traducer (Source: www.siapmicros.com).
Figure 3.11 Installation of the anemoscope on the support bracket.
43
Figure 3.12 Anemoscope calibration according to the heading direction.
Figure 3.13 Support sensor bracket installed on top of the mast.

44
Finally, for the acquisition of roll, pitch and yaw of the hull, or rotations in a three-axis
reference system (x-y-z), the inertial nine-axis platform was used, installed as an
additional external module to the Arduino. Before each test the system was reset so as to
calibrate the yaw angle to zero in correspondence with the direction of the bow, which
was positioned using a compass on the North.
Figure 3.14 Ultimate 9-axis orientation module (Source: [19]).
In order to validate the maximum speed prediction computation program (VPP) it was also
necessary to calculate the distance from the centre of gravity of the two sailors in the
maximum speed regime. This aims to assess the value of the heeling moment generated by
the wind and balanced by the weight of the sailors. To achieve this, two retractable wire
coils were provided, coloured in a different colour every ten centimetres, hinged on one
side to the centre of gravity of the boat and on the other to the life jackets of the sailors.
The distance was calculated by framing the boat using a GoPro Hero Session-type video
camera installed on the mast using a special support. The instant corresponding to the
maximum speed for each sailing was then taken into consideration and the lengthening of
the wire was measured for each of the two sailors. Furthermore, the orientation angle of
the two sailors in reference to the centre of gravity was calculated by means of a
goniometer drawn on the deck of the boat, in correspondence with the centre of gravity.
By means of suitable geometric calculations, the distance of the sailors from the centre of
gravity was obtained at a given moment of each steady state, which corresponds to the
maximum speed reached by the boat in those specific wind conditions.
45
Figure 3.15 Heeling moment.
46
Figure 3.16 Retractable wire coil and goniometer drawn on the deck of the boat.
47
In the images above you can see the sailors in action, thanks to the images recorded by the
on board video camera. In the first image it can be seen that both sailors are on the terraces
to balance the swinging moment generated by the force of the wind on the sails. While in
the second photo you can see only the bowman slightly protruded, while the helmsman is
hanging on the trapeze in the center of the deck. Furthermore, in both images you can see
the retractable cables and the protractor which indicate the position of both sailors at that
particular moment. After the post-processing phase of the data, only the moments in which
the boat speed will be maximum will be analyzed, in correspondence with a given sailing
which forms a specific beta angle with respect to the wind direction.
Following, the Atka sailboat is shown during the preparation phase before going out into
the water, where you can see the sensors installed on the masthead.
48
Figure 3.19 Atka skiff of the Polito Sailing Team with the sensor installed.
3.1.1 SENSORS DATASHEET
As can be expected, each acquisition system has different characteristics, which determine
its behaviour. Therefore, it is appropriate to describe the datasheets of each sensor used, so
as to be able to clarify its precision, resolution and accuracy in acquiring data. Below, the
table will show each of the above features for the sensors used.
49
ULTIMATE GPS MODULE SPECIFICATIONS
Position accuracy 1.8 m
Velocity Accuracy 0.1 m/s
Warm/cold start 34 seconds
Acquisition sensitivity -145 dBm
Tracking sensitivity -165 dBm
Maximum velocity 515 m/s
Table 3.1 Ultimate GPS module specifications (Source: [17])
ULTIMATE 9-AXIS ORIENTATION MODULE SPECIFICATIONS
acceleration 3 channels
magnetic field 3 channels

gauss magnetic field full-scale From ±1.3 to ±8.1 gauss
Linear acceleration measurement range 1/2/4/8/12 mg/LSB
Magnetic measurement range from 205 to 1100 LSB/gauss
Linear acceleration sensitivity ±0.01 %/°C
Magnetic gain setting ±60 mg
Linear acceleration sensitivity change vs. temperature ±0.5 mg/°C
Linear acceleration typical Zero-g level offset accuracy 220 ug/sqrt(Hz)
Magnetic resolution 2 mgauss
Operating temperature range from -40°C to +80°C
Table 3.2 Ultimate 9-axis orientation module specifications (Source: [20])
50
WIND SPEED SENSOR SPECIFICATIONS
Output 0.4V to 2V
Testing Range 0.5 m/s to 50 m/s
Start wind speed 0.2 m/s
Resolution 0.1 m/s
Accuracy Worst case: 1 m/s
Max Wind Speed 70 m/s
Table 3.3 Wind speed trasducer specifications (Source: [18])
WIND DIRECTOR TRASDUCER SPECIFICATIONS

Range 0 ÷ 360 °
Sensitivity <0.1 °
Accuracy ±1°
Operative range 0,25 ÷ 50 m/s
Transducer Wind vane and optical encoder
Working temperature -30 + 60°C

Start up time 5s
Response time 5 s (default); min = 1 s
Table 3.4 Wind director trasducer specifications (Source: www.siapmicros.com)
In the subsequent treatises from these tables it will be possible to deduce the systematic
error that characterizes the operation of each of these sensors and its relative influence on
the global uncertainty due to the experimental data. These will therefore differ from the
true values of a known fixed quantity.
As for table 3.1, in particular we will refer to position accuracy and velocity accuracy. In
order to avoid measurement errors, a static calibration of the velocity data provided by the
Ultimate GPS module was performed, recording a set of values in stationary boat
conditions, obtaining an average velocity accuracy of 0.0224 m/s. This result falls well
within the value indicated by the calibrations carried out by the manufacturer, Adafruit.
However, a position calibration has not been performed, as it is not in this study's interest
to record the position data of the boat, but the latter will only be used to indicate the type
of route taken during the acquisition phase.
51
While for table 3.2 only the 3 acceleration channels will be considered, since in this
discussion it was not possible to exploit the full potential of the 9-axis orientation module
due to technical impossibilities in the development of the code implementation. So, the
specifications that will affect the generation of a systematic error are the linear
acceleration sensitivity and its accuracy. In addition, in the construction of the contested
box, the set of sensors has a difficulty in positioning it perfectly aligned with the reference
axes, horizontal and vertical. For this reason, in the post-processing phase of the acquired
data, the measured offset, compared to a zero degree, was excluded.
• Offsetpitch = 2.5°;
• Offsetroll = 0.187°.
Finally, for the table 3.3 and table 3.4 all the present specifications will be taken into
consideration. As regards the measurement error, the values of accuracy and resolution
published by the manufacturer are indicated as reference, but also in this case an
experimental calibration was carried out producing a zero error for wind direction sensor,
but being difficult to correctly align the sensor flag with its zero position, the accuracy
data provided by the manufacturer Siap Micros is considered more conservative. Instead
as regards the wind speed sensor, it was not possible to perform a calibration as it was not
possible to have a comparison tool to validate the measurement. So, for the wind speed
sensor, see the accuracy and resolution results provided by the manufacturer, Adafruit.
3.2 GPS EXPERIMENTAL DATA ACQUIRED

The route taken by the boat during the data acquisition phase on Lake Como, performed at
Dongo, province of Como (Italy), is shown in Figure 3.20. What is evident, considering
the direction of origin of the wind mainly from the South-East, is that mainly close-hauled
sailing was carried out, with traits of beam-reach winds, alternated, in the final phase of
the acquisition, with short down winds, performed primarily for experimental purposes.
52
Figure 3.20 Ultimate GPS track sailboat on at Lake Como
This track is not intended to divide the sailing ways, but to provide an overall view of
what is obtained by using the GPS Ultimate module which may also be useful in future
acquisition phases to determine the real-time position of the boat. Instead, define how to
divide the various ways navigated it will then be the task of the subsequent post-
processing phase, establishing the criteria that allow to divide one pace from another. We
will not analyse all the instants, but only those that respect the constraints of comparisons
with the computational model subjected to verification and validation.
3.3 OVERVIEW OF VELOCITY PREDICTION PROGRAM

The VPP determines the boat speed for a given true wind angle and speed. The apparent
wind direction and speed are determined from the boat speed Vb and true wind direction
and speed Vr, and then the lift and drag forces due to the sail and structural windage are
calculated. These forces are resolved into drive force and side force; the drive force is then
maximized in order to achieve maximum speed, under the constraint that the heeling
moment may not exceed the maximum available righting moment given by the sailors.
53
Equations and numerical implementation: Check the equations in the VPP will be
impossible without defining the boat reference system that has 3 DOFs: x, y, z axes; the
rotations around these axes generate respectively roll moment, pitch and yaw.
Figure 3.21 3 DOFs Boat reference system. (Source: [3])
The study of the wind interaction with sails start from the follow velocity triangle, in
which is well shown how the apparent wind velocity (Va) is equal to the vector sum of
real wind velocity (Vr) and boat velocity (Vb):
Figure 3.22 Boat velocity triangle. (Source: [3])
54
From velocity triangle shown in Figure 3.22 is possible to extract the follow equations,
splitting the apparent wind components:
𝑉𝑎,𝑥 = −𝑉𝑎 𝑐𝑜𝑠𝛼 = −(𝑉𝑟 𝑐𝑜𝑠𝛽 + 𝑉𝑏 𝑐𝑜𝑠𝜙)
𝑉𝑎,𝑦 = 𝑉𝑎 𝑠𝑖𝑛𝛼 = (𝑉𝑟 𝑠𝑖𝑛𝛽 − 𝑉𝑏 𝑠𝑖𝑛𝜙)
A finest angular resolution won’t provide a better numerical solution neither more
information because of the uncertainty of the motion direction: it is impossible to
guarantee a perfect angle during the navigation, due to the sea and wind conditions and the
human behaviour. In addition, the most interesting information for the boat trend is not the
behaviour at a single wind direction but its performance at the main navigation conditions
(such as close hauled, beam wind, free and downwind).
Results of the VPP: One of the most important result of the VPP is to obtain polar trends
of boat speed at different wind speed. Remembering that the goal of the present work is to
evaluate the sailing yacht performance in low winds.
3.4 ASSUMPTIONS AND CONSIDERATIONS FOR

PROCESSING EXPERIMENTAL DATA
The VPP implementation aim to predict the performance of a small high-performance
sailing yacht in calm waters and low to medium wind speeds. The true wind speed has a
boundary layer depending on the location: the wind speed is considered constant and its
value is the average wind speed in the boundary layer between 0 and 8m (the mast high)
above still water. For simplicity the water current is neglected. This assumption will not
provide a huge error as the wind action on the current will be significant only with strong
winds which is not the case study. The simulation model also has the assumption of a
small heeling angle (its effects on the forces are neglected): it is assumed that the sailors
can maintain the yacht in its best condition at zero heeling angle. The distance between
boat and sailors is defined as the distance between the centre of gravity of the boat and the
centre of gravity of both sailors. The righting moment contributing to the roll equilibrium
is the moment of the sailors. For reasons of installation of the components on the boat, the
position of the centre of gravity position on the z axis is considered zero, this generates a
small error, however not predominant, as the component on the z-axis is very small
compared to the mast high. One of the major difficulties when developing the VPP is to
find the correct aerodynamic coefficients of the sails. For simplicity the main sail force
55
coefficients are included in a single pair of coefficients. This assumption will provide an
error because all the aerodynamic forces are applied in an equivalent center of pressure of
which the position is not accurately known. In order to correlate the drag coefficient of the
sails to the lift coefficient, a rigid wing assumption was necessary due to the lack of
experimental data because of the technical impossibility of acquiring them and due to the
lack of valid CFD simulations of the sail under consideration. Due to these sail
assumptions it is not possible to analyze the intrinsic instability of the air flow around the
real sails and mast. The hydrodynamic force model calculates the drag forces due to the
hull and appendices under the condition that the yacht adopts a leeway angle such that the
centerboard, rudder and hull provide enough lift to counter the sailing side force. The hull
in exam is not conventional and without towing tank tests it is impossible to obtain
experimental data from which evaluate the forces. To solve this problem, in the VPP a
CFD model is used to evaluate the drag of the hull at various speeds: the model has a
maximum error of 5% in the considered velocity range. It was not possible to verify and
validate this model as a towing tank test would have been necessary.
3.5 POST PROCESSING OF EXPERIMENTAL DATA

The validation phase of the theoretical model under examination (Velocity Prediction
Program) consists in comparing its results with those returned by the experimental tests. In
this regard it is appropriate to define the conditions and the ways in which these were
acquired. It follows the choice of the appropriate data that allow a sensible comparison
with the VPP outputs. Obviously before being able to choose the appropriate data it is
necessary to clarify which are the goals of the theoretical model. The output of a VPP is a
performance diagram (boat velocity plot with respect to true wind conditions) that states
the boats optimal (target) speed though the water as a function of the sailing conditions at
best possible trimming. The development of a static Velocity Prediction Program consists
to understand all the forces acting on a sailing yacht and to create a physical model of the
boat in various sailing conditions in order to write the governing equations in order to
model each force acting on the entire boat as a function of certain parameters and then
trying to acquire enough data (from both literature or existing projects) to reduce the
number of variables. Finally, after making assumptions on the sailing conditions of the
sailboats, it is possible to numerically solve the equations in various conditions and to
obtain a reasonable sailing yacht polar. Entering as data the dimensions and the weight of
the body-boat sailors, taking into account, through appropriate parameters derived from
literature and CFD, aerodynamics of the sails and hydrodynamics of the hull, according to
certain ranges of wind values, both in terms of speed and of direction, it is calculated the
forces equilibrium heeling moment e iteratively, respecting the physical constraint, it
maximize boat speed.
56
The acquired data is contaminated by moments in which the boat is stopped or by
moments in which the wind conditions do not fall within the range of values for which the
VPP is developed. In this context, the need arises to extrapolate only those values that
reflect our conditions, so we proceed by first excluding the data in the conditions of a
stopped boat and then excluding the surpluses according to a normal distribution with an
accuracy of 95.45% according to criteria shown in Figure 3.23. Finally, as in the VPP, an
iterative search of the maximum speed values is performed, only in the moments in which
correspondingly also the wind speed is in a constant condition.
Figure 3.23 Normal-Distribution curve (Source: [21])
Below is the flow chart of the procedures that are necessary to perform the entire post
processing phase, as described above.
57
Figure 3.24 Data processing scheme
58
3.5.1 OVERVIEW OF DATA CLEANING PROCESS
The data cleaning process begins with the exclusion of all those data that were acquired
when the boat was still at a standstill, or from the instant of the acquisition system set up
to the launch of the boat, more precisely up to its run by the sailors. After excluding values
at standstill, as well as those of wind speed and boat speed, which were at the extremes of
the probability distribution curve, resulting in surpluses, we proceeded to calculate the
maximum speed of the boat at the way followed by it.
To this end, intervals subdivided according to a range of angle values that the boat forms
with the real wind have been taken into consideration. The angle ranges were taken with a
2-degree step. This was necessary because it would not make sense to scan all the angles
step by step, because the variability of a wind angle does not identify a real variation in
the direction of the boat, but can be the result of oscillations due to the detection of the
wind sensor, or even to a normal and continuous fluctuation of the intensity and direction
of origin of the wind. Also, a further skimming of the found points was performed, which
however were not in the suitable conditions for a direct comparison with the results of the
computational model in question. That is, there was the need to consider only the points at
maximum speed of the boat in which at the same time even the wind speed is not very
variable. Considering the average wind speed, all those points that had a range of
variability included in ±30% of the average value for a given time interval, conventionally
chosen, equal to one minute were considered acceptable.
This is summarised graphically in the following flowchart:
59
Figure 3.25 Data cleaning diagram
60
It is easy to understand that these conditions are rare to find in reality, especially taking
into account the area in which the data acquisition was carried out and the period of the
year, at Lake Como, in autumn. The aforementioned cleaning of the speed data, in the
presence of variable wind, has led to a considerable reduction in the quantity of points at a
maximum comparable speed but making these decidedly more reliable and thus allowing a
timely comparison with the results of the VPP. Although it was not possible to perform an
instantaneous comparison with the trends of the VPP curves, since the latter considers the
constant wind, while in the case under examination the conditions were continuously
variable, a difference was however evaluated for each real point, which was reasonably ,
following the previous considerations, it was possible to consider acquired in the presence
of almost constant wind. At each point there is a given orientation of the boat with respect
to the wind direction and therefore the cases that could have been examined were only
those of the ways that actually could have been performed, by virtue of the conditions
found at the time of acquisition.
Having said this, the reason for the differences between the results obtained in the real
case and in the ideal one will be clearer, although apparently substantial, they are the
result of a series of discrepancies that have been found between the actual acquisition
conditions and those foreseen in the design phase.
61
4
RESULTS
4.1 EXPERIMENTAL DATA RESULTS

Following what described in the previous paragraph, regarding the procedures that were
necessary in order to obtain at least some results comparable with those of the VPP, in this
paragraph the effects of each approximation accepted in the post-processing and in the
cleaning data phases and any resulting difference are described and explained individually.
These phases have been carried out, as already mentioned, in order to take into account
any discrepancy detected in the acquisition phase, with respect to the conditions foreseen
in the design phase. In this regard it is intended to point out that the aforementioned
conditions foreseen in the design phase are steady-state way conditions in the presence of
constant wind for a sufficient time interval such as to allow the boat to perform all the
ways possible with respect to the wind direction, or we intend to follow trajectories that
form every possible angle with respect to the wind (beta angle), thus generating a beta
angle ranging from 0 ° to 180 °.
This condition envisaged in the design phase is difficult to find in reality, especially
considering that the data acquisition was carried out in a lake, which is generally
characterized by variable wind and possible sudden gusts of wind, all the more so in a day
not very windy, like the one found during the acquisition carried out, which therefore
causes the recorded data to have a very variable amplitude compared to a low average
wind value, thus influencing very much the speed reached by the boat in each different
instant analysed .
4.1.1 COMPARISON BETWEEN REAL-BOAT SPEED RESULTS

AND THOSE PROVIDED BY THE VPP
Following what is expressed in the introduction to this chapter, what the sensors provide,
in the form of acquired data, are values of speed of the boat while it traverses a given
trajectory with respect to the direction of the wind at a given value of its speed. Both
direction data (beta) and wind speed are variable almost for every given acquired, i.e.
62
almost every second of acquisition. It is therefore evident that obtaining a steady way in
the presence of a constant wind for each instant of time is not possible with the data
obtained from the sensors installed on the boat. This makes sense however in virtue of the
place and the conditions of acquisition. In this perspective, the boat speed data measured
by the sensors are necessarily different from what is obtained with the iterative
calculations for optimizing the maximum speed of the boat performed by the
computational model. Since in these terms a comparison between the experimental model
and the theoretical model is senseless, the latter has been brought back to the conditions
actually found in reality, therefore the maximum speed of the boat has been derived, no
longer in the presence of constant wind for each way, but rather corresponding to a given
angle of the bow with respect to the wind direction, to that given wind speed value present
in the instant analysed. Therefore, all those moments that provided for an equal angle but
with a lower speed of the boat were excluded. This made it possible to establish that that
specific point in analysis was actually a point at which the boat was at a steady way.
However, it should be pointed out that, excluding all the values in which a given angle
was repeated but with lower boat speeds, there is an exaggeratedly reduced set of values
available. Moreover, as described above, the data set has been further reduced to take into
account the variability of the wind and to try to exclude it as much as possible, keeping
only the data set that had wind speeds that respect the tolerances imposed, which however
lead to they too are errors, since they are defined on an experiential basis. From all this,
follows the small number of comparable points between the real and the theoretical case.
Below are shown results, for both cases, of the boat speed values obtained at a given beta
angle and of a real wind speed. The set of values reported are those that meet all the
conditions imposed, in order to obtain a comparison as rational as possible despite the
problems mentioned above.
63
Beta True wind Experimental boat VPP's predicted boat Speed difference Boat speed error
[°] speed [m/s] speed [m/s] speed [m/s] [m/s] [%]
37,47 1,91 2,08 1,14 0,94 45%
40,95 1,86 2,08 1,23 0,85 41%
44,88 1,83 2,04 1,38 0,67 33%
47,80 1,58 1,99 1,27 0,72 36%
48,80 1,63 1,96 1,30 0,66 34%
50,54 1,52 2,09 1,28 0,80 38%
52,91 1,72 2,04 1,43 0,61 30%
57,84 1,58 2,03 1,43 0,61 30%
59,45 1,56 2,08 1,45 0,64 31%
63,91 1,32 2,05 1,34 0,70 34%
79,72 1,11 2,09 1,20 0,89 42%
125,25 1,34 0,74 1,20 -0,47 64%
127,51 1,50 0,87 1,30 -0,43 49%
139,82 1,54 1,13 1,21 -0,08 7%
150,58 1,77 1,33 1,21 0,12 9%
153,01 1,81 1,24 1,20 0,04 3%
158,72 1,69 0,92 0,76 0,15 17%
161,00 1,55 1,14 0,71 0,43 38%
Average error
32%
Table 4.1 Comparison of experimental and theoretical results of boat speed
From the comparison it is clear that the post-processing and the data cleaning in a view to
reducing the differences between the two cases was not entirely sufficient to eliminate the
error.
Resume the equation 2.1, the validation comparison error E is defined as
E=S–D (4.1)
Thus,
• S = Simulation value;
• D = Experimental data.
In this case study:

• S = VPP's predicted value;
• D = Experimental data;.
64
The validation comparison error E is consistent but it is not surprising since it is
reasonable to expect a high value as it is due to multiple factors that come into play in the
comparison of two values in turn, as already mentioned, the result of simplifications and
assumptions and inevitable some errors of acquisition.
Going back to what is described in chapter 2, it is useful to re-propose equation 2.6, which
describes the components that make up the comparison error.

• the error δnum due to the numerical solution of the equations;
• the error δinput in the simulation result due to errors in the simulation input parameters.
These three categories of errors translate into:

• δmodel due to modeling assumptions and approximations;
• δnum due to the numerical solution of the equations. However, since the equations are
solved by MATLAB calculation software, this error is an infinitesimal value and therefore
negligible;
• δinput due to errors in the simulation input parameters and assuming that the equations
have been written correctly, it is sensible and lawful to consider this error null.
The last source of error is to be input to the experimental model

• δD due to errors in the experimental input parameters caused by acquisition errors
attributable to the characteristics of the sensors, as described in paragraph 3.1.1 where the
tables with the datasheets for each sensor used are listed.
Since the errors δnum , δinput and δD are not very influential in the generation of error E, it is
clear that the main source of error is to be found in the δmodel category in which the
numerous assumptions and approximations necessary come into play to describe the real
conditions found where the test took place.
The error δmodel is due not only to the assumptions made to consider steady way despite
the different wind conditions in the real case compared to what was assumed in the VPP,
but also to the calculation methods of the real wind speed in the experimental case, which
will be described in detail the process to obtain it, and to the presence of hydrodynamic
phenomena, such as the leeway angle, whose contribution has been neglected for the
reasons explained in section 3.3. In this regard it is also necessary to explain how the real
wind speed is obtained in the experimental case. In fact, this is not measured by a certain
sensor, but obtained through geometric calculations from the value acquired by the
apparent wind speed anemometer. It is for this reason that the effect of neglecting the
65
leeway angle now comes into play. To better clarify, it is advisable to go back to Figure
3.15, where the contribution of the leeway angle (ϕ) to the calculation of the real wind
speed (Vr) is easier to interpret, starting from the measurements of the apparent wind
speed (Va) and boat speed (Vb).
Figure 4.1 Resume Figure 3.22
The error provided by the assumption of zero leeway angle is relevant but not
preponderant as at low wind speeds, as in our case, the expected value of this angle, on an
experiential basis, is less than 5 degrees in correspondence of beta angles lower than 70
degrees and with increasing of this tending to zero. This is confirmed by what can be seen
from the results shown in the table, in which the error on the value of the speed of the boat
tends to reduce as the values of beta angles increase above 120 °. Moreover, always on an
experiential basis it can be said that it is almost never greater than 8 degrees even in strong
wind conditions. These considerations make this assumption acceptable. However, if we
wanted to completely eliminate this error, it would be necessary to be able to calculate the
heading angle value between two points using an inertial platform, knowing the latitude
and longitude provided by the GPS system. Although this would have been useful, it was
not possible as the acquisition system was not yet sufficiently developed to obtain this
important data. It can be concluded that the calculation of heading angle would be an
important starting point for future further experimental data acquisition phases.
Comparison of experimental and theoretical results of boat speed with a check on the
sailboat yaw direction. What is expressed in table 4.1 is the result of the data cleaning
process described in paragraph 3.5.1 which aims to identify points for a given beta angle
that respect the maximum boat speed and wind speed tolerances as much as possible
66
constant within a time range. In order to provide a complete view of the possible error
variations following the insertion of a further tolerance to be respected, the latter was
evaluated after adding a further check on the direction of the boat with respect to the wind,
using the data acquired by Ultimate 9-axis orientation module as regards the yaw, so as to
avoid considering moments in which the boat makes changes of direction that could
change its speed. The goal is to obtain a number of points, which although lower, obtained
by excluding as many sources of error, which would distort the data acquisition result.
What has been achieved is shown in the following table:
Beta True wind Experimental VPP's predicted boat Speed difference Boat speed error
[°] speed [m/s] boat speed [m/s] speed [m/s] [m/s] [%]
37,47 1,91 2,08 1,14 0,94 45%
44,88 1,83 2,04 1,37 0,67 33%
47,80 1,58 1,99 1,27 0,72 36%
48,80 1,63 1,96 1,30 0,66 34%
63,91 1,32 2,05 1,32 0,72 35%
79,72 1,11 2,09 1,20 0,89 42%
127,51 1,50 0,87 1,30 -0,43 49%
139,82 1,54 1,13 1,21 -0,08 7%

Average error
35%
Table 4.2 Comparison of experimental and theoretical results of boat speed with a
check on the yaw direction.
A quick analysis of the results shown in table 4.2 reveals an unexpected result regarding
the average error obtained, i.e. the latter increases about 10%, despite the objective being
the exclusion of sources of error. However, a more careful analysis allows us to
understand that reducing the sources of error that could distort the result of the acquisition
does not necessarily mean obtaining a minor final error, but a final error that represents
more faithfully what happens in reality, thus allowing to perform a more accurate and
truthful overall assessment of the results.
67
4.1.2 COMPARISON WITH THE VPP OF THE ESTIMATE
DISTANCE BETWEEN SAILORS AND THE CENTRE OF GRAVITY
OF THE BOAT
In order to validate the prediction computation program of the maximum speed (VPP), it
was also appropriate to calculate the distance from the centre of gravity of both sailors
during each way performed at maximum speed. This has the aim of assessing the value of
the heeling moment generated by the wind and balanced by the weight of the sailors. To
obtain this data, sensors were not used, but, as described in chapter 3, two retractable
spools of wire were provided, hinged on one side to the centre of gravity of the boat and
on the other to the life jackets of the sailors. The distance has been calculated, viewing the
videos made through GoPro Hero Session action-cam, installed on the mast, measuring
the variation in length of the wire, for each of the instants corresponding to the points that
satisfy the constraints imposed during the post processing of the data, which, as described
in the preceding paragraphs, must guarantee wind speeds with maximum boat speeds at
almost constant wind. Through appropriate geometric calculations, the distance of both
sailors from the centre of gravity was obtained, which however is not directly comparable
with that calculated by the VPP, because the latter represents the distance from the centre
of gravity of a single equivalent sailor at a given moment of each pace at regime
implemented. Therefore, the distances of the sailors were subsequently referred to a single
equivalent sailor, thus making a further small approximation so as to make the
experimental and simulated results comparable. However, this approximation is not
preponderant in the creation of the final error, since the weight of the two sailors taken
into consideration is comparable (approximately equal to 70kg) and moreover the distance
between the two, on board, during navigation, is negligible and even more negligible
compared to the overall distance that both have with respect to the centre of gravity. On
the contrary, the method of calculating the distance is not marginal, on the contrary, the
greatest cause of error, which necessarily results in a high risk of error because it is based
on the observation of the phenomenon via video. For the calculation of the distance of the
equivalent sailor from the centre of gravity of the boat, neglecting the altitude on the z-
axis of the centre of gravity, the positions on the x and y-axes of both sailors were
measured (where the x-axis is the longitudinal one to the boat and the y-axis the transverse
one), then through square root sum the distances of both sailors were obtained and having
both of the same weight the equivalent sailor distance was obtained as the average of the
two distances.
Table 4.3 below shows and compares the values of distance measured experimentally and
those simulated by the computational model in correspondence with the various speeds
achieved during the test phase, as already done in the previous paragraphs for the speed
values of the boat. The values predicted by the boat speed VPP do not appear because they
are not necessary for the purpose of this analysis. Instead, those of speed measured
experimentally by GPS are highlighted to show the trend of the latter in relation to the
68
angle formed with the direction of the real wind in order to show the evolution of the
moment that it is necessary to generate by both sailors to balance the skidding moment.
True wind Experimental Experimental VPP's predicted Distance

Beta speed boat speed distance SAILORS - distance SAILORS - deviation Error
[°] [m/s] [m/s] SAILBOAT GC [m] SAILBOAT GC [m] [m] [%]
37,47 1,91 2,08 0,20 0,21 -0,01 0,32%
40,95 1,86 2,08 0,20 0,22 -0,02 1,00%
44,88 1,83 2,04 0,20 0,27 -0,07 3,81%
47,80 1,58 1,99 0,53 0,21 0,32 16,40%
48,80 1,63 1,96 0,80 0,21 0,59 30,43%
50,54 1,52 2,09 0,53 0,20 0,32 16,61%
52,91 1,72 2,04 0,53 0,24 0,28 14,66%
57,84 1,58 2,03 0,53 0,22 0,31 15,94%
59,45 1,56 2,08 0,53 0,22 0,30 15,75%
63,91 1,32 2,05 0,43 0,18 0,25 12,79%
79,72 1,11 2,09 0,20 0,09 0,11 5,43%
125,25 1,34 0,74 0,01 0,03 -0,02 1,11%
127,51 1,50 0,87 0,01 0,03 -0,02 1,27%
139,82 1,54 1,13 0,01 0,01 0,00 0,25%
150,58 1,77 1,33 0,20 0,00 0,20 10,31%
153,01 1,81 1,24 0,20 0,00 0,20 10,45%
158,72 1,69 0,92 0,00 0,00 0,00 0,07%
161,00 1,55 1,14 0,00 0,00 0,00 0,01%
Average
error
8.70%
Table 4.3 Comparison of experimental and simulation results of the distance measure
between sailor-sailboat GC
From the results shown in the table above, it is noted that for upwind sailing, those for
which the beta angle, which the boat forms with respect to the real wind, is less than 90
degrees, in which higher boat speeds are obtained, even the excursion of the sailors to
balance the skidding moment increases, consequently, the distance of the same from the
centre of gravity also increases. However, even in the case of the experimental detection
69
of the distance, as well as for the determination of the maximum speed, shown in the
previous paragraph, the comparison error E comes into play, defined as:
E=S–D (4.3)
Where in this case:
• S = Simulation value of distance between sailor-sailboat GC;

• D = Experimental data of the measurement of the distance between sailor –
sailboat gravity centre.
This error is on average small, except for some sporadic cases, which may be attributable
to a measurement error. To better understand the reason for a lower error than that
obtained in calculating the maximum boat speed, as in the previous case, it is advisable to
divide the comparison error into different categories of sources of errors that compete to
create it. This will be useful to understand how many and what are the greatest possible
sources of error. Being:
So, in this case, unlike the previous one in which the prevalent error was due to the
assumptions and approximations present in the model, both simulated and experimental,
although there are assumptions and approximations in the experimental model, these are
marginal, as the only approximation made in the experimental phase concerns the position
of the centre of gravity, which considers the altitude on the z-axis zero. The reason for this
choice, in addition to the need for physical installation of the components, is that the same
approximation is also made in the simulation generated by the computational model in
order to simplify the treatment of the mathematical model, therefore this error is
simplified in both models. So the error δmodel as regards the experimental model cannot be
considered insignificant, but it will not be dominant, while it is not possible to affirm the
same thing for the simulated model, as the assumptions and approximations made for the
calculation of the maximum speed influence however, the program optimization function
which also takes care of calculating the distance from the centre of gravity of the boat of
the equivalent sailor, which in itself is an approximation, which although small, is not
negligible. On the contrary, in the experimental model, it is no longer possible to neglect
the acquisition error, which as already mentioned, is the result of experimental observation
of the phenomenon, therefore δD in this case will be the predominant error.
Having made these considerations, remembering what was said at the beginning of the
paragraph regarding the way taken, it is logical to expect that the error will be minimal in
cases where the steps performed have lower boat speeds, i.e. loose speeds, for which the
beta angle is greater by 90 °, since the overturning moment being minimal, the excursion
of the sailors to balance it will also be minimal, and consequently their distance from the
70
centre of gravity of the boat will be almost zero. This results in a minimal, almost
negligible experimental observation error. On the contrary, it will be maximum in upwind
and transverse cases, or beta angles of about 90 ° with respect to the wind, because the
distance of the sailors from the centre of gravity will be greater and therefore the
experimental observation error can easily increase.
Comparison of experimental and theoretical results of distance between sailor-
sailboat GC with a check on the sailboat yaw direction. Also in this case it was decided
to analyse the distance of both sailors from the centre of gravity of the boat for the only
points that also respect the constraint concerning the yaw direction of the boat, and also in
this case it was confirmed that the average error increases because points that could distort
the test are excluded, but the accuracy of the result benefits.
True wind Experimental experimental VPP's predicted Distance Distance

Beta speed boat speed distance SAILORS - distance SAILORS - deviation Error
[°] [m/s] [m/s] SAILBOAT GC [m] SAILBOAT GC [m] [m] [%]
37,47 1,91 2,08 0,20 0,21 -0,01 0,32%
44,88 1,83 2,04 0,20 0,27 -0,07 3,68%
47,80 1,58 1,99 0,53 0,21 0,32 16,40%
48,80 1,63 1,96 0,80 0,21 0,59 30,43%
63,91 1,32 2,05 0,43 0,17 0,26 13,37%
79,72 1,11 2,09 0,28 0,09 0,19 9,71%
127,51 1,50 0,87 0,00 0,03 -0,03 1,79%
139,82 1,54 1,13 0,00 0,01 -0,01 0,77%
Average
error
9.56%
Table 4.4 Comparison of experimental and simulation results of the distance measure
between sailor-sailboat GC with a check on the yaw direction.
Table 4.4 shows how the average error increases, as in the previous case, about 10%,
however excluding numerous loose ways, therefore with high beta angle, which
corresponded to points of error about zero, but which concealed possible steering
manoeuvres or jibe that did not necessarily have a meaning useful for the analysis of the
distance necessary to generate balance with respect to the opposite action generated by the
overturning moment, which is evidently small in the manoeuvre phase at low speed.
Consequently, as desired, a final error value was obtained which allows also in this second
analysis to perform a more accurate and truthful overall evaluation of the results.
71
Overall, the measurement of the distance of both sailors from the centre of gravity of the
boat, placed at a zero axis height, is in accordance with that calculated by the VPP unless
experimental observation errors of great weight, which however are equated with points at
less error, but also less effective and less accurate for the above analysis, corresponding to
ways at low speeds.
4.2 PLOTS EXPERIMENTAL SAILBOAT SPEED VALUE

Returning to the results mentioned in the previous paragraphs, showing graphically what
has been obtained in numerical form, and already listed in the tables, facilitates
understanding and highlights more the differences between the results of the experimental
model and those predicted by the simulated model. Furthermore, the use of plots allows to
broaden the vision of each single data to a complex of data, giving prominence to the
general trend of the various points, and easily allowing to exclude the population of points
that deviates excessively from the average trend, so as not to focus on the mere numerical
value of the average error, as could happen by observing the values in the table, because
the latter could distort the vision of the overall result, but focusing on the objective of
understanding with what precision and accuracy the simulated model replicates the reality
that has been analysed in the experimental phase and how this model could be improved to
solve the cases in which it presents the greatest discrepancies, as well as understand how
to improve the experimental data acquisition phase, to bring the expected conditions in the
design phase of the computational model closer with those found in the test phase. This
does not mean looking for ideal conditions in reality, which would obviously be senseless
and useless, on the contrary to understand if what was thought in the design phase could
actually be found in real conditions. To do this, numerous tests are needed, in numerous
different conditions. Observing carefully the results obtained is obviously indispensable to
achieve these objectives, because it allows us to understand in which conditions there are
gaps in the results, which do not allow us to say whether the computation program
reproduces reality or not, and with what reliability, and at the same time if the
experimental conditions acquired are actually congruous with what is real or if they are
the consequence of some errors in the acquisition phase or in the post processing phase.
4.2.1 POLAR PLOT OF EXPERIMENTAL SAILBOAT SPEED.

The first image to be shown obviously represents the main output of the VPP, i.e. the
maximum speed value of the sailing boat in steady state, subjected to a wind of constant
intensity, while forming a certain angle with respect to the direction of the real wind, i.e.
the direction that the wind would have with respect to the boat if it were considered
72
stationary, excluding the apparent wind condition due to the motion of the sailboat
detected by the sensor installed in the masthead.
What has been said is shown in the following figure:
Figure 4.2 Polar Plot of experimental sailboat speed at a given true wind angle
As shown in Figure 4.2, most of the points that respected the constraints imposed in the
post processing phase, to ensure comparability with the results of the VPP, were acquired
during sailing which formed angles of less than 90 degrees from the wind direction ,
therefore in close-hauled conditions, both with starboard tack and with port tack, i.e. with
wind from the right or left with respect to the direction of the sailboat. This type of way is
the most used in less windy conditions and still allows you to reach fair speeds. What is
mainly missing are ways with beam wind, as a single point represents this condition, and it
is the one for which maximum speeds would be obtained in the case of more windy
conditions than those found in the test phase. It is a type of way that is carried out when
the sailboat forms an angle with the wind direction that oscillates around 90 °. Finally, as
previously mentioned in chapter 3, some sailing at broad reach or with down-wind have
been created, which are represented by points with angles of about 120 ° - 180 °, also in
this case both with starboard tack and with port tack. This type of way is less preforming
73
at the maximum speed level and difficult to achieve with slightly windy conditions, as it is
better combined with the use with gennaker, which, however, due to the unsuitable
conditions found in the test phase, has not been used and therefore not considered even in
order to obtain the subsequent results that will be shown among VPP speed predictions. It
is noteworthy and useful also for the interpretation of the subsequent plots, the fact that for
beta angles between about 0 ° - 30 ° with respect to the wind direction, considered to come
with a beta angle equal to 0 °, sailing is impossible to realize, therefore we speak of dead
angle with both starboard tack and with port tack. Therefore, it is correct that values are
never found in that range of angles.
Polar Plot of experimental sailboat speed excluding points with excessive variations
of the sailboat yaw direction. As already shown in the tables of the initial paragraphs of
this chapter, the differences in the results obtained are emphasized considering the further
control with respect to the yaw variations greater than 45 °, which are senseless for the
purpose of a correct speed and therefore can conceal sudden changes of direction or turns
or jibes, which obviously must be excluded from the analysis of the results.
Figure 4.3 Polar Plot of experimental sailboat speed at a given true wind angle with a
check on the sailboat yaw direction
74
Figure 4.3 above allows to note that, excluding excessive variations of sailboat yaw
direction, some close-hauled way points have been removed, which from the tables it is
clearer to identify which ones with precision, therefore refer to the vision of tables 4.1 and
4.2 for a clearer interpretation, but with the plot it is more immediate to understand that
these are points with values of sailboat angles direction among the wind very close to each
other, between 45 ° - 60 °. Since the boat speeds are generally low, these points could be
apparently accurate, but in reality evidently they could be distorted by a possible sudden
change of direction, which does not necessarily have to change the speed, but which,
however, as already mentioned, for a more accurate analysis should not be considered. In
the same way it is clear to identify the exclusion of down-wind points, which being
running-port tack are at high risk for jibes, but for these it was easier to expect them to be
inaccurate because they also had very low speeds.
4.2.2 VELOCITY PLOT OF EXPERIMENTAL SAILBOAT SPEED

DURING TIME
To describe what emerges from the results mentioned above and to understand more easily
the reasons, the following plot that shows how the boat speed varies throughout the data
acquisition phase, in relation to the trend of direction and speed of the wind.
Figure 4.4 Velocity Plot of experimental sailboat speed during time at specified wind
speed and direction
75
If you want to describe what you learned from figure 4.4, you can start by clarifying the
direction of the arrows, which identify the direction of origin of the wind with respect to
the direction of the boat, which for graphic reasons was considered directed towards y-
axis . Then the arrows are rotated at an angle between 0-180 ° with respect to the vertical
axis, in order to identify at what angle the wind hit the sailboat. This allows us to
recognize the proper ways to which points at different speeds correspond, as shown in the
previous polar plots. However, the real goal of this graph is to show how the speed of the
boat is directly proportional, not only to the wind speed, which of course is expected, but
also to the direction from which the wind is coming in relation to the direction traveled by
the boat. This allows us to understand why the velocity values obtained and shown in the
polar plots in relation to the respective angle values. The higher the incidence of the wind
on the sail surface, the higher the achievable speed, even at the expense of a constant wind
speed. However, it should be borne in mind that in the real case there is motion inertia,
which therefore allows the sailboat to maintain speed despite a change in speed and wind
direction, especially considering that it is a racing sailboat, light and purely planing..
Polar Plot of wind speed and its direction among axis origin. To show in detail the
wind speed and direction components that give rise to the length of the arrows shown in
the graph above and their orientation in space, a specific polar plot can be used for the
aforementioned wind components, as done below.
Figure 4.5 Polar Plot of wind speed experimental value [m/s] at beta angle [°]
76
This additional plot is important for understanding how each arrow in the previous plot in
Figure 4.4 is oriented. In fact, the blue lines, in this second case, are rotated by a certain
beta angle, which, as mentioned several times previously, is the angle that the direction of
the longitudinal axis of the boat forms with the direction of the wind, when this angle is
reported on the Cartesian plane of the Velocity plot, it gives us the direction of the arrow
with respect to the y-axis.
Furthermore, this polar plot allows us to identify the entity of the numerical value of the
wind speed. The latter is of significant importance because it highlights the fact that all
blue lines, which represent the wind speed vector, have a similar length, allowing to say
with discrete precision that, in the points saved among the thousands analysed, they were
found the desired conditions of almost constant wind, despite the fact that they are fair
points spaced over time, as shown once again in the x-axis of the velocity plot.
However, at the same time, they provide further demonstration of how low the wind
intensity was, in fact it never reaches 2 m/s. In order to have a counter test of the
constancy of the wind values, the saved data are further checked, checking that no
moments of tacking or jibe have been considered, as done in the case of boat speeds.
Velocity Plot differences excluding sudden changes of sailboat yaw direction. As

mentioned above, also in this case we want to pay attention to the effects of a further
constraint on the values obtained and above all we pay attention to which values are
excluded.
77
Figure 4.6 Velocity Plot of experimental sailboat speed during time at specified wind
speed and direction with a check on the sailboat yaw direction
Looking at both figures 4.4 and 4.6 it can be seen that around the 30th minute, there is a
change of direction in the first figure which is rightly excluded in the second, this second
plot therefore allows us to say that it is correct and sensible to try to add the further
constraint regarding the yaw direction variations of the sailboat so as to have a comparison
on the effects of this constraint on the average error, which in itself is not very interesting,
but it is useful to note that the accuracy of the result found, i.e. the average error increases,
following the exclusion of some values, which thanks to this graph, we can say with
greater confidence to be distorted by the presence of changes of direction.
Polar Plot of wind speed and its direction among axis origin excluding sudden
changes of sailboat yaw direction. Check that any points that represent moments of turns
or jibes with the aim of verifying what the wind direction was in those moments and how
the trend of the wind speed varies, shown in the following graph, are not considered.
78
Figure 4.7 Polar Plot of wind speed experimental value [m/s] at beta angle [°]
Figure 4.7 shows that most of the instants removed are those characterized by running-
port tack, which, as already mentioned, are at high risk of sudden jibes. However, this
graph is not useful to define the variation of the average wind speed after the exclusion of
the distorted points, but however it is not useful in order to consider the wind speed
constant and for this reason not further investigated. What is useful for understanding how
much the wind speeds differ from one point to another is to analyse the standard deviation,
i.e. an indicator of dispersion of a distribution of values, both in the case without control
over the direction of the yaw and with control. If the value is greater than the previous
one, it will mean that the wind speed values will be more dispersed, on the contrary more
united. Tuttavia, tale calcolo non si può fare senza avere una popolazione sufficientemente
grande di dati. Per tale motivo ci si limita ad una più semplice valutazione della media.
The equation that represents the value of the standard deviation is expressed below:
∑n
i=1(𝑥𝑖 −𝑢)
2
𝜎=√ (4.5)
𝑁
79
The calculation of the standard deviation, by means of calculation software, provides the
following results:
m
𝜎𝑤𝑖𝑛𝑑 = 0.2059 [ s ] (4.6)
m
𝜎𝑤𝑖𝑛𝑑𝑦𝑎𝑤 = 0.2574 [ s ] (4.7)
Equation 4.6 represents the standard deviation value for the case without constraint on the
yaw direction, while equation 4.7 represents the other case. The differences between the
wind speed values and the average speed of this set of points is equal to about 0.2 [m/s],
therefore a small value compared to the average of the wind speed values and is therefore
considered almost constant for this set. Furthermore, it is clear that the removal of some
points that did not respect the aforementioned constraint led to the removal of wind values
that differed less from the rest of the population in question. However, the difference
between the two is minimal, therefore the statement that the wind speed is almost constant
in the points analysed remains valid.
A parameter that can be useful to consider the variability of the data and the difference
that is added by adding a constraint and therefore decreasing the sample population is the
coefficient of variation (C.V.), this represents the ratio of the standard deviation to the
mean of the intensity values of the wind speed in the various moments considered, and it
is a useful statistic for comparing the degree of variation from one data series to another,
even if the means are drastically different from one another. The most common use of the
coefficient of variation is to assess the precision of a technique and to know the
consistency of the data. By consistency we mean the uniformity in the values of the data
from the arithmetic mean of the distribution. The standard deviation of an exponential
distribution is equivalent to its mean, the making its coefficient of variation to equalize 1.
Distributions with a coefficient of variation to be less than 1 are considered to be low
variance, whereas those with a CV higher than 1 are considered to be high variance. The
coefficient of variation (CV) is defined as the ratio of the standard deviation to the mean.
Whereas the average μ:
∑n
i=1 𝑥𝑖
𝜇= 𝑛
(4.8)
So, the C.V. is defined as:
𝜎
𝐶. 𝑉. = 𝜇
(4.9)
80
The calculation of this parameter provides the following results:
𝐶. 𝑉. 𝑤𝑖𝑛𝑑 = 12.86 % (4.10)
𝐶. 𝑉. 𝑤𝑖𝑛𝑑𝑦𝑎𝑤 = 16.57 % (4.11)
Equation 4.10 represents the coefficient of variation for the case without constraint on the
yaw direction, while equation 4.11 represents the other case. This coefficient is much
lower than the unit, in both cases, but in the second it has slightly increased, this means
that values that differed less than the others compared to the average have been excluded.
However, these percentages are small and approximately similar to each other, indicating
sufficient accuracy of the results. This information is useful to confirm at least that the
wind speed in the instants considered is not very variable.
4.2.3 COMPARISON EXPERIMENTAL – VPP’S PREDICTED

SAILBOAT SPEED
After analysing and describing in detail, within the previous paragraphs, the behaviour
found in the experimental test phase of the sailboat, in the wind conditions described in
figures 4.4 and 4.6, it was ascertained that most of the available data concern close-hauled
ways and to a lesser extent also broad reach and down-wind ways, due to the limited wind
available on the test day. So, after entering the same wind conditions within the
computational program, the simulation performed produced as many sets of values, as
described in the aforementioned tables 4.1 and 4.2. In order to make the extent of the error
easy to interpret, an additional polar plot was created, which compares the experimental
and simulated results. Subsequently, to allow easier analysis of the error, the same plot is
re-proposed, but in a linear form.
81
Figure 4.8 Polar comparison of experimental – VPP’s predicted sailboat speed
From a quick observation of the polar plot of comparison between the values acquired
experimentally and those predicted by the VPP, an approximately constant error is noted
for all instants representing close-hauled way, therefore for beta angles between 40 ° - 60
°, while it is less in the case of broad-reach and down-wind ways. This can act as a starting
point for reflection on what has been done during the data acquisition phase. In fact, it
would have been reasonable to expect a constant error for each type of way, or at least it
would have been a good result for the purpose of a subsequent review of the VPP.
However, as already mentioned, the reduced wind intensity has prevented the creation of a
sufficiently large number of broad-reach and down-wind ways, since being the latter of
the running-port tack, they are effective and stable gaits only with the support of the use of
the gennaker. However, what can be said is that mainly the values simulated by the VPP
tend to be conservative with respect to the speeds actually recorded.
Linear plot of comparison between experimental – VPP’s predicted sailboat speed.

The following graph, which reproduces the same contents of the polar plot above, but in
linear form, has the aim of allowing easy reading of the numerical entity of the error,
represented by the red linear bar, which is generated as the difference between the values
recorded experimentally, in blue, and VPP's predicted, in black.
82
Figure 4.9 Linear comparison of experimental – VPP’s predicted sailboat speed.
Looking at figure 4.9, the need for further data acquisition phases is even more evident,
especially with regard to down-wind ways, so as to be able to verify or not the presence of
a constant error for all beta angles, or in any case the presence or less error offset for the
different types of way, close-hauled, beam-reach, broad-reach and down wind.
Comparison plot differences excluding sudden changes of sailboat yaw direction.

Once again it is interesting to analyse in detail the differences between the Polar
comparison plot without checking the direction of the boat and what takes it into account.
This is done with the intention of confirming whether or not the prediction of constant
error values in close-hauled way conditions and error with a non-constant offset in the
case of broad-reach and down-wind way.
83
Figure 4.10 Polar comparison of experimental – VPP’s predicted sailboat speed with a
check on the sailboat yaw direction.
As expected, this counter verification graph shows that the error has a constant offset in
the case of ways with angles between 40° - 60° and in the same way confirms the doubts
about the consistency of the results found for angles between 150° -180° , which in fact
are then excluded due to the presence of a sudden change of trajectory, as shown also by
the Velocity plot in figure 4.7. The need for further analysis and tests regarding the values
acquired in wind conditions with a direction between 120° - 150° is also confirmed, which
are not distorted by changes of direction, because it is not a matter of running-port tack,
but however, the entities of the error remain variable in such a narrow range of angles, as
shown in the following graph:
84
Figure 4.11 Linear comparison of experimental – VPP’s predicted sailboat speed with a
check on the sailboat yaw direction.
Resuming the discussion of the excessively variable amount of error in the case of broad-
reach way, after observing figure 4.11, the need to acquire further data in the future can be
further confirmed to check what the error produced by the VPP is actually. In fact, at the
moment it is not conservative as regards running-port tack, but this is in contrast with
what was observed instead in the case of close-hauled ways. And since the data on the
latter are much greater and therefore much more accurate and reliable, before reaching
hasty and perhaps incorrect conclusions, it is good to deepen with further tests.
What can be studied now with the data available, it is not so much the average deviation
or the error, which has already been evaluated in tables 4.1 and 4.2, but as done for the
wind speed component, it is the evolution of the deviation standard in the two cases
analysed for boat speed. Taking the standard deviation equation:
∑n
2
𝜎=√ (4.12)
𝑁
85
The calculation of the standard deviation, through calculation software, provides the
following results:
m
𝜎𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑 = 0.5172 [ s ] (4.13)
m
𝜎𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑𝑦𝑎𝑤 = 0.4857 [ s ] (4.14)
Equation 4.13 represents the standard deviation value for the case without constraint on
the yaw direction, while equation 4.14 represents the other case. Contrary to what
happened for the wind speed, it can be seen that the removal of some points that did not
respect the aforementioned constraint, led to the removal of sailboat speed values that
differed most from the rest of the population in question. However, the difference between
the two, even in this case, is minimal. So, not being able to say whether the error
generated is the result of a problem in the VPP or if more likely it is due to a low
population of data for certain ranges of angles, which was discussed above, it can however
be said, given the small difference of dispersion of the population between the two cases,
that a check on the yaw is not essential in evaluating the error on the speed of the boat, but
it was useful to formulate hypotheses on where to search for the error in the future.
A statistical indicator of relative dispersion is also assessed for boat speed, useful for
indicating the variability of a phenomenon in percentage terms. calculated as the ratio
between the standard deviation and the average of the velocity distribution in the various
instants. This dispersion coefficient turns out to be:
𝜎
𝐶. 𝑉. = 𝜇
(4.15)
It provides the following results:
𝐶. 𝑉. 𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑 = 31.14 % (4.16)
𝐶. 𝑉.𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑𝑦𝑎𝑤 = 27.34 % (4.17)
yaw direction, while equation 4.17 represents the other case. A discrete variability of boat
speeds is evident, which tends to decrease if the accuracy of the population under
examination is increased, decreasing the total number. The variability of the error between
the results provided by the VPP and the experimental ones will be assessed. The
evaluation of the absolute and relative standard deviation, for the speed of the boat in the
86
various instants, is useful for understanding if some outputs are influenced by instants not
present after post processing, but which can contribute negatively to the overall result,
falsifying the values assumed by the boat in subsequent positions, and this can be seen by
assessing the variability as was done in this paragraph.
Bar graph: rate error among VPP’s predicted boat speed and experimental one.
Finally, we want to show the error rate that is generated between the simulated and the
experimental speed, with reference to the latter, which being measured by GPS is
considered accurate, unless the GPS module offset datasheet. In summary it is the graphic
representation of the error shown in table 4.1.
Figure 4.12 Error bar among VPP’s boat speed and experimental one at given beta
angle
From the bar graph shown above, we note what has already been said by observing the
previous plots, i.e. that there are many error bars with a similar value as regards angles
corresponding to upwind ways, while for down-wind ways there are anomalous
percentages of error of speed, either very large or very small, so for these values there is
the need to investigate further before reaching conclusions. While for the values of
87
upwind ways it can be reiterated that the error, although consistent, is almost constant and
therefore it will be easier to consider it to try to remove it in the future.
One way to investigate and try to understand, if and which values in down-wind ways are
to be discarded, can be to add the aforementioned yaw direction check, so as to see more
clearly what the differences are compared to the previous graph. We therefore want to
show the error calculated in table 4.2.
Figure 4.13 Error bar among VPP’s boat speed and experimental one at given beta
angle with yaw direction check.
The error bars are now more homogeneous with each other, except for a very small error
case. Therefore, it cannot be said that surely the down-wind values are not reliable,
because there is no reference to compare them, but it can be safely stated that they are
excessively variable in order to use them for a program evaluation effort.
As mentioned above, we intend to evaluate the relative dispersion index of the error
between VPP and experimental results, which allows us to understand whether the
variability of the error benefits or not from a control on the yaw direction, i.e. whether or
not to increase the accuracy of the population in question at the expense of the number
brings advantages for the analysis of the error. The coefficient of variation is calculated
88
once again, with reference to the average error value at each instant taken into
consideration:
𝜎
𝐶. 𝑉. = 𝜇
(4.18)
To calculate it, first evaluate the standard deviation:
m
𝜎𝐸𝑅𝑅𝑂𝑅 𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑 = 0.28 [ s ] (4.19)
m
𝜎𝐸𝑅𝑅𝑂𝑅 𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑𝑦𝑎𝑤 = 0.27 [ s ] (4.20)
The average error is:

m
𝜇𝐸𝑅𝑅𝑂𝑅 𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑 = 0.55 [ s ] (4.21)
m
𝜇𝐸𝑅𝑅𝑂𝑅 𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑𝑦𝑎𝑤 = 0.64 [ s ] (4.22)
𝐶. 𝑉. 𝐸𝑅𝑅𝑂𝑅 𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑 = 52 % (4.23)
𝐶. 𝑉.𝐸𝑅𝑅𝑂𝑅 𝐵𝑜𝑎𝑡𝑆𝑝𝑒𝑒𝑑𝑦𝑎𝑤 = 43% (4.24)
yaw direction, while equation 4.24 represents the other case. Interesting results are
obtained in terms of error analysis. In fact, it can be noted that the average error is greater
in the case of control on the yaw direction, but then the coefficient of variation is, as
expected, lower, this confirms that taking into account a more accurate, albeit lower,
distribution of values number, improves the accuracy of the result. Subsequently, we will
also evaluate how the error varies by doing the reverse procedure, i.e. increasing the
number of the starting distribution, but decreasing its accuracy.
89
4.2.4 PLOT MEASURED DISTANCE BETWEEN SAILORS AND
SAILBOAT CENTER OF GRAVITY
After having described in detail what was recorded during the test phase with regard to the
parameters that affect the maximum speed of the boat reached during the specific sailing,
i.e. mainly the speed and direction of the wind with respect to the sailboat, we can now try
to interpret more easily the values obtained as regards the necessary distance that the
hypothetical equivalent sailor should have in order to balance the action generated by the
force of the wind on the surface sail. In this regard, it is interesting to observe the
following graph.
Figure 4.14 Polar plot: measured distance between sailors and sailboat GC [m].
Even with a simple and rapid observation of the plot in the figure above, it is clear what
has already been mentioned regarding the various types of ways and the consequent
behaviours assumed by the sailboat. It is evident that in close-hauled ways, since the boat
reaches higher speeds and the wind direction comes from the front, forces are generated
on the sails that tend to unbalance the sailboat more than in the running-port tack that
receiving the wind posteriorly does not subject the boat to a high heeling moment.
Consequently, the distance of the sailor from the centre of gravity of the boat is correct to
be greater in the close-hauled, for angles between 40° - 60°, while much lower, tending to
90
zero in some situations, in the case of running-port tack. But it is necessary to reiterate that
the running-port tack are ways that combine in many cases with the use of the gennaker, to
allow the sailboat to reach high speeds, however, as previously expressed it is appropriate
to take into consideration, when evaluating the results obtained, which are measured
values in exceptionally low wind conditions. Therefore, in order to analyse in greater
detail, the distances that are necessary for the running-port way, it is extremely necessary
to resort to further acquisitions, preferably in more windy conditions. To further reiterate
the partiality of these results, it should be noted that the almost completely missing ways
are beam-reach, which being performed with angles between the boat and the wind
between 80° - 110°, are the ones that subject the sailboat most to heeling moment, so we
expect excursion values at least equal to those for close-hauled ways. What has been said
does not mean that the results obtained are not valid, but that they are valid together with
these conditions, which however are not the only ones that can be found in reality.
Plot measured distance between sailors and sailboat centre of gravity excluding
sudden changes of sailboat yaw direction. Considering some possible sudden and
unexpected changes of direction to which the boat may be subjected during navigation,
generating an oscillation of the yaw direction, one could erroneously measure the values
of distances that the sailor assumes with respect to the centre of gravity of the boat, which
however are not due to balancing movements of the heeling moment, but to movements on
the boat to correct the direction of navigation. Therefore, for this purpose, even when
measuring the distance, the points that did not respect the additional constraint of the
tolerance on the yaw direction, equal to 45°, were excluded, as shown in the following
graph.
91
Figure 4.15 Polar plot: measured distance between sailors and sailboat GC with a check
on sailboat yaw direction [m].
From the observation of the plot in the figure above, we want to highlight the presence of
only zero distance values between the sailor and the centre of gravity of the boat at beta
angles greater than 90°, demonstrating that the non-null values, previously measured and
shown in figure 4.14, for that range of angles they are meaningless, since for low wind
intensities, the position of both sailors during running-port tack is necessarily on board,
given the absence of heeling moment. So, some of the distances measured were the result
of an error due to a displacement of the sailor for the performance of indispensable
manoeuvres to correct the direction of navigation and not a real excursion to balance a
wind action. It can therefore be reasonably concluded that at least the values measured
during the test phase, after excluding the values distorted by sudden movements of the
boat or sailors, faithfully reflect what happens during the navigation phase. This was not at
all certain because, as mentioned in the descriptive phase of the test in section 4.1.2, in
measuring this distance there was a high risk of acquisition error risk, given by the
experimental observation of the phenomenon.
Plot differences of the estimate distance between sailors and sailboat centre of
gravity among experimental value and VPP’s predicted. After having ascertained that
a large part of the measured GC sailor-sailboat distance values, limited to the test
92
conditions, is reasonable and in line with what could be expected for the ways realized, it
is considered appropriate and necessary to analyse graphically what is reported in table 4.3
comparison with VPP's predicted values, with the intent to find out which are the most
critical points to validate what is produced by the simulated model in question. In this
regard, the following polar plot can be observed.
Figure 4.16 Polar plot: comparison experimental and VPP’s predicted distance among
sailors and sailboat GC.
By carefully examining the position of the blue and black markers, which respectively
represent the measured and predicted distances, paying as much attention to the size of the
error bands, red, which identify how much the two results obtained differ and shown in the
figure above and considering that the length of the deck available for both sailors to move
is about 1 meter in width and two in length, an average deviation of about ten centimetres
(for more detailed values see table 4.3) is almost an irrelevant error and also the result of
the choice of sailors during sailing.
Comparison linear plot among distance sailor-sailboat GC. To enhance the differences
in the various points analysed, a linear plot is used which highlights the presence of points
where the difference is even zero, which will subsequently be further investigated to find
out how many of these are actually true values and how many instead they report a null
error because they are null themselves.
93
Figure 4.17 Linear plot: comparison experimental and VPP’s predicted distance among
sailors and sailboat GC [m].
As for the linear plot in figure 4.17, we want to point out the accuracy of the result in the
case of ways with beta angles of about 40°, but also overall, with reference to all close-
hauled sailing, an average value is obtained VPP's predicted distance next to the
experimental one. However, what would need to be investigated further is that a lower
value is obtained than that measured experimentally. Therefore, it is considered useful to
verify through future tests whether the VPP actually produces a non-conservative value
with respect to reality or if an experimental acquisition error is present.
Comparison plot sailors-sailboat GC distance among experimental value and VPP’s

predicted with a check on sailboat yaw direction. Perspective to investigate the points
mentioned above that have zero error, the plot of the only values characterized by a yaw
variation of less than 45° is shown. In this case we want to pay attention mainly to the
values of down-wind sailing, which, due to what has been expressed several times
regarding the bad wind conditions that were available on the test day, are not reliable
values.
94
Figure 4.18 Polar plot: comparison experimental and VPP’s predicted distance among
sailors and sailboat GC with a check on sailboat yaw direction.
From figure 4.18 it is easy to understand that the doubt about the validity of some points
acquired during running-port tack has been partially confirmed. In fact, almost all of them
have been excluded from the additional added constraint. However, from this polar plot it
is not clear what the values are, among those that had zero error, i.e. among those that did
not differ from the aforementioned VPP's values. It is not accidental that we refer to the
predicted values such as those taken as the basis for the comparison, unlike what has been
done so far. In fact, from a conceptual point of view it would be correct to consider those
acquired experimentally as valid and reliable, but in our case this is not possible and
therefore the values predicted by the simulated model are used to understand if and where
there could be an error in the experimental case, this will make it possible to improve this
phase in the future.
Comparison linear plot sailors-sailboat GC distance among experimental value and

VPP’s predicted. To identify which values have been saved from the further added
constraint, and therefore increase their accuracy, refer to the following plot.
95
Figure 4.19 Linear plot: comparison experimental and VPP’s predicted distance among
sailors and sailboat GC with a check on sailboat yaw direction.
In practice, the surviving null error points, if they can be defined as such, are only a few in
upwind sailing condition and the already mentioned points in down-wind sailing, which,
however, at the time of this validation, are not considered reliable. To evaluate the
consistency of the remaining results, one can resort to observing the variation in standard
deviation, which indicates the dispersion of this distribution of values before and after
checking on the yaw. If this were very different between the two cases it would mean that,
albeit few, the values left, net of the obvious possible acquisition errors, due to various
factors already listed, are much more accurate than the previous ones, on the contrary if it
were similar in both the cases then would mean that such control is superfluous because
the values on average are already sufficiently accurate. Taking the standard deviation
equation:
∑n
2
𝜎=√ (4.25)
𝑁
96
following results:
𝜎𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 = 24.52 [cm] (4.26)
𝜎𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒𝑦𝑎𝑤 = 27.16 [cm] (4.27)
Equation 4.26 represents the standard deviation value for the case without constraint on
the yaw direction, while equation 4.27 represents the other case. Contrary to what
happened for the speed of the boat, it can be seen that the removal of some points that did
not respect the aforementioned constraint resulted in the removal of sailor-sailboat GC
distance values that differed less from the rest of the population in question, this it is
congruous with the removal of many null values, but not very accurate and reliable, to
support this evaluation. However, the difference between the two, even in this case, is
minimal. Therefore, not being able to say whether the error generated is the result of a
problem in the VPP or if more likely it is due to a low population of data for certain ranges
of angles, it can nevertheless be stated, given the small difference in dispersion of the
population between the two cases, that a check on the yaw is not essential in the
evaluation of the error, but has proved useful in support of the hypotheses previously
formulated, on where to search for this error in the future.
A statistical indicator of relative dispersion is also assessed for the distance sailor-sailboat
GC, in order to indicate the variability in percentage terms, to understand once again if
there are excessively variable distances that conceal an error. This dispersion coefficient is
calculated as the ratio between the standard deviation and the average of the velocity
distribution in the various instants.
𝜎
𝐶. 𝑉. = 𝜇
(4.28)
𝐶. 𝑉. 𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 = 87% (4.29)
𝐶. 𝑉.𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒𝑦𝑎𝑤 = 89 % (4.30)
yaw direction, while equation 4.30 represents the other cases. A coefficient of variation
has been obtained close to the unit that denotes a much wider variability of results
97
compared to that obtained for wind and boat speeds. This makes sense because the
distance varies greatly according to the type of ways travelled, therefore it can be said that
an assessment of variability, as done for the speed of the boat, for this type of data does
not have much meaning. It is more useful to evaluate only the variability of the error that
is generated between VPP and experimental output.
Bar graph: rate error among VPP’s predicted sailor-sailboat GC and experimental
measured one. Also for the measurement of the distance necessary to balance the heeling
moment, we want to show the error rate that is generated between the simulated distance
and the one measured experimentally, in reference to the maximum excursion that the
sailors can achieve, that is about 2 m. In summary, this is the graphic representation of the
error shown in table 4.3.
Figure 4.20 Error bar among VPP’s sailor-sailboat GC and experimental measured one
at given beta angle
From the bar graph shown above, we note what has already been said for the error bar
regarding the speed of the boat, i.e. that there are many error bars with a similar value as
regards angles corresponding to upwind ways, while for down-wind ways there are some
anomalous percentages of error of speed, very small or tending to zero, so for these values
98
there is a need to investigate further before reaching conclusions. While for the upwind
ways’ values, we can confirm what has been said for the speed, that the error is almost
constant and therefore it will be easier to consider it in order to try to remove it in the
future. Furthermore, in the case of distance, this is a small average error, 15% in upwind
conditions.
One way to investigate and try to understand, if and which values in down-wind ways are
to be discarded, can be to add the aforementioned yaw direction check, so as to see more
clearly what the differences are compared to the previous graph.
at given beta angle with yaw direction check.
The error bars, unlike the speed obtained, are now less homogeneous with each other, not
resulting more constant in upwind and cancelling completely in down-wind. So in this
case there is certainly no acceptable value in the down-wind ways and there are more valid
values in upwind ways, however dirtied by an error bar of double length compared to the
others, which however is negligible in considering an average error because balanced by
an error bar of much shorter length than the others. Therefore, the average error for
upwind conditions is just over 10% compared to the maximum possible excursion. It can
be said that surely the down-wind values will have to be measured in future acquisitions,
99
but the error rate in upwind conditions guarantees a good first feedback for the distance
that the sailors have to keep with respect to the centre of gravity of the boat.
As mentioned about boat speed, we intend to evaluate the relative dispersion index of the
error between VPP and experimental results for the sailor- sailboat GC distance. The
coefficient of variation is calculated once again, with reference to the average error value
at each instant taken into consideration:
𝜎
𝐶. 𝑉. = (4.31)
𝜇
To calculate it, first evaluate the standard deviation:
𝜎𝐸𝑅𝑅𝑂𝑅 𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 = 17 [cm] (4.32)
𝜎𝐸𝑅𝑅𝑂𝑅 𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒𝑦𝑎𝑤 = 20 [cm] (4.33)
The average error is:

𝜇𝐸𝑅𝑅𝑂𝑅 𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 = 17 [cm] (4.34)
𝜇𝐸𝑅𝑅𝑂𝑅 𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒𝑦𝑎𝑤 = 18.5 [cm] (4.35)
𝐶. 𝑉. 𝐸𝑅𝑅𝑂𝑅 𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 = 109 % (4.36)
𝐶. 𝑉.𝐸𝑅𝑅𝑂𝑅 𝐺𝐶𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒𝑦𝑎𝑤 = 97 % (4.37)
yaw direction, while equation 4.37 represents the other case. In calculating the distance
from the centre of gravity, a standard deviation and an average almost equal to each other
are obtained, this indicates that the values are averagely equal spaced by the hypothetical
average line by an amount equal to the value of the average, generating a coefficient of
variation approximately equal to one. This means that we are in conditions of wide data
variability. In fact, there are values that generate an almost zero error compared to the
100
VPP and others that instead generate a much larger error. We need to investigate further
with future acquisitions what is the actual value of the error.
4.2.5 ERROR TREND AS THE NUMBER OF AVAILABLE

POPULATION CHANGES
Assuming to decrease the restrictive constraints on the wind speed, with an increase in the
range of values within which the wind can be considered of constant intensity, I obtain an
increase in the number of instants that fall within the imposed conditions. However, in
some cases these are points affected by a greater error since increasing the tolerance on the
wind speed will have a greater variability of the same. At the same time, the probability of
finding good values also increases because the sample population increases. To analyse
the effect of this change imposed during the post processing of the data, first of all the
Polar Plot of wind speed and its direction among axis origin is shown under the above
conditions.
Figure 4.22 Polar Plot of wind speed experimental value [m/s] at beta angle [°] with a
speed tolerance range of 1 [m/s]
It is clear from the plot above that the wind intensity is more variable than in the cases
analysed previously, while the data population is as in the previous cases missing for some
101
wind directions, such as for beta angles between 90° - 120° (remember that from 0° - 30°
you are in the dead corner where it is not possible to navigate) which therefore shows us
once again that, even by decreasing the impositions during the post processing phase, the
set of values acquired it is small and not very wide. In any case, in order to further
investigate the nature of the boat speed error produced between experimental and
simulated values, the graph showing the percentage trend of the error is shown below.
at given beta angle with yaw direction check.
What you can guess at first glance is a very wide oscillation of the error values, which
increases further for a set of values between 120° - 160°, the probable cause of which is
the low amount of data acquired in those conditions. What we want to analyse is how the
extent of this error has changed now that the population of points has increased. By a
simple calculation of the average, it is obtained that the error average in this case is equal
to:
𝑎𝑣𝑎𝑟𝑎𝑔𝑒 𝑏𝑜𝑎𝑡 𝑠𝑝𝑒𝑒𝑑 𝑒𝑟𝑟𝑜𝑟 (𝑤𝑖𝑛𝑑 𝑠𝑝𝑒𝑒𝑑 𝑟𝑎𝑛𝑔𝑒: 1 𝑚/𝑠) = 28% (4.38)
102
Therefore, it is decreased compared to the values previously found for the points subjected
to further constraints on the wind and on the yaw direction, which are shown below:
𝑎𝑣𝑎𝑟𝑎𝑔𝑒 𝑏𝑜𝑎𝑡 𝑠𝑝𝑒𝑒𝑑 𝑒𝑟𝑟𝑜𝑟 (𝑤𝑖𝑛𝑑 𝑠𝑝𝑒𝑒𝑑 𝑟𝑎𝑛𝑔𝑒: 0.5 𝑚/𝑠) = 32% (4.39)
𝑎𝑣𝑎𝑟𝑎𝑔𝑒 𝑏𝑜𝑎𝑡 𝑠𝑝𝑒𝑒𝑑 𝑒𝑟𝑟𝑜𝑟 (𝑐ℎ𝑒𝑐𝑘 𝑜𝑛 𝑦𝑎𝑤 𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛) = 35% (4.40)
As can be seen from the averages shown, as the sample population increases, the error
tends to decrease. This makes us understand that having a very wide and varied range of
values acquired, in wind conditions that are not excessively variable or at least more
windy, the error that will be obtained can be considerably less. These last considerations
allow us to conclude that numerous acquisition phases will certainly be necessary, but that
there are all the prerequisites for validating the simulated model.
Now we want to evaluate how the variability of the data around the average varies with
the increase of the number of the population taken into consideration in the post
processing phase, at the expense of accuracy, which evidently decreases if values that
have a more variable wind speed are considered acceptable. For this purpose, the standard
deviation and the C.V. for the wind speed and then also for the speed of the resulting boat.
The equation that represents the value of the standard deviation is expressed below:
∑n
2
𝜎=√ (4.41)
𝑁
following results:
m
𝜎𝑤𝑖𝑛𝑑 = 0. 28 [ s ] (4.42)
m
𝜎𝑏𝑜𝑎𝑡 𝑠𝑝𝑒𝑒𝑑 = 0.5 [ s ] (4.43)
Remembering that the coefficient of variance is a function of the ratio between standard
deviation and mean, considering the mean μ:
103
∑ni=1 𝑥𝑖
𝜇=
𝑛
(4.43)
So, the C.V. is defined as follows:
𝜎
𝐶. 𝑉. = (4.44)
𝜇
The calculation of these parameters provides the following results:
𝐶. 𝑉. 𝑤𝑖𝑛𝑑 = 17.5 % (4.45)
𝐶. 𝑉.𝑏𝑜𝑎𝑡 𝑠𝑝𝑒𝑒𝑑 = 31% (4.46)
Equation 4.45 represents the C.V. for the wind speed, which, compared to C.V. calculated
previously, it increases by about 5%, as was logical to expect, having reduced the
restrictions on wind variability. As for the speed of the boat, the C.V. remains unchanged
and this result indicates to us that a small variability of the wind does not strongly
influence the variability of the speed, which is therefore mainly influenced by the
direction that the boat travels with respect to the wind
To analyse how the error varies in this case between the speed provides by VPP and the
experimental one, the C.V. of the error on the speed, taking into account both the error
average and the standard deviation from the average. The calculation of these parameters
provides the following results:
𝐶. 𝑉. 𝐸𝑅𝑅𝑂𝑅 𝐵𝑜𝑎𝑡 𝑠𝑝𝑒𝑒𝑑 (𝑤𝑖𝑛𝑑 𝑠𝑝𝑒𝑒𝑑 𝑟𝑎𝑛𝑔𝑒: 1 𝑚/𝑠) = 66 % (4.47)
From equation 4.47 it is obtained that the variability of the error increases if the variability
of the wind speed increases, going from 52% in the case of equation 4.23 to 66% in this
case. Furthermore, in the case of control on the yaw direction, the C.V., as shown with
equation 4.24, was further reduced to 43%. It is therefore evident that the accuracy of the
starting data is fundamental to obtain a constant error as possible, but at the same time a
lower average error is obtained, therefore it is considered essential to increase the set of
available data, maintaining a high accuracy compared to the conditions foreseen in the
VPP design phase, so as to have a much lower error and at the same time constant.
104
4.2.5 OVERVIEW OF THE ASSESSMENTS ABOUT THE RESULTS
In this chapter the experimental data acquired and processed in the manner described in
the previous chapter were analysed and commented, as regards the speed and direction of
the wind that blows in the moments in which the maximum speed of the boat is reached,
finally the distance that sailors must maintain from the sailboat gravity centre, necessary,
instant by instant, to balance heeling moment, with the aim of establishing with what
precision they are faithfully reproduced by the VPP. In order to define the extent of the
accuracy of the analysed data, the error created between the simulated value and the one
acquired experimentally was taken into consideration for each VPP design variable. It is
appropriate to study the results of this analysis taking into account the problems
encountered in this first data acquisition experience, both for lack of previous experience
and for the test conditions found, far from ideal or suitable for the purpose of an
acquisition as much as possible complete, such as to provide wide availability of data, in
the sense of having wind values high enough to allow you to perform as many types of
sailing as possible, both with upwind and with down-wind. However, this was not the case
and therefore only the gaits that could be carried out were analysed and what emerged is
the result of the above. It is therefore not possible to state for all the ranges of values that
the VPP reproduces reality correctly, because it was not possible to have reliable feedback
values for each beta angle corresponding to a specific wind speed.
From the comparison carried out with the points representing certain sailing referring to
certain values of beta angles, which in turn are specific for a single wind speed sufficiently
high, an average error percentage value of 30% was found for the calculation of the
maximum boat speed , while for the sailor-sailboat distance GC we are in the order of
10%. However, these values will have to be corrected in the future by acquiring further
data in order to also evaluate beam-reach and down-wind speeds with a sufficient number
of values to confirm whether these error entities are constant or if they vary between the
various types of sailing. This would be useful to know the possible error offset and to be
able to correct it. While, as regards the upwind sailing, during this first acquisition it was
possible to acquire a greater number of instants, therefore it can be stated with sufficient
precision, that between experimental and simulated values, the error present is about 35%
for the calculation the speed of the boat, while 13% for the sailor-sailboat GC distance, but
also in this case it is not possible to verify whether the error is constant as the external test
conditions vary, so refer to this evaluation for further analysis to following future data
acquisitions.
Finally, the trend of the percentage error was shown, for the maximum boat speed,
between the measured and simulated values, this showed that the error tends to decrease as
the sample population increases, while remaining high since it was a population of data
with a lower accuracy due to an amplitude greater than the range of wind speeds
considered constant. So, although less accurate data was considered, the average error rate
decreased by a few percentage points simply by increasing the number of data population
105
to draw from. What has been obtained confirms the hypotheses previously formulated
regarding the need to make further data acquisitions that could demonstrate that the error
component due to the approximations of the model is not consistencies as the results of
this first test could believe. However, it is also advisable not to leave any intentions
possible to improve the VPP code, which due to the numerous approximations that have
been applied to it during the design phase, it is reasonable to think that it will carry a fixed
error within itself. The main approximations that are recommended to be removed concern
the rigid model of the sails and the relative lift and drag coefficients and the hydrodynamic
model of the hull which should be tested by towing tank test, in order to take into
consideration the appropriate viscous and induced components through appropriate
coefficients drag over residuary resistance and added resistance in waves.
106
5
CONCLUSIONS AND
RECOMMENDATIONS
5.1 CONCLUSIONS
The present work has been initiated to determine the accuracy of velocity prediction
program of R3 class skiff sailboat designed by the sailing team of Politecnico di Torino.
The quantification of the degree of simulation accuracy of validation variables at a
specified validation point for cases in which the actual conditions of the experiment are
simulated. To this end, first the target and output of the VPP was described, i.e. a
performance diagram (polar plot) that states the boats optimal (target) speed as a function
of the sailing conditions, under the constraint that the heeling moment may not exceed the
maximum available righting moment given by the sailors.
Secondly, the several methods provided by the literature, and recommended in the ASME
guide for verification and validation of a program, have been studied and described. Code
verification involves error evaluation from a known benchmark solution. Solution
verification involves error estimation, since the exact solution to the specific problem is
unknown.
The objective of verification is to establish numerical accuracy, independent of the

physical modeling accuracy that is the subject of validation, i.e. verification is the process
of evaluating software to determine whether the products of a given development phase
satisfy the conditions imposed at the start of that phase. In contrast, in the validation
process, a simulation result (solution) is compared with an experimental result (data) for
specified validation variables at a specified set of conditions (validation point). The
validation comparison error E is thus the combination of all of the errors in the simulation
result and the experimental result, and its sign and magnitude are known once the
validation comparison is made. We will denote the predicted value from the simulation
solution as S, the value determined from experimental data as D, and the true (but
unknown) value as T.
107

• the error δnum due to the numerical solution of the equations. However, since the
equations are solved by MATLAB calculation software, this error is an
infinitesimal value and therefore negligible;
• the error δinput due to errors in the simulation input parameters and assuming that
the equations have been written correctly, it is sensible and lawful to consider this
error null.
The last source of error is to be input to the experimental model:
• δD due to errors in the experimental input parameters caused by acquisition errors

attributable to the characteristics of the sensors, i.e. the datasheets for each sensor
used are listed.
Note that once D and S have been determined, their values are always different by the
same amount from the true value T. That is, all errors affecting D and S have become
“fossilized” and δD, δinput, δnum, and δmodel are all systematic errors.
The estimation of a range within which the simulation modelling error lies is a primary
objective of the validation process and is accomplished by comparing a simulation result
(solution) with an appropriate experimental result (data) for specified validation variables
at a specified set of conditions.
There can be no validation without experimental data with which to compare the result of
the simulation. Validation process must be preceded by code verification and solution
verification. In the case described, the code verification phase was not carried out, since
the code was previously checked during the design phase, therefore only a verification
solution was carried out, however there is a description of the methods indicated by
ASME in order to carry out the code verification if it is necessary for a possible future
validation of a different program.
The concern of a V&V is the specification of a verification and validation approach that
quantifies the degree of accuracy inferred from the comparison of solution and
experimental data for a specified variable at a specified validation point. The approach
uses the concepts from experimental uncertainty analysis to consider the errors and
uncertainties in both the solution and the data.
Next step was trying to acquire enough sensor data through experimental tests in real
navigation conditions, to allow a comparison between the polar plot obtained by
numerically solution and the one obtained with measured data to determine how close is
108
the simulation output with the experimental output. For this purpose, validation procedure
consists of an assessment of the errors, associated with physical model’s assumptions,
input data and numerical solutions. These errors will be compared with the experimental
numerical results and with possible errors associated. The difference between the two
models will confirm or not the consistency of the theoretical VPP’s model.
In order to obtain experimental comparison value, the data acquisition phase was carried
out on the Atka skiff of the Polito Sailing Team (PST). For the first time on this sailboat
sensors have been installed to allow data acquisition. The sensors have been developed
from the PST sensor area, with the aim of acquiring data regarding:
• Boat speed using GPS acquisition system;

• Wind speed and direction;
• Roll, pitch and yaw of the hull.
The acquisition system was installed inside a waterproof box, positioned near the gravity
centre of the boat, after a verification of the correct installation, we proceeded with a
calibration of the system, to compensate for the lack of accuracy due to systematic errors
present.
To consider the VPP’s constraint that the heeling moment may not exceed the maximum
available righting moment given by weight of the sailors, it was calculated the distance
from the sailboat centre of gravity and that one of the two sailors in the maximum speed
regime. This aims to assess the value of the heeling moment generated by the wind and
balanced by the weight of the sailors. To achieve this data, sensors were not used, two
retractable spools of wire were provided, hinged on one side to the centre of gravity of the
boat and on the other to the life jackets of the sailors. The distance has been calculated
viewing the videos made through GoPro Hero Session action-cam, installed on the mast,
measuring the variation in length of the wire. Through appropriate geometric calculations
the distance was obtained, were subsequently referred to a single equivalent sailor as did
in VPP, thus making a further small approximation so as to make the experimental and
simulated results comparable. However, this approximation is not preponderant in the
creation of the final error, since the weight of the two sailors taken into consideration is
comparable and moreover the distance between the two on board during sailing is
negligible.
The simulation model also has the assumption of a small heeling angle, because it is
assumed that the sailors can maintain the yacht in its best condition at zero heeling angle.
The acquired data is contaminated by moments in which the boat is stopped or by

moments in which the wind conditions do not fall within the range of values for which the
VPP is developed. In this context, first we proceed by excluding the data in the conditions
109
of a stopped boat and then excluding the surpluses according to a normal distribution with
an accuracy of 95.45% (bell curve).
Then, as in the VPP, an iterative search of the maximum speed values is performed, only
in the moments in which the wind speed is in a constant condition too. Even so, the
steady-state way conditions envisaged in the VPP’s design phase, i.e. presence of constant
wind for a sufficient time interval, such as to allow the boat to perform all the ways from
0° to 180° beta angle against wind direction, have not been found in reality. It is
reasonable considering that the data acquisition was carried out in a lake in a day not very
windy, thus influencing very much the speed reached by the sailboat in each different
instant analysed.
In order to evaluate the accuracy of the outputs, the error created between the simulated
value and the one acquired experimentally has been considered for each VPP design
variable. It is appropriate to study the results of this analysis taking into account the
problems encountered in this first data acquisition experience, both for lack of previous
experience and for the test conditions found, far from ideal or suitable for the purpose of
an acquisition as much as possible complete, such as to provide wide availability of data,
in the sense of having wind values high enough to allow you to perform as many types of
sailing as possible, both with upwind and down-wind.
Therefore, it is not possible to affirm for all the ranges of values that the VPP reproduces
reality correctly, because, due to the lack of data, it was not possible to have reliable
feedback values for each beta angle corresponding to a specific wind speed. From the
comparison carried out with the points representing certain sailing referring to certain
values of beta angles, which in turn are specific for a single wind speed sufficiently high,
an average error percentage value of 30% was found for the calculation of the maximum
boat speed, while for the sailor-sailboat distance GC we are in the order of 10%. While, as
regards the upwind sailing, during this first acquisition it was possible to acquire a greater
number of instants, therefore it can be stated with sufficient precision, that between
experimental and simulated values, the error present is about 35% for the calculation the
speed of the boat, while 13% for the sailor-sailboat GC distance, but also in this case it is
not possible to verify whether the error is constant as the external test conditions vary.
However, these values will have to be corrected in the future by acquiring further data in
order to also evaluate beam-reach and down-wind speeds with a sufficient number of
values to confirm whether these error entities are constant or if they vary between the
various types of sailing. For a measured variable, the total error is caused by multiple error
sources. The sum of these errors is the difference between the value of the measurement
determined in the experiment and the true value of the variable.
In experimental program, corrections to the measurements are made for those errors that
are known, as in the calibration process. For those errors where the magnitude and sign are
110
unknown, uncertainty estimates are made to represent the dispersion of possible values for
the errors. We use the standard deviation to calculate the uncertainty in the measured
variable. The uncertainties from error sources that contribute to the variability of the
measurement are classified as random and the uncertainties from error sources that remain
fixed during the measurement process are classified as systematic.
Confirm if the error entities are constant would be useful to know the possible error offset
and to be able to correct it and so refer to this assessment for subsequent analyses
following future data acquisitions.
5.2 RECOMMENDATIONS
From the problems encountered and insights gained during this thesis, the following
recommendations will be useful to improve the validation of this VPP or to validate
another different one and its associated topics:
• To achieve an ideal V&V, when it starts, the validation variables should be chosen
and defined with care. Each measured variable refers to certain conditions, and the
experimental result, that is determined from these measured variables, should be
compared with a predicted result that possesses the same characteristics. If this is
not done, such conceptual errors must be identified and corrected, or estimated in
the initial stages of a V&V effort, or substantial resources can be wasted, and the
entire effort compromised. If uncertainty contributions are considered that
examine all of the error sources in δnum, δinput and δD, then, δmodel includes only
errors arising from modeling assumptions and approximations. In practice, there
are numerous gradations that can exist in the choices of which error sources are
accounted for in δinput and which error sources are defined as an inherent part of
δmodel. The code used will often have more adjustable parameters or data inputs
than the analyst may decide to use. The decision of which parameters to include in
the definition of the computational model is somewhat arbitrary. The point here is
that the computational model which is being assessed consists of the code and a
selected number of simulation inputs which are considered part of the model. It is
crucial in interpreting the results of a validation effort that which error sources are
included in δmodel and which are accounted for in the estimation of validation
uncertainties be defined precisely and unambiguously.
• To improve this validation effort, first it will be appropriate to study a method to

be able to acquire GPS angle with respect to the North position in order to
evaluate the leeway angle of the sailboat and it would also be useful to implement
an image processing system of the acquired video files, using a triangulation of
several cameras, in order to prevent the software from making a mistake in
tracking the distance in order to reduce the measurement error of the sailor-
111
sailboat GC distance. Only following at least one of these improvements can new
data acquisition phases be carried out, in order to have a wide and varied
availability of experimental data. Finally, it might be interesting to evaluate the
possibility of performing a towing tank test to carry out hydrodynamic studies on
the hull, so as to evaluate the veracity of the VPP’s predicted results for Cl and Cd
parameters of the hull, and secondly it could be considered wind tunnel test to
evaluate the Cl and Cd parameters of the sails.
112
REFERENCES
[1] American Society for Mechanical Engineers (ASME). Guide for verification and
validation in computational solid mechanics. ASME V&V 10–2006, ASME, New
York, 2006.
[2] O. L. de Weck. Fundamentals of Systems Engineering, Verification and

Validation, Session 9. 2015. MIT OpenCourseWarehttp://ocw.mit.edu.
[3] F. Beltrame et all. Development of a Velocity Prediction Program for a high

performance eco sustainable SKIFF sailing yacht. Department of Mechanical and
Aerospace Engineering, Politecnico di Torino, Italy.
[4] B. W Boehm. Verifying and Validating Software Requirements and Design

Specifications. IEEE Software, 1(1), 75–88. (1984). doi:10.1109/ms.1984.233702
[5] D. R. Wallace, L. M. Ippolito, B. B. Cuthill. Reference Information for the

Software Verification and Validation Process. Information Systems Architecture
Division of National Institute of Standards and Technology. Gaithersburg, April
1996.
[6] D.R. Wallace and R. U. Fujii. (1989). Software verification and validation: an
overview. IEEE Software, 6(3), 10–17. doi:10.1109/52.28119
[7] American institute of aeronautics and astronautics (AIAA). Guide for verification
and validation of computational fluid dynamics simulations. AIAA G-077—1998,
AIAA, New York, 1998.
[8] H. W. Coleman, and F., Stern, F. Uncertainties and CFD code validation. Journal
of Fluids Engineering, Vol. 119, Dec. 1997, pp. 795–803.
[9] International Organization for Standardization (ISO). Guide to the expression of

uncertainty in measurement (corrected and reprinted 1995), ISO Geneva, 1993.
[10] M. Rouaud, Probability, Statistics and Estimation: Propagation of Uncertainties

in Experimental Measurement, (PDF), 2013.
[11] American society for mechanical engineers (ASME). Test uncertainty, ASME
PTC 19. 1–2005, ASME, New York, 2006.
113
[12] American institute of aeronautics and astronautics (AIAA). Guide for verification
and validation of computational fluid dynamics simulations. AIAA G-077—1998,
AIAA, New York, 1998.
[13] American society of mechanical engineers (ASME), Standard for verification and
validation in computational fluid dynamics and heat transfer, V&V20—2009
ASME, New York, 2009.
[14] H. W. Coleman, w. G. Steele. Engineering application of experimental

uncertainty analysis. John Wiley & sons, New Jersey,1995.
[15] P. J. Roache. Verification and validation in computational science and

engineering. Hermosa Publishers, Albuquerque, NM, 1998.
[16] H. W. Coleman, w. G. Steele. Experimentation, validation, and uncertainty

analysis for engineers. John Wiley & sons, inc. 3rd ed. Hoboken, New Jersey,2009.
[17] https://learn.adafruit.com/adafruit-ultimate-gps.
[18] https://www.adafruit.com/product/1733
[19] https://learn.adafruit.com/adafruit-bno055-absolute-orientation-sensor.
[20] https://www.mouser.com/datasheet/2/389/lsm303agr-954987.pdf
[21] S. Stahl. The Evolution of the Normal Distribution. Department of Mathematics

University of Kansas Lawrence, KS 66045, USA.
[22] American society for mechanical engineers (ASME). Test uncertainty, ASME
PTC 19. 1–2005, ASME, New York, 2006.
[23] IEEE. Standard for System, Software, and Hardware Verification and Validation,
in IEEE Std 1012-2016 (Revision of IEEE Std 1012-2012/ Incorporates IEEE Std
1012-2016/Cor1-2017), vol., no., pp.1-260, 29 Sept. 2017.
[24] C. Boehm. A Velocity Prediction Procedure for Sailing Yachts with a

hydrodynamic Model based on integrated fully coupled RANSE-Free-Surface
Simulations. Yacht Research Unit Kiel, https://doi.org/10.4233/uuid:27c98a96-
8b2e-4797-a9ea-76f5f5cbab48.
114
[25] N. Saccenti et all. Development of a Multi-Purpose Velocity Prediction Program
for Sailing Yachts. Yacht and Superyacht research group of Newcastle University.
2013. Massachusetts Institute of Technology. 2015
[26] S.S Knudsen, K. Wedersoe et all. Velocity Prediction of a Sailing Yacht Report.
Technical University of Denmark, MEK Institute. Juli 25, 2008.
[27] R. G. Sargent. Verification and Validation of Simulation Models. Syracuse

University, Department of Electrical Engineering and Computer Science, L. C.
Smith College of Engineering and Computer Science. Syracuse, NY 13244, U.S.A.
2011.
115
SUMMARY
The present study describes the verification methods and accuracy assessment procedures,
provided by ASME guide, of sailing yacht performance prediction program useful in order
to the develop better and more efficient racing sailboat. To this end the numerical results
of the Velocity Prediction Program (VPP) are compared with the feedback provided by the
instruments used for the experimental data acquisition, which come from the sensors
installed on a skiff designed and built by Polito Sailing Team. The goal is to determine
how close is the simulation output with the real one. The main output of a VPP is a
performance diagram (polar plot) that states the boats optimal target speed as a function of
the sailing conditions. From this it is possible to evaluate the best sailing condition for a
given design and it is also possible to give the sailor an advantage in knowing the strength
and weaknesses of the boat.
Validation procedure consists of an assessment of the simulation errors, associated with
physical model’s assumptions, input data and numerical solutions and then compared with
the experimental results and with possible errors associated. The difference between the
two models will confirm or not the consistency of the theoretical VPP’s model. The
estimation of a range within which the simulation modelling error lies, for specified
validation variables at a specified set of conditions, is a primary objective of the validation
process. There can be no validation without experimental data with which to compare the
result of the simulation. The data are processed by calculation software to obtain outputs
comparable with the values simulated by the VPP, where were excluded the values that
deviate excessively from the average of the data population and those that were acquired
in conditions other than those foreseen in the VPP's design phase, i.e. sailboat at top speed
regime with constant wind. To consider the VPP’s constraint that the heeling moment
may not exceed the maximum available righting moment given by weight of the sailors, it
was calculated the distance from the sailboat centre of gravity and that one of the two
sailors in the maximum speed regime, which is another VPP output. This aims to assess
the value of the heeling moment generated by the wind and balanced by the weight of the
sailors.
From the comparison carried out with the points representing certain sailing referring to
certain values of beta angles, which in turn are specific for a single wind speed, an average
error was found for each output variable and it is caused by multiple error sources. The
sum of these errors is the difference between the value of the measurement determined in
the experiment and the true value of the variable. The uncertainty of the results is
expressed as a standard deviation to establishing if the error entities are constant, so would
be useful to know the possible error offset and to be able to correct it. Even so, the steady-
state way conditions envisaged in the VPP’s design phase have not been found in reality.
Therefore, it is not possible to affirm for all the ranges of values that the VPP reproduces
reality correctly, because, due to the lack of accurate data, it was not possible to have
116
reliable feedback values. So, the accuracy of the results should be investigated and
improved in the future by acquiring further data in order to confirm, with a sufficient
number of values, whether these error entities are constant or if they vary between the
various sailing conditions.
117
ACKNOWLEDGEMENTS
"The fear of failing is always greater than the hope of succeeding, everyone is afraid, the
difference lies in the use you make of it." This is how this path started when a few months
ago I decided to go against any logical opinion and to want to continue this work.
The premises were not on my side, in the team I had only been a member for a few
months, few knew me well, and nobody had ever carried out an experimental validation
with sensors installed on the boat, moreover it had to be done within a few weeks, before
the arrival of winter. But some believed in me and in my enthusiasm to complete the work
I had set myself to do at the beginning of the experience with the team, starting with the
team leader, Domenico Castellano, who pushed first because the whole team support me,
professors Giovanni Bracco and Giuliana Mattiazzo who believed in the importance for
me and for the team of this work. This support came since the beginning from my head of
Dynamics, Lorenzo Liboà, he advised me in the work and endured from the first to the last
day the fears of failing and fears of making mistakes. but it is equally important to
remember that if the acquisition was successfully completed it is thanks to the Sensors
area of the team, in particular to the area manager, Mattia Glorioso, who took action from
the first day we met to talk about this project, he took on the commitment and
responsibility of completing all the necessary work on the sensors, from programming the
code to installing them on the boat, ready to provide technical support or advice during all
the months of work. Special thanks go to my wingman, the one who helped me daily in
the work, Giovanni Santacroce, my partner in the Dynamic area, who has worked with me
in these months of the validation effort, helping me to solve problems and difficulties, but
always with the smile, and I hope he will continue this work in the future. Last, but non
least, the two sailors, Alessandro Smerchinich and Alessio Giudice, the Shipbuilding
manager, Alessandro Lombardo, they were at the forefront of the first data acquisition
weekend, dedicating time and effort in preparing everything necessary. Finally, thanks to
all the team members who participated directly and indirectly in every single phase of this
work. In addition, a note of merit goes to those who have personally taken care of the
conception, design and implementation of the Velocity Prediction Program, all the
members of the Dynamic Polito Sailing Team area and all those who have been in the past
years, because a team evolves, the students move on, but what matters is what each of us
leaves to the team. I wanted to keep last, but surely they are the most important for
achieving this goal, all my family and friends who have always believed in me, regardless
of what you can achieve, always urging me to continue to putting effort and dedication
into what you do, never giving up, because happiness is not in the results you get, but in
looking back and admiring the path you have taken and specially with those who have
done it, and in this I was definitely lucky to always have my girl Elena next to me, even if
hundreds km away.
It is thanks to all of them if I managed to improve and grow up to be able to take that fear
of failure and turn it into satisfaction.
118

Tesi

Uploaded by

Copyright:

Available Formats

Tesi

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Tesi

Uploaded by

Copyright:

Available Formats

POLITECNICO DI TORINO

Corso di Laurea Magistrale in Ingegneria Meccanica

Master’s degree thesis

Experimental validation and accuracy assessment of

1.1 PROBLEM DEFINITION AND OBJECTIVES ............................................................................ 7

1.2 OVERVIEW OBJECT SAILING YACHT .................................................................................... 8

BACKGROUND OF VERIFICATION AND VALIDATION PROCEDURE........... 10

2.1 PREFACE ........................................................................................................................................ 10

2.2 OBJECTIVES OF V&V ................................................................................................................. 11

2.3 VALIDATION NOMENCLATURE AND APPROACH ............................................................ 16

2.4 CODE VERIFICATION AND SOLUTION VERIFICATION .................................................. 19

2.4.1 CODE VERIFICATION ............................................................................................................................. 20

2.4.2 SOLUTION VERIFICATION .................................................................................................................... 23

2.5 ESTIMATION OF SIMULATION UNCERTAINTY ................................................................. 28

2.5.1 SENSITIVITY COEFFICIENT (LOCAL) METHOD FOR A PARAMETER UNCERTAINTY

2.6 UNCERTAINITY OF AN EXPERIMENTAL RESULT ............................................................ 29

2.7 EVALUATION OF VALIDATION UNCERTAINITY ............................................................... 30

2.7.1 ESTIMATING VALIDATION UNCERTAINTY WHEN EXPERIMENTAL VALUE OF

2.7.2 ESTIMATING VALIDATION UNCERTAINTY WHEN EXPERIMENTAL VALUE OF VALIDATION

2.7.3 INTERPRETATION OF VALIDATION RESULTS ..................................................................................... 32

2.7.4 SOME PRATICAL POINTS ............................................................................................................................ 33

DATA ACQUISITION PROCEDURES.............................................................................. 35

3.1 OVERVIEW OF DATA ACQUISITION PREPARATION ....................................................... 35

3.1.1 SENSORS DATASHEET ............................................................................................................................ 49

3.2 GPS EXPERIMENTAL DATA ACQUIRED ............................................................................... 52

3.3 OVERVIEW OF VELOCITY PREDICTION PROGRAM ....................................................... 53

3.5 POST PROCESSING OF EXPERIMENTAL DATA .................................................................. 56

3.5.1 OVERVIEW OF DATA CLEANING PROCESS .......................................................................................... 59

4.1 EXPERIMENTAL DATA RESULTS ........................................................................................... 62

4.2 PLOTS EXPERIMENTAL SAILBOAT SPEED VALUE .......................................................... 72

4.2.1 POLAR PLOT OF EXPERIMENTAL SAILBOAT SPEED. .................................................................. 72

4.2.2 VELOCITY PLOT OF EXPERIMENTAL SAILBOAT SPEED DURING TIME ............................... 75

4.2.3 COMPARISON EXPERIMENTAL – VPP’S PREDICTED SAILBOAT SPEED ................................ 81

4.2.5 OVERVIEW OF THE ASSESSMENTS ABOUT THE RESULTS ....................................................... 105

CONCLUSIONS AND RECOMMENDATIONS............................................................107

5.1 CONCLUSIONS ............................................................................................................................ 107

1.1 PROBLEM DEFINITION AND OBJECTIVES

1.2 OVERVIEW OBJECT SAILING YACHT

Overall Overall Total sail

4.6 m 1.5 m 2.1 m 33 m2 110 kg

Table 1.1 Atka geometrical data. (Source: [3])

Figure 1.1 Atka sailing yacht CAD. (Source: [3])

• It uncovers high-risk errors early, giving the designer time to evolve a

2.2 OBJECTIVES OF V&V

• More commonly, it is a ‘description of systematic errors, a measure of statistical

Figure 2.1 Dataset Gauss Curve refers to true value.

Propagation of uncertainty (or propagation of error) is the effect of variables' uncertainties

Pr(𝜇 − 1𝜎 ≤ Х ≤ 𝜇 + 1𝜎) ≈ 0.6827

Pr(𝜇 − 2𝜎 ≤ Х ≤ 𝜇 + 2𝜎) ≈ 0.9545

Pr(𝜇 − 3𝜎 ≤ Х ≤ 𝜇 + 3𝜎) ≈ 0.9973

and similarly, the error in the experimental value D is

E = S − D = (T + δS) − (T + δD) = δS − δD (2.4)

• the error δmodel due to modeling assumptions and approximations;

E = δmodel + δnum + δinput − δD (2.6)

δmodel = E − (δnum + δinput − δD) (2.7)

Considering the relationship shown in Eq. (2.7),

2.4 CODE VERIFICATION AND SOLUTION

Eh = f(h) – fexact (2.9)