AIAA JOURNAL

Vol. 58, No. 11, November 2020

Airfoil Design Parameterization and Optimization

Using Bézier Generative Adversarial Networks

Wei Chen,∗ Kevin Chiu,† and Mark D. Fuge‡

University of Maryland, College Park, Maryland 20742
https://doi.org/10.2514/1.J059317
Global optimization of aerodynamic shapes usually requires a large number of expensive computational fluid
dynamics simulations because of the high dimensionality of the design space. One approach to combat this problem is
to reduce the design space dimension by obtaining a new representation. This requires a parametric function that
compactly and sufficiently describes useful variation in shapes. This paper proposes a deep generative model, Bézier-
GAN, to parameterize aerodynamic designs by learning from shape variations in an existing database. The resulted
new parameterization can accelerate design optimization convergence by improving the representation compactness
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

while maintaining sufficient representation capacity. The airfoil design is used as an example to demonstrate the idea
and analyze Bézier-GAN’s representation capacity and compactness. Results show that Bézier-GAN both 1) learns
smooth and realistic shape representations for a wide range of airfoils and 2) empirically accelerates optimization
convergence by at least two times compared with state-of-the-art parameterization methods.

Nomenclature blades. The bottleneck for most high-fidelity aerodynamic shape

CD = drag coefficient optimization lies in performing computational fluid dynamics (CFD)
CL = lift coefficient simulations given its high computational cost. To perform global
c = latent codes optimization, where we search the design space for global optima,
c
= optimal latent code the demand for CFD evaluations increases with the number of design
c† = near-optimal latent code solution to the first- variables (i.e., design space dimensionality). Although current state-of-
stage optimization the-art can leverage gradient-based techniques, such as adjoint meth-
D = discriminator ods, to compute shape derivatives and efficiently guide the search, such
d = latent dimension approaches are not always feasible, for example, in chaotic flows where
d0 = noise dimension adjoints may become numerically ill-conditioned [1] or for solvers that
G = generator do not calculate adjoints. In such cases, practitioners turn to gradient-
Ma = Mach number free methods. However, in gradient-free methods the need for CFD
n = Bézier degree evaluations can increase exponentially with the design space dimen-
P = control points of the rational Bézier curve sionality due to the curse of dimensionality [2,3]. Even with advanced
Pc = prior distribution of latent codes techniques to balance the exploration and exploitation of the design
Pdata = data distribution space [4–8], this can still make optimization intractable for complex
PG = generative distribution geometries.
Pz = prior distribution of noise variables To combat this curse of dimensionality, previous research has
Q = auxiliary distribution developed dimensionality reduction (DR) techniques to more com-
Re = Reynolds number pactly represent the original design space. This accelerates exploration
t = parameter variables of the rational Bézier by capturing only those dimensions that either affect the final design’s
curve performance [9–13] or capture major shape variability [14–21]. How-
w = weights of the rational Bézier curve ever, these DR models may not sufficiently or compactly capture the
x = design variables true variation that we observe in real-world designs; that is, they may
x
= optimal design require more dimensions than are needed. Also, the distribution of the
z = noise variables reduced design variables corresponding to valid designs is usually
z
= optimal noise variables unknown, making it difficult to efficiently explore or bound this
α = angle of attack reduced space. Meanwhile, machine learning researchers have con-
λ0 , λ1 , λ2 , λ3 , λ4 = regularization coefficients of Bézier-GAN ducted a vast amount of DR research using deep neural networks such
as variational autoencoders (VAEs) [22] and generative adversarial
networks (GANs) [23] to learn low-dimensional representations for
I. Introduction data from complex high-dimensional distributions. Compared with
traditional DR methods, these deep learning frameworks can learn
A ERODYNAMIC shape optimization is a necessary step in
designing parts like aircraft wings and (propeller/rotor/turbine) an arbitrary design data distribution parameterized by neural networks.
As a result, they can model the highly complex variability of designs
Presented as Paper 2019-2351 at the AIAA SciTech 2019 Forum and
using a compact set of variables with a known prior distribution.
Exposition, San Diego, CA, January 7–11, 2019; received 11 December In this paper, we apply GANs to learn an interpretable low-
2019; revision received 3 May 2020; accepted for publication 17 June dimensional space (i.e., the latent space) that encodes major shape
2020; published online 30 September 2020. Copyright © 2020 by the Ameri- variation of aerodynamic designs. Variations not captured by the latent
can Institute of Aeronautics and Astronautics, Inc. All rights reserved. All space (i.e., minor features) can be encoded via a noise space. The
requests for copying and permission to reprint should be submitted to CCC at separation of major and minor features allows fast design space
www.copyright.com; employ the eISSN 1533-385X to initiate your request. exploration while maintaining the representation capacity (i.e., the
See also AIAA Rights and Permissions www.aiaa.org/randp.
geometry information is preserved when reducing the design’s dimen-
*Research Assistant and Ph.D. Student, Department of Mechanical
Engineering; [email protected]. sionality). However, one cannot directly adopt standard GANs to
†
Research Assistant and Ph.D. Candidate, Department of Mechanical synthesize aerodynamic designs, because they are designed for gen-
Engineering. erating discrete representations such as images and text and do not
‡
Assistant Professor, Department of Mechanical Engineering. conserve continuity properties important for aerodynamic shapes.
4723
 4724 CHEN, CHIU, AND FUGE

Therefore, we extend the concept of GANs and propose the Bézier- biological evolution through mutation, recombination, and reproduc-
GAN for synthesizing smooth aerodynamic designs. tion of different designs. Work has also been done to augment GA
The specific scientific contributions of this paper are as follows: with the bees algorithm [37] and adaptive mutation rates [38], result-
1) A new type of generative model, Bézier-GAN, appropriate for ing in more accurate optimization and/or faster convergence. Other
smooth geometry (such as those expressed via splines or Bézier PBO methods applied in aerodynamic optimization are differential
curves) that improves the design synthesis quality and convergence evolution [39] and particle swarm optimization [34,40]. However,
rate compared with traditional GANs. It also enables a two-level due to the large number of CFD evaluations needed to form the
shape parameterization that separately controls the major and the population at each iteration, PBO methods can usually be prohibi-
minor shape deformation. tively expensive computationally, especially if every evaluation
2) A two-stage optimization (TSO) method that accelerates con- requires a high-fidelity CFD simulation [36].
vergence by prioritizing the optimization of major shape features. SBO uses an inexpensive surrogate model to approximate the
3) A study of the comparative optima and convergence rate while expensive CFD evaluations. Bayesian optimization (BO) is a com-
using several competing parameterization methods: our Bézier-GAN monly used SBO method. It consists of two components: a sampling
with different configurations, genetic modal design variables (GMDV) strategy (e.g., maximum expected improvement [41] or maximum
[24], singular value decomposition (SVD) [20], and Bézier surface upper confidence bound [42]) and a surrogate modeling method (e.g.,
free-form deformation (FFD) [25,26]. We discuss the representation Gaussian process regression, also known as kriging [43]). In each
capacity and compactness of each parameterization. iteration, the sampling strategy proposes a point in the design space
for evaluation, which is then used to update the surrogate model.
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

Compared with methods like GAs, SBO reduces the number of

II. Background expensive CFD evaluations needed in aerodynamic shape optimiza-
In this section, we introduce previous work on common algorithms tion [9,11,14,16,44]. However, for a high-dimensional design space,
used in aerodynamic design optimization (Sec. IV.B.1), parameter- the number of evaluations will still be inevitably high due to the curse
ization techniques (Sec. IV.B.2), and methods for reducing design of dimensionality. This creates a demand for shape parameterization
space dimensionality (Sec. IV.B.3). methods or design space DR techniques that can reduce the number
of design variables in optimization.
A. Aerodynamic Design Optimization There are other gradient-free methods beyond the above two
classes (e.g., simulated annealing [45,46]) but are not as commonly
Aerodynamic design is, in large part, an optimization problem. One
used in aerodynamic shape optimization. We direct interested readers
prototypical example is to find design variables that minimize the drag
to Ref. [36] for a thorough review of these methods.
coefficient CD , while maximizing or constraining the lift coefficient CL
[11,13,15]. In this section, we divide the optimization approaches into
two categories, namely, gradient-based and gradient-free methods, and B. Shape Parameterization
introduce the current state-of-the-art in each category. A parameterization maps a set of parameters to points along a
smooth curve or surface via a parametric function. Common param-
1. Gradient-Based Methods eterization for aerodynamic shapes includes splines (e.g., B-spline
Gradient-based methods search for the optimum based on the and Bézier curves) [47–49], FFD [25,26], class-shape transforma-
gradient of the objective function. When the objective is based on tions (CSTs) [50,51], parametric sections (PARSEC) [52,53], and
CFD simulations, automatic differentiation (AD) or adjoint methods Bézier-PARSEC [54]. Usually during design optimization, parame-
can be used to compute the gradients. Such methods provide a ters are sampled to generate design candidates [44]. There are two
relatively fast and exact method of calculating numerical gradients main issues when optimizing these parameters from conventional
whose computation cost can be independent of the number of design parameterization: 1) one has to guess the limits of the parameters to
variables and within an order of magnitude of the forward simulation form a bounding-box within which the optimization operates, and
pass. Because of this, previous work [27–33] has used such methods 2) the design space dimensionality is usually higher than the under-
for gradient calculations. Combined with optimization algorithms lying dimensionality for representing sufficient shape variability
such as sequential quadratic programming (SQP) [31,32], steepest [55]; that is, to capture sufficient shape variation, manually designed
descent [30], and Newton and quasi-Newton methods [27,28,33], shape parameterizations require higher dimensions than are strictly
such methods can drastically accelerate gradient calculations in the necessary. In contrast, this paper shows that learned parameteriza-
optimization process [27,32,33]. tions can often achieve much higher compactness without a loss in
However, adjoint methods can become numerically ill-posed when representation quality or optimization performance.
using simulation techniques like large-eddy simulation [1]. In addi-
tion, AD or adjoint methods require additional simulations beyond the C. Design Space Dimensionality Reduction
original forward pass to calculate the gradient at each design; depend- It is usually wasteful to search for optima in the spaces of aforemen-
ing on the implementation, this can add significant computational tioned shape parameters, because valid designs constitute only a small
costs. In terms of memory, AD or adjoint methods can be expensive portion of those spaces so that most CFD evaluations are performed on
compared with, for example, a finite difference method. Additionally, invalid designs. Past work has studied methods to obtain more compact
as a method of gradient calculation, AD or adjoint methods will still representations via DR. Factor screening methods [56,57] can select the
inherit some disadvantages of gradient-based algorithms, for exam- most relevant design variables for a design problem while fixing the rest
ple, converging to local minima in nonconvex problems. To mitigate as constant during optimization. These methods fail to consider the
this issue, Berguin et al. [12] use solutions to surrogate-based opti- correlation between design variables. In response, researchers have
mization (SBO) as starting points for AD methods, hoping to find studied ways to capture the low-dimensional subspace that identifies
good local optima. important directions with respect to the change of response (i.e., Quan-
tities of Interest or performance measure) [9–13]. This response-based
2. Gradient-Free Methods DR usually has several issues: 1) it requires many simulations when
As mentioned earlier, there are cases where the optimization collecting samples of response gradients; 2) variation in gradients can
problem is ill-conditioned or gradients of the objective are in- capture only nonlinearity rather than variability in the response, so extra
accessible. Gradient-free methods can side step this problem and heuristics are required to select latent dimensions that capture steep
can increase the likelihood of finding a global optimum [34]. Two linear response changes; 3) the learned latent space is not reusable for
classes of approaches are widely used for gradient-free aerodynamic any different design space exploration or optimization task (i.e., when a
shape optimization: population-based optimization (PBO) and SBO. different response is used); and 4) the linear DR techniques applied in
A popular PBO method in aerodynamic shape optimization is previous work may not model well responses with nonlinear correlation
the genetic algorithm (GA) [14,17,35,36]. It mimics the process of between partial derivatives.
 CHEN, CHIU, AND FUGE 4725

The first three issues can be avoided by directly applying DR on Both D and G components improve via training until the discrimi-
design variables without associating them with the response. Doing nator cannot differentiate between real (data) and fake (synthetic)
so assumes that if changes in a design are negligible, changes in the inputs, implying that the generative distribution resembles the data
responses are also negligible. In the area of aerodynamic design, distribution. We direct interested readers to [23] for a more detailed
researchers use linear models such as principal component analysis explanation of GANs.
(PCA) [16–18] or SVD [19,20,58] to reduce the dimensionality of Standard GANs do not have a way of regularizing the latent
design variables. Although those linear models provide optimal representation (noise); thus, the noise may end up being noninter-
solutions to the linear DR problem, their linear nature makes them pretable. This causes the noise variation not to reflect an intuitive
unable to achieve the most compact representation (i.e., use the least design variation, which impedes design space exploration. To com-
dimensions to retain similar variance in the data) when the data are pensate for this weakness, the InfoGAN encourages interpretable and
nonlinear, which is the most common case for real-world data. Non- disentangled latent representations by maximizing the mutual infor-
linear models like generative topographic mapping [14,15] can solve mation between the latent codes c and the generated samples x.~ Thus,
this problem to some extent, but are still limited to the assumption that InfoGAN’s generator takes c as an additional input; that is,
data follow a Gaussian mixture distribution, which is too strict in x~  Gc; z (Fig. 1). Unfortunately, it is hard to directly maximize
most real-world cases. Beyond these data-driven methods, GMDV the mutual information Ic; Gc; z, so, instead, an InfoGAN ap-
[24] generates airfoils through orthogonal modes derived from the proximates the solution by maximizing a lower bound. The mutual
reduced singular matrix of the third-difference matrix. None of these information lower bound LI is
DR methods encourage compactness of the reduced shape representa-
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

tion, where the volume of the latent space that maps to the domain of LI G; Q  Ex∼PG Ec 0∼Pcjx log Qc 0jx  Hc (2)
invalid designs is minimized. Complementary work in DR has been
done in other fields such as computer vision and computer graphics
[59,60], where DR is used for generating images or three-dimensional where Hc is the entropy of the latent codes, and Q is the auxiliary
(3D) shapes. Deep generative networks such as VAEs and GANs have distribution for approximating Pcjx. We direct interested readers to
been widely applied in those areas to learn the latent data representa- [61] for the derivation of LI . The InfoGAN’s loss function combines
tions. These methods are known for their ability to learn a compact LI with the standard GAN’s loss:
latent representation from complex high-dimensional data distribu-
tions, where the latent representation follows a simple, known distri- min max Ex∼Pdata log Dx  Ec∼Pc ;z∼Pz log1 − DGc; z
G;Q D
bution (e.g., a normal or uniform distribution). Our work extends this
class of techniques by considering the generation of smooth geometries − λLI G; Q (3)
such as those needed in spline-based representations.
Note that our assumption is that we already have a reasonably
where Pc is the prior distribution of latent codes and λ is a weight
distributed dataset and the target design is within or at least not far
parameter. In practice, Hc is treated as constant if the distribution of
from the coverage of the data distribution. This assumption exists for all
c is fixed. The auxiliary distribution Q is also learned by a neural
data-driven methods. For example, given a database of airfoils, a data-
network and is simply approximated by sharing all the convolutional
driven method may help solve an airfoil design problem, but would not
layers with D and adding an extra fully connected layer to D to
necessarily give a good solution for a hydrofoil design problem.
predict the conditional distribution Qcjx. Thus, as shown in Fig. 1,
As DR models map latent variables to shapes, we can treat the
the discriminator tries to predict both the source of the data and the
latent variables and the mapping as parameters and the parametric
latent codes§. Note that although the InfoGAN encourages disen-
function, respectively. Thus, in a broader sense, we will also refer to
tangled latent representation, it does not guarantee orthogonal latent
these methods as parameterizations throughout this paper, though
variables as PCA or SVD does. Instead, it guarantees statistical
these are distinct from prior work in parameterizations because they
independence among latent or noise variables when each variable
are inferred from data directly, rather than fixed a priori.
is independently sampled. This is similar in concept to what inde-
pendent component analysis (ICA) [62] does for linear DR. We build
upon the work of InfoGAN, extending it to spline-based geometry.
III. Learning Compact and Disentangled
Representations
In this section, we introduce two deep generative models: GAN IV. Bézier-GAN: Spline-Based Shape Synthesis
[23] and its variant, InfoGAN [61], which our proposed Bézier-GAN
builds upon. Given a set of existing aerodynamic designs (e.g., air- Typical approaches to generative shape models (such as GANs)
foils from the University of Illinois Urbana-Champaign (UIUC) represent shapes as a collection of discrete samples (e.g., as pixels or
database), a deep generative model like GAN can learn a mapping voxels) owing to the their original development in the computer
from a known distribution to the unknown distribution of existing vision community. For example, a naïve way of synthesizing aero-
designs. We call the set of existing designs the training data. In dynamic shapes like airfoils would be to generate this discrete
GANs, we usually set the known distribution as a normal or uniform representation directly using the generator, such as generating a fixed
distribution. Samples drawn from this distribution are called noise. number of coordinates sampled along the airfoil’s surface curve.
The noise dimension is typically much lower than the design space However, in practice, aerodynamic shapes typically possess substan-
dimension and hence we can treat the noise as a reduced representa- tial smoothness/continuity and are typically represented using para-
tion of designs. metric curve families like splines, Bézier curves, or nonuniform
A GAN consists of two components: a generator G and a discrimi- rational B-splines (NURBS) surfaces. The naïve GAN representation
nator D (Fig. 1). The generator takes in random noise z from some of predicting discretized curves from the generator usually 1) creates
known prior distribution Pz and generates data x~  Gz. The noisy curves that have low smoothness and 2) have parametric output
discriminator takes in a sample (either from the training data or that is harder for humans to interpret and use in standard CAD
synthesized by the generator) and predicts the probability of the packages compared with equivalent curve representations (e.g.,
sample coming from the training data. The generator tries to make Bézier curves). This creates problems, particularly in aerodynamic
the generative distribution PG look like the data distribution Pdata to shape synthesis.
fool the discriminator; the discriminator tries not to be fooled. GANs To solve this issue, we modified the InfoGAN’s generator such that
achieve this by solving the following minimax problem: it only generates smooth shapes that conform to Bézier curves. We
call this generative adversarial network a Bézier-GAN [63].
min max VD; G  Ex∼Pdata log Dx  Ez∼Pz log1 − DGz
G D §
Here we use the discriminator D to denote both Q and D, because they
(1) share neural network weights.
 4726 CHEN, CHIU, AND FUGE

Fig. 1 Architectures of GAN and InfoGAN.

Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

Fig. 2 Model architecture of the Bézier-GAN.

A. Architecture B. Regularization
As shown in Fig. 2, the overall architecture is adapted from the The rational Bézier representation (i.e., the choice of P, w, and t)
InfoGAN. However, instead of directly outputting discrete coordi- for a point sequence is not unique. For example, we have observed
nates along the curve, the generator synthesizes control points that the generated control points are dispersed and disorganized. The
fPi ji  0; : : : ng, weights fwi ≥ 0ji  0; : : : ng, and parameter var- weights vanish at control points far away from the surface points, and
iables f0 ≤ tj ≤ 1jj  0; : : : ; mg of rational Bézier curves, where n the parameter variables have to become highly nonuniform to adjust
is the Bézier degree, and the number of surface points to represent the the ill-behaved control points. To prevent Bézier-GAN from con-
curve is m  1. The last layer of the generator (the Bézier layer) verging to bad optima, we regularize these Bézier parameters.
converts this rational Bézier curve representation into discrete repre-
sentation x: 1. Control Points
Because the control points can be dispersed and disorganized,
Pn  n  i causing the weights and parameter variables to also behave abnor-
i0 i tj 1 − tj 
n−i mally, one way to regularize control points is to keep them close
P i wi
xj  P   ; j  0; : : : ; m (4) together. We use the average and maximum Euclidean distance
n i between each two adjacent control points as a regularization term:
i0 i tj 1 − tj 
n n−i w
i

1 XN X n  
P s − P s 
R1 G  (6)
Since xj is differentiable with respect to fPi g, fwi g, and ftj g, we can Nn s1 i1 i i−1

train the network using backpropagation. To improve numerical sta-

bility, the Bernstein polynomial is computed via its natural logarithm: where N is the sample size.
n 2. Weights
Bni t  ti 1 − tn−i  explog Γn  1 − log Γi  1
i To eliminate the effects of redundant control points and avoid
convoluted curves, we regularize the weights of control points except
− log Γn − i  1  i log t  n − i log1 − t (5)
for the first and the last ones:

where Γ denotes the gamma function. 1 XN Xn−1  

ws 
Particularly, as is shown in Fig. 3, an aerodynamic surface (here we R2 G  (7)
Nn s1 i1 i
use airfoil as an example) can be represented as an ordered sequence
of points. Thus, the parameter variables have to satisfy 0  t0 <
tj−1 < tj < tm  1; j  2; : : : ; m − 1. Instead of directly generating 3. Edge Alignment
ftj jj  0; : : : ; mg, we use a softmax activation to generate the inter- We represent the aerodynamic surface curve by two-dimensional
vals fδj  tj − tj−1 jj  1; : : : ; mg and then compute the cumulative (2D) points starting from and ending at the trailing edge, as shown in
sum of fδj g to get ftj g. Fig. 3. Thus, the first and last control points (P 0 and Pn ) should meet at
the trailing edge so that the surface curve can be closed. To enforce this
constraint, we add the following regularization term to the loss function:

1X N
R3 G  kP s − Ps
n k (8)
N s1 0

In addition, we use the following regularization term to ensure that

Fig. 3 Ordering of the airfoil representation. the surface curve does not self-intersect near the trailing edge:
 CHEN, CHIU, AND FUGE 4727

1X N n  o
R4 G  max 0; −10 Ps s
0y − P ny (9)
N s1

where P s
iy denotes the y coordinate of the ith control point in the jth a) Stage 1
airfoil sample.
With the above regularization terms, the loss function of Bézier-
GAN becomes

X
4
min max VD; G − λ0 LI G; Q  λr Rr G (10) b) Stage 2
G;Q D
r1
Fig. 4 Objective functions in both stages of two-stage optimization
(TSO).
where λi controls the weight of each corresponding regularization
term.
c† ; μz  as a warm start, we optimize both c and z to refine the
near-optimal solution:
V. Two-Stage Optimization over the Bézier-GAN
c
; z
 arg minhc; z
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

Parameterization (12)
c;z
For a normal parameterization F (e.g., FFD), we can synthesize an
airfoil design x (i.e., the Cartesian coordinates of surface points) where hc; z  fGc; z (Fig. 4b). We can then synthesize the
through some given parameters p (e.g., in FFD, p is the coordinates optimal shape x
 Gc
; z
.
of control points). This synthesis process can be expressed as For the first-stage optimization (i.e., optimizing c), we want to
x  Fp. The CFD simulator is also a function f that maps a design quickly find a good solution c† near the true underlying optimum. In
x to its performance metric y (e.g., the drag coefficient). We can write
this paper we use an SBO method called efficient global optimization
this process as y  fx  fFp. The optimization problem can
(EGO) [41]. This method minimizes the number of function evalu-
be expressed as
ations by only evaluating at the point that shows the maximum
expected improvement (EI) [41]. The value of EI is estimated by a
miny  fFp
p Gaussian process (GP) regressor [43]. At each iteration of EGO we
want to find a latent code that is expected to best improve upon the
If we use Bézier-GAN’s generator G as a parameterization, there current optimum and then evaluate at that point. Because it is also
will be two sets of parameters we must optimize over: the latent code possible to have latent codes corresponding to invalid designs, we are
c and the noise variables z. The synthesis process can be written as dealing with a constrained SBO problem. Inspired by Refs. [64,65],
x  Gc; z. Thus, the optimization becomes we solve the following constrained optimization problem at each
iteration:
miny  fGc; z
c;z
max EIc PrCc
c
But instead of optimizing c and z simultaneously, we propose a s:t: PrCc ≥ 0.5 (13)
TSO designed to exploit Bézier-GAN’s two-level parameterization
for faster convergence toward good solutions (Algorithm 1).
Because we maximize the mutual information of the lower bound where Cc is an indicator of feasibility at c (i.e., whether the con-
between the latent codes and the generated design, the latent codes c straints are satisfied or whether the objective function has definition).
will capture major shape variations, whereas minor variations are In practice, we use a GP classifier [43] to estimate PrCc, where
controlled by the noise z. This is further demonstrated in Sec. VI. Cc  0 indicates the shape x  Gc; μz  is self-intersecting or the
This two-level representation allows us to perform design space CFD simulation on shape x is not successful. At each iteration t, we
exploration in a more efficient way. Specifically, we first optimize find the solution ct to Eq. (13) and evaluate at ct . The hyper-
c with fixed z to quickly find a near-optimal solution c† : parameters of both the GP regressor and the GP classifier are opti-
mized by maximizing the log marginal likelihood (LML). We direct
c†  arg mingc (11) interested readers to [43] for details of implementing a GP regressor/
c classifier and using LML for hyperparameter optimization.
The last iteration of the first-stage optimization provides a latent code
where gc  fGc; μz  and μz  Ez∼Pz z (Fig. 4a). When Pz is a c† that roughly locks down the major features of an optimal design. The
normal distribution centered at 0, we have μz  0. Then using true optimal latent code is likely to be near c†. The second-stage

Algorithm 1: Two-stage optimization

1 ⊳ Given the latent dimension d, the noise dimension d 0,
the mean noise μz , a trained generator G, a CFD simulator f,
and evaluation budget T
2 procedure TSO d; d 0 ; G; f; T
3 T 1 ←Td∕d  d 0 , T 2 ←T − T 1 ⊳ Assign budget to each stage
4 For t  1: T 1, solve minc gc  fGc; μz  ⊳ First-stage optimization
5 c† ← arg minc gc
6 For t  1: T 2, solve minc;z hc; z  fGc; z with c† ; μz  as a warm start ⊳ Second-stage optimization
7 c
; z
← arg minc;z hc; z
8 x
←Gc
; z
 ⊳ Synthesize the optimal design
9 return x

10 end procedure
 4728 CHEN, CHIU, AND FUGE

optimization then refines this solution by jointly optimizing c ∈ Rd and exhibit smooth curvature profiles. We compute the curvature at a
0
z ∈ Rd . We set z ∼ Pz  N μz ; Σz , which means that to synthesize surface point xt; yt via
realistic designs we need to sample z near μz. Thus, in the second stage,
we start from c† ; μz  and use GA to search for a refined solution. x_ y −x y_
κt  (14)
x_ 2  y_ 2 3∕2
VI. Experiment: Airfoil Synthesis and Shape
where x, y, and their derivatives can be obtained from Eq. (4), given
Optimization
control points P1 ; : : : ; Pn and weights w1 ; : : : ; wn as the generator’s
In this section, we test the performance of the Bézier-GAN as a intermediate layer output. We also visualize the control points and the
parameterization in two aspects: 1) representation capacity, or the weights in Fig. 7. If one needs to constrain the curvature (e.g., to
ability to cover the design space, and 2) representation compactness, improve aerodynamic performance, reduce simulation complexity, or
or the ability of using the least number of parameters to cover a satisfy manufacturing tolerances), we can add another regularization
sufficient design space while every point in the parametric space term that contains κt to Bézier-GAN’s loss function in Eq. (10).
maps to a valid design. However, this is beyond the scope of the current work and we leave it
for future study.
A. Dataset and Preprocessing To study the effects of latent/noise dimensions on the Bézier-GAN
We use the UIUC airfoil database¶ as our training data for the parameterization and the optimization performance, we trained
Bézier-GAN. It provides the geometries of approximately 1600 real- multiple Bézier-GANs with different combinations of latent dimen-
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

world airfoil designs that cover applications from low-Reynolds- sion (d  2; 4; 6; 8; 10) and noise dimension (d 0  0; 10; 20). We
number airfoils for UAVs and model aircraft to jet transports and used kernel maximum mean discrepancy (MMD) [71] to evaluate the
wind turbines. Each design is represented by discrete 2D coordinates quality of generated designs. The MMD metric measures how well
along their upper and lower surfaces. From the dataset, we removed our generator approximates the real data distribution:
outliers with unrealistic appearance. The number of coordinates for
each airfoil is inconsistent across the database, so we use B-spline MMD2 Pdata ; PG 
interpolation to obtain consistent shape representations. Specifically,
we interpolate 192 points over each airfoil with the concentration of  Exd ;xd0 ∼Pdata ;xg ;xg0 ∼PG kxd ; xd0  − 2kxd ; xg   kxg ; xg0  (15)
these points along the B-spline curve based on the curvature [47]. The
preprocessed data are visualized at the top of Fig. 5. We have where kx; x 0   exp−kx − x 0 k2 ∕2σ 2  is a Gaussian kernel and σ
published the preprocessed data and the code for reproducing the is the kernel length scale that was set to 1.0. A lower MMD indicates
experimental results.** that the generator is better at generating realistic designs. Results
shown in Fig. 8 were collected via 10 runs for each latent and noise
B. Bézier-GAN Parameterization dimension configuration. It indicates that the latent dimension plays a
The latent codes and the noise were concatenated and fed into the key role in improving the generator. Also, no significant improve-
generator. The generator has two branches: one generates control ment is shown after the latent dimension reaches eight, indicating that
points and weights through dense layers and deconvolutional layers major features of the airfoil geometry can be adequately encoded with
[66] and the other generates parameter variables via only dense eight latent variables. Ideally, one would expect lower MMD values
layers. The Bézier layer combines the outputs of the two branches when the noise dimension is larger, as the noise can encode minor
and synthesizes 2D point coordinates along the surface curve. The features that are not captured by latent codes. However, in our results
discriminator takes in the coordinates and predicts the source of the that was true only when the latent dimension was two. One explan-
input and the latent codes with dense layers and convolutional layers. ation for this is that increasing the number of latent dimensions leaves
Batch normalization and Leaky ReLU activation were used at each less room for the noise variables to control meaningful shape varia-
intermediate layer. The number of training steps was 10,000 and the tion, as the latent codes have a higher priority of capturing shape
batch size was 32. We set Pc  Unif0; 1 and Pz  N 0; 0:5I. variation than the noise variables. Thus, the noise variables are less
The detailed implementation of Bézier-GAN can be found in our effective when the latent dimension is high.
code. For the loss function shown in Eq. (10), we used λ0  1 and We benchmark Bézier-GAN against three state-of-the-art parame-
λ1  λ2  λ3  λ4  10. We trained the Bézier-GAN using an terization approaches, namely, GMDV [24], SVD [20], and FFD
Adam optimizer [67] on a Nvidia Titan X GPU. The wall-clock [8,25]. We also perform a fitting test to evaluate a parameterization’s
training time is about 1 h, and the inference takes less than 15 s. ability to recover a wide range of existing airfoil designs. Specifically,
Figure 5 shows the synthesized airfoils by linearly interpolating we perform least-squares fitting to match the synthesized airfoils with
points in the latent space and the noise space using a trained Bézier- the UIUC airfoils under different parameterizations and numbers of
GAN. The 3D latent space captured large shape variations with design variables. The results are shown in Fig. 9 (we set Bézier-GAN’s
respect to features such as thickness and camber/cord line curvature. noise dimension to 10). Lower mean square error (MSE) indicates
Those are major features of the airfoil geometry. In contrast, shapes in better coverage of the UIUC data. Note that the fitting performance
the noise space show only small variations when fixing the latent is biased toward SVD and GMDV and will not solely depend on the
codes. This indicates that the noise space captured minor features. design space coverage, because both SVD and GMDV have analytical
Figure 6 compares synthesized shapes during the training proc- solutions to the least-squares problem, whereas Bézier-GAN’s solu-
esses of a Bézier-GAN and an InfoGAN. It demonstrates that the tion is approximate. This approximation is because the fitting problem
Bézier-GAN converged smooth and realistic airfoil shapes in far is nonanalytical and nonconvex for Bézier-GAN, requiring iterative
fewer training samples compared with the InfoGAN. methods to find the (possibly suboptimal) least-squares fit. Although
One other advantage of the Bézier-GAN over an InfoGAN or other this hinders Bézier-GAN fitting performance on the training data,
discrete parameterizations like SVD [20] is that it synthesizes continu- it will not necessarily affect its performance in design optimization,
ous Bézier curves through the Bézier layer, rather than directly gen- as we show in the next section. The fitting results show that all the
erating discrete surface point representations. This means that the tested methods converge to a plateau as the number of variables
resultant shapes will always have continuous curvature. This guarantee increases. As shown in Fig. 9a, SVD has the lowest MSE, which is
benefits aerodynamic performance, because past work has shown that reasonable as it uses UIUC airfoils as training data and we can obtain
aerodynamic performance is strongly dependent on the shape’s curva- the exact optima of the least-squares problem. Bézier-GAN with fixed
ture continuity [68–70]. As Fig. 7 shows, the synthesized airfoils noise has a result similar to GMDV. In Fig. 9b, we show two scenarios:
1) we only optimize latent codes and fix noise variables during
¶
https://github.com/IDEALLab/bezier-gan. the least-squares fitting, and 2) we optimize both latent and noise
**http://m-selig.ae.illinois.edu/ads/coord_database.html. variables. The figure shows that as the latent dimension increases,
 CHEN, CHIU, AND FUGE 4729
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

Fig. 5 Examples in the airfoil database and synthesized airfoil shapes in the latent space and the noise space (visualized by uniform slices of multiple two-
dimensional spaces).

Fig. 6 Synthesized shapes during the training processes for an InfoGAN (left) and a Bézier-GAN (right).

Fig. 7 Control points (square markers with sizes indicating the magnitudes of weights w) and curvatures of three randomly synthesized airfoils.

optimizing noise variables contributes less to lowering the MSE (i.e.,

has less control on shape variation). This result offers one indication or
test as to when performing the TSO is beneficial.
The elbow point in Fig. 9 (i.e., the number of design variables
where the corresponding fitting error drops with relatively small
difference) can indicate the lowest number of variables needed to
reasonably cover the original design space. For Bézier-GAN, the
elbow point suggests a latent dimension of 8. For SVD and GMDV,
their elbow points locate at 8 and 9, respectively. Thus, we will set the
number of modes for SVD and GMDVaccording to these numbers in
the rest of our experiments.
In addition to the design space coverage study, we also compared
the performance space coverage (i.e., how the synthesized airfoils
cover the space of CL and CD ) of Bézer-GAN, SVD, GMDV, and
FFD (Fig. 10). For FFD, we use a set of 3 × 4 control points and fix
their x coordinates based on [8]. It shows that the performance
Fig. 8 Maximum mean discrepancy (MMD) metrics of trained Bézier- distribution of Bézier-GAN airfoils best matches the UIUC database.
GANs. The error bars indicate 95% confidence intervals. SVD’s performance coverage is similar to the data, but contains a
 4730 CHEN, CHIU, AND FUGE

a) b)
Fig. 9 The fitting test results showing the performance of recovering the UIUC airfoils.
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

Fig. 10 Comparison of parameterizations’ performance spaces under random samples. Note that the plots are in different scales.

large portion of invalid performances, where shapes are self- The coverage of the design space and the performance space
intersecting or cause unsuccessful simulations (denoted by CL  indicate the level of representation capacity. Another crucial property
CD  0). Both GMDV and FFD have larger coverage of the perfor- of a parameterization is its representation compactness, which indi-
mance space compared with the input data. This is expected because cates the proportion of useful designs in its design space. Figure 11
GMDVand FFD are not explicitly trained on the input data, and thus shows airfoils synthesized by randomly sampling points under differ-
their parameterizations readily sample designs far beyond the origi- ent parameterizations (i.e., random samples of the design space).
nal design space. Specifically, the Bézier-GAN airfoils are synthesized by using latent

Fig. 11 Random airfoils synthesized by parameterizations.

 CHEN, CHIU, AND FUGE 4731

a) b)
Fig. 12 Optimization results for Bézier-GAN parameterization with (i.e., two-stage optimization or TSO) and without (i.e., one-stage optimization or
OSO) refining the noise variables. The noise dimension was set to 10.
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

codes and noise variables drawn from their respective distributions

(mentioned in Sec. VI.B). Knowing the distribution of design variables
corresponding to valid designs (valid design variables) can benefit
design space exploration. For the other three parameterizations where
the distribution of valid design variables is unknown, the airfoils are
synthesized by design variables drawn uniformly at random within
specific bounds. For GMDV, we set the bounds according to Ref. [24]
(more details can be found in the code). For SVD, we set the bounds as
the minimum bounding box of design variables corresponding to the
database. For FFD, we set the bounds to be 0.2 perturbation of the
NACA 0012 airfoil. All designs synthesized by the Bézier-GAN look
realistic, whereas other parameterizations contain a larger proportion of
invalid (e.g., self-intersected) designs. This indicates that Bézier-GAN Fig. 13 Optimization history under Bézier-GAN parameterization with
has a higher representation compactness than the other parameter- different latent dimensions. The noise dimension was fixed to 10.
izations, which is expected as Bézier-GAN’s objective forces each
point in the latent or noise space to map to a design resembling the
training data.
optimization history) of those results. Thus, different from other
parameterizations, the variance in Bézier-GAN’s optimization results
C. Optimization
also depends on the stochasticity of trained Bézier-GAN models.
In Sec. VI.B, we show the representation capacity of Bézier-GAN At the first-stage optimization, each latent code was bounded in
through the coverage of the design space and the performance space. −0.2; 1.2. After getting the near-optimal latent code c† , we perform
We also visualize the design space through randomly synthesized
the second-stage optimization by bounding each latent code in c†i −
designs, which provides an indication for representation compact-
ness. But why do either of these properties matter? In this section, we 0.1; c†i  0.1; i  1; : : : ; d and each noise variable in −1.0; 1.0.
demonstrate the benefits of such properties for accelerating an aero- Figure 12 compares the optimization performance under the TSO
dynamic optimization task. The performance of global optimization and the one-stage optimization†† (OSO, i.e., the case where we assign
is closely related to the parameterization’s representation capacity the overall budget to only optimizing the latent codes). Figure 12a
and compactness. The low representation capacity may lead to a shows that compared with OSO, TSO is more effective in terms of the
suboptimal solution, because a better one cannot be represented by final optimal CL ∕CD when the latent dimension is small. Figure 12b
the parameterization. Meanwhile, low representation compactness shows that the CL ∕CD improvement of TSO over OSO (ΔCL ∕CD )
can contribute to slow convergence due to the curse of dimensionality is much more significant throughout the optimization history when
and the vast amount of invalid designs an optimizer might evaluate; the latent dimension is lower. These results make sense because, as
this wastes the evaluation budget. already indicated by the MMD values (Fig. 8) and the fitting errors
The below optimization task is only a means to assess the impact of (Fig. 9b), the increase of the latent dimension leaves less room for the
each parameterization’s representation capacity and compactness. To noise variables to control shape variation, and hence optimizing noise
make the below experiments easy and fast to replicate and evaluate by variables at the second stage becomes less useful.
other researchers, we use XFOIL [72] to compute the lift and drag The optimization performance is affected by both a parameter-
coefficients CL and CD of candidate airfoils. However, our approach is ization’s representation capacity and compactness. While low repre-
not limited to XFOIL. One can apply our techniques to any CFD or sentation capacity may lead to solutions with limited performance,
performance code including Reynolds-averaged Navier–Stokes simula- low representation compactness may explore larger regions of invalid
tions or LES. The specific choice of CFD simulation method we use designs and hence be slower at finding an optimum. To some extent,
below is not central to evaluating the key contributions of the paper, we can evaluate a parameterization by comparing its corresponding
though evaluating the impact of better parameterizations on more expen- optimization history. As shown in Fig. 13, the optimization perfor-
sive and advanced flow simulations is an interesting area for future work. mance improved when increasing the latent dimension from two to
Specifically, for the following experiments our optimization objec- eight, but further increases in the latent dimension decrease the
tive is to maximize the lift to drag ratio; that is, fx  CL ∕CD . The convergence speed. This result is consistent with the aforementioned
operating conditions are set as follows: Reynolds number Re  MMD values (Fig. 8) and fitting errors (Fig. 9). All our results
1.8 × 106 , Mach number Ma  0.01, and angle of attack α  0. indicate that a latent dimension of eight suffices with respect to the
We trained 10 Bézier-GANs for each configuration of latent/noise representation capacity for our specific optimization problem.
dimensions. For each configuration, we obtained one optimization
result by using each of the 10 Bézier-GANs as the parametric model
††
and studied the statistics (e.g., mean and standard deviation of the We used EGO for the one-stage optimization.
 4732 CHEN, CHIU, AND FUGE

Figure 14 shows that although the noise dimension cannot affect noise dimension goes up, the representation capability will go up
the optimization performance as significantly as the latent dimension (which leads to an improved final optimal solution under the noise
does, having noise variables is still better than not. Note that as the dimension of 10 over 0), but the optimization convergence may slow
due to a reduced representation compactness and the curse of dimen-
sionality; for example, using a noise dimension of 20 converged
slower than when using 10.
We also compared our method to several state-of-the-art parameter-
ization methods (Fig. 15). The latent and noise dimensions of Bézier-
GAN were set to 8 and 10, respectively. For GMDV and SVD, we set
the number of design variables to 8 and 9, respectively, according to
their elbow points in the fitting test results (Fig. 9). We used 3 × 4
control points for FFD, according to Ref. [8]. The design variable
bounds for GMDV were set according to Ref. [24]. For SVD, we set the
design variable bounds as the minimum bounding box of design
variables corresponding to the database. For FFD, the design variable
bounds were 0.2 perturbation of the NACA 0012 airfoil. When
Fig. 14 Optimization history under Bézier-GAN parameterization with performing GA, the population size was 100, and the chance of
different noise dimensions. The latent dimension was fixed to 8. mutation (i.e., the probability of mutating an individual’s parameter)
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

was 0.1. In each generation, we chose the 30 best and 10 random

individuals for crossover, and produced 5 children for each pair. We
direct interested readers to our code for further implementation spe-
cifics. Besides the aforementioned parameterizations, we also tested
the optimization performances of NURBS [73] and PARSEC [52].
However, we do not show their results in the figure as their perfor-
mances were strictly worse than the SVD, GMDV, FFD, and Bézier-
GAN parameterizations. We direct interested readers to Ref. [21] for
these results. Each optimization history was averaged over 10 runs.
The results show that the average CL ∕CD value reached in 200
evaluations by the Bézier-GAN takes other methods at least 500
evaluations to reach. It indicates that the Bézier-GAN parameter-
ization has much higher compactness so that fewer resources were
Fig. 15 Optimization history under Bézier-GAN and other parameter- wasted in exploring invalid and poor-performance regions. Figure 16
izations. The performance reached by Bezier-GAN+TSO in 200 evalua- shows the optimal airfoils for the different parameterizations. It is
tions takes the next best method 500 evaluations to reach. notable that different from other parameterizations, the variance of

Fig. 16 Optimal airfoils for different parameterizations.

 CHEN, CHIU, AND FUGE 4733

the Bézier-GAN results comes not only from the optimization proc- implied, of the Defense Advanced Research Projects Agency or the
ess but also the trained Bézier-GAN model, because the training Department of Defense.
process is stochastic.
References
VII. Conclusions [1] Larsson, J., and Wang, Q., “The Prospect of Using Large Eddy and
Detached Eddy Simulations in Engineering Design, and the Research
The Bézier-GAN is used as a new parameterization for aerody- Required to Get There,” Philosophical Transactions of the Royal Soci-
namic designs that possesses high representation compactness and ety A: Mathematical, Physical and Engineering Sciences, Vol. 372,
sufficient representation capacity. The latent codes and noise varia- No. 2022, 2014, Paper 20130329.
bles encode major and minor shape variations, respectively. This https://doi.org/10.1098/rsta.2013.0329
paper then proposes using this parameterization as part of a two- [2] Bellman, R., “Dynamic Programming,” Science, Vol. 153, No. 3731,
stage design optimization method. Our results show that the Bézier- 1966, pp. 34–37.
GAN accelerates convergence and finds optimal designs with higher https://doi.org/10.1126/science.153.3731.34
[3] Regier, J. C., and Stark, P. B., “Mini-Minimax Uncertainty Quantifica-
performance than those found by other state-of-the-art parameter-
tion for Emulators,” SIAM/ASA Journal on Uncertainty Quantification,
ization methods. Vol. 3, No. 1, 2015, pp. 686–708.
It is believed that this phenomenon is induced by the representation https://doi.org/10.1137/130917909
capacity and compactness of parameterizations and is independent of [4] Koziel, S., and Leifsson, L., “Multi-Level Surrogate-Based Airfoil
the specific CFD approach used to evaluate the airfoil. That is, a space Shape Optimization,” 51st AIAA Aerospace Sciences Meeting including
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

that better describes airfoil variation should help during optimization, the New Horizons Forum and Aerospace Exposition, AIAA Paper 2013-
even if such a space was not directly designed to aid in that task. It is 778, 2013.
agreed though that the simulation environment in this paper is simple https://doi.org/10.1016/j.procs.2013.05.254
and that it is admittedly assumed that one would see similar optimi- [5] Anderson, G. R., Nemec, M., and Aftosmis, M. J., “Aerodynamic Shape
Optimization Benchmarks with Error Control and Automatic Parame-
zation improvements under different performance evaluators, such as
terization,” 53rd AIAA Aerospace Sciences Meeting, AIAA Paper 2015-
more advanced CFD. Likewise, it is only demonstrated this for 2D 1719, 2015.
airfoils, and this hypothesis has not been explicitly tested on 3D https://doi.org/10.2514/6.2015-1719
surfaces or larger wing segments. It is expected that, in 3D with even [6] Fusi, F., Quaranta, G., Guardone, A., and Congedo, P. M., “Drag
greater design freedom, representations with better compactness and Minimization of an Isolated Airfoil in Transonic Inviscid Flow by
coverage would become even more important. However, this has not Means of Genetic Algorithms,” 53rd AIAA Aerospace Sciences Meet-
been tested, and it might be an excellent area for future research. ing, AIAA Paper 2015-1722, 2015.
In general, having a parameterization that separates major and minor https://doi.org/10.2514/6.2015-1722
shape deformation is useful for design optimization. Parameters with [7] Amrit, A., Leifsson, L. T., Koziel, S., and Tesfahunegn, Y. A., “Effi-
cient Multi-Objective Aerodynamic Optimization by Design Space
larger impact on the geometry can have higher priority during opti-
Dimension Reduction and Co-Kriging,” 17th AIAA/ISSMO Multidis-
mization, such that major features, which are usually the primary cause ciplinary Analysis and Optimization Conference, AIAA Paper 2016-
of the performance change, can be determined first. Although in this 3515, 2016.
paper a two-way partitioning of design parameters is used, in an ideal https://doi.org/10.2514/6.2016-3515
case one would have all parameters ordered with respect to their [8] Masters, D. A., Taylor, N. J., Rendall, T., Allen, C. B., and Poole, D. J.,
importance on the geometry. Some linear DR methods like SVD can “Geometric Comparison of Aerofoil Shape Parameterization Methods,”
learn ordered representations. Efforts have also been made to extend AIAA Journal, Vol. 55, No. 5, 2017, pp. 1575–1589.
this capability to nonlinear methods [74]. Yet so far, to the best of our https://doi.org/10.2514/1.J054943
knowledge, there is no equivalent for deep generative models. Related [9] Lukaczyk, T. W., Constantine, P., Palacios, F., and Alonso, J. J., “Active
Subspaces for Shape Optimization,” 10th AIAA Multidisciplinary
to this, one would need a novel global optimization method to effi-
Design Optimization Conference, AIAA Paper 2014-1171, 2014.
ciently optimize the ordered parameters. Future research can fill in https://doi.org/10.2514/6.2014-1171
these gaps. [10] Berguin, S. H., and Mavris, D. N., “Dimensional Design Space Explora-
Although only the Bézier-GAN’s capability to parameterize air- tion of Expensive Functions with Access to Gradient,” 15th AIAA/ISSMO
foils is demonstrated here, this new generative model can be also Multidisciplinary Analysis and Optimization Conference, AIAA Paper
trained to synthesize other smooth geometries such as hydrodynamic 2014-2174, 2014.
shapes. https://doi.org/10.2514/6.2014-2174
In this paper, EGO and GA are used in the first-stage and second- [11] Berguin, S. H., and Mavris, D. N., “Dimensionality Reduction in
stage optimization, respectively. There are other ways to improve the Aerodynamic Design Using Principal Component Analysis with Gra-
dient Information,” 10th AIAA Multidisciplinary Design Optimization
optimal solution while maintaining fast convergence. For example,
Conference, AIAA Paper 2014-0112, 2014.
the optimum obtained by our method can be used as a good start point https://doi.org/10.2514/6.2014-0112
for gradient-based optimization methods (e.g., as in the work of [12] Berguin, S. H., Rancourt, D., and Mavris, D. N., “Method to Facilitate
Berguin et al. [12]). For future research, a trained Bézier-GAN High-Dimensional Design Space Exploration Using Computationally
generator and an automatic differentiation solver can be concatenated Expensive Analyses,” AIAA Journal, Vol. 53, No. 12, 2015, pp. 3752–
to obtain the gradient of the performance with respect to each of the 3765.
latent codes and noise variables directly. In this way, gradients can be https://doi.org/10.2514/1.J054035
propagated to the reduced representation and help solve gradient- [13] Grey, Z. J., and Constantine, P. G., “Active Subspaces of Airfoil Shape
based optimization problems on the compact reduced space. Parameterizations,” AIAA Journal, Vol. 56, No. 5, 2018, pp. 2003–2017.
https://doi.org/10.2514/1.J056054
Overall, it is our hope that this paper highlights the promising
[14] Viswanath, A., Forrester, A. I., and Keane, A. J., “Dimension Reduction
directions that learned geometric parameterizations can play in shape for Aerodynamic Design Optimization,” AIAA Journal, Vol. 49, No. 6,
and design optimization more generally, and how future work in such 2011, pp. 1256–1266.
techniques can complement traditional optimization methods used https://doi.org/10.2514/1.J050717
by the optimization community. [15] Viswanath, A., Forrester, A., and Keane, A., “Constrained Design
Optimization Using Generative Topographic Mapping,” AIAA Journal,
Vol. 52, No. 5, 2014, pp. 1010–1023.
Acknowledgments https://doi.org/10.2514/1.J052414
[16] Cinquegrana, D., and Iuliano, E., “Efficient Global Optimization of a
This work was supported by the Defense Advanced Re- Transonic Wing with Geometric Data Reduction,” 35th AIAA Applied
search Projects Agency (grant numbers 16-63-YFA-FP-059 and Aerodynamics Conference, AIAA Paper 2017-3057, 2017.
HR00111820009). The views, opinions, and/or findings contained https://doi.org/10.2514/6.2017-3057
in this paper are those of the authors and should not be interpreted [17] Cinquegrana, D., and Iuliano, E., “Investigation of Adaptive De-
as representing the official views or policies, either expressed or sign Variables Bounds in Dimensionality Reduction for Aerodynamic
 4734 CHEN, CHIU, AND FUGE

Shape Optimization,” Computers & Fluids, Vol. 174, Sept. 2018, and Technology, Vol. 32, No. 7, 2018, pp. 3191–3199.
pp. 89–109. https://doi.org/10.1007/s12206-018-0622-x
https://doi.org/10.1016/j.compfluid.2018.07.012 [36] Skinner, S., and Zare-Behtash, H., “State-of-the-Art in Aerodynamic
[18] Yasong, Q., Junqiang, B., Nan, L., and Chen, W., “Global Aerodynamic Shape Optimisation Methods,” Applied Soft Computing, Vol. 62, Jan.
Design Optimization Based on Data Dimensionality Reduction,” Chi- 2018, pp. 933–962.
nese Journal of Aeronautics, Vol. 31, No. 4, 2018, pp. 643–659. https://doi.org/10.1016/j.asoc.2017.09.030
https://doi.org/10.1016/j.cja.2018.02.005 [37] Tandis, E., and Assareh, E., “Inverse Design of Airfoils via an Intelligent
[19] Li, J., Bouhlel, M. A., and Martins, J., “A Data-Based Approach for Fast Hybrid Optimization Technique,” Engineering with Computers, Vol. 33,
Airfoil Analysis and Optimization,” 2018 AIAA/ASCE/AHS/ASC Struc- No. 3, 2017, pp. 361–374.
tures, Structural Dynamics, and Materials Conference, AIAA Paper https://doi.org/10.1007/s00366-016-0478-6
2018-1383, 2018. [38] Jahangirian, A. R., and Ebrahimi, M., “Airfoil Shape Optimization with
https://doi.org/10.2514/1.J057129 Adaptive Mutation Genetic Algorithm,” Journal of Aerospace Science and
[20] Poole, D. J., Allen, C. B., and Rendall, T., “Efficient Aero-Structural Technology, Vol. 11, No. 1, 2017, pp. 47–59, http://jast.ias.ir/article_51638
Wing Optimization Using Compact Aerofoil Decomposition,” AIAA .html.
Scitech 2019 Forum, AIAA Paper 2019-1701, 2019. [39] Liu, Z., Liu, X., and Cai, X., “A New Hybrid Aerodynamic Optimization
https://doi.org/10.2514/6.2019-1701 Framework Based on Differential Evolution and Invasive Weed Optimiza-
[21] Chen, W., Chiu, K., and Fuge, M., “Aerodynamic Design Optimization tion,” Chinese Journal of Aeronautics, Vol. 31, No. 7, 2018, pp. 1437–1448.
and Shape Exploration Using Generative Adversarial Networks,” AIAA https://doi.org/10.1016/j.cja.2018.05.002
Scitech 2019 Forum, AIAA Paper 2019-2351, 2019. [40] Ray, T., and Tsai, H., “Swarm Algorithm for Single- and Multiobjective
https://doi.org/10.2514/6.2019-2351 Airfoil Design Optimization,” AIAA Journal, Vol. 42, No. 2, 2004,
Kingma, D. P., and Welling, M., “Auto-Encoding Variational Bayes,” arXiv
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

[22] pp. 366–373.

preprint arXiv:1312.6114, 2013, https://arxiv.org/pdf/1312.6114.pdf. https://doi.org/10.2514/1.9099
[23] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., [41] Jones, D. R., Schonlau, M., and Welch, W. J., “Efficient Global Opti-
Ozair, S., Courville, A., and Bengio, Y., “Generative Adversarial Nets,” mization of Expensive Black-Box Functions,” Journal of Global Opti-
Advances in Neural Information Processing Systems 27, edited by mization, Vol. 13, No. 4, 1998, pp. 455–492.
Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. https://doi.org/10.1023/A:1008306431147
Weinberger, Curran Associates, New York, 2014, pp. 2672–2680. [42] Srinivas, N., Krause, A., Kakade, S., and Seeger, M., “Gaussian Process
https://doi.org/10.5555/2969033.2969125 Optimization in the Bandit Setting: No Regret and Experimental
[24] Kedward, L., Allen, C. B., and Rendall, T., “Towards Generic Modal Design,” Proceedings of the 27th International Conference on Inter-
Design Variables for Aerodynamic Shape Optimisation,” AIAA Scitech national Conference on Machine Learning, Omnipress, Madison, WI,
2020 Forum, AIAA Paper 2020-0543, 2020. 2010, pp. 1015–1022.
https://doi.org/10.2514/6.2020-0543 https://doi.org/10.5555/3104322.3104451
[25] Sederberg, T. W., and Parry, S. R., “Free-Form Deformation of Solid [43] Rasmussen, C. E., “Gaussian Processes in Machine Learning,” Advanced
Geometric Models,” ACM SIGGRAPH Computer Graphics, Vol. 20, Lectures on Machine Learning, Springer, Berlin, 2004, pp. 63–71.
No. 4, 1986, pp. 151–160. https://doi.org/10.1007/978-3-540-28650-9_4
https://doi.org/10.1145/15886.15903 [44] Han, Z.-H., Chen, J., Zhang, K.-S., Xu, Z.-M., Zhu, Z., and Song, W.-P.,
[26] Kenway, G. K., and Martins, J. R., “Buffet-Onset Constraint Formu- “Aerodynamic Shape Optimization of Natural-Laminar-Flow Wing
lation for Aerodynamic Shape Optimization,” AIAA Journal, Vol. 55, Using Surrogate-Based Approach,” AIAA Journal, Vol. 56, No. 7,
No. 6, 2017, pp. 1930–1947. 2018, pp. 2579–2593.
https://doi.org/10.2514/1.J055172 https://doi.org/10.2514/1.J056661
[27] Mani, K., Lockwood, B., and Mavriplis, D., “Adjoint-Based Unsteady [45] Wang, X., and Damodaran, M., “Aerodynamic Shape Optimization Using
Airfoil Design Optimization with Application to Dynamic Stall,” Annual Computational Fluid Dynamics and Parallel Simulated Annealing Algo-
Forum Proceedings–AHS International, Vol. 3, American Helicopter rithms,” AIAA Journal, Vol. 39, No. 8, 2001, pp. 1500–1508.
Soc., Virginia, 2012, pp. 1940–1952, http://scientific-sims.com/cfdlab/ https://doi.org/10.2514/2.1474
Dimitri_Mavriplis/HOME/assets/papers/AHS.2012.paper.pdf. [46] Manzo, J. E., Leylek, E. A., and Garcia, E., “Drawing Insight from
[28] Kenway, G. K. W., Kennedy, G. J., and Martins, J. R. R. A., “Scalable Nature: A Bat Wing for Morphing Aircraft,” ASME 2008 Conference on
Parallel Approach for High-Fidelity Steady-State Aeroelastic Analysis Smart Materials, Adaptive Structures and Intelligent Systems, Ameri-
and Adjoint Derivative Computations,” AIAA Journal, Vol. 52, No. 5, can Soc. of Mechanical Engineers Digital Collection, Fairfield, NJ,
2014, p. 935–951. 2008, pp. 671–678.
https://doi.org/10.2514/1.J052255 https://doi.org/10.1115/SMASIS2008-613
[29] Tesfahunegn, Y. A., Koziel, S., Leifsson, L., and Bekasiewicz, A., [47] Lepine, J., Guibault, F., Trepanier, J.-Y., and Pepin, F., “Optimized
“Surrogate-Based Airfoil Design with Space Mapping and Adjoint Nonuniform Rational B-Spline Geometrical Representation for Aero-
Sensitivity,” Procedia Computer Science, Vol. 51, Elsevier, The Nether- dynamic Design of Wings,” AIAA Journal, Vol. 39, No. 11, 2001,
lands, 2015, pp. 795–804; also International Conference on Computa- pp. 2033–2041.
tional Science, 2015. https://doi.org/10.2514/2.1206
https://doi.org/10.1016/j.procs.2015.05.201 [48] Venkataraman, P., “A New Procedure for Airfoil Definition,” 13th
[30] Schramm, M., Stoevesandt, B., and Peinke, J., “Simulation and Opti- Applied Aerodynamics Conference, AIAA Paper 1995-1875, 1995.
mization of an Airfoil with Leading Edge Slat,” Journal of Physics: https://doi.org/10.2514/6.1995-1875
Conference Series, Vol. 753, No. 2, 2016, Paper 022052. [49] Rogalsky, T., Kocabiyik, S., and Derksen, R., “Differential Evolution in
https://doi.org/10.1088/1742-6596/753/2/022052 Aerodynamic Optimization,” Canadian Aeronautics and Space Jour-
[31] Dhert, T., Ashuri, T., and Martins, J. R. R. A., “Aerodynamic Shape nal, Vol. 46, No. 4, 2000, pp. 183–190.
Optimization of Wind Turbine Blades Using a Reynolds-Averaged https://doi.org/10.1016/j.cja.2018.05.002
Navier–Stokes Model and an Adjoint Method,” Wind Energy, Vol. 20, [50] Nadarajah, S., Castonguay, P., and Mousavi, A., “Survey of Shape
No. 5, 2017, pp. 909–926. Parameterization Techniques and Its Effect on Three-Dimensional
https://doi.org/10.1002/we.2070 Aerodynamic Shape Optimization,” 18th AIAA Computational Fluid
[32] Wang, K., Yu, S., and Liu, T., “Airfoil Optimization Based on Isogeometric Dynamics Conference, AIAA Paper 2007-3837, 2007.
Discontinuous Galerkin,” Proceedings of the 2018 2nd International https://doi.org/10.2514/6.2007-3837
Conference on Algorithms, Computing and Systems, Assoc. for Computing [51] Kulfan, B. M., “Universal Parametric Geometry Representation Method,”
Machinery, New York, 2018, pp. 227–231. Journal of Aircraft, Vol. 45, No. 1, 2008, pp. 142–158.
https://doi.org/10.1145/3242840.3242856 https://doi.org/10.2514/1.29958
[33] Schramm, M., Stoevesandt, B., and Peinke, J., “Optimization of Airfoils [52] Li, P., Seebass, R., and Sobieczky, H., “Manual Aerodynamic Optimi-
Using the Adjoint Approach and the Influence of Adjoint Turbulent zation of an Oblique Flying Wing,” 36th AIAA Aerospace Sciences
Viscosity,” Computation, Vol. 6, No. 1, 2018, p. 5. Meeting and Exhibit, AIAA Paper 1998-598, 1998.
https://doi.org/10.3390/computation6010005 https://doi.org/10.2514/2.2547
[34] Venter, G., and Sobieszczanski-Sobieski, J., “Particle Swarm Optimi- [53] Sobieczky, H., Parametric Airfoils and Wings, Vieweg+Teubner Verlag,
zation,” AIAA Journal, Vol. 41, No. 8, 2003, pp. 1583–1589. Wiesbaden, Germany, 1999, pp. 71–87.
https://doi.org/10.2514/2.2111 https://doi.org/10.1007/978-3-322-89952-1_4
[35] Jeong, J.-H., and Kim, S.-H., “Optimization of Thick Wind Turbine [54] Derksen, R., and Rogalsky, T., “Bezier-PARSEC: An Optimized Aero-
Airfoils Using a Genetic Algorithm,” Journal of Mechanical Science foil Parameterization for Design,” Advances in Engineering Software,
 CHEN, CHIU, AND FUGE 4735

Vol. 41, Nos. 7–8, 2010, pp. 923–930. [66] Zeiler, M. D., Krishnan, D., Taylor, G. W., and Fergus, R., “Deconvolu-
https://doi.org/10.1016/j.advengsoft.2010.05.002 tional Networks,” 2010 IEEE Conference on Computer Vision and
[55] Chen, W., Fuge, M., and Chazan, N., “Design Manifolds Capture the Pattern Recognition (CVPR), Inst. of Electrical and Electronics Engi-
Intrinsic Complexity and Dimension of Design Spaces,” Journal of neers, New York, 2010, pp. 2528–2535.
Mechanical Design, Vol. 139, No. 5, 2017, Paper 051102. https://doi.org/10.1109/CVPR.2010.5539957
https://doi.org/10.1115/1.4036134 [67] Kingma, D. P., and Ba, J., “Adam: A Method for Stochastic Optimiza-
[56] Welch, W. J., Buck, R. J., Sacks, J., Wynn, H. P., Mitchell, T. J., and tion,” arXiv preprint arXiv:1412.6980, 2014, https://arxiv.org/pdf/1412
Morris, M. D., “Screening, Predicting, and Computer Experiments,” .6980.pdf.
Technometrics, Vol. 34, No. 1, 1992, pp. 15–25. [68] Korakianitis, T., Rezaienia, M., Hamakhan, I., Avital, E., and Williams,
https://doi.org/10.2307/1269548 J., “Aerodynamic Improvements of Wind-Turbine Airfoil Geometries
[57] Myers, R. H., and Montgomery, D. C., Response Surface Methodology: with the Prescribed Surface Curvature Distribution Blade Design
Process and Product in Optimization Using Designed Experiments, 1st (CIRCLE) Method,” Journal of Engineering for Gas Turbines and
ed., Wiley, New York, 1995, Chap. 8. Power, Vol. 134, No. 8, 2012, Paper 082601.
https://doi.org/10.5555/546074 https://doi.org/10.1115/1.4005969
[58] Allen, C. B., Poole, D. J., and Rendall, T. C., “Wing Aerodynamic Opti- [69] Song, Y., Gu, C.-W., and Xiao, Y.-B., “Numerical and Theoretical
mization Using Efficient Mathematically-Extracted Modal Design Varia- Investigations Concerning the Continuous-Surface-Curvature Effect in
bles,” Optimization and Engineering, Vol. 19, No. 2, 2018, pp. 453–477. Compressor Blades,” Energies, Vol. 7, No. 12, 2014, pp. 8150–8177.
https://doi.org/10.1007/s11081-018-9376-7
https://doi.org/10.3390/en7128150
[59] Lee, J. A., and Verleysen, M., Nonlinear Dimensionality Reduction,
[70] Shen, X., Avital, E., Paul, G., Rezaienia, M. A., Wen, P., and
Springer Science & Business Media, Berlin, 2007.
Korakianitis, T., “Experimental Study of Surface Curvature Effects on
Downloaded by UNIVERSITY OF GLASGOW on November 1, 2020 | http://arc.aiaa.org | DOI: 10.2514/1.J059317

https://doi.org/10.1007/978-0-387-39351-3
Aerodynamic Performance of a Low Reynolds Number Airfoil for Use
[60] Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y., Deep Learning,
in Small Wind Turbines,” Journal of Renewable and Sustainable
Vol. 1, MIT Press, Cambridge, MA, 2016, Chaps. 14, 20, http://www
.deeplearningbook.org. Energy, Vol. 8, No. 5, 2016, Paper 053303.
[61] Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and https://doi.org/10.1063/1.4963236
Abbeel, P., “Infogan: Interpretable Representation Learning by Infor- [71] Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., and Smola,
mation Maximizing Generative Adversarial Nets,” Advances in Neural A., “A Kernel Two-Sample Test,” Journal of Machine Learning
Information Processing Systems, Curran Associates, Inc., New York, Research, Vol. 13, March 2012, pp. 723–773.
2016, pp. 2172–2180. https://doi.org/10.5555/2503308.2188410
https://doi.org/10.5555/3157096.3157340 [72] Drela, M., “XFOIL: An Analysis and Design System for Low Reynolds
[62] Comon, P., “Independent Component Analysis, A New Concept?” Number Airfoils,” Low Reynolds Number Aerodynamics, edited by
Signal Processing, Vol. 36, No. 3, 1994, pp. 287–314. Mueller, T. J., Springer, Berlin, 1989, pp. 1–12.
https://doi.org/10.1016/0165-1684(94)90029-9 https://doi.org/10.1007/978-3-642-84010-4_1
[63] Chen, W., and Fuge, M., “BézierGAN: Automatic Generation of Smooth [73] Lepine, J., Trepanier, J.-Y., and Pepin, F., “Wing Aerodynamic Design
Curves from Interpretable Low-Dimensional Parameters,” arXiv preprint Using an Optimized NURBS Geometrical Representation,” 38th Aero-
arXiv:1808.08871, 2018, https://arxiv.org/abs/1808.08871. space Sciences Meeting and Exhibit, AIAA Paper 2000-669, 2000.
[64] Basudhar, A., Dribusch, C., Lacaze, S., and Missoum, S., “Constrained https://doi.org/10.2514/6.2000-669
Efficient Global Optimization with Support Vector Machines,” Structural [74] Rippel, O., Gelbart, M., and Adams, R., “Learning Ordered Represen-
and Multidisciplinary Optimization, Vol. 46, No. 2, 2012, pp. 201–221. tations with Nested Dropout,” International Conference on Machine
https://doi.org/10.1007/s00158-011-0745-5 Learning, Omnipress, Wisconsin, 2014, pp. 1746–1754.
[65] Gelbart, M. A., Snoek, J., and Adams, R. P., “Bayesian Optimization with https://doi.org/10.5555/3044805.3045087
Unknown Constraints,” Proceedings of the Thirtieth Conference on Uncer-
tainty in Artificial Intelligence, AUAI Press, Oregon, 2014, pp. 250–259. C. Pettit
https://doi.org/10.5555/3020751.3020778 Associate Editor