IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 1

Tree Modeling with Real Tree-Parts Examples

Ke Xie, Feilong Yan, Andrei Sharf, Oliver Deussen, Baoquan Chen, Hui Huang*

Abstract—We introduce a 3D tree modeling technique that utilizes examples of real trees to enhance tree creation with realistic
structures and fine-level details. In contrast to previous works that use smooth generalized cylinders to represent tree branches, our
method generates realistic looking tree models with complex branching geometry by employing an exemplar database consisting of
real-life trees reconstructed from scanned data. These trees are sliced into representative parts (denoted as tree-cuts), representing
trunk logs and branching structures. In the modeling process, tree-cuts are positioned in space in an intuitive manner, serving as
efficient proxies that guide the creation of the complete tree. Allometry rules are taken into account to ensure reasonable relations
between adjacent branches. Realism is further enhanced by automatically transferring geometric textures from our database onto tree
branches as well as by guided growing of foliage. Our results demonstrate the complexity and variety of trees that can be generated
with our method within few minutes. We carry a user study to test the effectiveness of our modeling technique.

Index Terms—Tree modeling, Data-driven modeling, Allometry

F
1 I NTRODUCTION

W ITHIN computer graphics, tree models are of special

interest due to their variety in nature, and have appli-
cations in many areas, including urban modeling, gaming,
We introduce an example-based tree modeling technique
which allows to design trees in an intuitive manner, to
generate models of high realism with complex branching
and movies. However, even with the significant advance- structures and fine-level geometric details, and yet without
ment of 3D tree modeling, it still remains a challenge to requiring a tedious interaction. At the core of our method
easily create trees with high level of realism. The reasons is a novel editing operator utilizing trees parts exemplars
to this are twofold: globally, trees have inherently complex representing trunk logs and branching structures (denoted
branching structures and intricate topology, while locally tree-cuts).
branches are prevalent with fine-level geometric detail and We first build a data-base of realistic 3D trees captured
nuances. Applications usually require the amount of user with a range scanner. In a preprocessing step, we reconstruct
knowledge and interaction effort be kept modest opting for the trees and compute tree-cuts which we extract from the
both flexibility and fidelity. Striking the balance among all tree branching structure and store in a repository. The tree
these factors is still a major challenge for a majority of tree editing operator allows the user to select and loosely posi-
modeling applications. tion tree-cuts in 3D space. These tree parts exemplars serve
Commonly, tree modeling approaches may be catego- as high-level geometric proxies, which locally constrain and
rized into three major classes: procedural, sketch-based and guide the tree creation process. As the user selects and in-
data-driven. Procedural approaches utilize grammar-based serts new tree-cuts, the overall tree geometry and topology
parametric methods allowing the creation of trees with ever updates interactively. New branching structures that confor-
increasing branching detail [1]. These approaches typically m with recent edits of the user, automatically emerge and
require expert knowledge and are manually intensive, pro- provide immediate visual feedback. The full tree is created
viding indirect means to control the tree modeling process. through an optimization process which balances between
Sketch-based approaches [2], [3], [4] allow for fast inter- the user-defined tree-cuts and allometric rules aiming at
active design of trees, however sketching the individual generating naturally-looking trees (Figure 1).
branches can be tedious, hence, other means have to be To further enhance realism, we transfer fine branch de-
applied for generating complex branching and fine details. tails in the form of geometric textures from our database
With the recent advance of 3D scanning technology, it has onto the new tree model branches. Additionally, the user
becoming a promising approach to scan real world trees models the foliage by drawing loose canopy profiles, which
and reconstruct their 3D models [5], [6]. With high fideli- guide the foliage growth from branch ends (Figure 2). In
ty reconstruction, the generated models capture branching essence, the process provides a holistic modeling solution
structures and geometric nuances that are necessary for for naturally looking full trees. Our modeling tool accounts
conveying life-like realism. In this work, we take advantage for both ease-of-use, as well as high levels of expressiveness
of such tree models captured from the real world to leverage and realism coming from the geometry of real trees.
the generation of new models that inherit natural realism
and variety.
2 R ELATED WORK
• K. Xie, F. Yan and H. Huang are with Shenzhen VisuCA Key Lab / SIAT
• A. Sharf is with Ben Gurion University Early works on tree modeling use simple rules, functions
• O. Deussen is with University of Konstanz and random variables to produce branching patterns [7].
• B. Chen is with Shandong University In his pioneering work, Bloomenthal [8] takes a geometric
*Corresponding author: Hui Huang ([email protected]) approach for modeling of principal tree structures. He uses
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2

Fig. 1. Using tree parts from our data-base (left) we model a complete detailed ficus (mid), and increase its realism with geometric texture and
foliage (right). The two zoomed images show the geometric complexity of the generated tree using only few tree-cuts.

(a) (b) (c) (d)

Fig. 2. Interactive modeling sequence of a Populus tree. Tree-cuts from a real tree are selected and placed in space (a) leading to a temporal trunk
model (b), to which the user adds more cuts yielding a full tree model (c). Lobes are scribbled to guide the foliage growth in the final tree (d).

implicit functions to model branchings and full trees. Op- within envelope surfaces created from a single image or
penheimer [9] uses a fractal model to describe the overall user sketches by applying some heuristics on the tree for-
geometry and bark structure of a tree. Holton [10] uses m. Palubicki et al. [18] generate realistic models of trees
Leonardo da Vinci’s proportion rules to describe a botani- and shrubs using a self-organizing process which simulates
cally valid branching pattern. He defines a tree by strands the competition of branches for light and space. Wither et
growing from the root to leaves. The cross sectional area of al. [19] sketch foliage contours to guide the distribution
a branch is defined by the sum of cross sectional areas of of 3D branches while accounting for additional botanical
its contained strands. Boudon et al. [11] present a method constraints. Their sketching system applies from the scale of
for managing the multitude of parameters involved in pro- a leaf to the scale of an entire tree.
cedural plant modeling. They represent trees as multi-scale With scanning technology becoming available, ap-
graphs and design interactive tools to edit trees on them. proaches for 3D tree reconstruction from point sets emerged.
Nevertheless, such generation methods are time consuming, Xu et al. [20] cluster edges in a spanning graph to recon-
hard to control and require a high level of expertise. In con- struct the tree skeleton while leaves are randomly added
trast, our method is straightforward and intuitive, allowing to the fine branches. Livny et al. [5] use this technology and
to easily guide and predict the resulting tree models. reconstruct tree skeletons from point clouds of multiple trees
There have been many attempts to utilize 3D recon- by computing minimal spanning graphs. In a subsequent
struction techniques from 2D photos in the context of tree work, they separate the base tree skeleton and the proce-
modeling. Reche-Martinez et al. [12] use registered photos durally reconstructible smaller parts of the tree [6]. Pirk
to generate a volumetric representation of the tree canopy et al. [21] convert this representation into a dynamic data
and its branches and twigs. Neubert et al. [13] improve this structure that allows trees to react to their environment.
by using only loosely arranged input images and a particle- Bucksch et al. [22], [23] use a space partitioning method
system to produce small branches. Other approaches [14], to cluster points and form a skeleton by connecting adjacent
[15] extract visual hulls from the input images and use L- clusters. Côté et al. [24] synthesize minor tree and leaf
Systems to synthesize branches within these hulls. geometry on reconstructed branches based on light scatter-
Other methods [16], [17] infer the branching structures ing properties obtained from intensity data. Roumonen et
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 3

(a) (b)

Fig. 4. Cut retrieval: from a simple user sketch (a) we compute a

descriptor (a)-right which is used to retrieve a set of best matching cuts
from the repository, sorted according their similarity (b).

3 OVERVIEW
In a preprocessing step, we create a database of 3D real-
life trees which we capture using scanning devices and
reconstruct into 3D meshes. Our method focuses on the
modeling of branch geometries and therefore trees in the DB
Fig. 3. A snapshot of part of our tree-cuts repository currently consisting
are stored without the foliage. We allow the user to create
of more than 200 exemplars from a variety of 15 tree species.
tree-branching exemplars simply by drawing 2D scribbles
around tree parts and cutting them out. For each tree
al. [25] reconstruct branches by locally reconstructing patch- species, its tree-cuts (including branches and ramifications)
es that are then combined into the branches. Recently Zhang are stored in a data repository and suggested to the user
et al. [26] model a 3D tree by reconstructing cylindrical while modeling a tree. Having 100-200 cuts for each species
branches from a single scan of a tree combined with user is typically sufficient to create expressive trees with large
assistance. variety and complex structures (Figure 3).
Modeling operations merely consists of selecting tree-
Two-dimensional sketches have been previously used
cuts from the repository and positioning them in 3D space.
for interactive design and editing of 3D trees and branch-
To effectively browse and suggest tree-cut candidates from
ings. Okabe et al. [27] reconstruct 3D branching skele-
the repository, the user may coarsely sketch a branch shape
tons from 2D sketches by maximizing distances between
and topology which is then matched against our DB, retriev-
branches. Additional gesture-based editing allows the user
ing the top matching candidates (Figure 4).
to apply further modification of the tree skeleton. Chen et
As tree cuts are selected and positioned in the scene, they
al. [3] model trees from 2D sketches while accounting for ad-
connect to existing structures based on shortest-distance.
ditional tree constraints from a data-base using probabilistic
Our method provides the user with immediate feedback by
optimization.
computing an interpolatory surface approximation of the
Recently, Longay et al. [28] combined procedural trees final tree branch structure on-the-fly. Essentially, a tree cut
with user interaction to generate complex, realistic-looking may connect to one or more cuts, forming various bifur-
tree models. Thus, models are generated through a self- cations in a generalized manner. During this process scale
organization procedural process which the user controls and orientations of tree cuts are continuously updated while
through several indirect parameters as well as directly guid- imposing allometry rules to account for botanical realism.
ing its overall form. In general, these methods utilize 2D Once the required overall branching structure is reached,
sketches as means to guide tree model creation, however the user finalizes the model and a fine tree structure is gen-
explicit control of the tree structure is limited. In contrast, erated. In this step, we compute a valid mesh by removing
our method is inherently 3D, using tree examples to explic- intersecting branches and remeshing discontinuous bifur-
itly control the tree creation process. Thus, the user directly cations. To increase the realism levels, branches’ endpoints
models the complex branching structures through the u- are enhanced with leaves and foliage. The user controls
tilization of existing tree-exemplars in a relatively simple their growth by scribbling lobes that are automatically filled
manner. with foliage, in the spirit of [6] . Furthermore, we transfer
Within our framework we can utilize reconstructed trees geometric texture from real-life 3D bark exemplars in our
as well as artificial ones as input in an example-based tree repository onto the synthesized branches.
creation framework. Similarly, Sharf et al. [29] and Harary
et al. [30] use shape parts as exemplars for hole filling
and shape completion. In the same context, Funkhouser et 4 T ECHNICAL D ETAILS
al. [31], Merrel et al. [32] and Marechal et al. [33] synthesize In a preprocessing step, we create a database of real-life
the 3D geometry of parts for the creation of new shapes 3D trees by scanning and reconstructing the tree models.
in an example-driven manner. Our work follows in this We use a commercial handheld mid-range structure light
path, using real-life tree parts to enhance tree creation and scanner and capture a tree by manually scanning it from
leverage the level of realism. In contrast to general synthesis different views. We then register the scans together and
approaches, ours accounts for tree-specific properties such reconstruct the 3D tree model using Poisson reconstruc-
as cylinder-shaped branches, fine bark geometry and allom- tion [34]. Since trees vary to a large extent in their shape,
etry rules. even within one species, and to enrich our database we
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 4

(a) (b) (c) (d)

Fig. 6. Connecting tree-cuts. We compute correspondence and similar-

ity transformation (T,R) between two branches contours (a). Next, we
compute their interpolation (b) and smooth mesh (c). In (d) we transfer
Fig. 5. Tree-cut generation. Cuts may be automatically generated by ran- geometric texture onto the smooth surface.
domly cutting segments along the tree skeleton (bottom), or manually,
using user scribbled loops (top).
(Figure 4). Thus, a user may define the desired tree-cut
shape and structure by loosely 2D sketching the branch
collect multiple tree models for each species. In practice we shape and retrieving the best matching 3D shape from
collect 15 different tree species, among others of a Cactus, the DB. Since branching structures are typically complex,
Ficus, Cercis, etc. The whole DB creation was an elaborate we develop a novel 2D sketch-based retrieval tool for this
process which took several days as tree scans were typically problem which is both easy-to-use and intuitive..
noisy, hard to register and properly reconstruct. There are many works on 3D model retrieval which
Our basic modeling operation involves manipulating allow users to sketch the contour of their desired object and
small tree parts. Thus, we process tree models by cutting thus guide a DB search [36], [37], [38], [39], [40]. In these
them into multiple parts which we store in our database works, the user is required to sketch the 2D shape of the
repository. A tree-cut defines a cylinder-like connected part object as seen from one or more viewpoints. In our case,
of tree, determined by at least two simple boundary-loops. sketching the shape of tree branches would require some
Thus, the simplest tree-cut is of cylinder topology while effort as well as certain drawing skills.
complex cuts define arbitrary tree-like cylindrical branch- Thus, the user sketches a rough 2D approximation of
ings. Cuts may be of arbitrary size, ranging from small to the desired branch structure which is matched against 2D
large fractions of the tree. projections of tree cuts skeletons from our DB. We compute
Tree-cuts may be automatically generated by random- a distance field originating from the user sketches using a
ly cutting principled segments of the tree model. For an 30X30 grid resolution 2D image. We use this 2D field to
automatic production, we first compute the skeleton of a search for best matching structures in the tree-cuts DB.
tree model using robust skeletonization [35] and define Since tree-cuts in the DB are essentially 3D, we project
random segments along the tree skeleton yielding various their skeletons onto 2D planes in a preprocessing step. To
subtrees. First, we sample a random starting point by climb- obtain a sufficient representation we compute projections
ing upwards from the root (we assume a known upward of the skeleton from multiple views. In our experiments,
direction). A randomly given length defines the size of 5 different views were sufficient to sample the view space
our subtree along the tree skeleton. The tree-cut is then around the tree-cut to a sufficient extent. Thus, we compute
produced by cutting the tree geometry at their endpoints for each skeleton a set of 5 corresponding 2D distance fields.
with perpendicular planes to the local skeleton (Figure 5). Our system then retrieves the top 10 best matching tree-
In addition, we allow the user to manually create tree- cuts w.r.t. the distance between their distance-fields. Among
cuts by loosely scribbling close loops on the tree. We project these candidates, the user selects one tree-cut which may be
a loop onto the tree skeleton and find intersection endpoints. loosely translated, rotated and scaled to a desired position
Note that our method requires that loop scribbles and their in the 3D scene.
intersection with the tree skeleton yields a single connected
subtree. We then proceed as in the automatic step and cut
the tree geometry with perpendicular planes at the intersec- 4.2 Connecting tree-cuts.
tion endpoints. For each tree cut, we compute its boundary Once a tree-cut is positioned, it is matched to other tree-cuts
loops of connected vertices and skeleton, and store them in in the scene by computing the shortest Euclidean distance
the repository. between the endpoints of the cuts. The user may also
manually specify pairwise correspondences by clicking and
marking the desired cuts.
4.1 Modeling interface. Given two tree-cuts defined by their boundary loops
To model a tree, the user selects tree-cuts from the repository (w.r.t. upward direction) (bi , bj ), we compute the inter-
by browsing through the tree-cut gallery. polating surface connecting them ζ(bi , bj ) (Figure 6). We
If a specific part or approximate structure is required, uniformly resample the two boundary loops with an equal
the system provides a sketch-based tree-cut retrieval tool number of points ( i.e., |bi | = |bj | = 100), and compute a
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 5

(a) (b) (c) (d)

Fig. 8. Given an input bifurcation (a), we change the diameter and angle
Fig. 7. Bifurcation generation. Original bifurcation with intersection (left),
of its branches conforming to allometry constrains (b). In (c-d) we show
its mesh reconstruction (mid) and zoom in on remeshed intersection
their 3D counterpart.
(right).

correspondence between them. To establish a full correspon-

dence between two boundary loops, we translate, rotate
and scale the boundaries using the method presented by
Horn [41]. We select the best transformation (denoted T )
by minimizing the sum of squared distances between the
transformed points and their corresponding points on the
other loop. We select the best matching pair of points in
T (bi ) and bj and compute the full correspondence by walk-
ing on both boundary loops in counter-clockwise direction (a) (b)
and matching sample points.
Next, we compute the interpolation surface ζ(B1 , B2 ) Fig. 9. Given an arbitrary tree-model (a), we apply allometry rules to
further refining its branching structure (b).
by creating k intermediate cross sections that interpolate
bi and bj . First, we compute a skeleton α(bi , bj ) as the
Hermite curve which interpolates the boundary loop centers branches, we apply Poisson reconstruction [34] in the in-
and orientations. Then, we interpolate between bi and bj tersection area.
in a gradual manner and generate a set of intermediate
boundary loops {b0ij , b2ij , ..., bkij }.
We position the intermediate loops uniformly along 4.4 Allometric constraints
α(bi , bj ) yielding a set of interpolating cross sections. Thus, Our modeling process incorporates botanical constraints to
an intermediate loop center center(blij ) coincides with enhance the realism of the final result. We utilize allometric
α(bi , bj ) at a regular interval. In addition, we rotate the rules to determine branching diameters and angles as de-
intermediate loop such that center(blij ) is perpendicular to fined by Holton [10]. Thus, our modeling scheme jointly
the skeleton α(bi , bj ) at the current position. accounts for user cuts and allometric rules to optimize
Finally, we compute the surface ζ(B1 , B2 ) passing branching diameters and angles of the tree (Figures 8, 9).
through boundary loops simply by connecting correspond- In real trees, branching diameter and angle are influ-
ing points between neighboring loops (specifically, we con- enced by the species, the location of the tree, bifurcation
nect pm ∈ bk−1 ij and qm ∈ bkij ). This yields a quadrangulated orientation, and tropisms (growth tendencies). For simplici-
boundary surface which may be further triangulated upon ty and effective computation, we ignore these factors here.
requirement. Given a user-modeled tree structure (not necessarily
final), we extract the father-son relations for its branches
4.3 Generation of bifurcations by simply traversing the tree skeleton starting from its root
to the branch endpoints.. Let d be the diameter of the father
Branching bifurcations are created by connecting a tree-cut
branch and d1 , d2 the diameters of its children. Furthermore,
to two or more different cuts. The number of boundary
let a be the angle between the child branches, and a1 ,a2 the
loops in a tree-cut predominate the number of connections
angles between child branches and the main direction of the
with other tree-cuts. Thus, a bifurcation can be seen as
father branch (thus, a = a1 + a2 ). Having this, we follow
the generalization of connecting two boundary loops to a
Leonardo da Vinci’s diameter rule [42]: d2 ≈ d21 + d22 .
multiple cut connection. Bifurcations are modeled as su-
perpositions of multiple connections between simple cuts The relation between diameters and angles follows
which are independently matched and interpolated by their Holton’s model which models branches as a set of strand-
boundary loops. s [10] and is defined by:
Nevertheless, the generated interpolating surfaces may
 γ
S2 A2 d2
intersect each other and additional processing is required a1 = a ≈ a = a, (1)
S1 A1 d1
to generate a valid bifurcation surface. Thus, we search
for mesh intersections, compute intersection curves on the where Ai = πd2i /4 is the cross-sectional area of a branch. γ
surface and remove mesh parts in the intersection region is a species-specific constant close to two. We adapt Holton’s
(Figure 7). To gracefully blend and connect intersecting model by assuming strands of equal diameter and define the
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 6

TABLE 1
Modeling time for the different trees.

Figure # Cuts # Triangles Time

1 (ficus) 25 1145985 20 m
12 (ficus) 18 734987 13 m
13 (bonsai) 16 81435 12 m
13 (Cercis) 25 707467 15 m

target cylinder with multiple instances. Since geometry is

irregular, the transition between tiles is negligible.
To avoid inconsistent results at tree-cut boundaries, we
let textures of neighboring cuts overlap and blend the over-
lapping maps at the boundary regions.

4.6 Modeling of the foliage

(a) (b) (c) (d) To finalize the tree model, we allow the user also to control
Fig. 10. Given a tree-cut in our DB (a), we generate its geometric texture
the foliage production, i.e., the generation of twigs and
illustrated as a gray-scale height map (b). We scale the texture into a leaves, on the so-far created branching structures.
larger area (c) and transfer the geometric details onto a new cylindrical To relieve the user from the tedious task of manually
structure (d). modeling the foliage we take a sketch-based approach simi-
lar to Longay et al. [28]. We define the tree foliage using the
main branches and a set of envelope lobes [6]. Thus, a user
cross-sectional area of a branch by the number of strands S .
loosely scribbles 2D loops to define major lobe contours and
Simple rearrangement of the terms yields:
to guide foliage growth. While Longay’s method utilizes
r
a1
q sketches to control the overall tree from, our sketched lobes
d2 = d, d1 = dγ − dγ2 , a2 = a − a1 . (2) define only the space of twigs and leaves which grow from
a
the nodes of the existing branch skeleton (Figure 11).
Having d, d1 , d2 and a for the input bifurcation, Eq. (2) al- To compute the shapes of the 3D lobes from the 2D scrib-
lows us to compute the new proportions for the branchings bles, we simply extrude their volume in the view direction
by linking together a1 , a2 , and d1 , d2 . We apply allometry and round them to avoid sharp edges. Then lobes are filled
rules and adjust the tree structure on-the-fly after each by randomly distributed sample points, which represent the
user edit. Allometry typically yields a well-proportioned growth space of the foliage.
tree with thinner branches at higher branching levels. Fig- Twigs grow based on these sample points from nodes
ures 8(a-b) show the global effect of applying allometric of the already existing branch skeleton similar to Runions et
constraints on a bifurcation. al. [16]. Specifically we use the skeleton nodes of the existing
branches as buds and grow twigs from them. In a recursive
4.5 Geometry transfer manner, we use the new twigs as buds and grow additional
twigs from their nodes. Thus, we generate twigs for several
Tree cuts are connected using an interpolating surface that
iterations, until all sample points are assigned to twigs (w.r.t.
allows the creation of twisted and gnarled branches through
their distance). Finally, we add tiny branches and leaves
modification of the tree-cut position and orientation. Never-
in the spirit of Livny et al. [6] where a set of branchlets
theless, the resulting interpolation is essentially a smooth
is defined by using simple L-Systems and is stored in a
generalized cylinder (inherently parameterized) that lacks
repository.
fine details and realism. Thus, we enhance the interpolat-
ed surface with detailed geometry by applying geometric
textures from our database. We do this by extracting the 5 R ESULTS AND D ISCUSSION
fine details from a given example and map them onto the We demonstrate our method by modeling a large variety of
smooth surface (Figure 10). trees, focusing on their complex branching structures and
Given a sample tree cut from our repository, we transfer geometric details (see Figure 1). All results were modeled
its fine geometric details into a texture which we then apply by non-professionals with limited botanic and modeling
to our surface. First, we compute a cylindrical parametriza- expertise. To qualitatively and quantitatively evaluate our
tion of the tree cut by fitting cylinders along the tree- method, we provide some visual results and comparisons
cut skeleton. We then project the fine geometry to this as well as performing a user study.
cylinder and obtain a cylindrical parametrization of the Modeling of complex trees consisting of detailed branch-
geometry which is represented as a displacement map from es and geometrical structures stayed well below 20 minutes.
the cylindrical base. A similar cylindrical parametrization is This, however, did not include the foliage computation
applied to the branch surface, thus reducing detail transfer within the lobes, as it was done in a preprocessing step and
to a conventional texture mapping and scaling problem. We took up to one hour using our non-optimized code.
transfer a geometric texture simply by scaling it to fit the Table 1 summarizes statistics of our modeling interac-
target cylinder. If scaling is too large, we simply tile the tion. The number of cuts that the user was required to
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 7

(a) (b) (c) (d)

Fig. 11. Interactive foliage modeling. Main branches computed from user tree-cuts (a) are enhanced with user sketched lobes (b). In the lobe volume
we grow twigs in a recursive manner for 4 iterations (c). Tiny branchlets and leaves are added to twigs endpoints (d).

Fig. 12. Modeling a Ficus tree with complex branching structures and geometric variety. Left-to-right are the tree-cuts, detailed views on the cuts
and their completion, full tree structure and complete tree with foliage.

position is very low even for complex trees, demonstrating to represent the Ficus faithfully.
the expressiveness of our technique. We run our method on We demonstrate that our method can be used to enhance
a QuadCore Intel i5-3210M CPU 2.5Ghz with 4GB RAM. All the realism of results generated by state-of-art systems. In
trees are well detailed, represented by a large number of Figure 14(a) we inflate a cylindrical skeleton and transfer
triangles. geometric details from our database, resulting in a realistic
Figure 1 shows excerpts from a modeling sequence tree with rich surface geometries. In Figure 14(b), we re-
demonstrating the mid-grain level of our tool, balancing model a tree generated by the commercial software Xfrog.
between fine-level quality details and effectiveness. Here, Obviously, since we use tree-cuts from real tree examples,
only a small number of tree cuts is sufficient to generate a our method generates more compelling, realistic and com-
detailed and complex branching structure. In Figure 2 some plex tree geometry. In Figure 14(c), we re-model a bonsai tree
more details about the tree-cuts and the 2D sketching for the created using Google Sketchup. The side-by-side compari-
lobe construction are given. son suggests utilizing our method to transform low-quality
Figure 13 demonstrates the modeling power of our tech- tree models into realistic models with high-level details and
nique compared with real-life trees. In 13 (top) we model a complex structures.
bonsai tree by following and approximating its 3D structure Our tree-cut editing framework is not restricted to cut
from a 2D photograph. Similarly, in 13 (bottom), we model connecting operations. In Figure 15 we demonstrate con-
a dense Cercis tree. We are also able to reproduce even the necting a set of tree-cuts to arbitrary regions on the surface
highly dense branching structure at the top of the tree by of a main trunk. Our method is general and merely requires
simply positioning small branches without much effort. to define positions on the surface. Our method then defines
In Figures 1, 12 we modeled two realistic Ficus trees, a local loop on the surface which connects to the tree-cuts
zooming into their geometric fine details and branch struc- using the the same process.
tures. From a relatively small number of interactive edits, We also compare our method with professional tree
mostly positioning of tree cuts, we obtain a fairly complex editing systems. We asked an expert tree modeler to model
branching structure with large variety and enough details a tree similar to 16 (a,b) in terms of number of branches
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 8

Fig. 13. Modeling of Bonsai (top) and Cercis (bottom) trees from given 2D photographs. Left-to-right, the 2D photograph, input tree-cuts, full tree
structure and with foliage.

(a) (b) (c)

Fig. 14. Realism enhancement of existing trees: in (a) we enhance a simple skeleton (left) with geometric texture (right). Tree models from Xfrog
(left of (b)) and Google Sketchup (left of (c)) are re-modeled using our tree-cuts and geometric detail (right of (b) and (c)).

and their configuration. Using Xfrog and ZBrush it took connection of the cuts and the lobe-based modeling of small
the modeler more than 2 hours to create the tree model structures easy and intuitive.
in Figure 16 (c,d). Their main efforts lied in editing and To evaluate controllability and expressiveness factors,
adjusting the tree to match the target tree. In contrast, we designed two tests for each participant. First, modeling
using our method, it took approximately 20 minutes to a tree from a photo within a bounded time of 5 minutes. We
generate similar looking branching structures in terms of observe that the resulting models bear good resemblance
their configuration and complexity. with the target photo (see Figure 17). Second, creating at
least two complex and expressive trees in a bounded time
5.1 User Study. of 20 minutes. Figure 18 summarizes these results where
eight out of ten participants are able to generate three trees
We performed a user study to obtain feedback and evaluate
given 20 minutes. The gallery in Figure 18 demonstrates the
our system. Ten non-professional participants received a
large expressivity of our tool as a large variety of complex
short introductory to the system (10 minutes). Our tool
trees was accomplished in a relatively short time.
requires not more than two basic interaction actions of cut-
positioning in 3D and specifying branch connections. The
subjects had basic modeling experience, thus they familiar- 6 C ONCLUSIONS AND FUTURE WORK
ized easily with the system and reported no problems with We presented an example-based method for generating
positioning and orienting cuts in 3D. All subjects found the realistic tree models with complex branching structures.
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 9

Fig. 15. Connecting branches arbitrary. Our system allows connecting branches to arbitrary surface regions. Here, the user loosely positions
tree-cuts in space which connect regions in the surface of a simple trunk.

Fig. 16. We compare a tree created by our tool (a,b) with a tree created professionally by Xfrog (c,d). While both trees resemble in terms of their
branching structures and geometric complexity, our tree was created faster and required less expertise.

Fig. 17. User study excerpts, showing photography guided tree models achieved by users. Small figures left to trees depict the applied tree-cuts.

From a collection of example tree models, sub-structures ning more trees from the real-world, but also by creative
(named tree-cuts) are extracted such that a new tree model editing of existing examples. Second, we plan to further
is produced after the user manually placed selected tree- make the user interaction more intuitive and effortless, so
cuts. Our system is capable of generating tree models that that the user is able to concentrate more on creativity. This
inherit geometry nuances of the example models, which are can be achieved by leveraging state-of-the-art procedural
normally missing in the existing systems, but are important methods and using the underlying procedural principles to
to human eyes for the perceived realism. suggest the placement of new tree-cuts. Third, our interac-
We were only able to acquire a subset of existing trees, tion could be further enhanced using a suggestive system
due to the limitations in scanning large tree structures in to assist tree editing. Specifically, our system would analyze
uncontrolled environments. Additionally, our system cur- the tree on-the-fly and suggest tree cuts at different locations
rently ignores the interrelations between branches, which based on the current status. Cuts may be automatically po-
may be computed using geometrical, physical, or biologi- sitioned and aligned at feasible locations, thus significantly
cal constraints. The system could be further amended by reducing the user’s workload.
tropisms and other global factors.
There are several ways to further improve our system.
First, our example-base can be enriched not only by scan-
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 10

Fig. 18. A gallery of 28 complex tree models generated by 10 user study participants using our interactive tree modeling tool within 20 minutes.

ACKNOWLEDGMENTS R EFERENCES

We thank all the reviewers for their valuable comments and [1] O. Deussen and B. Lintermann, Digital Design of Nature:
constructive suggestions. We are grateful to Jianwei Guo Computer Generated Plants and Organics. Springer, 2010.
[2] M. Okabe, S. Owada, and T. Igarashi, “Interactive design of botan-
for his help on tree rendering. This work was supported in ical trees using freehand sketches and example-based editing.”
part by NSFC (61522213, 61232011, 61502471), 973 Program Computer Graphics Forum, pp. 487–496, 2005.
(2015CB352501), Guangdong Science and Technology Pro- [3] X. Chen, B. Neubert, Y.-Q. Xu, O. Deussen, and S. B. Kang,
gram (2015A030312015, 2014B050502009, 2014TX01X033), “Sketch-based tree modeling using markov random field,” ACM
Trans. on Graphics, pp. 109:1–109:9, 2008.
Shenzhen Innovation Program (JCYJ20151015151249564, [4] Speedtree, “Speedtree web site, interactive data visualization, inc.”
CXB201104220029A), National Foreign 1000 Talent Plan http://www.speedtree.com/.
(WQ201344000169) and the Israel Science Foundation. [5] Y. Livny, F. Yan, M. Olson, B. Chen, H. Zhang, and J. El-Sana,
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 11

“Automatic reconstruction of tree skeletal structures from point [31] T. Funkhouser, M. Kazhdan, P. Shilane, P. Min, W. Kiefer, A. Tal,
clouds,” ACM Trans. on Graphics, pp. 151:1–151:8, 2010. S. Rusinkiewicz, and D. Dobkin, “Modeling by example,” in Proc.
[6] Y. Livny, S. Pirk, Z. Cheng, F. Yan, O. Deussen, D. Cohen-Or, of ACM SIGGRAPH, 2004, pp. 652–663.
and B. Chen, “Texture-lobes for tree modelling,” ACM Trans. on [32] P. Merrell, “Example-based model synthesis,” in Proceedings of
Graphics, pp. 53:1–53:10, 2011. the 2007 Symposium on Interactive 3D Graphics and Games, ser.
[7] P. Prusinkiewicz and A. Lindenmayer, The Algorithmic Beauty of I3D ’07, 2007, pp. 105–112.
Plants (The Virtual Laboratory). Springer, 1996. [33] N. Marechal, E. Galin, E. Gurin, and S. Akkouche, “Component-
[8] J. Bloomenthal, “Modeling the mighty maple,” Proc. of ACM based model synthesis for low polygonal models,” in Proceedings
SIGGRAPH, pp. 305–311, 1985. of Graphics Interface 2010, 2010, pp. 217–224.
[9] P. E. Oppenheimer, “Real time design and animation of fractal [34] M. Kazhdan and H. Hoppe, “Screened poisson surface reconstruc-
plants and trees,” Proc. of ACM SIGGRAPH, pp. 55–64, 1986. tion,” ACM Trans. on Graphics, vol. 32, no. 3, p. 29, 2013.
[10] M. Holton, “Strands, gravity and botanical tree imagery,” [35] O. K.-C. Au, C.-L. Tai, H.-K. Chu, D. Cohen-Or, and T.-Y.
Computer Graphics Forum, pp. 57–67, 1994. Lee, “Skeleton extraction by mesh contraction,” ACM Trans. on
Graphics, pp. 44:1–44:10, 2008.
[11] F. Boudon, P. Prusinkiewicz, P. Federl, C. Godin, and R. Karwows-
[36] T. Funkhouser, P. Min, M. Kazhdan, J. Chen, A. Halderman,
ki, “Interactive design of bonsai tree models.” Comput. Graph.
D. Dobkin, and D. Jacobs, “A search engine for 3d models,” ACM
Forum, pp. 591–600, 2003.
Trans. on Graphics, vol. 22, pp. 83–105, 2003.
[12] A. Reche-Martinez, I. Martin, and G. Drettakis, “Volumetric recon-
[37] J. Lee and T. Funkhouser, “Sketch-based search and composition
struction and interactive rendering of trees from photographs,”
of 3d models,” in Proceedings of Sketch-Based Interfaces and
ACM Trans. on Graphics, vol. 23, no. 3, pp. 720–727, 2004.
Modeling, ser. SBM’08, 2008, pp. 97–104.
[13] B. Neubert, T. Franken, and O. Deussen, “Approximate image- [38] S. M. Yoon, M. Scherer, T. Schreck, and A. Kuijper, “Sketch-based
based tree-modeling using particle flows,” ACM Trans. on 3d model retrieval using diffusion tensor fields of suggestive
Graphics, vol. 26, no. 3, pp. Article 71, 8 pages, 2007. contours,” in Proceedings of the International Conference on
[14] I. Shlyakhter, M. Rozenoer, J. Dorsey, and S. Teller, “Reconstructing Multimedia, 2010, pp. 193–200.
3d tree models from instrumented photographs,” IEEE Comput. [39] T. Shao, W. Xu, K. Yin, J. Wang, K. Zhou, and B. Guo, “Discrimina-
Graph., vol. 21, no. 3, pp. 53–61, 2001. tive sketch-based 3d model retrieval via robust shape matching.”
[15] P. Tan, G. Zeng, J. Wang, S. B. Kang, and L. Quan, “Image-based Comput. Graph. Forum, vol. 30, pp. 2011–2020, 2011.
tree modeling,” ACM Trans. Graph., pp. Article 87, 8 pages, 2007. [40] M. Eitz, R. Richter, T. Boubekeur, K. Hildebrand, and M. Alexa,
[16] A. Runions, B. Lane, and P. Prusinkiewicz, “Modeling trees with “Sketch-based shape retrieval,” ACM Trans. on Graphics, vol. 31,
a space colonization algorithm,” in Proceedings of Eurographics no. 4, pp. 31:1–31:10, 2012.
Workshop on Natural Phenomena 2007, 2007, pp. 63–70. [41] B. K. P. Horn, “Closed-form solution of absolute orientation using
[17] P. Tan, T. Fang, J. Xiao, P. Zhao, and L. Quan, “Single image tree unit quaternions,” Journal of the Optical Society of America A,
modeling,” ACM Trans. on Graphics, vol. 27, no. 5, pp. Article 108, vol. 4, no. 4, pp. 629–642, 1987.
7 pages, 2008. [42] P. Jaccard, “Eine neue Auffassung ueber die Ursachen des Dick-
[18] W. Palubicki, K. Horel, S. Longay, A. Runions, B. Lane, R. Měch, enwachstums der Baeume,” Naturwiss. Z. fuer. Landwirtschaft,
and P. Prusinkiewicz, “Self-organizing tree models for image no. 13, pp. 321–360, 1913.
synthesis,” in Proc. of ACM SIGGRAPH, 2009, pp. 58:1–58:10.
[19] J. Wither, F. Boudon, M.-P. Cani, and C. Godin, “Structure from
silhouettes: a new paradigm for fast sketch-based design of trees.”
Comput. Graph. Forum, pp. 541–550, 2009.
[20] H. Xu, N. Gossett, and B. Chen, “Knowledge and heuristic-
based modeling of laser-scanned trees,” ACM Trans. on Graphics,
vol. 26, no. 4, pp. Article 19, 13 pages, 2007.
[21] S. Pirk, O. Stava, J. Kratt, M. A. M. Said, B. Neubert, R. Měch,
B. Benes, and O. Deussen, “Plastic trees: interactive self-adapting Ke Xie is Ph.D. Candidate of Shenzhen Insti-
botanical tree models,” ACM Trans. on Graphics, pp. 50:1–50:10, tutes of Advanced Technology (SIAT), Chinese
2012. Academy of Sciences(CAS), majoring in com-
[22] A. Bucksch and R. Lindenbergh, “Campino – a skeletoniza- puter graphics, focusing on point cloud scanning,
tion method for point cloud processing,” ISPRS journal of processing, 3D modeling, being co-supervised
photogrammetry and remote sensing, vol. 63, no. 1, pp. 115–127, by Prof. Baoquan Chen and Prof. Hui Huang.
2008.
[23] A. Bucksch, R. Lindenbergh, and M. Menenti, “Skeltre - fast
skeletonisation for imperfect point cloud data of botanic trees,”
in Proceedings of Eurographics Workshop on 3D Object Retrieval,
2009, pp. 13–27.
[24] J.-F. Côté, J.-L. Widlowski, R. A. Fournier, and M. M. Verstraete,
“The structural and radiative consistency of three-dimensional
tree reconstructions from terrestrial lidar,” Remote Sensing of
Environment, pp. 1067 – 1081, 2009.
[25] P. Raumonen, M. Kaasalainen, S. Kaasalainen, H. Kaartinen,
M. Vastaranta, M. Holopainen, M. Disney, P. Lewis, and et al.,
“Fast automatic precision tree models from terrestrial laser scan-
ner data,” Remote Sensing, vol. 5, pp. 491–520, 2013.
[26] X. Zhang, H. Li, M. Dai, W. Ma, and L. Quan, “Data-driven Feilong Yan received the bachelor degree in
synthetic modeling of trees,” IEEE Trans. Vis. & Mach. Comp. Computer Science from Xidian University in
Graphics, pp. 1214–1226, 2014. 2008, and received his Ph.D degree in Computer
[27] M. Okabe, S. Owada, and T. Igarashi, “Interactive design of botan- Science from Shenzhen Institutes of Advanced
ical trees using freehand sketches and example-based editing,” in Technology (SIAT), Chinese Academy of Sci-
ACM SIGGRAPH 2006 Courses, 2006. ences (CAS), co-supervised by Prof. Baoquan
Chen and Prof. Hui Huang. His research inter-
[28] S. Longay, A. Runions, F. Boudon, and P. Prusinkiewicz,
ests include computer graphics, plant modeling
“Treesketch: Interactive procedural modeling of trees on a tablet,”
and object capture and reconstruction.
in Proceedings of the International Symposium on Sketch-Based
Interfaces and Modeling, 2012, pp. 107–120.
[29] A. Sharf, M. Alexa, and D. Cohen-Or, “Context-based surface
completion,” ACM Trans. on Graphics, pp. 878–887, 2004.
[30] G. Harary, A. Tal, and E. Grinspun, “Context-based coherent
surface completion,” ACM Trans. on Graphics, pp. 5:1–5:12, 2014.
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 12

Andrei Sharf received a B.Sc. in Computer Sci-

ence from the Technion Institute in 1999, and
an M.Sc in Computer Science from the Tel-Aviv
University in 2002. After then he completed his
Ph.D. on Surface Reconstruction Techniques for
Imperfect Raw Data in 2007 under the supervi-
sion of Daniel Cohen-Or and Ariel Shamir and
did his PostDoc with Prof. Nina Amenta at the
Institute for Data Analysis and Visualization (I-
DAV), University of California at Davis. His cur-
rent research interests include surface recon-
struction, geometric modeling, interactive applications and parallel al-
gorithms. He is awarded as the CAS Foreign Young Scientist in 2009,
2010 and 2013.

Oliver Deussen Prof. Deussen graduated at

Karlsruhe Institute of Technology and is profes-
sor at University of Konstanz (Germany) and
visiting professor at the Chinese Academy of
Science in Shenzhen. He serves as Co-Editor
in Chief of Computer Graphics Forum and is
Vice-President of Eurographics Association. His
areas of interest are modeling and rendering of
complex biological systems, non-photorealistic
rendering as well as Information Visualization.
He also contributed papers to geometry pro-
cessing, sampling methods, and image-based modeling.

Baoquan Chen is a Professor of School of Com-

puter Science and Technology, Shandong Uni-
versity. He received his MS in Electronic Engi-
neering from Tsinghua University, Beijing (1994),
and a second MS (1997) and then PhD (1999)
in Computer Science from the State Universi-
ty of New York at Stony Brook. He is the re-
cipient of the Microsoft Innovation Excellence
Program 2002, NSF CAREER award 2003, M-
cKnight Land-Grant Professorship 2004, IEEE
Visualization Best Paper Award 2005, Chinese
Academy of Sciences 100 Talents Program 2008, and NSFC Out-
standing Young Researcher program in 2010. His research interests
generally lie in computer graphics, visualization, and human-computer
interaction, focusing specifically on large-scale city modeling, simulation
and visualization.

Hui Huang is a Professor of Shenzhen Insti-

tutes of Advanced Technology (SIAT), Chinese
Academy of Sciences (CAS), where she is also
the Deputy Director of the Institute of Advanced
Computing and Digital Engineering and the Di-
rector of the Visual Computing Research Center
and the Director of the Shenzhen Key Lab of
Visual Computing and Visual Analytics. She re-
ceived her PhD in Applied Math from The Univer-
sity of British Columbia in 2008 and another PhD
in Computational Math from Wuhan University
in 2006. She is the recipient of NSFC Excellent Young Researcher
program and Guangdong Technology Innovation Leading Talent award
in 2015. Her research interests are mainly on Computer Graphics and
Scientific Computing. She is now the associate Editor-in-Chief of The Vi-
sual Computer and serves on the editorial board of Computer Graphics
Forum and Frontiers of Computer Science.