1

A Constrained Shortest Path Scheme for

Virtual Network Service Management
Dmitrii Chemodanov* , Flavio Esposito†* , Prasad Calyam* , Andrei Sukhov‡
*
University of Missouri, USA; † Saint Louis University, USA; ‡ Samara National Research University, Russia;
Email: [email protected], [email protected], [email protected], [email protected]

Abstract—Virtual network services that span multiple data infrastructure [3], [4]. Another management protocol example
centers are important to support emerging data-intensive applica- pertains to Network Function Virtualization service chaining
tions in fields such as bioinformatics and retail analytics. Success- (NFV-SC) [5], [6], in which a network manager is required
ful virtual network service composition and maintenance requires
flexible and scalable ‘constrained shortest path management’ to place Virtual Network Functions (e.g., firewalls, load bal-
both in the management plane for virtual network embedding ancers, etc) and setup a path across corresponding software-
(VNE) or network function virtualization service chaining (NFV- defined middleboxes to guarantee different high-level traffic
SC), as well as in the data plane for traffic engineering (TE). In constraints i.e., policies. While virtual network services are
this paper, we show analytically and empirically that leveraging operating, network managers need to also deploy, within the
constrained shortest paths within recent VNE, NFV-SC and TE
algorithms can lead to network utilization gains (of up to 50%) ‘data plane’, traffic engineering (TE) techniques to maintain
and higher energy efficiency. The management of complex VNE, profiles or improve network utilization [7].
NFV-SC and TE algorithms can be, however, intractable for Successful virtual network service composition and mainte-
large scale substrate networks due to the NP-hardness of the nance in both management plane and data plane mechanisms
constrained shortest path problem. To address such scalability
challenges, we propose a novel, exact constrained shortest path
requires scalable and flexible ‘constrained shortest path man-
algorithm viz., ‘Neighborhoods Method’ (NM). Our NM uses agement’ for the following reasons. The constrained shortest
novel search space reduction techniques and has a theoretical path problem is the NP-hard problem of finding the shortest
quadratic speed-up making it practically faster (by an order path that satisfies an arbitrary number of end-to-end (or path)
of magnitude) than recent branch-and-bound exhaustive search constraints [8], and its existing exact solutions are commonly
solutions. Finally, we detail our NM-based SDN controller im-
plementation in a real-world testbed to further validate practical
based on branch-and-bound exhaustive search algorithms [8],
NM benefits for virtual network services. [9], [10] or integer programming. These techniques have
exponential complexity, and hence, limit scalability of the
Index Terms—Constrained Shortest Path, Virtual Network
Embedding, NFV Service Chaining, Traffic Engineering.
constrained shortest path management. Such scalability lim-
itations are exacerbated by the complexity of VNE, NFV-SC
and TE algorithms as well as the potentially large scale of sub-
I. I NTRODUCTION
strate networks. Although some heuristics or approximation

T HE advent of network virtualization has enabled new

business models allowing infrastructure providers to
share or lease their physical networks that span multiple data
algorithms can find constrained shortest paths in polynomial
time (at expense of optimality [11]), these algorithms support
only a specific number of constraints, or a specific cost to
centers. Network virtualization is being increasingly adopted optimize [12], [13], [14]. Thereby, they limit the flexibility of
to support data-intensive applications within enterprises (e.g., the constrained shortest path management.
retail analytics) and academia (e.g., bioinformatics and high By leveraging constrained shortest paths, we show how we
energy physics) over wide-area federated infrastructures such can enhance network utilization and energy efficiency of VNE,
as the VMware Cloud [1] and the Global Environment for NFV-SC and TE services by both analytical and empirical
Network Innovations (GENI) [2]. A major challenge for in- means. Reasons for the observed benefits are as follows:
frastructure providers is to offer virtual network services that Firstly, by using a constrained shortest path, we can minimize
meet the application Service Level Objective (SLO) demands a provider’s cost associated with flows allocation subject to
on shared (constrained) physical networks. Examples of SLO the application SLO constraints in TE [15], [16]. Such costs
demands can refer to technical constraints such as bandwidth, potentially hinder long-term infrastructure providers’ revenue
high reliability, or low latency. maximization; this can be understood using a simple example:
In order to compose and maintain virtual network services, a path comprising of two physical links to maintain a single
infrastructure providers need to run management protocols flow has a higher allocation cost than an alternative solution
(within the ‘management plane’) such as e.g., Virtual Network that uses only one physical link. For example, if the flow SLO
Embedding (VNE) to satisfy users’ virtual network requests. demand is 10 Mbps, a path composed by a single physical
VNE is the NP-hard graph matching problem of mapping link would need to provision only 10 Mbps, while a path
a constrained virtual network on top of a shared physical composed by two links would require 20 Mbps. Secondly, a
This work has been partially supported by the National Science Foundation more scalable and flexible constrained shortest path approach
awards CNS-1647084, CNS-1647182, the Coulter Foundation Translational can be also beneficial for VNE and NFV-SC [17], [18], [19]
Partnership Program and by RFBR according to the research project 16-07- when virtual links are subject to an arbitrary number of
00218a and the public tasks of the Ministry of Education and Science of the constraints such as bandwidth, latency and loss rate. Other
Russian Federation (2.974.2017/4.6). Any opinions, findings or conclusions
expressed in this publication are those of the author(s) and do not necessarily types of constraints can also be imposed by optimization
reflect the views of the funding agencies. methods, such as the column generation approach, typically
Digital Object Identifier: 10.1109/TNSM.2018.2865204

1932-4537 c 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
 2

used to speed-up integer programming [8], [17], [18]. This Section III shows importance of the constrained shortest path
is because in the aforementioned cases, constrained shortest management in VNE/NFV-SC and TE. In Section IV, we
paths can be part of the optimal solution, i.e., such paths can present details of our NM approach. The complexity improve-
best improve the objective value under arbitrary constraints. ments of our NM w.r.t. recent branch-and-bound exhaustive
Our Contributions. To achieve the constrained shortest path search algorithms are presented in Section V. Section VI
benefits and address their management flexibility and scalabil- describes our NM prototype implementation. Section VII
ity challenges in virtual network services, we propose a novel describes our evaluation methodology, performance metrics
and exact constrained shortest path algorithm viz., ‘Neighbor- and results. Section VIII concludes the paper.
hoods Method’ (NM). Our NM is based on a novel double-
pass search space reduction technique that synergizes dynamic II. T HE C ONSTRAINED S HORTEST PATH P ROBLEM
programming with a branch-and-bound exhaustive-search. In The constrained shortest path problem is the NP-hard
addition, we propose a novel infeasibility pruning technique problem of finding the shortest path subject to an arbitrary
viz., “Look Back”, which benefits from NM’s double pass set of hop-to-hop and end-to-end constraints. In this section,
design to further ease the constrained shortest paths finding we define this problem using optimization theory. In the
in practice. Our NM is solving an NP-hard problem, so it is subsequent section, we motivate the importance of its flexible
exponential in its general form. However, our computational and scalable management in diverse virtual network services.
complexity analysis shows that NM has a quadratically lower
complexity upper-bound (halved exponent) than alternative A. Problem Overview
methods. Moreover, when synergistically used with existing
The problem of providing a (shortest) path with multiple
search space reduction techniques [8], [9], [10], our scalability
(SLO) constraints is NP-hard [8], and its complete survey can
evaluation results indicate how NM is faster by an order of
be found in [11]. Herein, we mention a few representative
magnitude than recent branch-and-bound exhaustive search
solutions that help us present our novel contributions. Most
methods, and hence, scales better.
heuristics group multiple metrics into a single function reduc-
Furthermore, NM is flexible due to its adaptable perfor- ing the problem to a single constrained routing problem [24],
mance and applicability to different constrained shortest path and then solve the routing optimization separately, e.g., us-
scenarios with an arbitrary set of (SLO) constraints and an ing Lagrangian relaxation [25]. The exact pseudo-polynomial
arbitrary cost function. For example, when we allocate traffic algorithm proposed by Jaffe et al. [14] offers a distributed
flow requests with a single path and multiple link constraints, path finder solution limited to a two-path constraints problem.
NM can find some constrained shortest path variants in Wang et al. [20] use an extended version of Dijkstra algo-
polynomial time. Thus, it can substitute diverse existing path rithm (EDijkstra), where all links with infeasible hop-to-hop
finder algorithms such as the extended version of Dijkstra [20] constraints are excluded. EDijkstra runs in polynomial time
and the iterative version of Bellman-Ford [12]. In its general but may omit any (path hop count) minimization, desirable
form, NM can be also used to speed-up finding of all loop- in network virtualization to optimize the physical network
free, k-constrained shortest or Pareto-optimal paths [9] from utilization. To minimize the path hop count under a single path
the source to the destination. Thus, NM is also applicable to constraint (e.g., delay) an iterative modification of Bellman-
diverse virtual network services including those with splittable Ford (IBF) algorithm was proposed in [12]. Our approach is
and unsplitable flows. We demonstrate such flexibility with not limited to a single path constraint and can be adapted to
an extensive numerical simulation campaign, testing NM over subsume both EDijkstra and IBF as we discuss in Section V.
diverse network topologies for both online VNE/NFV-SC The authors in [10] propose an exact algorithm for the
with unsplittable flows and for TE with splittable flows. We constrained shortest path problem, and apply several search
found that the number of embedded VN requests and energy space reduction techniques such as dominated paths (pruning
efficiency (and thereby the providers’ revenue) can increase by dominance and bound) and the look-ahead (pruning by
when the constrained shortest path management is used for infeasibility) notion for the exponential complexity exhaustive
tested VNE/NFV-SC solutions: either one-shot centralized search, utilizing the k-shortest path algorithm. We also apply a
(i.e., with joined node and link embedding) [17], [18], [19] similar technique to reduce the constrained path search space,
or two-stages distributed (i.e., with separate node and link though without any look-ahead, since it is computationally
embedding) [21]. When using the constrained shortest path expensive. Instead, our design uses a more efficient “Look-
management within either linear programming [22] or greedy- Back” pruning technique (see Section IV-B). In [26], the
based TE solutions [15], [16], our simulation results indicate authors propose an Exhaustive Breath-First Search (EBFS)
gains of up to 50% in network utilization and lower energy based approach to solve the constrained path finder problem,
consumption in some cases. focussing on delay. Another more recent work [9] also uses
Finally, we implement an open-source Software-Defined EBFS with a dominant path space reduction technique to find
Networking (SDN) based NM controller that is available multi-criteria Pareto-optimal paths. Alternatively, an exhaus-
at [23]. Our GENI [2] evaluation experiments with our imple- tive Depth-First Search (EDFS) can be used as a branch-and-
mentation prototype confirm our analytical and empirical find- bound algorithm. For example, the authors in [8] proposed
ings in real-world settings and show no constrained shortest the “pulse” algorithm that uses EDFS with dominated paths
path overhead (sought by NM) on the virtual network service and look-ahead search space reduction techniques. Both of
management and data plane mechanisms at large scale. EBFS and EDFS algorithms have exponential worst case time
Paper organization. In Section II, we formally state the complexity. Our solution however quadratically reduces the
constrained shortest path problem using optimization theory. worst case complexity of these algorithms (see Section V).
 3

B. Constrained Shortest Path Problem all fij = 0, which corresponding physical edge eij capacity
uij is not sufficient to allocate flow demand D. The same
Let l be the number of hop-to-hop or link constraints for
applies for other link constraints (see Equation 3).
min/max network metrics, e.g., bandwidth, and p be the num-
ber of end-to-end path constraints for additive/multiplicative In the next section we show how finding a flexible and
network metrics, e.g., delay or loss. Moreover, we denote with scalable solution to Problem 1 benefits a wide range of path
l ⊕ 1 paths with multiple links and a single path constraints, finder subproblems to manage virtual network services.
and with and l ⊕ p paths with multiple links and multiple
path constraints. Given the above notation, we define the III. C ONSTRAINED S HORTEST PATH FOR V IRTUAL
constrained shortest path problem as follows: N ETWORK S ERVICE M ANAGEMENT
Using optimization theory, in this section we motivate the
Problem 1 (constrained shortest path). Given a physical
need for a flexible and scalable constrained shortest path to
network graph G = (V, E), where V is the set of vertices
manage virtual network services such as Traffic Engineering
and E the set of edges; let us denote with D the flow demand
(TE), Virtual Network Embedding (VNE) and NFV Service
to be transferred and let uij denote a capacity of the directed
Chaining (NFV-SC). We also show how such a constrained
edge eij ; let fij be a binary variable fij ∈ {0, 1} denoting
shortest path scheme does not introduce any additional inter-
a ratio of flow on the edge eij , and let cij denote a cost
operability issues for aforementioned virtual network services
of transferring a unit of flow through such edge; finally, let
¯l and p̄ denote vectors of link (hop-to-hop) and path (end- with respect to traditional shortest path schemes.
to-end) constraints, excluding capacity constraints, where ¯l A. Finding Virtual Paths in Resource Constrained Scenarios
l̄
corresponds to min/max edge eij weights wij (i.e., ≥ or ≤,
We begin by considering a variant of the constrained short-
respectively), and p̄ corresponds to additive edge eij weights 1 est path that operates on a resource constrained scenario, e.g.,
p̄
wij ; the problem of finding constrained shortest path between a natural or man made disaster scenario where connectivity
source vs and destination vt vertices can be formulated as is scarce. In those cases, one aim is to minimize the overall
follows: X physical resource consumption of virtual paths. To this end,
minimize cij Dfij (1) we define the resource optimal constrained path by modifying
eij ∈E the objective of Problem 1 as follows:
subject to Problem 2 (resource optimal constrained path). The resource
Flow Conservation Constraints: optimal constrained path is a path that satisfies an arbitrary

X X 1,
 i=s set of link/path constraints using minimal amount of physical
fij − fki = 0, i=6 s or t , ∀vi ∈ V (2) bandwidth: X
vj ∈V vk ∈V

−1, i = t minimize Dfij (7)
eij ∈E
Capacity Constraints:
where fij , eij , D and E are as defined in Problem 1.
Dfij ≤ uij , ∀eij ∈ E (3)
Note that by defining an equal weight c to all edges we
Other Link Constraints: seek the minimum hop path that satisfies an arbitrary set of
link/path constraints (e.g., imposed by SLO).
l
wij fij ≤ l, ∀eij ∈ E, l ∈ ¯l (4)
Path Constraints: B. Traffic Engineering
p
X
wij fij ≤ p, ∀p ∈ p̄ (5) TE techniques today can be roughly divided into two
eij ∈E groups: oblivious i.e., no a-priori knowledge of the SLO
demands [28], and demands-aware, when such knowledge is
Existential Constraints:
available [15], [16]. Moreover, the later has a superior per-
fij ∈ {0, 1}, ∀eij ∈ E (6) formance (e.g., can better utilize substrate network resources)
than the former [28] at the expense of having a centralized
Finding a shortest path (without constraints) has a poly- forwarding (or routing) control [15], [16], [29].
nomial time complexity: consider Equations 3, 4 and 5: We broadly classify demands-aware traffic engineering so-
in absence of any link or path constraints, the constraint lutions (see e.g. [15], [16]) with the following network utility
matrix of the above optimization problem is unimodular [27]. maximization problem:
This condition allows us to solve the optimization problem
using (polynomial) linear programming. Such time complexity maximize[min f airnessi (fi )] (8)
fi ∈F
bound does not necessarily hold in presence of at least a single
link or path constraint (¯l 6= 0 or p̄ 6= 0). In that case, we where F denotes a set of all demands (or commodities);
have to solve the above optimization problem using (NP-hard) in [16], for example, such commodities are {src, dst, SLO}
integer programming [27]. Note that we can always avoid tuples; fi ∈ [0, 1] is continuous variable that denotes the ratio
specifying capacity constraints in Equation 3 by simply setting of flow for commodity i with bandwidth demand Di ; and
f airnessi (f ) is a linear piecewise-defined function whose
1 Note that multiplicative constraints (e.g., packet loss) can be converted to definition is based on path service’s demands SLO constraints.
additive by composing them with a logarithmic function to avoid nonlinearity. For a complete problem formulation we refer to [22], [30].
 4

Flow1 [A-B], SLOs: 10 Gbps, 20 ms C. Embedding Virtual Networks and Service Chains
Flow2 [A-C], SLOs: 10 Gbps, 20 ms
Embedding a virtual network (service chain) requires a con-
A A A
10G 10G 0G strained virtual service request to be mapped on top of a physi-
15ms 15ms 15ms
cal network hosted by a single infrastructure provider, or by a
10G 0G 0G
B 5ms B 5ms B 5ms federation of providers. To solve this (NP-hard [32]) graph
matching problem, earlier work proposed centralized (see
10G 0G 10G
5ms C 5ms C 5ms C e.g. [17], [18], [19], [33], [34], [35], [36]) and distributed [21],
[37], [38] algorithms. Such solutions either separate the node
Constrained
Shortest Path Shortest embedding from the link embedding phase [21], [33], [37],
Path
Allocated Flows: Flow1, Flow2 Flow1, Flow2 [38], or simultaneously apply the two phases [17], [18], [19],
[36]. When link embedding is considered separately, paths
Fig. 1: Applying constrained shortest paths to allocate traffic flows improves can be found dynamically or precomputed for each virtual
network utilization: the widely applied shortest paths, e.g., to allocate traffic
flows, in this case F low1, can hinder allocation of other flows; allocating
link using Dijkstra or k-shortest path algorithms [21], [33],
F low1 on a constrained shortest path as in Problem 2 instead permits the [38]. When instead node and link embedding are considered
allocation of F low2 as well. jointly, recent work has shown how embedding problems can
be formulated as known multi-commodity flow problems [17],
Constrained shortest path relevance. There are two standard
[18], [19]. On the one hand, all these problems show how
ways of formulating the TE optimization problem shown in
there is a need to minimize providers’ management costs when
Equation 8 — the arc-based [30] and path-based [22] formu-
allocating the virtual network or the service chain [18] as
lations. In practice, due to hardware granularity limitations,
shown in the following objective:
the arc-based linear programming solution can be infeasible X X X X
st
to implement, or require use of NP-hard integer programming minimize cij Dst fij + ci Ds xsi (9)
which can be intractable even for moderate size networks. est ∈EV eij ∈ES vs ∈VV vi ∈VS
The shortest path algorithms such as Dijkstra (e.g., within
k-shortest path algorithms [31]) are currently used to find a where VS and ES (VV and EV ) denote sets of physical
set of paths as an input for the path-based linear programming (virtual) vertices and edges, respectively; cij and ci denotes
formulations or for their simplified greedy solutions [15], [16]. unitary bandwidth and CPU cost on physical edge eij , respec-
st
The hope is also to map the highest priority flows to the tively; fij ∈ [0, 1] is continuous variable that models the ratio
minimum latency paths first. However, if we are aware of the of flow from vs to vt virtual vertices transferred through the
services’ SLO demands (e.g., bandwidth, latency, loss rate, physical edge eij when a flow with demand Dst is splittable,
st
etc.), we can use them as constraints to optimize physical or, in its binary form, fij ∈ {0, 1} for unsplittable flows.
s
network utilization (and hence the flow fairness) as described Finally, xi ∈ {0, 1} is the binary variable that denotes whether
in Problem 2. the virtual vertex vs with computation demand Ds is assigned
Motivating example. Consider Figure 1: two flows F low1 to the physical vertex vi or not. For a complete problem
and F low2 with bandwidth and latency constraints are to be formulation we refer readers to [17], [18], [19]. On the other
allocated on a physical network of Figure 1-left. Attempting hand, when virtual network (or service chain) requests are
an allocation of F low1 by merely considering the shortest unknown in advance, the cost minimization strategy presented
path algorithm to minimize the latency (Figure 1-center) — as in Equation 9 can cause physical network partitioning, that in
current TE solutions do, — the hosting physical path would turn can lead to lower physical network utilization [17], [19].
have to use up to two physical links (A → C → B), thus To maximize the long term provider’s revenue i.e. to allocate
preventing the allocation of the subsequent requested flow more virtual network requests, often a load balancing objective
F low2. If instead a constrained shortest path is used to allocate is sought [17], [19]:
F low1 and F low2 as shown in Figure 1-right, then path X X cij Dst X X ci Ds
st
minimize fij + xs (10)
A → B could be found, leaving capacity for the allocation uij ui i
est ∈EV eij ∈ES vs ∈VV vi ∈VS
of F low2 (resulting in its higher fairness) as well.
On the contrary, when shortest path algorithms are used to where uij and ui denote available capacities of physical edge
find min-hop paths, such algorithms can lead to an infeasible eij and vertex vi , respectively.
solution even when a feasible one exists, leading to TE Constrained shortest path relevance. To cope with integer
performance degradation. For example, consider the mapping programming intractabilities for solving path-based multi-
of F low1 with a latency constraint of 10 ms (tighter than 20 commodity flow problems, the well-known column genera-
in Figure 1). The min hop (shortest) path A → B violates the tion approach can be applied as in [17], [18]. The column
10 ms latency constraint, preventing the allocation of F low1, generation approach iteratively adds only those paths (i.e.,
whereas the constrained shortest path algorithms would find flow variables or columns) to the problem formulation that
the feasible and optimal A → C → B path even though improve objective. In [17], [18] the Dijkstra shortest path
more hops are needed to satisfy such tighter SLO demand. algorithm is used to find the best shadow price paths (columns)
Note, however, that the above problem solution of allocating for each virtual link to include them in the formulation.
sequential flow requests can be suboptimal w.r.t. the problem Although such approach best improve the objective value of
solution of allocating such flows when they are known in e.g., Equation 10, it can be suboptimal when virtual links
advance. We dissect such network utilization gains when have additional constraints such as latency or loss rate. In
constrained shortest paths are applied for TE in Section VII. this scenario, a path between any two physical nodes (found
 5

Input-1:
Physical Graph Neighborhoods Method

Optimal Path
-Vertices V

Output:
Build Neighborhoods
-Edges E + Look Back
Input-2: Constrained
Perform Backward Pass
Shortest Path Request +Dominated Paths + Look Back
-<src, dst> (a) (b) (c)
Validate Constraints
-cost & constraints

Fig. 2: Neighborhood Method Workflow in l ⊕ p case: The kth -hop

Neighborhood Build (forward pass) first (using label-correcting), and the
Backward Pass later find all simple paths of k length (with any instance of
branch-and-bound exhaustive search); in the third step, we check candidates
feasibility; we then repeat recursively the backward pass with the k + 1 (d) (e)
neighborhoods, until the optimal (constrained shortest) path is found or all
candidates have been eliminated. To improve the time to solution, we couple Fig. 3: Running example of NM in l ⊕ p case: (a) An example network
NM with a dominated path and a look-back search space reduction technique. configuration with [bandwidth, delay, cost] link metrics; (b) NM forward pass
phase estimates the min hop count distance to Y during the first iteration,
by solving Problem 1) can be a part of the optimal solution, and (c) backward pass identifies all the shortest path candidates, which in
even if it has a worse objective value than the shortest path, this case do not contain the optimal path; (d) the forward pass adds more
but satisfies all virtual link constraints. Generally, for any neighborhoods to find a path with longer distance to Y. (e) The Backward
pass identifies all paths of a given new length; in this case the path contains
two stage or one-shot VNE algorithm, constrained shortest the optimal X → B → A → Y solution.
paths can be used to best improve the objective value while
satisfying an arbitrary number of virtual link constraints. In (ii) a backward pass (with any instance of branch-and-bound
Section VII, we confirm such intuition empirically, by studying exhaustive search), and a final (iii) constraints validation
the allocation ratio improvements (a proxy for provider’s phase. During the forward pass, NM builds the neighborhoods
revenue) when our method is used to find paths for embedding to estimate the path length. The backward pass is used to
services. find end-to-end paths with a given length (hop count). The
final constraints validation phase is used to keep the best path
IV. N EIGHBORHOODS M ETHOD candidate and decide whether or not the path search should be
extended to longer path candidates involving more neighbors.
To solve the NP-hard constrained shortest path problem with
an arbitrary set of (e.g., SLO) constraints in practical time, we Example 1. Consider a network consisting of 4 nodes X,
propose the novel Neighborhoods Method (NM). Y , A and B, as shown in Figure 3a. On the link (X, A),
Why the name “Neighborhoods Method”? Most path seek- we denote with the first value 5 the link constraint (in this
ing algorithms require at least two inputs for each node: (i) case bandwidth), and with the second 5 we refer to the first
knowledge of neighbors, and (ii) awareness of all adjacent path constraint (in this case end-to-end delay); with the third
link costs, often dictated by policies, or SLO constraints. value 4 we refer to the second path constraint (in this case
Such constraints are then used by the path seeking algorithm an arbitrary cost). In its general case, NM finds a path from
to compute the lowest-cost paths. The Dijkstra algorithm X to Y satisfying the three constraints bw ≥ 5, delay ≤
e.g., recursively finds the shortest path traversing the source 5 and cost ≤ 5 as follows: in the first phase, NM builds
neighbors and the neighbors of their neighbors. This recursive all neighborhoods starting from the source node X until the
notion leads to our definition of “neighborhoods” in NM, i.e, a destination node Y is reached (Figure 3b). Upon reaching
set of nodes that can be reached from the source node with the Y , NM begins the backward pass to find the full set of min
same number of hops, where each “neighborhood” (being a hop paths (Figure 3c). If the set contains the optimal path (in
set) contains unique elements. Based on Bellman’s “Principle this case for Problem 2), NM terminates with a solution. If
of Optimality” [39], such node repetitive structures can be no such path is found, NM builds an additional neighborhood
used for label-correcting dynamic programming that we apply as shown in Figure 3d and performs another backward pass
to reduce a number of exhaustive search path candidates. to check for optimality among paths that are one hop longer
than at the previous iteration (Figure 3e). NM iterates until
A. The NM General Case (l ⊕ p case) either the solution is found, or the maximum path length is
violated. In this example, NM returns the constrained shortest
As the exact constrained shortest path algorithm, the worst
path X → B → A → Y .
case time complexity of NM when accepting an arbitrary set of
hop-to-hop and end-to-end constraints (l ⊕ p) is exponential. Forward pass for l ⊕ p. Given an arbitrary graph G(V, E),
Our NM complexity analysis shows that the time complex- where |V | and |E| represent the number of vertices and
ity exponent is, however, halved with respect to common edges, respectively, in this phase NM successively builds
branch-and-bound path finders methods based on exhaustive neighborhoods < N H > from the source X to the destination
search [8], [9], [10] (see Section V). Note also that in addition Y . Algorithm 1 describes the forward pass of NM. To build
to the constrained shortest path, NM can simplify the process a new neighborhood N H, we add therein neighbors (adjacent
of finding all simple, k-constrained shortest or Pareto-optimal vertices) of each vertex u in the current neighborhood cN H
paths [9] from the source to the destination. (line 6). For example, the first N H includes nearest neighbors
The general workflow of our NM algorithm is shown in of the source X (which is in the zero N H), and the second
Figure 2: NM is executed in three phases: (i) a forward N H contains nearest neighbors of vertices in the first N H,
pass or neighborhoods building (by using label-correcting), and so on. The first phase ends as soon as the destination Y
 6

appears in cN H (line 4), or < N H > size is more or equal we check their feasibility and update the best known path
to |V | (line 13). candidate, if needed. Similarly to IBF [12], we keep iterating
while the candidate path length is less than the number of
Algorithm 1: Build Neighborhoods (l ⊕ p case) vertices |V | and then return the optimal solution.
Input: X:= src, Y := dest
We close this subsection with three important remarks: (1)
Output: The list of neighborhoods < N H > from X to Y NM can be used to find k-constrained shortest paths:
1 begin the backward path at each iteration returns all possible path
2 cN H ←− X
3 < N H >←−< N H > ∪cN H candidates of the same hop count. To find k-constrained
4 while Y ∈ / cN H do shortest paths we need to keep not a single best (shortest)
5 N H ←− ∅
6 foreach Vertex u ∈ cN H do path candidate, but a set of k best path candidates at each
7 N H ←− N H ∪ adjacent(u) iteration and update this set if needed. Clearly, as in the worst
8 end
9 if < N H > .size < |V | then case NM traverses all possible path candidates, its upper bound
10 < N H >←< N H > ∪N H complexity does not change (see Section V). (2) NM can be
11 cN H ← N H
12 else
applied to both directed and undirected graphs making it
13 Return Y is unreachable. an interesting solution even for NFV chain instantiations.
14 end When traversing directed graphs, NM simply uses vertices’
15 end
16 end outgoing neighbors during the forward pass and incoming
neighbors during the backward pass. (3) A distinctive feature
of our NM is the construction of the intersection of two
Backward pass for l ⊕ p. The best exhaustive search strategy neighborhoods. This leads to a quadratic reduction of path
may depend on the network topology, constraint and cost candidates for exhaustive search algorithms (see Section V).
functions, and we leave it as a policy for NM. In this
paper, we use EBFS for the backward pass of NM detailed
in Algorithm 2. Note however, that this phase can use any B. NM Search Space Optimizations
exhaustive search (e.g., EBFS [9], EDFS [8] or exhaustive k- In this subsection we show how NM can be coupled with
shortest path [10]) with the only difference being that we do existing search space reduction techniques, i.e., dominant
not process all neighbors (adjacent vertices) of each vertex u paths or look ahead [8], [9], [10], to speed up its time to
but only those which are within previous N H (line 10). The solution. By leveraging the NM’s double pass, we also propose
first step is to find the intersection between neighbors of the a variant of the look ahead technique, viz., “Look Back”
destination Y with its previous N H (line 5). This intersection without complexity overhead. We first describe the dominated
is not an empty set, it contains at least one vertex v. For paths method. Observe that the dominant paths technique is
all obtained vertices we again build the intersection of their not applicable in case we wish to use NM as k-constrained
neighbors with the penultimate N H (line 16). The second shortest paths, as it removes suboptimal candidates which can
phase ends as soon as we hit the zero N H (line 6), and as be among k paths.
a result we obtain the collection of all paths with a length of Dominated paths (pruning by dominance or bound). The
< N H > size between the source X and the destination Y . basic idea behind dominated paths states that an algorithm
can avoid evaluating candidate paths when their (multidimen-
Algorithm 2: Perform Backward Pass (l ⊕ p case) sional) distance is longer than other candidates distance, since
Input: The list of neighborhoods < N H > from X to Y they cannot be a part of the shortest solution [9], [10]. Consider
Output: All paths < path > from X to Y of < N H > .size length for example Figure 3a: note how path X → A → Y with
distance vector d¯ = [6, 5]T is dominated by X → B → Y
1 begin
2 path ←− Y
3 < path >←−< path > ∪path path with distance vector d¯ = [2, 5]T and hence it can be
4 k ←− 1
5 N H ←−< N H > [size − k]
excluded to avoid unnecessary time-consuming processing.
/* EBFS: */ Path dominance is formally defined as follows:
6 while N H 6=< N H > [0] do
7 < tempP ath >←− ∅ Definition 1 (dominant path). We say that path P1 dominates
8 foreach path ∈< path > do
9 Vertex u ←− path[1] path P2 if and only if:
10 foreach Neighbor v ∈ adjacent(u) ∩ N H do
11 < tempP ath >←− v ∪ path ∃ i ∈ 1, . . . , p, c : di (P1 ) < di (P2 ) ∧ dj (P1 ) ≤ dj (P2 ),
end
12
13 end
∀j 6= i ∈ 1, . . . , p, c
14 < path >←−< tempP ath >
15 k ←− k + 1 where di is a distance corresponding to pi path constraint or
16 N H ←− N H[size − k] to c path cost.2
17 end
18 end In its general form, we can reduce NM’s search space by re-
moving the dominated paths. This is accomplished by keeping
track of only non-dominated paths during its backward pass
Constraints validation for l ⊕ p. In this last phase, we check
(Algorithm 2, line 11). In particular, a path is added to a vertex
for candidates optimality, i.e., we check whether or not a
v if it is not dominated by any other path to v, or if it is not
candidate path satisfies all l ⊕ p constraints, and keep the
dominated by other paths from the source to the destination.
best candidate. At each consequent iteration, we first build
an additional (N + 1) neighborhood, repeat the backward pass 2 Note how, if only the cost distance d (P ) is used in Equation 1, pruning
c
and subsequently obtain all paths of length N + 1, and then by path dominance is the well-known pruning by bound technique [8].
 7

the complete algorithm description in those cases, we refer

reader to our earlier work on NM [40].

D. NM Optimal Solution
(a) (b) We conclude this section stating an important result, whose
Fig. 4: NM running in its in l ⊕ p general case reduce its search space with proof (by contradiction) is reported in our technical re-
a Look Back. (a) Forward pass: excludes vertices violating at least a path port [41].
constraint; (b) Backward pass: NM looks back and removes a path candidate
which sum of its current delay and the best (estimated during the forward Theorem 1. (Theorem of the Optimal Solution) The Neigh-
pass) residual delay violates the delay constraint (i.e., 1 + 5 > 5). borhood Method always finds the optimal path if it exists.
Look ahead (pruning by infeasibility). We can further reduce Proof. see technical report [41].
NM’s search space by omitting path candidates with endpoint
v, if the sum of their current path distances and the residual
best distances from v to the destination violates any of the path V. A SYMPTOTIC C OMPLEXITY A NALYSIS
constraints. This technique is known as “look ahead” [10] and
requires a run of Dijkstra or Bellman-Ford algorithm p times to In this section, we provide a complexity analysis compari-
pre-compute best distances (corresponding to path constraints son among our NM and related algorithms such as: IBF [12],
p) from all vertices to the destination. EDijkstra [20], EDFS [8] and EBFS [9], [26] — common
Looking backwards (pruning by infeasibility). To avoid branch-and-bound exhaustive search approaches. Table I sum-
running e.g., Dijkstra algorithm p + 1 times, in this paper marizes the main results of this comparison, where |V | is the
we propose a more efficient look back technique that reduces total number of vertices, and |E| is the total number of edges.
the search space. The best distance could be estimated now Theoretical results summary. The major benefits of NM arise
from the intermediate vertex v to the source (instead of the when we are seeking the NP-hard [8], [9] constrained shortest
destination) for each neighborhood containing v during the paths in l⊕1 and l⊕p cases. We show how, under l constraints,
NM forward pass. We then use such information during the EDijkstra finds constrained (shortest) paths faster than IBF
backward pass to exclude paths from v at neighborhood N and NM. We also show how NM is an alternative for IBF
that violate corresponding path constraints. to find the resource optimal constrained path when accepting
Note that when our looking backward technique is used, we l ⊕ 1 constraints (and thus flexible), and how EDijkstra loses
can learn the best distance for each node in each neighborhood. any path optimality guarantees in that case. Finally, when
Moreover, as physical nodes may appear in more than one accepting l ⊕ 1 and l ⊕ p constraints, we show how time and
neighborhood, we have more chances to prune path candidates space EBFS and EDFS complexities are quadratically higher
than with the look ahead search space reduction technique. In with respect to our NM.
addition, as we coupled the look back space reduction with
the NM forward pass, we do not introduce additional overhead
with respect to the look ahead reduction (i.e., running Dijkstra A. Complexity Analysis for the l and l ⊕ 1 Cases
p + 1 times).
For lack of space and due to the fact that most of the l/l ⊕
Example 2. In this example we revisit Example 1, to describe 1 complexity results are based on a simple extension of the
how NM in its the general case coupled with a Look Back original Dijkstra, Bellman-Ford, DFS and BFS algorithms, we
search may reduce the search space when finding a path from only show complexity analysis for the general l ⊕ p case. The
X to Y . We assume that two path constraints delay ≤ 5 in-depth complexity analysis for l/l ⊕ 1 cases however can be
and cost ≤ 5 need to be satisfied. During the forward pass found in our technical report [41]. Note how the constrained
(Figure 4a) NM estimates the path length to Y, keeping track shortest path in the l ⊕ 1 case is NP-hard [8] and can be found
of all constraint-satisfying distances for each vertex and at only by the general l ⊕ p case algorithms.
each neighborhood. Vertices whose estimated distance violate
at least one path constraint are instead removed. During the
backward pass, NM removes A → Y path candidate whose B. Complexity Analysis for the l ⊕ p Case
sum of the current delay (which equals to 1) and the best EDijkstra and IBF complexities (l ⊕ p case). For complete-
estimated delay from the source to A (which equals to 5) ness, we mention that neither EDijkstra nor IBF are applicable
violates delay constraint 1 + 5 > 5 (Figure 4b). to any variant of the constrained shortest path in the l ⊕p case.
EDFS and EBFS complexities (l ⊕ p case). Both EDFS and
C. NM for l links and a single path constraint (l/l ⊕ 1 cases) EBFS can find any variant of the constrained shortest path or
The worst case running time of NM becomes polyno- k such paths in exponential time by exhaustive search. For
mial (by avoiding an exhaustive search) if either a general each potential path, these algorithms check the satisfaction of
constrained shortest path (see Problem 1) with only l link all p constraints by calculating p + 1 new path and new cost
constraints or the resource optimal constrained path (see metrics in O(2p). All l constraints violating edges could be
Problem 2) with l links and a single path constraint (l ⊕ 1) are pruned prior to running EDFS or EBFS. The total number of
sought (see Section V). This is because unnecessary iterations path candidates can be bound as O(bd ) [42], where b is the
and phases (including the exhaustive search) are avoided. For number of neighbors, and d is the maximum loop-free path
 8

TABLE I: Virtual path embedding algorithms complexity

Case: EDijkstra [20] IBF [12] EDFS [8], EBFS [9], [26] NM
Time: O(|V | + |E| · l) Time: O(|V | + |E| · l) Time: O(|V | + |E| · l) Time: O(|V | + |E| · l)
l Space: O(|V | + |E|) Space: O(|V | + |E|) Space: O(|V | + |E|) Space: O(|V | + |E|)
resource optimal constrained path resource optimal constrained path resource optimal constrained path resource optimal constrained path
|E|

Time: O |V | |V | + |E| · l


Time: O(|V |log|V | + |E| · l) Time: O(|V ||E| + |E| · l) Time: O(|V ||E| + |E| · l)
|E|
Space: O |V | |V |



l/l ⊕ 1 Space: O(|V | + |E|) Space: O(|V ||E|) Space: O(|V ||E|)
constrained shortest path/ constrained shortest path/ constrained shortest path/
constrained shortest path
constrained path only resource optimal constrained path resource optimal constrained path
|V |
|E|
|V |
|E|


Time: O |V |
+ |E| · l
p Time: O |V | 2 p + |E| · l
l⊕p N/A N/A |E|
|E|
|V |

Space:O |V | |V | p



Space: O |V | 2 p
constrained shortest path constrained shortest path

hop count. The average number of neighbors per vertex b can

Neighborhood Method Prototype
be obtained from the hand-shaking lemma3 :
Web Interface
PV
(neighbors of vertex v) 2|E| Input: Virtual Link Output: Virtual Link Status,
b = i=v = . (11) Requests with Constraints Physical Network State
|V | |V | RESTful API
Using Equation 11 and based on the fact that the maximum External Module
loop-free path hop count equals to |V − 1|, EDFS and EBFS Physical Graph Path Mapping Path Allocation
time complexities Ol⊕p are: Discovery Service Service Service
 |E| |V |  Floodlight API
Ol⊕p = O(2p · bd + |E|l) = O p + |E| · l (12) Physical Link Controller Circuit Pusher
|V | Discovery Service Engine Service
 |V | 
|E|
Floodlight Controller
The space complexity equals to O |V | p due to the fact OpenFlow Protocol
that we have to store p metrics for each path. Physical Network
NM complexity (l ⊕p case). Similar to EDFS and EBFS, NM
can find any variant of the constrained shortest path or k such
paths with l ⊕ p constraints in exponential time utilizing any Hosts OpenFlow Switches Hosts
instance of an exhaustive search. The neighborhoods contains Fig. 5: System architecture of our NM prototype (which is a module of the
non-unique nodes, and the total number of their nodes can Floodlight OpenFlow controller [43]) includes four main logical components:
a physical graph discovery service, a path mapping service, a path allocation
be up to |V |2 . For each neighbor, NM checks if it already service and a user web interface that interacts with the controller. The
appears in the neighborhood O(b). Taking into account the prototype source code is publicly available under a GNU license at [23].
edge pruning phase, to ensure the constraints satisfaction
and to reduce a search space, the time complexity of the larly as for EDFS and EBFS, the total space complexity is
  |V2 | 
neighborhoods building step O1l⊕p is: O |V |E|
p .
|
O1l⊕p = O(|V |2 b + |E|l) = O(|V ||E| + |E| · l ) (13)
During the backward pass, NM builds all possible paths from VI. NM P ROTOTYPE I MPLEMENTATION
the destination node using any exhaustive search methods such In this section, we establish the practicality of our approach
as EDFS or EBFS. However, there is a difference in that, for network virtualization with a prototype implementation
we process only those vertex neighbors which appear in its over a Software Defined Networking infrastructure. Our source
previous neighborhood. This allows us to significantly reduce code is publicly available at [23]. In particular, our proto-
the total number of the processing paths: instead of processing type implementation extends the Floodlight OpenFlow con-
1 + b + b2 + ... + bd or O(bd ) paths, due to double pass (i.e., troller [43]. Our system architecture is shown in Figure 5. Our
forward and backward passes) we process only: 1 ∩ bd + b ∩ prototype includes four main logical components: a physical
d d
bd−1 +b2 ∩bd−2 +...+b 2 ∩b 2 +...+bd−2 ∩b2 +bd−1 ∩b+bd ∩1 ≤ graph discovery service, a path mapping service, a path
d d
1 + b + b2 + ... + b 2 + ... + b2 + b + 1 or O(b 2 ) paths. allocation service and a user web interface that interacts with
Hence, the time complexity of the backward pass step O2l⊕p is the controller. In the rest of the section we describe with some
quadratically lower than for EDFS or EBFS (see Equation 12): more details each of the four components of our prototype.
d
 |E|  |V2 |  Physical graph discovery. Upon bootstrapping a SDN setup,
O2l⊕p = O(2p · b 2 ) = O p (14) all configured OpenFlow switches connect to the controller
|V |
allowing a dynamic switch-to-switch link discovery. After this
  |V2 | phase, the NM module tracks all unknown incoming packets
|E|
The total time complexity of the NM is O |V | p+ to detect hosts and their corresponding host-to-switch links.
|V |
   2
|E|
 We specify the main physical link properties, such as capacity
|V ||E| + |E| · l or just O |V | p + |E| · l . Simi- and cost (e.g., delay) through an XML configuration file. The
XML configuration file indirection allows our NM prototype to
3 Without loss of generality, we can assume undirected network graphs as easily interact with foreign measurement services, for example
|E|
in a directed graph, b equals to the average outdegree: b = |V | . for real-time awareness of its link properties. The information
 9

collected by the path discovery service is then used in the path and in the physical network topology. In most of our physical
mapping and the path allocation steps. topologies, the average node degree equals to 4, a known
Path mapping. To map constrained virtual link requests, common value within Internet topologies [47].
the path mapping module of our prototype uses information Results summary. Efficiency and Provider’s revenue: During
from the physical path discovery service and runs one of the the virtual network formation (management plane), we found
following routing algorithm policies: NM for l and l ⊕ 1 cases that using NM within recent VNE or NFV-SC algorithms
(default policy), NM for l ⊕ p cases, EDijkstra, IBF or EBFS. increases their allocation ratio while improving the network
In the current version of our prototype the routing policy is utilization by better load balancing (close to the optimal),
static and has to be set via our XML configuration file before which in turn decreases energy consumption. Network Uti-
starting the Floodlight controller. lization: Our results evaluating NM within the data plane
Path allocation. In this last phase of the path embedding, the instead show how utilizing a set of paths found with NM is
NM module sets all appropriate flow rules in all switches via beneficial for TE in terms of minimum path hop count, net-
the OpenFlow [44] protocol, along with the computed path work utilization and in some cases even energy consumption.
obtained during the path mapping phase. Specifically, using Time to Solution (Convergence Time): When we attempted to
OpenFlow protocol messages the module assigns active flows allocate flows (virtual links) with multiple link and multiple
to corresponding queues, i.e., applies an ENQUEUE action, path constraints over large scale physical networks using NM
for guaranteed bandwidth provisioning. In our XML config- with the proposed search space reduction techniques, we found
uration file, users may specify also the type of timeout for a path computation speed-up of almost an order of magni-
each flow i.e., the flow duration time can start from either the tude w.r.t. common exhaustive search algorithms. Moreover,
previously matched packet or from the first flow instantiation. we also found almost 3 orders of magnitude running time
To estimate the available bandwidth on each physical link, improvement w.r.t. the same integer programming problem
the NM module uses both the capacity information derived solved with CPLEX [50]. Prototype Evaluation: Finally, using
from the XML configuration file, and its allocated bandwidth our NM prototype over GENI, we were able to reproduce
queried from the flow stored in the OpenFlow switch. our main results. Moreover, our measurements show how the
Web interface. To request virtual links and/or to monitor constrained shortest path computation (virtual link or flow
physical network states, we have implemented a web user mapping phase in NM) is up to an order of magnitude faster
interface, which uses a RESTful API to communicate with our than the path allocation phase (i.e., setting up appropriate flow
NM prototype module. The user interface uses technologies rules within all switches along the found path) on small scale
such as HTML, PHP, AJAX, JavaScript, and CSS. networks. This along with our scalability results confirm how
at large scale the time needed for a path computation with NM
VII. P ERFORMANCE E VALUATION will have the same order of magnitude as the time needed for
In this section, we evaluate NM’s scalability and flexibility the path allocation introducing no significant bottleneck for
performance through simulations and our prototype imple- the end-to-end virtual link embedding.
mentation in the context of its applicability to several com-
plementary virtual network services. To assess the flexibility A. Management Plane Evaluation
of NM, we compare, with and without it: (i) the embedding Simulation settings. To assess the impact of NM on the
performance of several online VNE and real-time NFV-SC virtual network embedding, we include results obtained with
mechanisms within the management plane; (ii) several traffic simulated physical networks of 20 nodes (as in similar earlier
engineering solutions within the data plane. Our goal is to studies [17]), following Waxman connectivity model, where
show how our NM can be used to improve the overall each physical node and each physical link have uniformly
network utilization (allocation ratio or total flow throughput), distributed CPU and bandwidth from 0 to 100 units, respec-
optimality (load balancing or fairness of flows), as well as tively. Note that we use fairly small scale physical networks
energy consumption within both planes; (iii) To assess NM due to complexity of the integer programming formulation.
scalability, we then compare it with related solutions under We attempt to embed a pool of 40 VN (service chain)
different network scales and service requests/topology scenar- requests with 6 virtual nodes and random (linear) virtual
ios when accepting multiple link and multiple path constraints; topologies. Each virtual node and each virtual edge have
(iv) Finally, we confirm our main simulation results with our uniformly distributed CPU and bandwidth demand from 1
NM prototype running over the GENI testbed [2]. to 10 units, respectively. When we tested NM within virtual
Evaluation settings. In our simulations, we used a machine network embedding algorithms, we vary the virtual node
with an Intel Core i5 processor with dual core CPU of 2.7 GHz degree from 1 (the VN has linear topology) to 5 (VN is a
and 8GB RAM. We use the BRITE [45] topology generator fully connected topology). Moreover, we also assume that each
to create our physical and virtual networks. Our results are virtual edge has a propagation delay stretch latency constraint
consistent across physical networks that follow Waxman and uniformly distributed between 1 to 4. We defined propagation
Barabasi-Albert models [46], that are known to approximate delay stretch the ratio between the propagation delay and the
well subsets of Internet topologies [47]. For lack of space we propagation delay encountered traversing the diameter of the
only show results relative to Waxman connectivities. In our physical network. When we tested NM within the real-time
NM prototype evaluation instead, we use a physical network service chaining use case, we vary the latency constraint of
obtained with the GENI testbed [2]. All our results show virtual links (service-to-service communication) from ∞ (i.e.,
95% confidence interval, and our randomness lays in both SC is not real-time sensitive) to 1/4 (i.e., SC is highly real-
the service request (i.e., in its type and constraints to accept) time sensitive) of the propagation delay stretch.
 10

Opt PGen PGen+NM CAD CAD+NM

Optimality Gap [%]

0.8 0.8

Energy Delta [%]

15
Allocation Ratio

Opt
100
12 PGen
0.6 0.6 PGen,NM

PDF
9 10 CAD
0.4 0.4
6 CAD,NM
0.2 1
3 0.2
0 0 0 0
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Avg node degree of VN Avg node degree of VN Avg node degree of VN Path Hop Count

(a) (b) (c) (d)

Optimality Gap [%]

0.72 0.8

Energy Delta [%]

8
Allocation Ratio

Opt
100 PGen
0.7 6 0.6 PGen,NM

PDF
10 CAD
4 0.4
0.6 CAD,NM
2 1 0.2
0.3
0 0 0 0
2 1 0.5 0.25 2 1 0.5 0.25 2 1 0.5 0.25 1 2 3 4
Prop. Delay Stretch Prop. Delay Stretch Prop. Delay Stretch Path Hop Count

(e) (f) (g) (h)

Fig. 6: Virtual network (VN) embedding and real-time NFV service chaining (SC) results obtained with physical networks of 20 nodes following Waxman
connectivity model: by addressing the constrained shortest path problem with NM versus using commonly adopted shortest path algorithms (e.g., Disjktra),
the allocation ratio of VNE (a) and real-time NFV-SC (e) can be improved when the virtual node degree increase or when requests are not highly sensitive to
delays. Optimality gap of VNE (c) and NFV-SC (g) can be also improved (resulting in our case into a better load balancing) which leads to a lower energy
consumption (b) and (f) relative to the network idle state [49], respectively. NM’s benefits are due to its ability of finding more path with a lower hop count
that can satisfy latency demands and simultaneously improve objective value for full-mesh VNs (d) and moderate real-time sensitive SC (h) pools.

Evaluation metrics. To demonstrate the advantages of using revenue loss by specifying the fraction of VN request accepted
NM within the virtual network embedding (VNE) and real- over the VN requested (allocation ratio), to what extent physi-
time service chaining (SC) mechanisms, we compared four cal links were utilized (link utilization) and how many paths of
representative VNE algorithms. To compare them, we replace the particular length in total were used per VN pool. Finally,
their (original) Dijkstra-based shortest path with our NM. We we used the idle energy model proposed previously [49] to
have chosen to compare against [17], [19] as the optimal VNE access the energy consumption of the network:
scheme formulated as integer programming multi-commodity X
flow is the best to our knowledge solution for online VNE (yet Energy Consumption = (M − E0 )Ue + E0 (15)
e∈E
intractable for large scale networks). We refer to this solution
as Optimal and we denote it as Opt. Note however, that Opt where E is a set of physical edges, Ue is an edge e utilization,
solution can be still suboptimal with respect to the optimal and M and E0 are numerical values taken from [49] that
solution of the offline VNE problem, where all requests are correspond to the maximum and idle energy consumption
known in advance. Opt in fact attempts to minimize the ratio of the switch interface, respectively. We use M = 2 and
between the provider’s costs of embedding a VN request and E0 = 1.7 maximum and idle energy consumptions (measured
the available substrate resources provided for this request, in Watts) assuming gigabit channel communications. In our
with the aim of balancing the network load. We also compare results, we show an energy consumption increase relative to
Opt against its version where a path (or column) generation the idle network state (in %).
approach is used to make Opt more scalable [17]. We refer NM improves VNE/NFV-SC allocation ratio and energy
to this scheme as PathGen and, even in this case, substitute efficiency. Figures 6a and 6e show how including constrained
its original shortest path algorithm (used to find new paths shortest paths (e.g., found with NM) during column generation
within the multi-commodity flow) with our path management of the PathGen approach can improve overall VNE and NFV-
solution (see details in Section III-C). Note that we used SC acceptance ratio. Particularly, the highest acceptance (i.e.,
the one-shot VNE approximation algorithm proposed in [36] allocation) ratio gains arise in the case ofdense (e.g., full-
as an initial solution for the column generation approach mesh) VNs or under moderate real-time sensitivity of NFV-
to avoid two stage VNE limitations when physical network SCs. Moreover, we can see how in some cases, utilizing NM
is initially unbalanced [17]. Finally, we compare against a within PathGen leads to a lower energy consumption (see
Consensus-based Auction mechanism (CAD) [5], [21], the first Figures 6b and 6f ). This is due to an improved network
policy-based distributed VNE approximation algorithm with utilization because of a better (closer to the optimal) load
convergence and optimality guarantees. Note that a version balancing (see Figures 6c and 6g).
of CAD can be also used to solve the NFV-SC problem [5]. These optimality gains in turn arise due to NM’s ability
The link embedding of CAD is a policy that runs a shortest to find more paths that can satisfy all virtual link constraints
path algorithm to either pre-compute the k-shortest paths or to (e.g., bandiwdth and latency) and simultaneously improve the
find these paths dynamically. For fairness of comparison, we objective value (see Figures 6d and 6h). These results confirm
assume that the latter holds and as in the PathGen case, we our expectations in Section III-C. At the same time, optimality
substitute the currently used shortest path algorithm, Dijkstra, improvements with NM demonstrate no significant benefits (in
with our NM constrained shortest path finder solution. terms of allocation ratio or energy efficiency) for the CAD over
standard shortest path management. This is due to the fact that
In this simulation scenario we have tested the potential in our settings, separate node and link embedding approach
 11

TE,low SLO TE+NM,low SLO TE,mid SLO TE+NM,mid SLO TE,high SLO TE+NM,high SLO
Total Flow TPut [Tbps]

1 12

Fraction of Flows

Energy Delta [%]

5

Path Hop Count

6
4 0.8 9
3 0.6 4
6
2 0.4
2 3
1 0.2
0 0 0 0
1 5 10 15 20 25 0 0.25 1 2.5 5 10 20 1 5 10 15 20 25 1 5 10 15 20 25
Num. of Paths per Flow Flow TPut Fairness [Gbps] Num. of Paths per Flow Num. of Paths per Flow

(a) (b) (c) (d)

Total Flow TPut [Tbps]

Fraction of Flows

Energy Delta [%]

20

Path Hop Count

12 12
16 0.8
9 9
12 0.6
6 6
8 0.4
4 0.2 3 3
0 0 0 0
8 7 6 5 4 3 2 1 0 0.25 1 2.5 5 10 20 8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1
Avg node degree of PN Flow TPut Fairness [Gbps] Avg node degree of PN Avg node degree of PN

(e) (f) (g) (h)

Fig. 7: Performance analyses of LP-based (Top) and greedy (Bottom) max-min fairness Traffic Engineering (TE) algorithms [15], [16] utilizing the constrained
shortest path management with NM versus their original shortest path management with Dijkstra on Waxman topologies in terms of: (a) and (e) total gained
flow throughput; (b) and (f) cumulative distribution of flow throughput for 25 paths per flow and for average node degree 4 (common for Internet [47]),
respectively; (c) and (g) energy consumption increase relative to the network idle state [49]; and (d) and (h) number of average path hops per flow.

demonstrates the worst performance caused by significantly Evaluation metrics. To evaluate the performance of NM for
limited feasible space for the virtual link mapping. Such data plane TE solutions, we compare NM with (extended)
limitations are due to randomized capacities of physical nodes Dijkstra algorithm within the greedy-based TE (i.e., in the
and edges (i.e., due to initially unbalanced physical network) second scenario), and their corresponding k-shortest path [31]
further exacerbated by randomized virtual link capacity and and k-constrained shortest path (that uses general version of
latency constraints. NM coupled with Look Back technique) algorithms within
LP-based TE (i.e., in the first scenario). We remark that
B. Data Plane Evaluation NM’s superior performance w.r.t. Dijkstra-based TE is ex-
pected due to its ability of minimizing provider’s associated
In the next set of results we analyze the benefits of using our cost for flow allocation e.g., minimizing provisioned physical
solution within the data plane by evaluating NM performance bandwidth (see Problem 2) under an arbitrary set of (e.g.,
within standard Traffic Engineering schemes [15], [16], un- SLO) constraints. This difference is expected to degrade in
der different physical network topologies and under different the first scenario (where LP-based formulation is used) with
severity of service level objectives. either number of maximum paths per flow or SLO severity
Simulation settings. To evaluate the impact of NM (used to increase, as in this case the k-shortest path set converge
compute constrained shortest paths) within Traffic Engineer- to the k-constrained shortest path set resulting in the equal
ing [15], [16], we use a physical network topology of 10, 000 LP formulation. However, in the second scenario, as we
nodes, where each physical link has bandwidth uniformly allocate bandwidth for flows on their most preferable tunnels
distributed between 1 and 10 Gbps. We attempt to allocate (best paths) first, that superior performance is expected to be
flows for 1000 random source destination pairs by solving preserved and can vary with different node degree or SLO
max-min fairness problem shown in Equation 8 with the fixed severity. For comparison we use the following four metrics:
latency SLO demands. To evaluate the maximum possible total gained throughput of all flows, cumulative distribution
gains, we assume infinite bandwidth demands of the flows of flows’ throughput which corresponds to their fairness,
and we omit any constraints imposed by hardware granularity energy consumption relative to the network idle state (see
due to rule count limits or flow quantization limitations [15], Equation 15) and path hop count.
[16]. For clarity, we also assume that all flows have the same
priority. Thus, the fairness of the flow is its total allocated Path hop savings lead to network utilization and flow
throughput. We denote with low, medium and high delay SLO fairness gains. In Figure 7a, we show how the correlation
constraints, 4, 1.5, and 1 times of a propagation delay stretch between the total gained throughput across all flows and the
defined in Section VII-A, respectively. maximum number of paths available per each flow is a loga-
In the first simulation scenario, we use LP-based solution rithmic function — increasing number of paths linearly brings
(that is costly to address in practice [15], [16]) with fixed a logarithmic growth to the total gained throughput. Figure 7e
average physical node degree equal to 4 (common for the shows how the total gained throughput of all allocated flows
Internet [47]), where we vary the maximum number of paths changes when multiple physical links become available (the
available for each flow allocation. In the second scenario, we average physical node degree increases). This dependence is an
use instead scalable greedy solution proposed in [16] with affine function: the maximum possible total gained throughput
unrestricted number of paths per flow. To this end, once the increases linearly with the available physical links.
best currently available path (or tunnel) gets fully saturated, In both scenarios, we can see how the total flow through-
we find the next best path dynamically. In this scenario, we put and the resulting flow fairness (e.g., the particular flow
vary average physical node degree from 8 to 1. throughput) are higher for NM than for Dijkstra-based TE
 12

(see Figures 7a, 7b, 7e and 7f ). These results demonstrate CPLEX NM NM+LB EBFS EBFS+LA
how minimizing the total physical bandwidth provisioned for

Path Comp. Time [ms]

Traversed Paths per VL
4 3
10 10
a single flow with NM can significantly benefit even traffic 103 2
10
engineering solutions. In particular, due to the path hop count 102 1
10
optimization under SLO constraints within the data plane, 101 0
10
NM gains up to 50% of total flow throughput under low 10
0
1 2 3 4 1 2 3 4
10 10 10 10 10 10 10 10
SLO in the first (LP-based) scenario, and up to 20% of total Nodes Num. of PN Nodes Num. of PN
flow throughput under mid SLO in the second (greedy-based)
(a) (b)
scenario w.r.t. Dijkstra-based TE (see Figures 7a and 7e).

Path Comp. Time [ms]

Traversed Paths per VL
3
Such gains allow, in turn also to improve flow fairness (see 103 10

Figures 7b and 7f ). Note that such large throughput gains in 2 102

10 1
the first scenario (i.e., up to 50%) are partly due to the k- 1
10
0
10 10
shortest path [31] and the general version of NM (that finds 0 -1
10 10
k-constrained shortest paths) algorithms difference, i.e., the 10
1
10
2
10
3
10
4
10
1
10
2
10
3
10
4

former has a higher probability of finding paths with more Nodes Num. of PN Nodes Num. of PN
shared edges then the later. As expected, NM gains decrease (c) (d)
with the node connectivity, as less physical path choices are Fig. 8: Performance analyses of general NM versus EBFS with dominant
available to map virtual links. Also, for LP-based scenario paths, look-back for NM (NM+LB) and look-ahead for EBFS (EBFS+LA)
search space reduction techniques, and versus IBM CPLEX solver for the
these gains decrease with increase of the maximum number constrained shortest path formulation in Problem 2 for low (top row) and
of paths or SLO severity, as both shortest and constrained medium (bottom row) SLO constraints in terms of: (a,c) number of traversed
shortest path sets converge to each other resulting in equal LP paths to find the optimal virtual path; (b,d) the virtual path computation time.
formulations. links (or traffic flows). In particular, we generate physical net-
Average path length and energy consumption tradeoff. We work topologies of 10, 100, 1K and 10K nodes, where each
further investigate the reasons why we observed such gains physical link has bandwidth uniformly distributed between 1
in total throughput of all flows w.r.t. Dijksra-based TE. In and 10 Gbps. In addition, we set each physical link with a cost
particular, observing Figures 7d and 7h we note that there are uniformly distributed between 1 and 10. We attempt to find
≈ 2 − 3 hops difference in the average path length between the constrained shortest path variant — the resource optimal
NM and Dijkstra-based TE when allocating low SLO flows. At constrained path for 10% of physical network nodes random
the same time, for the medium SLO constraints this difference source-destination pairs, where for each pair we allocate as
is reduced to circa one hop. Finally, when the SLO constraints many virtual links as possible with the fixed demands. We
are high, there is no significant physical path length difference. denote with low and medium bandwidth constraints, 1 and 4
To understand why the average path length changes with Gbps, respectively; these values represent approximately 10%
the constraint severity, note how the longer is an end-to-end and 45% of the maximum physical link capacity. Similarly,
physical path, the lower is the probability that the entire path we denote with low and medium (propagation) delay SLO
satisfies the SLO constraints. On the other hand, the higher constraints, 4 and 2.5 times of a propagation delay stretch
the number of hops, the higher is the number of candidates defined in Section VII-A, respectively. In addition, we denote
paths, and so the higher is the probability of finding one with low and medium cost constraints, 100 and 50 that
which satisfies these constraints. This explains the trade-offs represent 10 and 5 times of the maximum physical link cost.
in average path length behavior observed in Figure 7h. Scalability evaluation metrics. To evaluate the NM scalabil-
The hop count savings minimize the physical bandwidth ity, we compare the general version of NM with the EBFS
provisioned for a single path, allowing the provider to accept (common branch-and-bound exhaustive search) algorithm and
more flows or allocate more bandwidth for a single flow. As a with the CPLEX [50] performance (that uses 4 parallel
result, the overall link utilization increases leading to a higher threads) of solving the common arc-based constrained shortest
energy consumption (see Figures 7c and 7g). We observe path formulation. Both NM and EBFS are coupled with
one exception when throughput gains are low and path hop dominant paths search space reduction techniques. Moreover,
count savings are high (as observed in the management plane we couple EBFS with a Look Ahead (EBFS+LA) search space
scenario in Section VII-A). An example of such situation can reduction technique [10] and NM with a Look Back (NM+LB)
be also observed in the second scenario for dense physical search space reduction technique - a variant of Look Ahead
networks (with average node degree ≥ 5) when allocating without complexity overhead (see Section IV-B).
flows with low SLO demands. In that case, we can see a small We compare NM with EBFS and CPLEX across two metrics:
reduction (of ≈ 2%) in the energy consumption simultaneously the number of traversed paths required to find the constrained
with low throughput gains (of ≈ 5%). shortest path and the average path computation time. Note that
C. Scalability Results in case of CPLEX, the number of traversed paths corresponds
In the next set of results we test the scalability perfor- to the total number of iterations.
mance of NM when accepting multiple link and multiple path Dominant paths prevent intractabilities. Figures 8a and 8c
constraints (l ⊕ p case). We remark that in this case only show that dominant paths technique reduces the number of
exponential exact solutions exist for the constrained shortest traversed paths per virtual link to a linear function of a
path problem due to its NP-hardness [8]. physical network size for both EBFS and NM. Moreover,
Scalability simulation settings. To assess NM scalability, due to its “double pass” technique, NM traverses up to two
we simulate on-line requests for allocating constrained virtual orders of magnitude paths less then EBFS. However, we can
 13

see how EBFS works slightly faster than NM for large scale ED IBF NM l+1 EBFS NM l+p

Total VL TPut [Mbps]

VL Embed. Time [ms]

20

VL Path Hop Count

2
physical networks with medium link and path constraints (see 5
10

Figure 8d). That can be explained with the more expensive 15 4

forward pass of NM for larger physical networks (≥ 10K 10 3 101

2
nodes). The NM’s unnecessary iterations can be however 5
1
reduced by the proposed Look Back technique. 0 0 10
0
Mapping Allocation
NM scalability with backward pass and look back. Our
(a) (b) (c)
experiments show that when NM backward phase is coupled
Fig. 9: Performance analysis of the shortest path algorithm such as the
with a Look Back technique, the number of traversed paths by extended version of Dijkstra (ED) versus constrained shortest path algorithms
NM further reduces. This reduction is almost independent from such as NM (in both l ⊕ 1 and l ⊕ p cases), EBFS and IBF on a reserved in
the size of the physical network (see Figures 8a and 8c). Using GENI small SDN testbed in terms of: (a) total gained throughput; (b) number
of path hops per VL; and (c) average time per VL embedding, i.e., VL path
the Look Back search space optimization does not introduce computation (mapping) and its consequent allocation.
any significant path computation overhead, while the same
cannot be said for the Look Ahead search space reduction random < src, dst > pairs of hosts, where for each pair of
technique (see Figures 8b and 8d). endpoints we allocate as many virtual links as possible.
Even though CPLEX uses 4 parallel threads (instead of Experiment metrics. For each virtual link request we again
a single thread for NM and EBFS) and traverses moderate measure the total gained throughput and the virtual link path
number of path (similar to NM without Look Back technique), hop count. In addition, we measure the time required to
it shows the worst performance in all cases. That is due to the compute a path (virtual link mapping), and the time to allocate
fact, that finding constrained shortest paths with the commonly the computed path (i.e., set appropriate flow rules within
utilized arc-based integer programming formulation is NP- OpenFlow switches along the computed path). Note that the
hard and no existing techniques can reduce that complexity overall time for end-to-end virtual link embedding includes
to pseudo-polynomial. On the contrary, NM and EBFS com- both virtual link mapping and its allocation. Our experiment
plexities can be reduced to pseudo-polynomial by applying goals are twofold: first, we want confirm our simulation results
the dominant paths search space reduction technique [10]. in real settings; secondly, we want to estimate an overhead of
Proposed novel double pass and Look Back search space addressing constrained shortest path problem in real settings.
NM gains are confirmed experimentally. Using real-world
reduction techniques further reduce the practical complexity
settings, we were able to confirm constrained shortest path
of finding constrained shortest paths. Thus, NM is almost an
algorithms (i.e., IBF, NM and EBFS) for the online TE pro-
order of magnitude faster in comparison with EBFS and is
duce superior performance even on a small physical network
almost 3 orders of magnitude faster than CPLEX for large-
scale. This is similar to superior results of the offline TE
scale physical networks, and hence scales better. We remark
which utilizes the constrained shortest path algorithms (see
that such scalability improvements over existing constrained
Section VII-B). Specifically, IBF, NM and EBFS show gains
shortest path algorithms are essential for the virtual network
of up to 12% in total VL throughput (network utilization) and
service management at large scale. For example, the column
find almost 1 hop shorter VL path in average, w.r.t. extended
generation approach can generate tens of thousands paths
Dijkstra (ED) shortest path scheme as shown in Figures 9a
per a single VN request at large scale. Thus, it will take
and 9b. Note however that IBF is applicable only for in l and
100 ms x 10K ≈ 17 minutes when EBFS is used. On the
l ⊕ 1 cases (see Table I).
contrary, we need only 10 ms x 10K ≈ 100 seconds with NM
NM running time scales well with physical network size.
which significantly reduces VN request blocking probability.
Figure 9c shows how VL mapping is an order of magnitude
Note that additional scalability results comparison of our NM
faster for small scale physical networks (of ≈ 101 nodes)
with respect to the EDijkstra shortest path scheme can be
than its allocation for all routing schemes that have been
found in our prior work [40].
implemented. This is because the path computation is a local
D. Prototype Evaluation (in-memory) operation but the virtual link allocation requires
setting up of flow rules within all switches along the loop-free
In this final set of results, we use our NM prototype to
underlying physical path found. Hence, its speed depends on
estimate the impact of the on-demand constrained shortest
the Round Trip Time between switches and the OpenFlow
path computation on the end-to-end virtual link embedding
controller. In reference to Figures 8b and 8d, we can see
performance. We also confirm our main simulation results.
how for large scale networks (≥ 10K nodes), the running
Experiment settings. Our setup for the performance exper-
time of classical constrained shortest path algorithms such as
iments includes 15 virtual machines (VMs) from the GENI
EBFS can become prohibitive (up to two orders of magnitudes
testbed [2]: Ten of these VMs are OpenFlow Virtual Switches
larger than in a case of a single path computation for small
(OVS) [51], and others are hosts. Each host-to-switch physical
scale networks). As a result, the VL mapping time becomes a
link has 10 Mbps bandwidth and a 0 arbitrary cost, and each
bottleneck. On the contrary, NM is just an order of magnitude
switch-to-switch physical link has both bandwidth (measured
slower at large-scale than at small scale. Thus, NM does not
in Mbps) and an arbitrary cost uniformly distributed between
bottleneck the end-to-end VL embedding at large scale.
1 and 10. Note, that our arbitrary cost is an additive metric
and therefore can represent any path metric, e.g., delay, losses, VIII. C ONCLUSION
jitter, etc. We request virtual links with low SLO constraints, In this paper, we motivated the problem of achieving a
i.e., ≥ 1 Mbps bandwidth and ≤ 50 arbitrary cost (5 times flexible and scalable constrained shortest path management
greater than the maximum physical link cost), between 5 approach for virtual network services deployed across multiple
 14

data centers. To cope with constrained shortest paths NP- [23] Neighborhood Method Prototype - https://bitbucket.org/naas cloud
computing project/nm-of-controller/overview; Last accessed Aug. 2018.
hardness (that limits both its scalability and flexibility), we first [24] P. Khadivi, et al., “Multi-constraint QoS routing using a new single
introduced a novel algorithm viz., ‘Neighborhoods Method’ mixed metrics”, Elsevier JNCA, 2008.
[25] A. Juttner, et. al., “Lagrange Relaxation based Method for the QoS
(NM), which utilizes a double pass (a synergy of dynamic Routing Problem”, IEEE INFOCOM, 2001.
programming and a branch-and-bound exhaustive search) and [26] R. Widyono, “The Design and Evaluation of Routing Algorithms for
Real-time Channels”, ICSI, UC Berkeley, 1994.
“Look Back” search space reduction techniques. Our computa- [27] M. S. Bazaraa, et al., “Linear programming and network flows”, John
Wiley & Sons, 2011.
tional complexity analysis indicates NM’s quadratically lower [28] M. Chiesa, et al., “Lying Your Way to Better Traffic Engineering”, ACM
complexity upper-bound than for recent branch-and-bound ex- CONEXT, 2016.
[29] S. Vissicchio, et al., “Central control over distributed routing”, ACM
haustive search methods, and our scalability evaluation results SIGCOMM CCR, 2015.
show how NM is faster by an order of magnitude than these [30] D. Nace, et al., “Max-min fairness in multi-commodity flows”, Elsevier
COR, 2008.
methods. Thus, NM scales better for large scale networks of [31] D. Eppstein, “Finding the k shortest paths”, SIAM SICOMP, 1998.
[32] E. Amaldi, et al., “On the computational complexity of the virtual
≥ 10, 000 nodes. Via numerical simulations of diverse network network embedding problem”, Elsevier ENDM, 2016.
topology scenarios, we were able to show how the constrained [33] M. Yu, et al., “Rethinking Virtual Network Embedding: Substrate
Support for Path Splitting and Migration”, ACM SIGCOMM CCR, 2008.
shortest path management improves the network utilization [34] Y. Zhu, et al., “Algorithms for Assigning Substrate Network Resources
(gains were seen up to 50%) and in some cases even in the en- to Virtual Network Components”, IEEE INFOCOM, 2006.
[35] J. Londono, A. Bestavros, “Netembed: A Network Resource Mapping
ergy efficiency of the recent management plane algorithms for Service for Distributed Applications”, IPDPS, 2008.
[36] J. Lischka, H. Karl, “A Virtual Network Mapping Algorithm based on
virtual network embedding or network function virtualization Subgraph Isomorphism Detection”, ACM VISA, 2009.
service chaining. Additionally, we found improvements in the [37] F. Samuel, et al., “PolyViNE: Policy-based Virtual Network Embedding
across Multiple Domains”, Springer JISA, 2013.
recent data plane algorithms for traffic engineering. Thus, we [38] M. T. Beck, et al., “Distributed and scalable embedding of virtual
demonstrated that our proposed NM is also flexible and can be networks”, Elsevier JNCA, 2015.
[39] R. Bellman, “On a Routing Problem”, AMS QAM, 1958.
applied to diverse virtual network service scenarios. Finally, [40] D. Chemodanov, et. al., “A General Constrained Shortest Path Approach
for Virtual Path Embedding”, IEEE LANMAN, 2016.
we were able to reproduce our main simulation results in a [41] D. Chemodanov, et al., “A Constrained Shortest Path Scheme for Virtual
real-world GENI testbed with an NM implementation, whose Network Service Management”, arXiv:1808.03031, 2018.
[42] R. Korf, “Depth-first Iterative-deepening: An Optimal Admissible Tree
source code is publicly available under a GNU license at [23]. Search”, Elsevier AI, 1985.
As part of future work, we aim to develop new VNE/NFV- [43] Floodlight SDN controller - http://www.projectfloodlight.org; Last ac-
cessed Aug. 2018.
SC algorithms that can better utilize our proposed constrained [44] N. McKeown, et al., “OpenFlow: Enabling Innovation in Campus
Networks”, ACM SIGCOMM CCR, 2008.
shortest path scheme at larger network scales. To obtain a good [45] A. Medina, et al., “BRITE: An Approach to Universal Topology Gen-
lower bound solution within large network scale simulations, eration”, IEEE MASCOTS, 2001.
[46] B. Waxman, “Routing of Multipoint Connections”, IEEE JSAC, 1988.
we can relax integrality constraints of the optimal VNE/NFV- [47] M. Faloutsos, et al., “On Power-Law Relationships of the Internet
Topology”, ACM SIGCOMM CCR, 1999.
SC integer programs to e.g., use efficient linear programming. [48] B. White, et. al., “An Integrated Experimental Environment for Dis-
tributed Systems and Networks”, USENIX OSDI, 2002.
R EFERENCES [49] A. P. Bianzino, et al., “Energy-aware routing: a reality check”, IEEE
GLOBECOM, 2010.
[1] VMware Cloud vServices - http://www.vmware.com/cloud-services.html; [50] IBM CPLEX solver - http://www.ibm.com/analytics/data-science/
Last accessed Aug. 2018. prescriptive-analytics/cplex-optimizer; Last accessed Aug. 2018.
[2] M. Berman, et. al., “GENI: A federated testbed for innovative network [51] Open vSwitch - http://openvswitch.org; Last accessed Aug. 2018.
experiments”, Elsevier ComNet, 2014.
[3] F. Esposito, et al., “Slice Embedding Solutions for Distributed Service Dmitrii Chemodanov received his MS degree from
Architectures”, ACM Comput. Surv., 2013.
[4] A. Fischer, et al., “Virtual Network Embedding: A Survey”, IEEE CST, the Department of Computer Science at Samara
2013. State Aerospace University, Russia in 2014. He is
[5] F. Esposito, “CATENA: A Distributed Architecture for Robust Service currently a PhD student in the Department of Electri-
Function Chain Instantiation with Guarantees”, IEEE NetSoft, 2017. cal Engineering and Computer Science at University
[6] J. G. Herrera, J. F. Botero, “Resource allocation in NFV: A comprehensive of Missouri-Columbia. His current research interests
survey”, IEEE TNSM, 2016. include distributed and cloud computing, network
[7] A. Mendiola, et al., “A survey on the contributions of Software-Defined and service management, and peer-to-peer networks.
Networking to Traffic Engineering”, IEEE CST, 2016.
[8] L. Lozano, A. L. Medaglia, “On an exact method for the constrained Flavio Esposito is an Assistant Professor in the
shortest path problem”, Elsevier COR, 2013. Computer Science Department at SLU and a Visiting
[9] X. Chen, et al., “Multi-criteria Routing in Networks with Path Choices”, Research Assistant Professor in the EECS Depart-
IEEE ICNP, 2015.
[10] P. Van Mieghem, F. Kuipers, “Concepts of exact QoS routing algo- ment at University of Missouri-Columbia. He re-
rithms”, IEEE/ACM ToN, 2004. ceived his Ph.D. in CS at Boston University in 2013,
[11] R. G. Garroppo, et al., “A survey on multi-constrained optimal path and his MS in Telecommunication Engineering from
computation: Exact and approximate algorithms”, Elsevier ComNet, 2010. University of Florence, Italy. His research interests
[12] K. Nahrstedt, S. Chen, “Coexistence of QoS and best-effort flows- include network management, network virtualization
routing and scheduling”, IEEE TIW, 1998. and distributed systems.
[13] X. Yuan, X. Liu, “Heuristic Algorithms for Multi-constrained Quality
of Service Routing”, IEEE INFOCOM, 2001. Prasad Calyam received his MS and PhD degrees
[14] J. M. Jaffe, “Algorithms for Finding paths with Multiple Constraints”, from the Department of Electrical and Computer
IEEE Network, 1984. Engineering at The Ohio State University in 2002
[15] C. Y. Hong, et al., “Achieving high utilization with software-driven and 2007, respectively. He is currently an Associate
WAN”, ACM SIGCOMM CCR, 2013. Professor in the Department of Electrical Engineer-
[16] S. Jain, et al., “B4: Experience with a globally-deployed software defined ing and Computer Science at University of Missouri-
WAN”, ACM SIGCOMM CCR, 2013.
[17] R. Mijumbi, et al., “A path generation approach to embedding of virtual Columbia. His current research interests include dis-
networks”, IEEE TNSM, 2015. tributed and cloud computing, computer networking,
[18] Y. Wang, et al., “A branch-and-price framework for optimal virtual and cyber security. He is a Senior Member of IEEE.
network embedding”, Elsevier ComNet, 2016. Andrei Sukhov received his PhD degree in Moscow,
[19] M. Chowdhury, et al., “Vineyard: Virtual network embedding algorithms
with coordinated node and link mapping”, IEEE/ACM ToN, 2012. in Physics and Mathematics in 1993. In 2007 he
[20] Z. Wang, J. Crowcroft, “QoS Routing for Supporting Resource Reser- received Dr.Sc. degree in computer networking at
vation”, IEEE JSAC, 1996. Moscow State University of Electronics and Math-
[21] F. Esposito, et al., “On Distributed Virtual Network Embedding with ematics (MIEM HSE). He is currently a Professor
Guarantees”, ACM/IEEE ToN, 2014. in the Department of Supercomputers and General
[22] E. Danna, et al., “A practical algorithm for balancing the max-min fair- Informatics at Samara National Research University.
ness and throughput objectives in traffic engineering”, IEEE INFOCOM, His current research interests include computer net-
2012. works and security.