Computer Networks 81 (2015) 79–95

Contents lists available at ScienceDirect

Computer Networks
journal homepage: www.elsevier.com/locate/comnet

Survey Paper

Software-Defined Networking: A survey

Hamid Farhady, HyunYong Lee ⇑, Akihiro Nakao
Graduate School of Interdisciplinary Information Studies, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan

a r t i c l e i n f o a b s t r a c t

Article history: Software-Defined Networking (SDN) is considered promising to simplify network manage-
Received 30 January 2014 ment and enable research innovations based on the decomposition of the control and data
Received in revised form 16 February 2015 planes. In this paper, we review SDN-related technologies. In particular, we try to cover
Accepted 17 February 2015
three main parts of SDN: applications, the control plane, and the data plane anticipating
Available online 24 February 2015
that our efforts will help researchers set appropriate and meaningful directions for future
SDN research.
Keywords:
Ó 2015 Elsevier B.V. All rights reserved.
Software-Defined Networking
Programmable networks
OpenFlow
Control and data planes

1. Introduction To overcome such limitations, the Software-Defined

Networking (SDN) concept has been proposed. SDN can
Modern Internet infrastructure consists of a set of net- be defined as ‘‘an emerging network architecture where the
working devices with purpose-built application-specific network control is decoupled and separated from the forward-
integrated circuits (ASICs) and chips that are used to achieve ing mechanism and is directly programmable’’ [1]. In SDN,
high throughput, thus realizing hardware-centric network- there is a logically centralized controller that has a net-
ing. However, the current hardware-centric Internet infras- work-wide view and controls multiple packet-forwarding
tructure suffers from several shortcomings, such as devices (e.g., switches) that can be configured via an inter-
manageability, flexibility, and extensibility. Networking face (e.g., ForCES [2] and OpenFlow [3]). For example, an
devices usually support a handful of commands and con- OpenFlow switch has one or more forwarding tables that
figurations based on a specific embedded operating system are controlled by a centralized controller, thus realizing
(OS) or firmware. As a result, network administrators are programmability in the control plane. Forwarding tables
limited to a set of pre-defined commands, even though it are used to control packets (e.g., forwarding or dropping).
would be easier, simpler, and more efficient to support more Therefore, according to the controller policy that manages
protocols and applications if it were possible to program the forwarding tables, an OpenFlow switch can act as a
network controls in ways that are more responsive and flex- router, switch, NAT, firewall, or exhibit similar functions
ible. In addition, researchers usually have to make their own that depend on packet-handling rules. Due to its decoupled
testbeds or take advantage of simulations rather than real nature, SDN is believed to be a new networking technology
world implementation scenarios to realize their ideas. In that simplifies today’s network operation and manage-
other words, innovation and research is costly under the ment and also enables network innovations and new net-
current condition of hardware-centric networking. work designs. Because of the potential benefits of SDN in
the current Internet and future Internet architectures, such
⇑ Corresponding author. as information-centric networking [4], it has gained con-
E-mail addresses: [email protected] (H. Farhady), ifjesus7@gmail. siderable attention from the community.
com (H. Lee), [email protected] (A. Nakao).

http://dx.doi.org/10.1016/j.comnet.2015.02.014
1389-1286/Ó 2015 Elsevier B.V. All rights reserved.
80 H. Farhady et al. / Computer Networks 81 (2015) 79–95

An SDN instance consists of three major parts: applica-

tion, control plane, and data plane (Fig. 1). The application
label indicates a part that exploits the decoupled control
and data plane to achieve specific goals, such as a security
mechanism [5] or a network measurement solution [6].
Applications communicate with a controller at the control
plane via the northbound interface of the control plane. The
control plane is the part that manipulates forwarding
devices through a controller to achieve the specific goal
of the target application. The controller uses the south-
bound interface of the SDN-enabled switch to connect to
the data plane. The data plane is the part that supports a
shared protocol (e.g., OpenFlow) with the controller and
handles the actual packets based on the configurations that
are manipulated by the controller. Therefore, we believe
that deeply understanding each part and investing bal-
anced research attention into each of the three parts is
important to maximize the potential benefits of SDN. Fig. 2. High-level illustration of networking paradigms.
In this paper, we survey existing efforts on each part. In
a similar spirit, several SDN survey papers [7–10] have
been published recently. However, those papers mainly (and even new versions of existing protocols) is a challenge
focus on the control plane and the application part, par- because all the switches need to be updated or replaced. In
ticularly OpenFlow-related work. On the other hand, in this contrast, in SDN, switches become simpler (in terms of
paper, we try to cover all three parts anticipating that a removing the control plane from the device) forwarding
survey of the three parts will help researchers set appro- devices and a centralized controller derives the control
priate and meaningful objectives for future SDN research. mechanism of the network. This decomposition of control
The rest of the paper is organized as follows. In Sec- and data planes allows easier deployment of new protocols
tion 2, we briefly introduce the history of SDN and several and services because the decomposition enables us to pro-
SDN-related terms. Section 3 introduces examples of the gram switches via the controller. Finally, a hybrid approach
application and Section 4 surveys the control plane related supports both distributed and centralized control planes.
work. Section 5 surveys the data plane technologies. Final- For example, common commercial OpenFlow switches
ly, after we discuss several future directions for SDN (see Section 5.1.2) are hybrid switches that support Open-
research in Section 6, Section 7 concludes this paper. Flow in addition to traditional operation procedures and
protocols.

2. Background 2.2. Historical foundations

2.1. Networking paradigm Even though SDN is popular nowadays, several SDN
concepts have been around for many years. In the follow-
Networking paradigms can be divided into three types ing, we briefly review each of them.
according to the deployment of control and data planes
(Fig. 2). In the traditional hardware-centric networking,
2.2.1. Active networks
switches are usually closed systems that have their own
In active networks, each packet carries a program rather
control and data planes and support manufacturer-specific
than raw data. When a network node receives a packet, the
control interfaces. Therefore, in the traditional hardware-
program inside the packet is executed and then different
centric networking, deploying new protocols and services
types of actions can be triggered against the packet (e.g.,
forward or drop) based on the data plane design. The idea
of active networks relates to some in-network processing
services and tries to treat network devices as an environ-
ment that reacts based on what the packet carries rather
than passively transmitting bits from one node to another
[11].
The active networking approach shows less interest in
the control plane and is instead focused on providing a
smart environment similar to end-point PCs compared
with current dumb switches that can execute a limited
set of procedures. That is, if we consider the end-points
of a network, which are PCs, and servers as smart devices,
then we can consider network controls (e.g., switches and
routers) as dumb devices. They are dumb because they can
Fig. 1. Components of SDN. execute limited types of tasks, albeit they do so rapidly.
 H. Farhady et al. / Computer Networks 81 (2015) 79–95 81

Therefore, active networks help to create smarter network definition platform to automatically manipulate a large
controls and intermediate nodes. In active networks, pack- number of routers. Other examples include solutions for
ets are the entities that control the behavior of a node, scalable control and management of an ATM network
starting with booting it up and proceeding onto the (such as DCAN [26]). As a complementary version of SNMP,
method of handling a single packet. A series of studies fol- NETCONF [27] allows the definition of a new API for net-
lowed the idea of active networking including network work controls to modify configuration data.
management (e.g., Smart packets [12]), dynamic protocol
deployment (e.g., ANTS [13]), and network architecture 2.3. SDN-related terms
studies (e.g., SwitchWare [14,15]). The main shared char-
acteristic between SDN and active networks is fostering There are several terms that are related to SDN. For
programmability. However, the current SDN definition clarity’s sake, we will introduce those terms.
mostly focuses on control plane programmability whereas OpenFlow. There is a misconception that SDN and
active networks are more focused upon data plane OpenFlow are equivalent. This is mainly due to the fact
programmability. that the term SDN was coined after the introduction of
OpenFlow [28] and OpenFlow is one representative API
2.2.2. Open signaling to setup an SDN instance. However, OpenFlow is an API
In the 1990s, through a series of workshops as well as used for the communication between the controller and
DARPA’s developments in active networking, the Open Sig- switches, as shown in Fig. 1. In other words, OpenFlow is
naling (OPENSIG) community expanded. As the name sug- one way of controlling network-wide forwarding behavior
gests, OPENSIG takes a telecommunication approach under an SDN concept. Moreover, different APIs and tools
toward programmability. OPENSIG suggests modeling of (e.g., JunOS SDK [29] and Cisco ONE [30]) can be used to
communication hardware using open programmable inter- achieve similar objectives.
faces that lead to open access to switches and routers; it Network Virtualization. SDN and Network Virtualiza-
acts as a facilitator for third-party software providers to tion (NV) [31] are closely related to each other in some
enter the telecommunications software market. An inter- aspects. NV enables simultaneous coexistence of multiple
esting shared perspective between OPENSIG and SDN is a network instances on a shared physical infrastructure;
clear distinction between concerns. OPENSIG establishes thus, NV can be used to run an SDN solution. SDN can also
a clear distinction between transport, control, and manage- be used to realize NV by distinguishing and forwarding
ment in the network [16]. The idea first defines open pro- flows to different slices.
grammable interfaces for network controls and then lets Network Function Virtualization. Network Function
service providers manipulate the network state using a Virtualization (NFV) [32] refers to the virtualization of
middleware toolkit such as common object request broker specific in-network functions (e.g., firewalls, WAN optimiz-
architecture (CORBA) to be able to introduce new services ers, load balancers, and VPN gateways) that may be con-
easier and faster. nected together to provide value-added services. A
virtualized network function can be implemented based
2.2.3. Decomposition of control and data planes on one or more virtual networking devices running differ-
Although active networks seek programmability in net- ent software. Similar to active networks, NFV mainly focus-
working, some projects in the community look for separa- es on data plane programmability. In this regard, NFV may
tion of data and control planes to simplify the network be able to extend SDN in terms of data plane programma-
architecture and provide some sort of abstraction. Tempest bility because current SDN is more concerned with control
[17] was an ATM virtualization project intended to enable plane programmability. Furthermore, an NFV solution (e.g.,
multiple coexisting virtual networks on a shared physical a virtual appliance) can operate in SDN.
infrastructure including a software controller to manage
forwarding decisions. Forwarding and Control Element
Separation (ForCES [2]) is an Internet Engineering Task 3. Applications
Force (IETF) standard which provides an interface between
control and data plane. Routing Control Platform (RCP The logically centralized controller-based SDN para-
[18]), Path Computation Element, (PCE [19]) SoftRouter, digm makes network information collection and dynamic
[20] and Intelligent Route Service Control Protocol (IRSCP networking policy changes easier. As a result, SDN tech-
[21]), all foster a centralized approach to controlling the niques can be exploited to achieve specific purposes in var-
network. Security incentives also encourage centralization ious areas.
of the control plane. SANE [22] and Ethane [23] both
devote some consideration to 4D [24] projects, and pro- 3.1. Specific purposes
pose flow-level access control in the network. The original
implementation of the OpenFlow API is based on the 3.1.1. Network management
Ethane switch. An efficient network management requires information
about current network status and timely change of net-
2.2.4. Other work work control. Aster⁄x [33] uses OpenFlow to measure the
There are dozens of works in the literature that share state of the network and directly control the forwarding
objectives with, SDN but work from different perspectives. paths. In [34], the authors utilize the network information
For example, XMILE [25] is a dynamic XML-based policy (collected by the centralized controller) to improve the
82 H. Farhady et al. / Computer Networks 81 (2015) 79–95

network utilization and reduce packet losses and delays. In network targets and launch attacks. In this approach, the
particular, they focus on the case where SDN is incremen- OpenFlow controller dynamically allocates a random virtu-
tally introduced into an existing network. In addition, they al IP (translated to/from the real IP of the host) to each host
formulate the SDN controller’s optimization problem for to avoid exposing an authentic IP that could be used by the
traffic engineering with partial deployment. To enrich net- attacker. In [51], the authors implement several traffic
work management services, FleXam [35] enables the anomaly detection algorithms using OpenFlow compliant
OpenFlow controller to access packet-level information, switches and an NOX controller to deal with network secu-
and the authors in [36,37] try to incorporate application rity problems in home and office networks. Through
awareness into an SDN that is currently agnostic to appli- experiments, they show that the SDN-supported approach
cations. There have been some works that propose new results in more accurate identification of malicious activ-
network management systems using SDN technologies. In ities versus the ISP-driven approach. Resonance [52]
[38], the authors combine legacy network management secures enterprise networks where the switches enforce
functions (e.g., discovery and fault detection) with the dynamic access control policies based on flow-level infor-
end-to-end flow provisioning and control enabled by mation. Resonance allows the switches to take actions
SDN. In [39], based on SDN technologies, the authors try (e.g., dropping) to enforce high-level security policies.
to improve various aspects of network management, such Some studies use SDN capabilities to develop applications
as enabling frequent changes to network conditions and such as edge-based authentication gateways [53] or secu-
state, providing support for network configuration in a rity application development frameworks [54].
high-level language, and better visibility and control over
tasks such as network diagnosis and troubleshooting. 3.2.1. Others
OpenSketch [6] is a software-defined traffic measurement Virtual Machine (VM) migration is a promising way to
architecture proposed to support more customized and ensure load balancing and power saving by VM realloca-
dynamic measurement while guaranteeing the measure- tion. However, VM migration is a difficult task because it
ment accuracy. To achieve this goal, OpenSketch separates requires updates of the network state; this may lead to
the measurement data plane (supporting many measure- inconsistency and violations of service level agreement.
ment tasks) from the control plane (configuring and allo- In [55], the authors exploit the abilities of SDN (i.e., run-
cating different measurement tasks). In [40], the authors ning algorithms in a logically centralized controller and
propose an inter-domain routing solution using an NOX- manipulating the forwarding layer of switches) to handle
OpenFlow architecture that was originally created only VM migration issues. In particular, given the network
for routing in enterprise networks. Plug-n-Serve [41] is a topology, SLA requirements, and set of VMs that are to be
web traffic load-balancing solution that leverages an Open- migrated, along with their new locations, the algorithms
Flow controller to dynamically change switch configura- (running in the SDN controller to orchestrate these
tion. Real-time policy checking [42,43] on the controller changes within the network) output an ordered sequence
is proposed to keep OpenFlow rules consistent and thus of VMs to migrate, as well as a set of forwarding state
control the network traffic in a stable way. changes. To support VM migration that is transparent to
applications, LIME [56] clones the data plane state to a
3.1.2. Middlebox new set of switches and then incrementally migrates
Middleboxes (such as a load balancer) and in-network VMs by leveraging SDN technologies. The traditional IP
services (such as in-network caching) are very popular in multicast technique based on IGMP is not scalable because
today’s networks [44]. SDN technologies can be exploited it consumes a large amount of resources such as IP multi-
to produce better middlebox-based services. SIMPLE [45] cast tables and CPU time. In [57], the authors propose to
is an SDN-based policy enforcement layer for efficient mid- manage an IP multicast in overlay networks using Open-
dlebox-specific traffic engineering. SIMPLE exploits SDN Flow. In particular, they eliminate periodic Join/Leave mes-
technologies to ensure that the traffic is directed through sages and achieve more than 4,000 tenants. It is also
the desired sequence of middleboxes to realize efficient possible to combine SDN and NFV to provide a virtual net-
middlebox applications. FlowTags [46] allows middleboxes working lab for computer science education without any
to add tags to outgoing packets so that tags are used on simulation [58].
switches and middleboxes for systematic policy enforce-
ment. SDN facilitates to embed various in-network ser- 3.3. Networks
vices, such as advertisement targeting [47], where a
software control in the middle of the network injects relat- 3.3.1. Data center network
ed advertisements based on the session content. The ElasticTree [59] is a network-wide energy optimizer
general framework for developing in-network services or that monitors data center traffic conditions and chooses
middlebox functionalities is discussed in [48,49]. the set of network elements that must stay active to meet
the performance requirements. The authors discuss several
3.2. Security strategies to find minimum-power network subsets. Scis-
sor [60] tries to save energy by removing redundant traffic.
The centralized control of SDN is useful for implement- Scissor replaces the redundant header information with a
ing security applications. OpenFlow Random Host Muta- flow ID to be used for the forwarding. Scissor leverages
tion (OFRHM) [50] is a technique that hides network SDN technologies to dynamically allow switches to route
assets from external/internal scanners that try to discover packets based on the flow IDs. NCP [61] supports scalable
 H. Farhady et al. / Computer Networks 81 (2015) 79–95 83

service replication in data centers through SDN. NCP iden- support various functionalities (e.g., authentication, mobi-
tifies flows based on network addresses and ports, and spe- lity, and AP association decision). Odin allows a network
cifies a replication target for each identified flow. NCP then operator to implement enterprise WLAN services as net-
determines the ideal switch for replication and installs cor- work applications while not requiring client-side modifica-
responding forwarding rules so that the identified flow tions. OpenRadio [78] proposes a programmable wireless
goes to a designated server. In [62], the authors exploit data plane by providing modular and declarative program-
wildcard rules of switches to direct large aggregates of cli- ming interfaces across the entire wireless stack. OpenRadio
ent traffic to server replicas in a scalable way. can be used to realize modern wireless protocols, such as
Some research tries to satisfy the need for customized WiFi and LTE, while providing flexibility to modify the
routing and management of distributed data centers that PHY and MAC layers. OpenRoads [79,80] aims to create
cannot be easily achieved by a traditional WAN architec- an open platform where various mobility solutions, net-
ture. B4 [63] exploits OpenFlow to connect Google’s data work controllers, and routing protocols are examined.
centers to satisfy massive bandwidth requirements, maxi- OpenRoads provides control of the data path and device
mize average bandwidth, and enable rate limiting and mea- configuration using OpenFlow and SNMP, respectively.
surement at the edge. The centralized traffic engineering of The centralized control of SDN can also be used to provide
B4 drives links to nearly 100% utilization. By exploiting the flow-based routing and forwarding capabilities in wireless
centralized control, SWAN [64] aims at boosting the utiliza- mesh networks [81], to abstract all base stations in a local
tion of inter-data center networks. SWAN controls when geographic area [82], or to achieve complete virtualization
and how much traffic each service sends, and re-configures and programmability in radio access networks [83]. In [84],
the data plane to match current traffic demand. M2cloud the authors argue that SDN can simplify the design and
[65] aims to support global network performance isolation management of cellular data networks and enable new
for concurrent tenants by providing scalable network con- services. In [85], the authors exploit OpenFlow to optimize
trol for multi-site data centers. M2cloud employs two-level handovers in heterogeneous wireless environments, par-
controllers to provide each tenant with flexible virtualiza- ticularly for media-independent handover procedures.
tion support in both intra- and inter-data center networks.
3.3.4. Home network
3.3.2. Optical network Small networks, such as home networks, are becoming
The use of SDN technologies in optical transport net- more complex to manage because the market is presently
works may offer many benefits (e.g., improved network saturated with low-cost networking devices, which are
control and management flexibility) [66]. The potential popular among end users. SDN can be utilized to manage
benefits and challenges of extending SDN concepts to var- those networks as well. The authors in [51] argue that
ious transport network architectures (including optical the advent of SDN provides a unique opportunity to effec-
wavelength, and fiber and circuit switches) are discussed tively detect and contain network security problems in
in [67]. In [68], the combination of an SDN controller and home networks and shows that SDN-enabled approaches
optical switching is exploited to optimize application per- can outperform alternatives. The authors in [86] present
formance and network utilization of big data applications. a design for a home router that focuses on monitoring
It is shown that the combination has great potential to and controlling network traffic flows to provide a user with
improve application performance with relatively small control over his or her network behavior. A home network
configuration overhead. In [69], it is argued that the use data recorder (prototyped after NOX) is proposed to man-
of SDN in the transport layers can enable packet-optical age and troubleshoot home networks [87]. By exploiting
integration and improves transport network applications. the programmable network switches that enable flexible
The authors discuss extensions to OpenFlow v1.1 to remote management, the authors in [88] propose a way
achieve control of switches in the multi-technology trans- to outsource the management of home networks to a third
port layer. Furthermore, OpenFlow and GMPLS control party with operations expertise and a broader view of net-
plane integration is exploited for software-defined packet work activity.
over optical networks [70]. Moreover, OpenFlow-based
control plane architectures for optical SDN [71] and 3.3.5. Information-Centric Networking
Flexi-Grid optical networks [72] are proposed. In [73], the Information-Centric Networking (ICN) [4] has been pro-
authors introduce a QoS-aware unified control protocol posed to solve fundamental limitations of the current
for optical burst switching. OpenFlow is also used to Internet in supporting content-oriented services. Because
dynamically create a bidirectional wavelength circuit for ICN takes a revolutionary approach, its development
a TCP flow [74] or wavelength path control for light-path requires a new network architecture. In this regard, SDN
provisioning in transparent optical networks [75]. A simple is a valuable tool to build a testbed for ICN [89,90]. In
programmable architecture is proposed in [76] to abstract [91], the authors discuss how ICN functionalities can be
a core transport node as a programmable virtual switch implemented over an OpenFlow network and how Open-
that meshes well with the SDN paradigm while leveraging Flow should be modified to better suit ICN functionalities.
OpenFlow protocol for control. The question of how SDN and ICN could concretely be com-
bined, deployed, and tested is addressed in [92]. The
3.3.3. Wireless network authors outline a possible realization of a novel design
By exploiting a light virtual AP abstraction, Odin [77] for ICN solutions and point to possible testbed deploy-
introduces programmability in enterprise WLANs to ments for future testing. SDN features can also be utilized
84 H. Farhady et al. / Computer Networks 81 (2015) 79–95

to realize ICN capabilities in an efficient manner. In [93], flow-based forwarding by keeping one or more flow tables.
the authors present an OpenFlow-based architecture that Flow tables are used to determine how the given packets
performs efficient caching for ICN where caching strategy are handled. Each flow table entry contains a set of packet
might dramatically influence performance and efficiency fields to match (e.g., Ethernet addresses and IP addresses),
of ICN. C-flow [94] seeks to achieve efficient content deliv- and a set of actions to be applied upon a match (e.g., send-
ery by leveraging the current OpenFlow functionalities. C- out-port, modify-field, or drop). Flow entries also maintain
flow can deliver content to mobile hosts by using the byte- counters to collect statistics for particular types of flow
range option of the HTTP header. In addition, multicast/ (e.g., the number of transmitted bytes). When the switch
anycast is naturally supported by mapping between files receives a packet that causes a miss in the forwarding table
and their corresponding IP addresses. matching process, the action to be taken depends on the
table-miss flow entry. That packet can be forwarded to
4. Control plane the controller, passed to the next flow table, or dropped.
When the packet is forwarded to the controller, the con-
The control plane acts as an intermediary layer between troller then decides how to handle this packet. The con-
applications and the data plane. The control plane in SDN is troller can drop the packet, or it can add a flow entry
called a controller; it communicates with the application via that gives the switch direction on how to forward similar
the northbound interface and with the switches via the packets in the future. From version 1.1, OpenFlow supports
southbound interface. Therefore, in the control plane, it is multiple flow tables and pipeline processing. Therefore,
important to design interfaces and the controller itself in continuing the matching process on the next flow table
an efficient way. Thus, in this section, we survey existing can be one option for the non-matching packets. Various
efforts to realize and improve various aspects of the control OpenFlow versions have been released, and features of
plane. In particular, we focus on OpenFlow-related research each version are as follows. Version 1.2 supports a more
because OpenFlow is prominently successful, whereas flexible matching structure, which used to be a static
other approaches are not as successful in practice. In fact, fixed-length structure, and more protocols such as IPv6.
other approaches for the control plane (e.g., IETF ForCES Version 1.3 provides more support for protocols such as
[2]) have not been adopted by the major router vendors, MPLS BoS bit and IPv6 extension headers. It also includes
which has hampered incremental deployment [8], even a better expression of switch capabilities (i.e., refactor
though they can be used to achieve the same goals using a capabilities negotiation) and improves metering facilities
somewhat different SDN model [95]. (e.g., per flow metering). Version 1.4 improves the Open-
Flow extensible match introduced in v1.2 that gives more
4.1. OpenFlow flexible classification capabilities to the user to match
packet header fields and classify flows. Version 1.4 also
The OpenFlow protocol defines an API for the communi- supports optical ports. For more detailed description about
cation between the controller and the switches (Fig. 3). the OpenFlow protocol, please refer to [96].
Therefore, both the controller and the switches should
Understand the OpenFlow protocol. The controller manip- 4.2. Controller implementations
ulates the flow tables of the switch by adding, updating,
and deleting flow entries. This happens reactively (when To date, various OpenFlow (compatible) controllers
the controller receives a packet from the switch) or proac- have been publicly released. Unless otherwise stated, the
tively according to the OpenFlow controller implementa- controllers we review here support OpenFlow v1.0 API.
tion. The controller communicates with the switches via
a secure channel. The OpenFlow switch supports the  NOX [97] (C++/Python, Nicira): NOX is the first Open-
Flow controller. NOX Classic were written in C++ and
Python and current NOX is written in C++.
 POX [98] (Python, Nicira): POX is a general SDN con-
troller that supports OpenFlow controller.
 SNAC [99] (C++, Nicira): SNAC is an OpenFlow controller
based on NOX. It supports a graphical user interface and
a policy definition language.
 Maestro [100] (Java, Rice University): Maestro tries to
improve the system throughput by exploiting multicore
processors and parallelism.
 Ryu [101] (Python, NTT): Ryu is a component-based
SDN framework that supports OpenFlow v1.0, v1.2,
and v1.3.
 MUL [102] (C, Kucloud): MUL supports a muli-threaded
infrastructure and a multi-level northbound interface. It
supports OpenFlow v1.0 and v1.3.
 Beacon [103] (Java, Stanford University): Beacon is a
cross-platform and modular OpenFlow controller. It
Fig. 3. OpenFlow. supports event-based and threaded operation.
 H. Farhady et al. / Computer Networks 81 (2015) 79–95 85

 Floodlight [104] (Java, BigSwitch): Floodlight supports a proposed in [119] so as to satisfy the state consistency
broad range of virtual and physical OpenFlow switches and durability requirements. HOTSWAP [120] supports
and can handle mixed OpenFlow and non-OpenFlow the upgrade of an SDN controller to a new version in a dis-
networks. ruption-free manner by maintaining a history of network
 IRIS [105] (Java, ETRI): IRIS is a recursive OpenFlow con- events managed by the old controller. HOTSWAP boot-
troller that aims to support scalability, high availability, straps the new controller by replaying the history. In the
and multi-domain support. case where multiple controllers are used, HyperFlow
 OESS [106] (Perl, NDDI): OESS is a set of softwares to [121] localizes the decision-making to individual con-
configure and control dynamic VLAN networks using trollers by synchronizing network-wide views of Open-
OpenFlow-enabled switches. Flow controllers to minimize the control plane response
 Jaxon [107] (Java, independent developer): Jaxon is a time for data plane requests. The controller uses transac-
NOX-dependent OpenFlow controller. tional semantics to achieve consistent packet processing
 NodeFlow [108] (JavaScript, independent developer): [122]. Some other works try to understand various aspects
NodeFlow is an OpenFlow controller written for of the state consistency. In [123], it is shown that control-
Node.JS. state inconsistency significantly degrades the performance
 ovs-controller [109] (C, independent developer): ovs- of logically centralized control applications. The trade-off
controller is an OpenFlow reference controller packaged between update time and rule-space overhead [124] and
with Open vSwitch [110]. the trade-off between maintaining consistency during con-
 Flowvisor [111] (C, ON.LAB): As a transparent proxy figuration updates and the update performance [125] are
between OpenFlow switches and multiple OpenFlow also studied.
controllers, Flowvisor allows multiple tenants to share
the same physical infrastructure by dividing traffic flow 4.3.2. Scalability
space into slices. The centralized control of SDN naturally faces scalabil-
 RouteFlow [112] (C++, CPQD): RouteFlow provides vir- ity issues. Many works try to improve the scalability, par-
tualized IP routing services over OpenFlow-enabled ticularly by reducing the overhead of the centralized
hardware. RouteFlow is composed by an OpenFlow con- controller in various aspects. DIFANE [126] keeps all traffic
troller, an independent RouteFlow server, and a virtual in the data plane by selectively directing packets through
network environment. intermediate switches that store the necessary rules, so
 Helios [113] (C, NEC): Helios is an extensible OpenFlow as to avoid a bottleneck at the controller and achieve better
controller that provides a programmable shell for per- performance and scalability. For this, the controller parti-
forming integrated experiments (not publicly tions the forwarding rules over the switches. Similarly,
available). DevoFlow [127] handles short flows at the OpenFlow
switch while only large flows are directed to the controller.
4.3. Controller designs Kandoo [128] applies two layers of controllers to limit the
overhead of frequent events on the control plane so as to
In OpenFlow, a physically or logically centralized con- realize scalable SDN. Controllers at the bottom layer try
troller manages multiple switches. Therefore, the con- to handle most of the frequent events using the state of a
troller design significantly affects the overall performance single switch. A logically centralized controller of the top
of the network. Due to this, there has been much research layer is only used to handle the events that cannot be han-
on enhancing the controller design to improve the perfor- dled by the controllers at the bottom layer. A hybrid con-
mance of various aspects. trol model that combines both central control and
distributed control is proposed in [129] to provide more
4.3.1. State consistency relaxed and scalable control. By defining new features such
OpenFlow controllers and switches need to keep the as the proactive flows to be activated under certain condi-
same forwarding policy for stable forwarding. FlowAdapter tions, they try to reduce the overhead of the central con-
[114] allows a consistent forwarding policy to be fitted into troller. The scalability issue in specific areas is also
different types of hardware by converting the flow entry studied. SDN can be a natural platform for network virtual-
rules of the controllers to switch the hardware flow table ization, but scalability becomes an issue in supporting a
pipeline. A similar approach is discussed in [115], which large number of tenants. To handle this scalability issue,
proposes a set of axioms for policy transformation to FlowN [130] exploits a database technology to efficiently
enable rewriting of rules across multiple switches while provide each tenant the illusion of its own topology and
preserving the forwarding policy. Palette distribution controller. To support scalable SDN slicing, AutoSlice
framework [116] decomposes large SDN tables into small [131] introduces a distributed hypervisor architecture that
ones and then distributes them across the network while can handle a large number of flow table control messages
preserving the overall SDN policy semantics. Efficient from multiple tenants. OpenFlow protocol can be used to
rule-placement algorithms are also proposed in [117] to realize a load balancer for data centers by dividing traffic
distribute forwarding policies across SDN-inspired net- over the server replicas. However, the current OpenFlow
works while managing rule-space constraints. ONOS approach may lead to a huge number of rules for the
[118] maintains consistency across the distributed state switches and a heavy load on the controller. In [62], the
by providing a network topology database to the con- authors argue that the controller should exploit the wild-
troller. The idea of a network information base is also card rules to direct large aggregates of client traffic to
86 H. Farhady et al. / Computer Networks 81 (2015) 79–95

server replicas in a more scalable way. They present sever- the SDN vulnerabilities and sketch the design of a secure
al methods to compute wildcard rules to achieve a target and dependable SDN control platform.
distribution of the traffic and to adapt to changes in load-
balancing policies. Several scalability trade-offs in SDN 4.3.5. Availability
design space are discussed in [132]. The OpenFlow controller and switches should be robust
in various circumstances. In [144], the authors discuss a
4.3.3. Flexibility runtime system that automates failure recovery by spawn-
Some works try to support the same functions in a more ing a new controller instance and replaying inputs
efficient and flexible way. ElastiCon [133] manages the observed by the old controller. The controller can install
controller pool that is dynamically expanded or reduced static rules on the switches to verify topology connectivity
according to traffic conditions. In addition, the load is and locate link failures based on those rules [145]. For fast
dynamically shifted across controllers to gracefully handle recovery, frequent issuing of monitoring messages is need-
the traffic. The reconfigurable match tables model is pro- ed, but this may place a significant load on the controller.
posed in [134] to permit the forwarding plane to be chan- To realize scalable fault management, in [146], the authors
ged without modifying hardware, and to enable the propose to implement a monitoring function in the switch-
programmer to modify all header fields much more com- es so that those switches can send monitoring messages
prehensively than in OpenFlow. To realize far more flexible without placing a load on the controller. RuleBricks [147]
processing of counter-related information, the software- is a system to embed high availability support in existing
defined counters that utilize general-purpose CPUs rather OpenFlow policies by introducing three key primitives:
than ASIC-based inflexible counters is introduced in drop, insert, and reduce. In this work, the authors describe
[135]. In a similar spirit, a CPU in the switches is used to how these primitives can express various flow assignments
handle not only the control plane but also data plane traffic and backup policies.
to overcome the limitations of the ASIC-based approach
[136]. In [137], the authors present research directions that 4.3.6. Placement
could significantly reduce TCAM and control plane require- In the controller-driven SDN architectures, the position
ments through classifier sharing and reuse of existing of the controller may impact the performance of an SDN. In
infrastructure elements. In [138], the authors propose the [148], the authors try to answer the following question:
extensible session protocol to proactively install flow ‘‘given a topology, how many controllers are needed, and
entries based on the requirements of the application. The where should they go?’’ To answer this question, they
goal of this work is to provide a general and extensible pro- examine fundamental limits to controlling the plane
tocol for managing the interaction between applications propagation latency. Through experiments, they show that
and network-based services, as well as between the the answers depend on the topology and that one con-
devices. Modularity is one of the key factors in managing troller location is often sufficient to meet existing reac-
complexity of any software system, including SDN. In tion-time requirements. In [149], the authors try to
[139], the authors introduce new abstractions for building understand the impact of the latency between an Open-
applications from multiple, independent modules that Flow switch and its controller. They show that latency
jointly manage network traffic to ensure that rules drives the overall behavior of the network and bandwidth
installed to perform one task do not override other rules. affects the number of flows that the controller can process.
In [150], the authors solve the controller placement issue
4.3.4. Security in such a way that the reliability of control networks is
OpenFlow architecture may suffer from trust issues on maximized. They present several placement algorithms
OpenFlow applications because it allows third-party devel- along with a metric to characterize the reliability of SDN
opment. The abuse of such trust could lead to various types control networks.
of attacks that impact the entire OpenFlow network. One
general way to deal with this potential trust issue is to con- 4.4. Development tools
strain the applications. PermOF [140] allows a minimum
privilege to the applications. The authors summarize a set There have been many approaches to assisting various
of permissions to be enforced at the API entry of the con- aspects of SDN development.
troller. Similarly, ForNox [141] provides role-based autho-
rization and security constraint enforcement for the NOX 4.5. Simulators and frameworks
OpenFlow controller. FRESCO [54] provides a programming
framework to execute and link security-related applica-  Mininet [151]: Mininet emulates multiple virtual Open-
tions. Some works try to unveil potential vulnerabilities of Flow switches, end hosts, and controllers on a single
SDN. In [140], the authors discuss several vulnerabilities machine. Mininet supports OpenFlow v1.3 and NOX.
of the OpenFlow protocol as it is currently deployed by  ns-3 [152]: ns-3 supports OpenFlow v0.89.
hardware and software vendors. In [142], the authors intro-  OMNeT++ [153]: OMNeT++ supports OpenFlow v1.2
duce a new attack that fingerprints SDN networks and through a plugin.
launches further efficient-resource- consumption attacks  EstiNet 8.0 OpenFlow network simulator [154,155]: It
(e.g., DDOS-like attacks that issue many requests to the cen- can simulate thousands of OpenFlow v1.3.2 and v1.0.0
tralized controller supporting the reactive-mode). In [143], switches and run the real-world NOX, POX, Floodlight,
the authors describe several threat vectors that may exploit and Ryu controllers.
 H. Farhady et al. / Computer Networks 81 (2015) 79–95 87

 Trema [156]: Trema is a framework for developing 4.7. Configuration languages

OpenFlow controllers in Ruby and C.
 Mirage [157]: Mirage is an exokernel for constructing Some works try to assist SDN development by providing
network applications across a variety of platforms and high-level abstractions. To maintain the simplicity of the
supports OpenFlow. centralized programming model of SDN that can be com-
promised by challenges in managing low-level details,
Maple [170] simplifies SDN programming by allowing a
4.6. Debuggers
user to decide the behavior of an entire network based
on a standard programming language. In addition, it pro-
 Cbench [158]: Cbench tests OpenFlow controllers by
vides an abstraction that runs on every packet entering a
emulating a bunch of OpenFlow switches and generat-
network to overcome the challenge of translating a high-
ing packet-in events for new flows.
level policy into sets of rules on distributed switches. To
 NICE [159,160]: Based on the combination of the model
enable fault-tolerant network programs, FatTire [171] sup-
checking and symbolic execution, NICE tests OpenFlow
ports a new programming construct based on regular
controller programs for the NOX controller platform to
expressions that allow a user to specify the set of paths
find potential programming errors that make communi-
that packets may take through the network and the degree
cations less reliable.
of fault tolerance required. Some studies suggest policy
 SDN Troubleshooting System (STS) [161]: To find the
declaration languages such as FML [172], which uses
erroneous codes that lead the controller software to
NOX to calculate and deploy flow table entries on switches,
break in an automatic way, this tool simulates the
or Procera [173], which also handles the controller archi-
devices of a given network and allows a user to pro-
tecture. The Frenetic project [174] proposes ideas for
grammatically generate tricky test cases.
designing a compiler, run-time environment, and modular
 Oflops [162,163]: Oflops quantifies an OpenFlow switch
programming constructs for SDN [172], as well as con-
performance in various aspects by supporting a modu-
structs for certain tasks such as updates [175].
lar framework for adding and running implementa-
tion-agnostic tests.
 OFTest [164]: OFTest is a Python-based framework that 4.8. Testbeds
tests basic functionality of OpenFlow switches.
 OFRewind [165,166]: OFRewind enables a user to Several testbeds support OpenFlow protocol.
record the control and data traffic and to replay it at
an adaptive rate for network-level troubleshooting  Planetlab Europe [176]: It supports OpenFlow through
and debugging. sliver-ovs that is a modified version of Open vSwtich.
Experimenters can create an OpenFlow overlay network
There also have been several works on improving by specifying the links between PlanetLab Europe
OpenFlow development. Inspired by gdb (i.e., the debug- nodes.
ger for software programs), ndb [167] is proposed to  OpenFlow in Europe: Linking Infrastructure and Appli-
debug network control software in SDN. ndb pinpoints cations (OFELIA) [177]: OFELIA allows researchers to
the sequence of events that lead to a network error by not only experiment on a test network but to control
using familiar debugger actions such as breakpoint and and extend the network itself precisely and
backtrace. To troubleshoot bugs in SDN control software, dynamically.
the authors in [168] propose cross-layer correspondence  National LambdaRail (NLR) [178]: Experimental (i.e.,
checking (to find what problems exist and where in the NetFPGA) and commercial (i.e., Hewlett Packard 6600)
control software the problem first developed) and simula- OpenFlow platforms are deployed within several NLR
tion-based causal inference (to identify the minimal set of Points of Presence (PoPs) to enable researchers to create
events that triggered the problem). VeriFlow [42] tries to and run experiments using realistic scenarios.
check network-wide invariants in real time without  Internet2 [179]: It supports the NDDI-substrate mostly
affecting the network performance. As a layer between consisting of interconnected OpenFlow-capable switch-
the controller and the switches, VeriFlow dynamically es. NEC G8264 switch and NOX controller are used.
checks for network-wide invariant violations while each  Research Infrastructure for large-Scale network Experi-
forwarding rule is inserted. ATPG [169] is an approach ments (RISE) [180]: This testbed is a large scale Open-
for testing and debugging networks including SDNs. Flow testbed on JGN-X network in Japan.
Based on router configurations, ATPG generates a set of  SURFnet [181]: It is an OpenFlow testbed linked to the
test packets to exercise network links and rules to detect Internet2 network.
both functional (e.g., incorrect forwarding rules) and per-  COTN [182]: COTN deploys OpenFlow-enabled switches
formance problems (e.g., a congested queue). HSA [43] is into the backbone of CENIC’s CalREN network and con-
a protocol-agnostic framework that allows checking of nects with emerging OpenFlow testbeds within the
network specifications and configurations to identify an national research networks such as NLR and Internet2.
important class of network failures, including those of  FIBRE [183]: FIBRE project is a Future Internet research
SDNs (e.g., reachability failures), regardless of the proto- platform that hosts joint experiments between Brazilian
cols running. and European researchers.
88 H. Farhady et al. / Computer Networks 81 (2015) 79–95

Moreover, the original Emulab does not support Open- section, we survey existing efforts and implementations
Flow, but there is a research that tries to add functionality for the data plane. Please note that there are not many
to support OpenFlow [184]. SDN-related works in this aspect due to the early vision
of SDN, as we mentioned before. Consequently, the follow-
ing survey includes works that have been performed in dif-
4.9. Industry standardizations and forums
ferent areas. We introduce these works because we believe
that these technologies can be used to improve or realize
There have been several approaches for standardizing,
the functions of the data plane.
coordinating, and implementing SDN and related technolo-
gies such as NFV and cloud computing. ONF [185], backed
5.1. OpenFlow switches
by some university research centers such as ONRC [186],
introduces an OpenFlow standard that is a vital element
5.1.1. Software switches
of the current SDN architecture. In particular, they focus
Several OpenFlow software switch implementations are
on standardizing the northbound and southbound inter-
publicly available. Unless otherwise stated, OpenFlow v1.0
faces. The open source software community, including
is supported.
OpenDaylight [187], OpenStack [188], and CloudStack
[189] is developing basic building blocks for SDN imple-
 OpenFlow reference implementation [194] (C): This is a
mentation for the advancement of SDN. For example,
minimal OpenFlow stack that is used to track OpenFlow
OpenDaylight is intended to be extensible and configurable
specification. The code base is now considered as an
to potentially support emerging SDN open standards (e.g.,
archive (supports v1.1.0 specification) and transferred
OpenFlow, I2RS, VxLAN, PCEP). The IETF Overlay (NVO3,
to the ONF website [185].
L2VPN, TRILL, LISP, PWE3), API (NETCONF, ALTO, CDNI,
 Open vSwitch [110] (C/Python): Open vSwitch is a an
XMPP, SDNP, I2AEX), Controller (PCE, FORCES), Protocol
OpenFlow stack that is used both as a vswitch in virtu-
(IDR, IS-IS, OSPF, MPLS, CCAMP, BFD), and Interface to the
alized environments and has been ported to multiple
Routing System (I2RS) [190] working groups are also
hardware platforms. Open vSwitch currently supports
involved with SDN-related activities. For example, the IETF
multiple virtualization technologies including Xen/Xen-
FORCES working group [2] defines protocols and interfaces
Server, KVM, and VirtualBox.
for the separation of IP control and forwarding, centralized
 CPQD ofsoftswitch13 [195] (C/C++): ofsoftswitch is an
network control, and the abstraction of network infrastruc-
OpenFlow compatible user-space switch implementa-
ture. Furthermore, the IRTF initiated an SDN working group
tion. It supports OpenFlow v1.3.
to contribute to the community. Even though IEEE is still
not involved deeply with SDN, we can see some 802.1
Table 1
Overlay Networking projects that look at SDN-related con- Control plane implementations.
cepts. The Software-Defined Networking Research Group
(SDNRG) [191] studies SDN from various perspectives Name Language Institute Version

(e.g., scalability, abstractions, and programming languages) OpenFlow NOX C++/Python Nicira 1.0
to identify approaches that can be used in near-term and controllers POX Python Nicira 1.0
SNAC C++ Nicira 1.0
future research challenges. Some ITU-T [192] study groups
Maestro Java Rice Univ 1.0
are working on requirements for SDN. ITU-T SG13 is inves- Ryu Python NTT 1.3
tigating SDN-related questions and network virtualization. MUL C Kucloud 1.3
ITU-T SG11 is developing signaling requirements and pro- Beacon Java Stanford 1.0
Floodlight Java BigSwitch 1.0
tocols for SDN, and this work aligns with the functional
IRIS Java ETRI 1.0
requirements and architectures developed by ITU-T SG13. OESS Perl NDDI 1.0
Joint Coordination Activity on SDN (JCA-SDN) is estab- Flowvisor C ON.LAB 1.0
lished for coordinating and helping plan work to ensure RouteFlow C++ CPQD 1.0
that the ITU-T SDN standardization progresses in a well- Jaxon Java – 1.0
NodeFlow JavaScript – 1.0
coordinated manner among relevant SGs (e.g., SG13 on
ovs- C – 1.0
use-cases, requirements, and architecture; SG11 on proto- controller
cols and interoperability). In addition, the Metro Ethernet Helios C NEC 1.0
Forum (MEF) [193] is now introducing SDN technologies Tools Simulation Mininet, ns-
to the Carrier Ethernet paradigm. Finally, ETSI owns an ini- & 3,
tiative on Network Function Virtualization. Framework OMNeT++,
Trema
Table 1 summarizes existing efforts for the control
EstiNet 8.0,
plane implementations. Mirage,
Wakame-
vdc
5. Data plane Debuggers OFlops, Cbench, NICE, STS OFTest,
OFRewind
One basic and primary function of the data plane is Testbeds PlanetLab Europe, OFELIA, NLR, Internet2, RISE,
packet forwarding. In addition, the data plane can support SURFnet, COTN, FIBRE
some types of in-network services to realize various ser- Standards ONF, IETF, SDNRG, ITU-T, IRTF, IEEE, ETSI, MEF
vices such as in-network caching and transcoding. In this
 H. Farhady et al. / Computer Networks 81 (2015) 79–95 89

 Indigo [196] (C): Indigo supports OpenFlow on physical Table 2

and hypervisor-based switches. OpenFlow switches.

 OpenFaucet [197] (Python): As a pure Python imple- Name Language Ver.

mentation of OpenFlow protocol, OpenFaucet can Open Open vSwitch C/Python 1.0
implement both switches and controllers. source ofsoftswitch13 C/C++ 1.3
 Pantou [198] (C): Pontou is an OpenFlow port to the Indigo C 1.0
OpenWRT wireless environment. OpenFaucet Python 1.0
Pantou C 1.0

5.1.2. Commercial switches Commercial Vendor Model Ver.

Currently available commercial OpenFlow switches are Arista 7050 series 1.0
as follows. Brocade 1.0
Dell Force10 Dell Force10 Z9000 and S- 1.0
Series S4810
 Arista: 7050 series (v1.0) Extreme Summit X440, X460, X480, 1.0
 Brocade Communications: MLX Series, NetIron CES ser- and X670
ies, NetIron XMR series, ICX 7750 switch (v1.0) HP 3500, 3800, 5400, 6200, 1.0
 Dell Force10: Dell Force10 Z9000, Dell Force10 S-Series 6600, and 8200 series
IBM RackSwitch G8264 1.0
S4810 (v1.0)
Juniper MX-series 1.0
 Extreme networks: Summit X440, X460, X480, and NEC NoviFlow 1248, 1132 1.3
X670 (v1.0) Pica8 P-3290, P-3295, P-3780, and 1.3
 HP: 3500 series, 3800 series, 5400 series, 6200 series, P-3922
Larch OpenFlow switch 1.0
6600 series, and 8200 series (v1.0)
networks
 IBM: RackSwitch G8264 (v1.0)
 Juniper: MX-series (v1.0)
 Larch networks: the extendible Linux-based OpenFlow
parallelism, SP4 can achieve a reasonable throughput.
switch (v1.0)
FLARE [204] exploits many core general purpose proces-
 NEC: PF5240, PF5248, PF5820, and PF1000 virtual
sors and an optimized version of Click. FLARE can process
switch (v1.0)
40 Gbps of traffic and inherits all the programmability
 NoviFlow: NoviSwitch 1248 and NoviSwitch 1132
characteristics of Click. Some works [205,206] study how
(v1.3)
parallelism affects the performance of software-based
 Pica8: P-3290, P-3295, P-3780, and P-3922 (v1.3)
packet forwarding. In [206], the authors investigate differ-
ent architectural setups of a software router using Click to
Table 2 summarizes OpenFlow-compatible software
ascertain the main performance limitation. They conclude
and commercial switches.
that memory access delay is the primary bottleneck of
the system, and software routers are sensitive to this para-
5.2. Data plane technologies
meter. In [205], an optimization framework is proposed for
parallel processing of packets. The framework produces an
5.2.1. Packet forwarding
optimal mapping of flows to process cores based on server
To improve OpenFlow switching, network processor-
resources and the resource consumption of flows. RouteB-
based acceleration cards are used in [199]. The authors
ricks [207] realizes a high-speed software router based on
show that their proposed approach reduces the packet
multiple servers. Servers are connected using a full mesh of
delay by 20% while maintaining comparable packet for-
wires and use some randomization to pass packets from
warding throughput compared to other existing OpenFlow
the servers that receive the packet to the server that sends
reference designs. The authors in [200] describe an
the packet out of the router. A load-balanced routing algo-
approach to improve lookup performance of OpenFlow
rithm spreads traffic between sender and receiver nodes in
switching in Linux using a standard commodity network
multiple paths without any centralized scheduling
interface card. They show that their approach improves
mechanism. HyperSplit [208] achieves 100 Gbps of packet
the packet switching throughput by 25% compared to reg-
classification throughput based on a tree data structure of
ular software-based OpenFlow switching. In [201], the
rules. Each level of the tree forms a step of the classifica-
authors compare OpenFlow switching, layer-2 Ethernet
tion pipeline on the FPGA. HyperSplit processes a pair of
switching, and layer-3 IP routing performance. In par-
packets in each cycle and supports 50,000 rules in the tree.
ticular, forwarding throughput and packet latency in
underloaded and overloaded conditions with different traf-
fic patterns are analyzed. System scalability is also ana- 5.2.2. In-network services
lyzed using different forwarding table sizes. Data plane programmability is the key to supporting
Parallelism is one promising way to improve forward- various in-network services. ClickOS [209] supports var-
ing performance. PacketShader [202] exploits the paral- ious applications such as NATs, rate-limiters, and applica-
lelism of a GPU to handle computation and memory- tion accelerators by exploiting the Click modular router.
intensive tasks and achieves a traffic handling capacity of FLARE [204] seeks to enable programmability of the data
40 Gbps. SP4 [203] is a software-based programmable plane so as to realize arbitrary in-network services in a
packet processing platform. It leverages multicore proces- modular manner based on the Click router architecture.
sors and parallelism. Using a pipelined architecture plus NetOpen switch [210] is a programmable networking
90 H. Farhady et al. / Computer Networks 81 (2015) 79–95

switch node that supports in-network processing. The support for both planes, we may be able to define SDN as
NetOpen switch node prototype is based on NOX. OpenRa- a network architecture that decomposes the control and
dio [78] supports a programmable wireless data plane by data planes to provide and penetrate programmability dee-
dividing wireless protocols into the processing and deci- ply and comprehensively into the networking stack. We
sion planes. This decoupling allows a user to program the refer to the total solution that covers programmability of
platform while hiding the underlying complexity of execu- both the data and control planes as Deeply Programmable
tion. Odin [77] is a programmable AP for WLAN service Network (DPN) (Fig. 4). One of the current efforts to realize
applications. The AP is virtualized using a light hypervisor DPN is FLARE [204]. FLARE is a deeply programmable net-
(i.e., OpenWrt) and then two services are run on top. Open- work node using many core general-purpose processors as
Flow is used to pass incoming packets to appropriate ser- the hardware platform. FLARE aims to enable data and con-
vice VMs and then back to the network. The virtualized trol plane programmability while guaranteeing high perfor-
WiFi AP can also be used to realize Ad targeting that mance and scalability of the device.
embeds advertisement into the result set of multiple Data plane programmability has the potential to enrich
search engines [47]. SDN applications. For example, because the data plane (i.e.,
In-network services require additional processing that switches) can look inside the packet, data plane pro-
consumes the switch’s resources, such as CPU, memory, grammability enables us to realize in-network caching,
and storage. Therefore, we need an elaborative resource transcoding, compression, and encryption efficiently.
allocation method that does not degrade the performance Without data plane programmability, a flow may need to
of the switch. In [211], the authors propose an offline pre- be forwarded to the controller for those applications; send-
diction method that predicts the performance degradation ing a packet back and forth to a controller may take time
of a resource on a specific workload in the case of resource and cause inefficiency in the network. The data plane pro-
contention among applications. Rather than using an grammability may also enable us to handle new protocols.
expensive, purpose-built devices for in-network services, To realize data plane programmability in an efficient way,
software-based tools can be exploited. SSLShader [212] is we may need to concern ourselves with the following. i)
a software SSL accelerator that exploits the parallelism of APIs: Opening up some interfaces that expose resources
a GPU and achieves processing speeds of about 13 Gbps. of a switch such as storage (e.g., for in-network caching),
processors (e.g., for transcoding and encryption), and pack-
et queues (e.g., dynamic adjustment of queue sizes for
6. Future directions specific applications) could be useful. Furthermore, stan-
dard and open interfaces for resources on network controls
6.1. Data plane programmability such as storage, processing, or packet queues are a handy
tool for SDN data plane application programmers to devel-
From our survey shown in previous sections, we can see op their solution faster and more portable across hardware
that research on data plane programmability is absent from platforms. ii)Tools: Similar to the tools for the control plane
most SDN-related work, even though supporting data plane programmability, we need tools to simulate, debug, and
programmability is technically possible. We believe that test data plane applications and APIs. Moreover, unlike
this is mainly due to the early vision for SDN that focused the debuggers for the control plane that use formal meth-
on control plane programmability. However, we argue that ods to debug forwarding table entries, we need different
research on both the data and control planes is required to types of debuggers that can test data plane APIs because
enrich SDN services and applications. With well-balanced they mostly deal with packets and payloads themselves.

Fig. 4. Deep programmability within network.

 H. Farhady et al. / Computer Networks 81 (2015) 79–95 91

6.2. Platform independency a full SDN deployment. But, in the real world, only part of
the network can be upgraded at a time when budgets are
Not depending on specific hardware or protocols constrained. In this case, one challenge is to support com-
enables us to be more productive and innovative. Tradi- patibility between existing network components and SDN-
tional hardware-centric networking and current Open- enabled components (e.g., [219]). Another issue is deter-
Flow-compatible switches both make networking mining which existing switches or routers need to be
dependent on a special family of hardware or software, upgraded to maximize the benefits of SDN (e.g., [220]).
which hampers innovation and extensibility. Therefore, We also may need to consider inter-networking across
using commodity hardware (e.g., CPU, GPU, and NPU) can multiple SDN domains. Most current SDN-related work
be an important move for creating and independent SDN focuses on the context of a single administrative domain.
data plane. Similarly, in terms of software-related issues, The current focus is well-suited to SDN’s logically central-
we can develop SDN In such a way that it suffers from less ized control. However, the logically centralized control
dependencies. For example, protocol-oblivious forwarding may not be suited to the case of multiple SDN networks
[213] is proposed to remove any dependency of protocol- where the network controls are independently driven by
specific configurations on the forwarding elements and their own controllers. For example, a federation of
enhance the data-path with new stateful instructions or physically distributed SDN networks (including testbeds)
actions [214] to support genuine SDN behavior [215,216]. needs agreement from the corresponding administrators.
ONF is still in early development stages of concepts such In addition, the controller may need to be extended to cov-
as protocol independent/oblivious/agnostic forwarding er inter-networking.
(i.e., PiF, PoF, and PaF) and they are all products of data
plane programmability. We believe that we need to depart
from the current hpattern-match, actioni architecture of 7. Conclusion
SDN.
SDN is becoming popular due to the interesting features
6.3. User-driven control it offers that unlock innovation in how we design and orga-
nize networks. However, there are still important chal-
Most SDN APIs are developed to be used by network lenges to be solved before realizing successful SDN. In
operators and/or administrators. Although these APIs are this survey, we introduce existing SDN-related technolo-
useful and should be considered, there can be some APIs gies and also argue that data plane programmability needs
implemented by users (i.e., authorized users who can pro- to be considered in the SDN definition. In particular, we
gram their own software controls to execute their tasks provide data plane technologies and discuss several future
based on hardware and software interfaces of the network directions to realize data plane programmability. In fact,
operator) (e.g., [217,218]). APIs for end-users can be uti- current southbound APIs are not flexible and are mostly
lized to realize on-demand services. For example, an intru- translated as OpenFlow. If we can define a better south-
sion detection application on a user machine could request bound API, then we can add interesting operation and
the network to filter traffic from a specific source. Alterna- management features and facilities to the data plane. We
tively, a MapReduce-style application could request band- believe the SDN community should consider data plane
width guarantees to improve performance of its shuffle programmability as a facilitator for new SDN applications.
phase. These examples mean that an API needs to exist
between the network control plane and its client applica- References
tions. These principals need both read and write access
to learn the network’s status and make independent con- [1] O.N. Foundation, Software-Defined Networking (SDN) Definition.
figuration changes for their own benefit, respectively <http://goo.gl/O2eTti>.
[2] A. Doria, R. Gopal, H. Khosravi, L. Dong, J. Salim, W. Wang,
[218]. For the user-defined control, several challenges need Forwarding and Control Element Separation (Forces) Protocol
to be resolved. Trust may play an important role in such Specification. <https://ietf.org/wg/forces/charter/>.
APIs because we must give a portion of network control [3] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,
J. Rexford, S. Shenker, J. Turner, OpenFlow: enabling innovation in
to a semi-external entity. We also may need to resolve con- campus networks, ACM SIGCOMM CCR 38 (2) (2008) 69–74.
flicts across user requests while maintaining baseline [4] B. Ahlgren, C. Dannewitz, C. Imbrenda, D. Kutscher, B. Ohlman, A
levels of fairness and security. survey of information-centric networking, IEEE Commun. Mag. 50
(7) (2012) 26–36.
[5] S. Shin, P. Porras, V. Yegneswaran, M. Fong, G. Gu, M. Tyson, Fresco:
6.4. Deployment modular composable security services for software-defined
networks, in: Proceedings of Network and Distributed Security
Symposium, 2013.
Early SDN deployments mainly focused on university
[6] M. Yu, L. Jose, R. Miao, Software defined traffic measurement with
campus networks and data centers. However, recent works opensketch, in: USENIX NSDI, vol. 13, 2013.
have explored applications of SDN to a wider range of net- [7] B. Nunes, M. Mendonca, X. Nguyen, K. Obraczka, T. Turletti, A
survey of software-defined networking: past, present, and future of
works, including optical, home, wireless, and cellular net-
programmable networks, IEEE Commun. Surv. Tutorials PP (99)
works, as well as ICN as described in Section 3. Because (2014) 1–18.
each network has its own settings, applying SDN to those [8] N. Feamster, J. Rexford, E. Zegura, The Road to SDN: An Intellectual
networks may introduce new opportunities and challenges History of Programmable Networks. <http://goo.gl/X8TCjy>.
[9] A. Lara, A. Kolasani, B. Ramamurthy, Network Innovation using
to be solved [8]. Another potential issue is an incremental OpenFlow: a survey, IEEE Commun. Surv. Tutorials 16 (1) (2014)
deployment. Most existing works that exploit SDN assume 493–512.
92 H. Farhady et al. / Computer Networks 81 (2015) 79–95

[10] F. Hu, Q. Hao, K. Bao, A survey on Software Defined Networking [38] P. Sharma, S. Banerjee, S. Tandel, R. Aguiar, R. Amorim, D. Pinheiro,
(SDN) and OpenFlow: from concept to implementation, IEEE Enhancing network management frameworks with SDN-like
Commun. Surv. Tutorials PP (99) (2014) 1. control, in: IFIP/IEEE IM, 2013, pp. 688–691.
[11] D.L. Tennenhouse, D.J. Wetherall, Towards an active network [39] H. Kim, N. Feamster, Improving network management with
architecture, ACM SIGCOMM CCR 26 (2) (1996) 5–17. software defined networking, IEEE Commun. Mag. 51 (2) (2013)
[12] B. Schwartz, A.W. Jackson, W.T. Strayer, W. Zhou, R.D. Rockwell, C. 114–119.
Partridge, Smart packets for active networks, in: IEEE OPENARCH, [40] R. Bennesby, P. Fonseca, E. Mota, A. Passito, An inter-as routing
1999, pp. 90–97. component for software-defined networks, in: IEEE NOMS, 2012,
[13] D.J. Wetherall, J.V. Guttag, D.L. Tennenhouse, Ants: a toolkit for pp. 138–145.
building and dynamically deploying network protocols, in: IEEE [41] N. Handigol, S. Seetharaman, M. Flajslik, N. McKeown, R. Johari,
OPENARCH, 1998, pp. 117–129. Plug-n-serve: Load-balancing web traffic using OpenFlow, in: ACM
[14] D.S. Alexander, W.A. Arbaugh, M.W. Hicks, P. Kakkar, A.D. SIGCOMM Demo, 2009.
Keromytis, J.T. Moore, C.A. Gunter, S.M. Nettles, J.M. Smith, The [42] A. Khurshid, W. Zhou, M. Caesar, P. Godfrey, Veriflow: verifying
switchware active network architecture, IEEE Network 12 (3) network-wide invariants in real time, ACM SIGCOMM CCR 42 (4)
(1998) 29–36. (2012) 467–472.
[15] K.L. Calvert, S. Bhattacharjee, E. Zegura, J. Sterbenz, Directions in [43] P. Kazemian, G. Varghese, N. McKeown, Header space analysis:
active networks, IEEE Commun. Mag. 36 (10) (1998) 72–78. static checking for networks, in: USENIX NSDI, 2012, pp. 9–9.
[16] A.T. Campbell, H.G. De Meer, M.E. Kounavis, K. Miki, J.B. Vicente, D. [44] J. Sherry, S. Hasan, C. Scott, A. Krishnamurthy, S. Ratnasamy, V.
Villela, A survey of programmable networks, ACM SIGCOMM CCR Sekar, Making middleboxes someone else’s problem: network
29 (2) (1999) 7. processing as a cloud service, ACM SIGCOMM CCR 42 (4) (2012)
[17] J.E. van der Merwe, S. Rooney, L. Leslie, S. Crosby, The tempest-a 13–24.
practical framework for network programmability, IEEE Network [45] Z.A. Qazi, C.-C. Tu, L. Chiang, R. Miao, V. Sekar, M. Yu, SIMPLE-fying
12 (3) (1998) 20–28. Middlebox Policy Enforcement Using SDN, in: ACM SIGCOMM,
[18] M. Caesar, D. Caldwell, N. Feamster, J. Rexford, A. Shaikh, J. van der 2013.
Merwe, Design and implementation of a routing control platform, [46] S. Fayazbakhsh, V. Sekar, M. Yu, J. Mogul, Flowtags: Enforcing
in: USENIX NSDI, 2005, pp. 15–28. network-wide policies in the presence of dynamic middlebox
[19] A. Farrel, J.-P. Vasseur, J. Ash, A path computation element (PCE)- actions, ACM SIGCOMM HotSDN.
based architecture, in: RFC 4655, IETF, 2006. [47] Y. Nishida, A. Nakao, In-network ad-targeting through wifi ap
[20] T. Lakshman, T. Nandagopal, R. Ramjee, K. Sabnani, T. Woo, The virtualization, in: 2012 International Symposium on
softrouter architecture, in: ACM SIGCOMM HotNets, 2004. Communications and Information Technologies (ISCIT), IEEE,
[21] J. Van der Merwe, A. Cepleanu, K. D’Souza, B. Freeman, A. 2012, pp. 1092–1097.
Greenberg, D. Knight, R. McMillan, D. Moloney, J. Mulligan, H. [48] J. Lee, J. Tourrilhes, P. Sharma, S. Banerjee, No more middlebox:
Nguyen, et al., Dynamic connectivity management with an integrate processing into network, ACM SIGCOMM CCR 41 (4)
intelligent route service control point, in: Proceedings of the (2011) 459–460.
SIGCOMM workshop on Internet network management, 2006, pp. [49] M. SHIMAMURA, T. IKENAGA, M. TSURU, A design and prototyping
29–34. of in-network processing platform to enable adaptive network
[22] M. Casado, T. Garfinkel, A. Akella, M.J. Freedman, D. Boneh, N. services, IEICE Trans. Inf. Syst. E96-D (2) (2013) 238–248.
McKeown, S. Shenker, Sane: a protection architecture for enterprise [50] J.H. Jafarian, E. Al-Shaer, Q. Duan, OpenFlow random host mutation:
networks, in: USENIX Security Symposium, 2006. transparent moving target defense using software defined
[23] M. Casado, M.J. Freedman, J. Pettit, J. Luo, N. McKeown, S. Shenker, networking, in: ACM SIGCOMM HotSDN, 2012, pp. 127–132.
Ethane: taking control of the enterprise, in: ACM SIGCOMM CCR, [51] S.A. Mehdi, J. Khalid, S.A. Khayam, Revisiting traffic anomaly
vol. 37, 2007, pp. 1–12. detection using software defined networking, in: Recent Advances
[24] A. Greenberg, G. Hjalmtysson, D.A. Maltz, A. Myers, J. Rexford, G. in Intrusion Detection, Springer, 2011, pp. 161–180.
Xie, H. Yan, J. Zhan, H. Zhang, A clean slate 4D approach to network [52] Resonance. <http://resonance.noise.gatech.edu/>.
control and management, ACM SIGCOMM CCR 35 (5) (2005) 41–54. [53] M. Suenaga, M. Otani, H. Tanaka, K. Watanabe, Opengate on
[25] C. Mascolo, W. Emmerich, H. De Meer, An XML based OpenFlow: system outline, in: 2012 Fourth International
programmable network platform, in: ICSE Workshop on Software Conference on Intelligent Networking and Collaborative Systems,
Engineering and Mobility, 2001. IEEE, 2012, pp. 491–492.
[26] Devolved Control of atm Networks. <http://www.cl.cam.ac.uk/ [54] S. Shin, P. Porras, V. Yegneswaran, M. Fong, G. Gu, M. Tyson,
research/srg/netos/old-projects/dcan/>. FRESCO: modular composable security services for software-
[27] Network Configuration Protocol (netconf). <http://tools.ietf.org/ defined networks, in: Proceedings of Network and Distributed
html/rfc624>. Security Symposium, 2013.
[28] K. Greene, TR10: Software-Defined Networking – MIT Technology [55] S. Ghorbani, M. Caesar, Walk the line: consistent network updates
Review, 2009. <http://goo.gl/SWpjW3>. with bandwidth guarantees, in: ACM SIGCOMM HotSDN, 2012, pp.
[29] Develop with the junos sdk. <http://goo.gl/jIQ3Wd>. 67–72.
[30] Cisco Open Network Environment. <http://www.cisco.com/ [56] E. Keller, S. Ghorbani, M. Caesar, J. Rexford, Live migration of an
web/solutions/trends/open_network_environment>. entire network (and its hosts), in: ACM SIGCOMM HotNets, 2012,
[31] N. Chowdhury, R. Boutaba, A survey of network virtualization, in: pp. 109–114.
Elsevier Computer Networks, 2010. [57] Y. Nakagawa, K. Hyoudou, T. Shimizu, A management method of ip
[32] European Telecommunications Standards Institute, Network multicast in overlay networks using OpenFlow, in: ACM SIGCOMM
Functions Virtualisation, 2012. <http://portal.etsi.org/NFV/NFV_ HotSDN, 2012, pp. 91–96.
White_Paper.pdf>. [58] M. Casado, N. McKeown, The virtual network system, in: ACM
[33] N. Handigol, S. Seetharaman, M. Flajslik, R. Johari, N. McKeown, SIGCSE Bulletin, vol. 37, 2005, pp. 76–80.
Aster⁄x: Load-balancing as a network primitive, in: Proceedings of [59] B. Heller, S. Seetharaman, P. Mahadevan, Y. Yiakoumis, P. Sharma, S.
ACLD, 2010. Banerjee, N. McKeown, ElasticTree: Saving Energy in Data Center
[34] S. Agarwal, M. Kodialam, T. Lakshman, Traffic engineering in Networks, in: USENIX NSDI, vol. 3, 2010, pp. 19–21.
software defined networks, in: IEEE INFOCOM, 2013, pp. 2211– [60] K. Kannan, S. Banerjee, Scissors: Dealing with header redundancies
2219. in data centers through SDN, in: IEEE CNSM, 2012, pp. 295–301.
[35] S. Shirali-Shahreza, Y. Ganjali, FleXam: flexible sampling extension [61] V. Mann, K. Kannan, A. Vishnoi, A.S. Iyer, Ncp: Service replication in
for monitoring and security applications in OpenFlow, in: ACM data centers through software defined networking, in: IFIP/IEEE IM,
SIGCOMM HotSDN, 2013, pp. 167–168. 2013, pp. 561–567.
[36] K. Takagiwa, S. Ishida, H. Nishi, SoR-Based Programmable Network [62] R. Wang, D. Butnariu, J. Rexford, OpenFlow-based server load
for Future Software-Defined Network, in: IEEE COMPSAC, 2013, pp. balancing gone wild, in: USENIX Hot-ICE, 2011, pp. 12–12.
165–166. [63] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh, S.
[37] Z.A. Qazi, J. Lee, T. Jin, G. Bellala, M. Arndt, G. Noubir, Application- Venkata, J. Wanderer, J. Zhou, M. Zhu, B4: Experience with a
awareness in SDN, in: Proceedings of the ACM SIGCOMM, 2013, pp. globally-deployed software defined WAN, in: ACM SIGCOMM,
487–488. 2013, pp. 3–14.
 H. Farhady et al. / Computer Networks 81 (2015) 79–95 93

[64] C.-Y. Hong, S. Kandula, R. Mahajan, M. Zhang, V. Gill, M. Nanduri, R. [90] S. Salsano, N. Blefari-Melazzi, A. Detti, G. Morabito, L. Veltri,
Wattenhofer, Achieving high utilization with software-driven Information centric networking over SDN and OpenFlow:
WAN, in: ACM SIGCOMM, 2013, pp. 15–26. Architectural aspects and experiments on the OFELIA testbed,
[65] Z. Liu, Y. Li, L. Su, D. Jin, L. Zeng, M2cloud: software defined multi- Elsevier Comput. Networks 57 (16) (2013) 3207–3221.
site data center network control framework for multi-tenant, in: [91] L. Veltri, G. Morabito, S. Salsano, N. Blefari-Melazzi, A. Detti,
ACM SIGCOMM, 2013, pp. 517–518. Supporting information-centric functionality in software defined
[66] Optical Transport Working Group in ONF. <http://goo.gl/YkdUP6>. networks, in: IEEE ICC, 2012, pp. 6645–6650.
[67] S. Gringeri, N. Bitar, T.J. Xia, Extending software defined network [92] D. Syrivelis, G. Parisis, D. Trossen, P. Flegkas, V. Sourlas, T. Korakis, L.
principles to include optical transport, IEEE Commun. Mag. 51 (3) Tassiulas, Pursuing a software defined information-centric
(2013) 32–40. network, in: IEEE EWSDN, 2012, pp. 103–108.
[68] G. Wang, T. Ng, A. Shaikh, Programming your network at run-time [93] X.N. Nguyen, D. Saucez, T. Turletti, Efficient caching in content-
for big data applications, in: ACM SIGCOMM HotSDN, 2012, pp. centric networks using OpenFlow, in: INFOCOM Student Workshop,
103–108. 2013.
[69] M. Shirazipour, W. John, J. Kempf, H. Green, M. Tatipamula, [94] J. Suh, H. Jung, T. TaekyoungKwon, Y. Choi, C-flow: Content-
Realizing packet-optical integration with SDN and OpenFlow 1.1 oriented networking over OpenFlow, in: Open Networking Summit,
extensions, in: IEEE ICC, 2012, pp. 6633–6637. 2012.
[70] S. Azodolmolky, R. Nejabati, E. Escalona, R. Jayakumar, N. [95] T. Tsou, X. Shi, J. Huang, Z. Wang, X. Yin, Analysis of Comparisons
Efstathiou, D. Simeonidou, Integrated OpenFlow-gmpls control between OpenFlow and Forces. <http://tools.ietf.org/html/draft-
plane: an overlay model for software defined packet over optical wang-forces-compare-openflow-forces-00/>.
networks, in: European Conference and Exposition on Optical [96] OpenFlow Switch Specification. <http://goo.gl/1DYxw6>.
Communications, Optical Society of America, 2011. [97] N. Gude, T. Koponen, J. Pettit, B. Pfaff, M. Casado, N. McKeown, S.
[71] D.E. Simeonidou, R. Nejabati, M. Channegowda, Software defined Shenker, NOX: towards an operating system for networks, ACM
optical networks technology and infrastructure: enabling software- SIGCOMM CCR 38 (3) (2008) 105–110.
defined optical network operations, in: Optical Fiber [98] POX: A Python-Based OpenFlow Controller, author = Mccauley, J.
Communication Conference, Optical Society of America, 2013. <http://www.noxrepo.org/pox/about-pox/>.
[72] J. Zhang, J. Zhang, Y. Zhao, H. Yang, X. Yu, L. Wang, X. Fu, [99] SNAC. <http://www.openflowhub.org/display/Snac/>.
Experimental demonstration of OpenFlow-based control plane for [100] E. Ng, Maestro: A System for Scalable OpenFlow Control.
elastic lightpath provisioning in flexi-grid optical networks, Optics <https://code.google.com/p/maestro-platform/>.
express 21 (2) (2013) 1364–1373. [101] Ryu, An Operating System for Software Defined Network. <http://
[73] A.N. Patel, P.N. Ji, T. Wang, QoS-aware optical burst switching in osrg.github.com/ryu/>.
OpenFlow based Software-Defined Optical Networks, in: ONDM, [102] Mul. <http://sourceforge.net/p/mul/wiki/Home/>.
2013, pp. 275–280. [103] Beacon. <https://openflow.stanford.edu/display/Beacon/Home/>.
[74] V.R. Gudla, S. Das, A. Shastri, G. Parulkar, N. McKeown, L. Kazovsky, [104] Floodlight. <http://floodlight.openflowhub.org/>.
S. Yamashita, Experimental demonstration of OpenFlow control of [105] IRIS. <http://openiris.etri.re.kr/>.
packet and circuit switches, in: Optical Fiber Communication [106] OESS. <https://code.google.com/p/nddi/>.
Conference, Optical Society of America, 2010. [107] Jaxon. <http://jaxon.onuos.org/>.
[75] L. Liu, T. Tsuritani, I. Morita, H. Guo, J. Wu, OpenFlow-based [108] Nodeflow. <http://garyberger.net/?p=537/>.
wavelength path control in transparent optical networks: a proof- [109] ovs-controller. <http://openvswitch.org/cgi-bin/ovsman.cgi?page=
of-concept demonstration, in: IEEE ECOC, 2011, pp. 1–3. utilities%2Fovs-controller.8/>.
[76] A. Sadasivarao, S. Syed, P. Pan, C. Liou, A. Lake, C. Guok, I. Monga, [110] Open vSiwtch. <http://openvswitch.org/>.
Open transport switch – a software defined networking [111] Flowvisor. <https://github.com/OPENNETWORKINGLAB/flowvisor/
architecture for transport networks, in: ACM SIGCOMM HotSDN, wiki/>.
2013. [112] Routeflow. <https://sites.google.com/site/routeflow/>.
[77] L. Suresh, J. Schulz-Zander, R. Merz, A. Feldmann, T. Vazao, Towards [113] Helios by NEC. <http://www.nec.com/>.
programmable enterprise wlans with odin, in: ACM SIGCOMM [114] H. Pan, H. Guan, J. Liu, W. Ding, C. Lin, G. Xie, The flowadapter:
HotSDN, 2012, pp. 115–120. enable flexible multi-table processing on legacy hardware, in: ACM
[78] M. Bansal, J. Mehlman, S. Katti, P. Levis, Openradio: a SIGCOMM HotSDN, 2013, pp. 85–90.
programmable wireless dataplane, in: ACM SIGCOMM HotSDN, [115] N. Kang, J. Reich, J. Rexford, D. Walker, Policy transformation in
2012, pp. 109–114. software defined networks, in: Proceedings of the ACM SIGCOMM,
[79] K.-K. Yap, M. Kobayashi, R. Sherwood, T.-Y. Huang, M. Chan, N. 2012, pp. 309–310.
Handigol, N. McKeown, Openroads: empowering research in [116] Y. Kanizo, D. Hay, I. Keslassy, Palette: Distributing tables in
mobile networks, ACM SIGCOMM CCR 40 (1) (2010) 125–126. software-defined networks, in: IEEE INFOCOM Mini-conference,
[80] K.-K. Yap, R. Sherwood, M. Kobayashi, T.-Y. Huang, M. Chan, N. 2013.
Handigol, N. McKeown, G. Parulkar, Blueprint for introducing [117] N. Kang, Z. Liu, J. Rexford, D. Walker, Optimizing the one big switch
innovation into wireless mobile networks, in: ACM SIGCOMM abstraction in software-defined networks, in: ACM CoNEXT, 2013.
VISA, 2010, pp. 25–32. [118] Network OS: ONOS. <http://onlab.us/tools.html#os/>.
[81] P. Dely, A. Kassler, N. Bayer, OpenFlow for wireless mesh networks, [119] T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu, R.
in: IEEE ICCCN, 2011, pp. 1–6. Ramanathan, Y. Iwata, H. Inoue, T. Hama, Onix: A distributed
[82] A. Gudipati, D. Perry, L.E. Li, S. Katti, SoftRAN: Software Defined control platform for large-scale production networks, in:
Radio Access Network, in: ACM SIGCOMM HotSDN, 2013. Proceedings of USENIX OSDI, vol. 10, 2010, pp. 1–6.
[83] M. Yang, Y. Li, D. Jin, L. Su, S. Ma, L. Zeng, Openran: a software- [120] L. Vanbever, J. Reich, T. Benson, N. Foster, J. Rexford, Hotswap:
defined ran architecture via virtualization, in: ACM SIGCOMM, Correct and efficient controller upgrades for software-defined
2013, pp. 549–550. networks, in: ACM SIGCOMM HotSDN, 2013.
[84] L.E. Li, Z.M. Mao, J. Rexford, Toward software-defined cellular [121] A. Tootoonchian, Y. Ganjali, Hyperflow: A distributed control plane
networks, in: IEEE EWSDN, 2012, pp. 7–12. for OpenFlow, in: USENIX INM/WREN, 2010, pp. 3–3.
[85] C. Guimaraes, D. Corujo, R.L. Aguiar, F. Silva, P. Frosi, Empowering [122] P. Peresini, M. Kuzniar, N. Vasic, M. Canini, D. Kostic, Of. cpp:
software defined wireless Networks through Media Independent Consistent packet processing for OpenFlow, Tech. rep., EPFL, 2013.
Handover management, in: IEEE GLOBECOM, 2013, pp. 2204–2209. [123] D. Levin, A. Wundsam, B. Heller, N. Handigol, A. Feldmann, Logically
[86] R. Mortier, T. Rodden, T. Lodge, D. McAuley, C. Rotsos, A.W. Moore, centralized?: state distribution trade-offs in software defined
A. Koliousis, J. Sventek, Control and understanding: Owning your networks, in: ACM SIGCOMM HotSDN, 2012, pp. 1–6.
home network, in: IEEE COMSNETS, 2012, pp. 1–10. [124] N.P. Katta, J. Rexford, D. Walker, Incremental consistent updates, in:
[87] K.L. Calvert, W.K. Edwards, N. Feamster, R.E. Grinter, Y. Deng, X. ACM SIGCOMM HotSDN, 2013, pp. 49–54.
Zhou, Instrumenting home networks, ACM SIGCOMM CCR 41 (1) [125] N.P. Katta, J. Rexford, D. Walker, Time-Based Updates in Software
(2011) 84–89. Defined Networks, in: ACM SIGCOMM HotSDN, 2013.
[88] N. Feamster, Outsourcing home network security, in: ACM [126] M. Yu, J. Rexford, M.J. Freedman, J. Wang, Scalable flow-based
SIGCOMM HomeNets, 2010, pp. 37–42. networking with DIFANE, ACM SIGCOMM CCR 41 (4) (2011) 351–
[89] N. Blefari-Melazzi, A. Detti, G. Mazza, G. Morabito, S. Salsano, L. 362.
Veltri, An OpenFlow-based testbed for information centric [127] A.R. Curtis, J.C. Mogul, J. Tourrilhes, P. Yalagandula, P. Sharma, S.
networking, Future Network Mobile Summit (2012) 4–6. Banerjee, DevoFlow: Scaling flow management for high-
94 H. Farhady et al. / Computer Networks 81 (2015) 79–95

performance networks, in: ACM SIGCOMM CCR, vol. 41, 2011, pp. [161] SDN Troubleshooting Simulator (STS). <http://ucb-sts.github.io/sts/
254–265. >.
[128] S. Hassas Yeganeh, Y. Ganjali, Kandoo: a framework for efficient [162] Oflops. <http://archive.openflow.org/wk/index.php/Oflops/>.
and scalable offloading of control applications, in: ACM SIGCOMM [163] C. Rotsos, N. Sarrar, S. Uhlig, R. Sherwood, A.W. Moore, Oflops: An
HotSDN, 2012, pp. 19–24. open framework for OpenFlow switch evaluation, in: Passive and
[129] M. Othman, K. Okamura, Hybrid control model for flow-based Active Measurement, Springer, 2012, pp. 85–95.
networks, in: IEEE COMPSAC, 2013, pp. 765–770. [164] OFTest. <http://archive.openflow.org/wk/index.php/
[130] D. Drutskoy, E. Keller, J. Rexford, Scalable network virtualization in OFTestTutorial/>.
software-defined networks, IEEE Internet Comput. 17 (2) (2013) [165] OFRewind. <http://archive.openflow.org/wk/index.php/OFRewind/
20–27. >.
[131] Z. Bozakov, P. Papadimitriou, Autoslice: automated and scalable [166] A. Wundsam, D. Levin, S. Seetharaman, A. Feldmann, OFRewind:
slicing for software-defined networks, in: ACM CoNEXT student Enabling Record and Replay. Troubleshooting for Networks, in:
workshop, 2012, pp. 3–4. USENIX ATC, 2011.
[132] S.H. Yeganeh, A. Tootoonchian, Y. Ganjali, On scalability of [167] N. Handigol, B. Heller, V. Jeyakumar, D. Mazieres, N. McKeown,
software-defined networking, IEEE Commun. Mag. 51 (2) (2013) Where is the debugger for my software-defined network? in: ACM
136–141. SIGCOMM HotSDN, 2012, pp. 55–60.
[133] A. Dixit, F. Hao, S. Mukherjee, T. Lakshman, R. Kompella, Towards [168] R.C. Scott, A. Wundsam, K. Zarifis, S. Shenker, What, Where, and
an elastic distributed sdn controller, in: ACM SIGCOMM HotSDN, When: Software Fault Localization for SDN, Tech. rep., UCB/EECS-
2013, pp. 7–12. 2012-178, UC Berkeley, 2012.
[134] P. Bosshart, G. Gibb, H.-S. Kim, G. Varghese, N. McKeown, M. Izzard, [169] H. Zeng, P. Kazemian, G. Varghese, N. McKeown, Automatic test
F. Mujica, M. Horowitz, Forwarding metamorphosis: Fast packet generation, in: Proceedings of the 8th International
programmable match-action processing in hardware for SDN, in: Conference on Emerging Networking Experiments and
Proceedings of the ACM SIGCOMM, 2013, pp. 99–110. Technologies, 2012, pp. 241–252.
[135] J.C. Mogul, P. Congdon, Hey, you darned counters!: get off my [170] A. Voellmy, J. Wang, Y.R. Yang, B. Ford, P. Hudak, Maple:
ASIC!, in: ACM SIGCOMM HotSDN, 2012, pp. 25–30. Simplifying SDN programming using algorithmic policies, in:
[136] G. Lu, R. Miao, Y. Xiong, C. Guo, Using cpu as a traffic co-processing ACM SIGCOMM, 2013, pp. 87–98.
unit in commodity switches, in: ACM SIGCOMM HotSDN, 2012, pp. [171] M. Reitblatt, M. Canini, A. Guha, N. Foster, Fattire: Declarative Fault
31–36. Tolerance for Software-Defined Networks.
[137] K. Kogan, S. Nikolenko, W. Culhane, P. Eugster, E. Ruan, Towards [172] T.L. Hinrichs, N.S. Gude, M. Casado, J.C. Mitchell, S. Shenker,
efficient implementation of packet classifiers in SDN/OpenFlow, in: Practical declarative network management, in: ACM SIGCOMM
ACM SIGCOMM HotSDN, 2013, pp. 153–154. WREN, pp. 1–10.
[138] E. Kissel, G. Fernandes, M. Jaffee, M. Swany, M. Zhang, Driving [173] A. Voellmy, H. Kim, N. Feamster, Procera: a language for high-level
Software Defined Networks with XSP, in: IEEE ICC, 2012, pp. 6616– reactive network control, in: ACM SIGCOMM HotSDN, 2012, pp.
6621. 43–48.
[139] C. Monsanto, J. Reich, N. Foster, J. Rexford, D. Walker, Composing [174] N. Foster, R. Harrison, M.J. Freedman, C. Monsanto, J. Rexford, A.
software defined networks, in: Proceedings of USENIX NDSI. Story, D. Walker, Frenetic: a network programming language, ACM
[140] X. Wen, Y. Chen, C. Hu, C. Shi, Y. Wang, Towards a secure controller SIGPLAN Notices 46 (9) (2011) 279–291.
platform for OpenFlow applications, in: ACM SIGCOMM HotSDN, [175] M. Reitblatt, N. Foster, J. Rexford, D. Walker, Consistent updates for
2013, pp. 171–172. software-defined networks: change you can believe in!, in: ACM
[141] P. Porras, S. Shin, V. Yegneswaran, M. Fong, M. Tyson, G. Gu, A SIGCOMM HotNets, 2011, p. 7.
security enforcement kernel for OpenFlow networks, in: ACM [176] PlanetLab Europe. <http://www.planet-lab.eu/openflow/>.
SIGCOMM HotSDN, 2012, pp. 121–126. [177] OFELIA, OpenFlow in Europe: Linking Infrastructure and
[142] S. Shin, G. Gu, Attacking software-defined networks: a first Applications. <http://www.fp7-ofelia.eu/about-ofelia/>.
feasibility study, in: ACM SIGCOMM HotSDN, 2013, pp. 165–166. [178] National LambdaRail. <http://www.nlr.net/testbeds.php/>.
[143] K. Benton, L.J. Camp, C. Small, OpenFlow vulnerability assessment, [179] Internet2-based NDDI. <http://inddi.wikispaces.com/Internet2-
in: ACM SIGCOMM HotSDN, 2013, pp. 151–152. based+NDDI/>.
[144] M. Kuzniar, P. Perešıni, N. Vasic, M. Canini, D. Kostic, Automatic [180] RISE testbed. <http://www.jgn.nict.go.jp/english/info/technologies/
failure recovery for software-defined networks, in: ACM SIGCOMM openflow.html/>.
HotSDN, 2013. [181] SURFnet. <http://goo.gl/QDPm7W>.
[145] U.C. Kozat, G. Liang, K. Kokten, Verifying forwarding plane con [182] COTN, The California OpenFlow Testbed Network. <http://www.
tivity and locating link failures using static rules in software cenic.org/page_id=143/>.
defined networks, in: ACM SIGCOMM HotSDN, 2013, pp. 157–158. [183] The FIBER Project. <http://www.fibre-ict.eu>.
[146] J. Kempf, E. Bellagamba, A. Kern, D. Jocha, A. Takacs, P. Skoldstrom, [184] M.F. Schwarz, M.A. Rojas, C.C. Miers, F.F. Redigolo, T.C. Carvalho,
Scalable fault management for OpenFlow, in: IEEE ICC, 2012, pp. Emulated and software defined networking convergence, in: IFIP/
6606–6610. IEEE IM, 2013, pp. 700–703.
[147] D. Williams, H. Jamjoom, Cementing high availability in OpenFlow [185] Open Networking Foundation. <https://www.opennetworking.org/
with rulebricks, in: ACM SIGCOMM HotSDN, 2013, pp. 139–144. index.php?lang=en/>.
[148] B. Heller, R. Sherwood, N. McKeown, The controller placement [186] Open Networking Research Center. <https://onrc.stanford.edu>.
problem, in: ACM SIGCOMM HotSDN, 2012, pp. 7–12. [187] OpenDayLight. <http://www.opendaylight.org/>.
[149] K. Phemius, M.B. Thales, OpenFlow: Why latency does matter, in: [188] OpenStack. <http://www.openstack.org/>.
IFIP/IEEE IM, 2013, pp. 680–683. [189] Apache CloudStack. <http://cloudstack.apache.org>.
[150] Y. Hu, W. Wendong, X. Gong, X. Que, C. Shiduan, Reliability-aware [190] Interface to the Routing System (I2RS). <http://datatracker.ietf.org/
controller placement for software-defined networks, in: IFIP/IEEE wg/i2rs>.
IM, 2013, pp. 672–675. [191] Software-Defined Networking Research Group. <http://irtf.org/
[151] Mininet. <http://mininet.org/>. sdnrg/>.
[152] ns-3. <http://www.nsnam.org/>. [192] ITU-T. <http://www.itu.int/en/ITU-T/Pages/default.aspx/>.
[153] D. Klein, An OpenFlow extension for the omnet++ inet framework. [193] Metro Ethernet Forum (MEF). <http://metroethernetforum.org>.
[154] Estinet 8.0 OpenFlow Network Simulator and Emulator. <http:// [194] OpenFlow Reference Implementation. <http://archive.openflow.
www.estinet.com/products.php/>. org/wk/index.php/Source_Code_Exploration>.
[155] S. Wang, C. Chou, C.-M. Yang, Estinet OpenFlow network simulator [195] Dpqd ofsoftswitch13. <https://github.com/CPqD/ofsoftswitch13/>.
and emulator, IEEE Commun. Mag. 51 (9) (2013) 110–117. [196] Indigo. <http://www.projectfloodlight.org/indigo/>.
[156] Trema. <http://trema.github.io/trema/>. [197] OpenFaucet. <http://rlenglet.github.io/openfaucet/index.html/>.
[157] Mirage. <http://www.openmirage.org/>. [198] Pantou. <http://archive.openflow.org/wk/index.php/Pantou_:_
[158] Cbench. <http://www.openflowhub.org/display/ OpenFlow_1.0_for_OpenWRT>.
floodlightcontroller/Cbench+(New)/>. [199] Y. Luo, P. Cascon, E. Murray, J. Ortega, Accelerating OpenFlow
[159] NICE. <https://code.google.com/p/nice-of/>. switching with network processors, in: ACM/IEEE ANCS, 2009, pp.
[160] M. Canini, D. Venzano, P. Peresini, D. Kostic, J. Rexford, A nice way 70–71.
to test OpenFlow applications, in: USENIX NSDI, 2012, pp. 10–10. [200] V. Tanyingyong, M. Hidell, P. Sjödin, Improving PC-based OpenFlow
switching performance, in: ACM/IEEE ANCS, 2010, p. 13.
 H. Farhady et al. / Computer Networks 81 (2015) 79–95 95

[201] A. Bianco, R. Birke, L. Giraudo, M. Palacin, OpenFlow switching: [220] D. Levin, M. Canini, S. Schmid, A. Feldmann, Incremental SDN
Data plane performance, in: IEEE ICC, 2010, pp. 1–5. Deployment in Enterprise Networks, in: ACM SIGCOMM, 2013.
[202] S. Han, K. Jang, K. Park, S. Moon, PacketShader: a GPU-accelerated
software router, ACM SIGCOMM CCR 40 (4) (2010) 195–206.
[203] H. Gill, D. Lin, L. Sarna, R. Mead, K.C.T. Lee, B.T. Loo, SP4: scalable
programmable packet processing platform, in: Proceedings of the Hamid Farhadi received the M.S. degree in
ACM SIGCOMM, 2012, pp. 75–76. computer science from Sharif University of
[204] Akihiro Nakao, FLARE: Open Deeply Programmable Switch, in: The Technology, Iran, in 2010. He is currently
16th GENI Engineering Conference, 2012. <http://www.pilab.jp/ pursuing his Ph.D. in the University of Tokyo,
ipop2013/exhibition/panel/iPOP2013_uTokyo_panel.pdf>. Japan. His research interests include software-
[205] M. Dobrescu, K. Argyraki, G. Iannaccone, M. Manesh, S. Ratnasamy, defined networking, network virtualization
Controlling parallelism in a multicore software router, in: ACM and cloud computing.
PRESTO, 2010.
[206] N. Egi, A. Greenhalgh, M. Handley, M. Hoerdt, F. Huici, L. Mathy,
Towards high performance virtual routers on commodity
hardware, in: Proceedings of ACM CoNEXT, 2008, p. 20.
[207] M. Dobrescu, N. Egi, K. Argyraki, B.-G. Chun, K. Fall, G. Iannaccone,
A. Knies, M. Manesh, S. Ratnasamy, RouteBricks: exploiting
parallelism to scale software routers, in: Proceedings of the ACM
SIGOPS SOSP, 2009, pp. 15–28.
[208] Y. Qi, J. Fong, W. Jiang, B. Xu, J. Li, V. Prasanna, Multi-dimensional
packet classification on FPGA: 100 Gbps and beyond, in: IEEE ICFPT, HyunYong Lee received the M.S. degree and
2010, pp. 241–248. the Ph.D. degree in computer science from the
[209] M. Ahmed, F. Huici, A. Jahanpanah, Enabling dynamic network Gwangju Institute of Science and Technology
processing with clickos, in: Proceedings of the ACM SIGCOMM, (GIST), Korea, in 2003 and 2010, respectively.
2012, pp. 293–294. He is currently Research associate of the
[210] N. Kim, J.-Y. Yoo, N.L. Kim, J. Kim, A programmable networking University of Tokyo, Japan. His research
switch node with in-network processing support, in: IEEE ICC, interests include P2P traffic control and
2012, pp. 6611–6615. in-network caching and user incentive
[211] M. Dobrescu, K. Argyraki, S. Ratnasamy, Toward predictable
mechanism of the information-centric net-
performance in software packet-processing platforms, in: USENIX
working.
NSDI, 2012, pp. 11–11.
[212] K. Jang, S. Han, S. Han, S. Moon, K. Park, SSLShader: cheap SSL
acceleration with commodity processors, in: USENIX NSDI, 2011,
pp. 1–1.
[213] H. Song, Protocol-oblivious forwarding: unleash the power of SDN
through a future-proof forwarding plane, in: ACM SIGCOMM
HotSDN, 2013, pp. 127–132. Akihiro Nakao received B.S. (1991) in Physics,
[214] H. Farhadi, P. Du, A. Nakao, User-defined actions for SDN, in: ACM M.E. (1994) in Information Engineering from
CFI, 2014. the University of Tokyo. He was at IBM
[215] H. Farhady, A. Nakao, Tagflow: efficient flow classification in sdn, Yamato Laboratory/at Tokyo Research
IEICE Trans. Commun. 97 (11) (2014) 2302–2310. Laboratory/at IBM Texas Austin from 1994 till
[216] L. Liang, H. Farhady, D. Ping, A. Nakao, Ouroboros: protocol 2005. He received M.S. (2001) and Ph.D.
independent forwarding for sdn, IEICE Trans. Commun. 97 (11)
(2005) in Computer Science from Princeton
(2014) 2278–2285.
University. He has been teaching as an Asso-
[217] A.D. Ferguson, A. Guha, J. Place, R. Fonseca, S. Krishnamurthi,
Participatory networking, in: USENIX Hot-ICE, vol. 12, 2012. ciate Professor in Applied Computer Science,
[218] A. Ferguson, A. Guha, C. Liang, R. Fonseca, S. Krishnamurthi, at Interfaculty Initiative in Information Stud-
Participatory Networking: An API for Application Control of SDNs, ies, Graduate School of Interdisciplinary
in: ACM SIGCOMM, 2013. Information Studies, the University of Tokyo
[219] P. Lin, J. Hart, U. Krishnaswamy, T. Murakami, M. Kobayashi, A. Al- since 2005. (He has also been an expert visiting scholar/a project leader at
Shabibi, K.-C. Wang, J. Bi, Seamless interworking of SDN and IP, in: National Institute of Information and Communications Technology (NICT)
ACM SIGCOMM, 2013, pp. 475–476. since 2007).