Extreme Networks White Paper

The Four Pillar Strategy for

Next Generation Data Centers

Abstract: The transformation from a dedicated, or siloed, application-specific infrastructure

to a shared and outsourced cloud infrastructure is not a one-size-fits-all
proposition. Enterprises today are taking advantage of the changes in incre-
mental steps, in essence migrating from a purely physical infrastructure to a
highly efficient virtualized infrastructure and potentially to a fully enabled
cloud environment. This migration, and the incremental steps, promises to
greatly enhance the efficiency and scalability of data center operations
over time.

© 2010 Extreme Networks, Inc. All rights reserved. Do not reproduce.

 Extreme Networks White Paper

Overview
The data center is changing rapidly. Enterprise and service The blade switch allows communication between blade
providers alike are looking to provision applications on the servers within a blade chassis enclosure and provides
fly, scale applications seamlessly, and simply submit jobs uplinks from the blade chassis enclosure to the rest of the
without worrying about server capacity, memory or even physical network infrastructure such as ToR or EoR
location—all while saving cost through server consolidation, switches. The net effect of adding the virtual switch and
reduced power consumption and highly automated operations. the blade switch to the data center switching infrastructure
is that the network architecture for the data center
A confluence of technologies is affecting the way data increases from a two- or three-tier architecture, to a four- or
centers are being built and operated. Blade and rack five-tier architecture.
servers, multi-core processors and virtualization are driving
compelling economic and transformative applications that Rapid Propagation of Switches in
are revolutionizing the data center environment. Business the Network
drivers, such as consolidation of locations, reducing physical
Server virtualization has created an explosion in the number
footprint, reducing power consumption and achieving easier
of switches in the network because each server has its own
provisioning and maintenance of applications are leading to
switch embedded in the hypervisor. Previously, a single
compelling CAPEX and OPEX savings as well as greatly
Ethernet switch could support up to 48 servers in a ToR
increased service and maintenance velocity within enter-
environment or thousands of servers in a middle-of-row or
prises and service provider networks.
EoR environment. Since each server has a dedicated virtual
switch, an environment with 1,000 servers would result in
The transformation from a dedicated, or siloed, application-
1,000 virtual switches. In this scenario, the cost overhead in
specific infrastructure to a shared and outsourced cloud
terms of both OPEX and CAPEX is much greater than the
infrastructure is not a one-size-fits-all proposition. Enter-
comparable 20 ToR switches or a single modular chassis-
prises today are taking advantage of the changes in
based switch in a non-virtualized environment. The
incremental steps, in essence migrating from a purely
dramatic increase in the number of switches greatly increas-
physical infrastructure to a highly efficient virtualized
es the complexity of the switching infrastructure.
infrastructure and potentially to a fully enabled cloud
environment. This migration, and the incremental steps,
promises to greatly enhance the efficiency and scalability of Management Complexity
data center operations over time. The complexity of switching in the data center has increased
not only in terms of number of tiers and switches in the
Challenges in Realizing the Promise network, but also in terms of complexity of switching
management in the data center. For example, one issue is
of Physical to Virtual to Cloud
the configuration and management of a variety of vendors’
virtual switches since standards do not currently exist for
Virtualization and cloud computing have created a number
virtual switches and each vendor has its own set of func-
of challenges in the data center including the increasing
tionalities and capabilities. Virtual switches from many
number of network tiers, complexity of the switching
different vendors must coexist, not just within the data
infrastructure, blurring of the lines between server, network
center, but even within a single server.
and storage, and the mobility of operating systems and
applications in what was previously a relatively static
Another issue created by the inclusion of switches in the
computing and networking environment.
server, whether a virtual switch or a blade switch, is which
functional group has responsibility for configuration and
Increasing Number of Network Tiers
management of the switch in the server. Traditionally, the
Traditional data center architectures have included networking group has been responsible for switches, but
Top-of-Rack (ToR) or End-of-Row (EoR) switches which with virtualization there has been a shift in the control and
connect into either an aggregation or a core switch. management of networking-level functions from the
Typically networks are deployed as two-tier or three-tier network to the server group. The blurring of the line
architectures. With the adoption of virtualization, a new between the server and the networking groups can increase
switching tier has been introduced into the network by way the likelihood of configuration errors, change management
of the virtual switch. The virtual switch is a software switch issues and can lead to more complicated maintenance and
that sits inside the hypervisor and allows Virtual Machines troubleshooting for the server and networking groups.
(VMs) to communicate with each other. With the adoption
of blade servers, it is becoming increasingly common to use
a blade switch within the blade server enclosure that
provides an additional layer of aggregation within the
network.

© 2010 Extreme Networks, Inc. All rights reserved. The Four Pillar Strategy for Next Generation Data Center—2
 Extreme Networks White Paper

Mobility The ability to dynamically provision new workloads relies on

being able to move workloads around, yet networks are
VM mobility is a benefit of virtualization. Disaster and site static. Traditionally, network configurations were set up and
recovery processes, as well as load-balancing applications, rarely changed, but the need to provision workloads
count on the ability to dynamically move workloads across dynamically means that network configurations must also
the data center infrastructure. Resiliency and fault toler- be dynamic. VM mobility requires the network configuration
ance depend heavily on this dynamic capability as well. (VLAN, QoS, ACLs, rate limiting, etc.) to dynamically follow
the virtual machine as it moves through the network.

Extreme Networks® Four Pillars Solution

Physical Efficient Scalable Automated Customized

Network Integration with Provision 1,000s Automated

Topology VM Platforms of Switches Configuration
across
Reduce Network Profiles Multiple Sites User Generated
Network Tiers for VMs Scripts
VEPA
Bandwidth and Heterogeneous XML-Enabled
Performance (Best-of-Breed) Infrastructure
Support for 1G → 10G →
Fixed and Virtual Machines 40G → 100G Open APIs
Modular Platforms
EPICenter® Application and Program and
Robust and Single Pane Awareness Application
Flexible Stacking of Glass Support Integration

Foundation for Achieving Value in the Data Center

5413_01

Figure 1

The Four Pillar Strategy – Migrating from 1. The first pillar is called “Physical”.
This represents the physical network that most data
Physical to Virtual to Cloud
centers are using today. As this physical network is the
To realize the compelling benefits of the move from physical foundation for all virtualization, it is important that this
to virtual to cloud, Extreme Networks® has developed a infrastructure be robust, easy to manage and scalable.
network infrastructure solution that will accommodate the
2. The second pillar is called “Efficient”.
evolving technology landscape in the data center and This is where the efficiencies of virtualization are realized.
address the complexities introduced by virtualization and
consolidation in the data center. 3. The third pillar is called “Scalable”.
This pillar provides support for dramatic scaling of
• This solution will allow users to migrate from a tradi- switches and virtual machines. It also includes the highly
tional or “physical” infrastructure to a virtual one, and scalable Extreme Networks Direct Attach™ architecture
later to a cloud environment without forcing particular that supports switching of the virtual machines in the
technologies or an operating methodology on the user network, rather than on the server.
• This approach applies to enterprise data centers and 4. The last pillar on the far right is called
hosting centers of any scale “Automated and Customized”.
This provides a model for data center managers to
customize their environment and automate routine tasks
The Extreme Networks solution for migrating from physical
which could otherwise be very time consuming and
to virtual to cloud is represented by Four Pillars (see Figure
prone to errors at scale.
1). Each pillar represents an advanced data center imple-
mentation phase.

© 2010 Extreme Networks, Inc. All rights reserved. The Four Pillar Strategy for Next Generation Data Center—3
 Extreme Networks White Paper

Pillar 1: Physical Network Infrastructure The BlackDiamond 8900-G96T-c blade, for example,
incorporates 96 gigabit copper connections on a single I/O
The physical data center networking infrastructure is the switch module. Utilizing MRJ21 cabling technology allows
foundation for the migration from traditional network six Ethernet cables to be consolidated into one cable.
designs to highly efficient virtualized environments and the
cloud. As the data center evolves, the physical infrastruc- In this architecture, a pass-through blade can be used for
ture must have the capability to meet the needs of evolving the blade server enclosure and a 6:1 cable consolidation
virtualized and cloud models. The infrastructure must be ratio can be achieved, thus significantly reducing cable
able to handle the increasing bandwidth requirements, complexity within the rack. Furthermore, by connecting the
address virtualization efficiently in the network, and deploy blade servers via the pass-through module directly to the
newer, scalable network architectures. The physical Ethernet ports of the BlackDiamond-G96T-c module in an
infrastructure for both stackable and modular chassis-based EoR configuration, the blade switch is eliminated, as well
platforms needs to be able to address these evolving as the active ToR switch, which eliminates two switching
technologies without requiring the network manager to “rip tiers in the network. The advantages of this solution
and replace” their existing infrastructure. are significant:

With its Direct Attach architecture, Extreme Networks • Eliminates oversubscription at the blade switch and ToR
provides the bandwidth and speed required with its layers by directly attaching the servers to the EoR switch
high-performance, highly scalable fixed and modular • Eliminates the management overhead involved with
systems. Extreme Networks stackable Summit® series blade switches and heterogeneity across switching layers
switches provide high-performance, line rate GbE and
10 GbE switches that can scale to support 100 GbE in the • Eliminates the conflict between server and network
future. The BlackDiamond® modular chassis-based management organizations regarding managing and
products provide the performance and fan-out to enable a troubleshooting of problems related to the blade switch
significant reduction of network tiers through highly • Eliminates additional switching latencies associated with
scalable line cards, including a 96-port GbE services line the blade switch and ToR switch
card. These products create a compelling architecture for
data center managers. • Reduces costs by eliminating expensive active
network elements
Reducing Network Tiers
• Reduces cabling overhead and management issues
Reducing network tiers is a key element of the Four Pillar
strategy for the data center. The Direct Attach architecture • Reduces power consumption in the data center
enables the reduction of network tiers in non-virtualized,
highly virtualized and cloud environments. There are two Pillar 2: Efficient Virtualization Lifecycle
components to the Direct Attach architecture that enable Management
the reduction of network tiers. The first includes high-
Today’s virtualized data centers are highly inefficient with
performance, high fan-out line cards with cable consolida-
respect to server-level virtualization because the network
tion, connector technology that reduces network tiers from
and server are isolated. The next step in the Four Pillar
five- and four-tier networks to three- and two-tier networks
strategy is to efficiently manage a highly virtualized network
and reduces cabling in the data center by up to 83%. The
environment with tight integration between the server
second component in reducing network tiers is moving
environment and the network. This integration is required
virtual machine switching from the server, where it exists
to enable mobility in the network as well as to provide
today, back into the network.
network administrator-level insight and control into
virtualization. Also critical to the success of achieving
The first step to simplifying the data center network is to
efficiencies are:
eliminate the blade switch tier and in some architectures,
the ToR switch. Eliminating these network tiers greatly • Tight integration with virtualization management platforms
reduces oversubscription and end-to-end latency in the
• Support for heterogeneous hypervisor environments
network. It also eliminates heterogeneity between switching
layers in the network. However, the alternative to the blade • Management of virtualization mobility through Virtual Port
switch—the pass-through blade—introduces wiring and Profiles (VPPs)
cabling complexity that can be onerous.
• Full lifecycle management of virtual machines from
creation to end of life
Extreme Networks has introduced products that can address
the cabling challenges of the pass-through blade and ToR
consolidation without introducing the oversubscription,
latency and management overhead of a blade switch and
ToR architecture.

© 2010 Extreme Networks, Inc. All rights reserved. The Four Pillar Strategy for Next Generation Data Center—4
 Extreme Networks White Paper

To enable efficient virtualized networks, Extreme Networks • XNV allows the application of network-level capabilities
has developed XNV™ (ExtremeXOS® Network Virtualization) such as ACLs (Access Control Lists), QoS (Quality of
for VM lifecycle management. XNV is a set of software- Service), rate limiting, and more down to the individual
loadable modules that delivers network-level visibility, virtual machine level as well as the automatic enforcement
control and automation of virtual machines into the hands of these capabilities in the network as the virtual machines
of the network administrator. move dynamically from server to server

• XNV allows network operators to incorporate the same

Typically, virtual machines are managed through server
level of hardened network security already deployed across
management platforms and tools, providing little or no
the corporate network to a highly virtualized data center
insight to the network for network administration teams.
This results in significant operational inefficiencies when it • The set of XNV loadable software modules applies across
comes to network configuration, monitoring and trouble- the entire data center product portfolio, including the
shooting in a highly virtualized and dynamic environment. Summit X450a, Summit X480 and Summit X650 fixed-
configuration switches and BlackDiamond 8800 series
XNV brings complete network-level insight and control into chassis with 8900-series modules specifically developed for
the virtualization process through integration with server data center applications.
virtualization technologies such as those from VMware and
Microsoft (see Figure 2).

Server 1 Server 2

A B C C

App App App App

Operating Operating Operating Operating

System System System System

MACA MACB MACC MACC

IPA IPB IPC IPC

Virtualization Layer Virtualization Layer

Physical Server Physical Server

96-port 10/100/1000BASE-T, MPU21

MACA 96-port 10/100/1000BASE-T, MPU21

MACB
MACC MACC MACC MACC MACC MACC MACC

BlackDiamond® 8800 VPPA BlackDiamond 8800

Series Series

VPPB

VPPC VPPC VPPC VPPC VPPC VPPC VPPC

ExtremeXOS® Switch 1 ExtremeXOS Switch 2

5414_01

Figure 2

© 2010 Extreme Networks, Inc. All rights reserved. The Four Pillar Strategy for Next Generation Data Center—5
 Extreme Networks White Paper

Pillar 3: Scalable Virtualized Environments At the hardware level, Extreme Networks has enabled
support of both scale and capacity through the introduction
Large data center operators are quickly running into scale of the Summit X480 ToR switches and BlackDiamond 8900-xl
issues in their facilities. These scale issues manifest blades. These products provide unprecedented Layer 2 and
themselves in three distinct areas: explosion of the number Layer 3 table size support of up to 512K MAC or IP address-
of virtual machines in the network, Layer 2 and Layer 3 es. This capability enables highly virtualized environments
scalability needed to handle this explosion and increased to support the rapid growth of virtualization. This is
demand for higher speed connectivity in the aggregation particularly important in managed hosting and cloud data
and networking core. center environments and gives enterprises the headroom to
handle virtualization in their environments.
Extreme Networks has created an architecture that allows
switching to move back into the network, provides a data Virtualization has also created an explosion of the amount
center product portfolio that can handle 40 GbE today while of bandwidth required on a single server. As servers change
preparing data centers for 100 GbE, and enables Layer 2 from hosting a single application to tens of thousands of
and Layer 3 scalability that can meet the requirements of applications, the need for 10 GbE connectivity has increased.
the burgeoning number of virtual machines. Ten GbE interfaces are native on blade servers now and
10 GbE LAN on Motherboard (LOM) is expected this year
Today, virtualization in the data center is achieved by having on 1RU and 2RU rack servers.
a software switch in the server. This creates network
scalability issues, as well as complexity and performance This bandwidth increase at the server level is creating the
issues at the server level including inefficient use of server need for 40 GbE and 100 GbE at the aggregation points in
resources to run switching functionality in the network. As the network. Extreme Networks data center solutions
networks scale to tens of thousands of virtual machines per provide unprecedented scale both on the Summit stackable
server, the switching overhead on the server can lead to and BlackDiamond modular chassis-based products with
significant degradation of server performance and erratic support for both 40 GbE today and 100 GbE when available
application performance. in the market.

For this reason, the industry is looking to remove switching

functionality from the server and put it back in the network,
where it belongs. Server and application performance are one
of many reasons for moving switching back into the network; Direct Attach Switching Architecture
other reasons include management simplicity as well as
security. Extreme Networks Direct Attach architecture enables
virtual machine switching to move back into the network and
out of the server domain.

This architecture allows administrators to leverage mature,

well-understood and fully capable network switches at wire
speed for virtual machine switching, while still enjoying the
benefits of server virtualization. The Direct Attach architec-
ture allows virtual machines to be directly attached to the
network without going through the software switch on the
server (see Figure 3).

With this unique approach, Direct Attach enables a reduc-

tion in switching tiers by eliminating the virtual switch tier,
which in turn reduces the switching burden on the server,
reduces cost, reduces end-to-end latency, reduces oversub-
scription in the network and simplifies management.
VM VM
Finally, Direct Attach allows for uniform, network-based
enforcement of security, compliance and regulatory policies
in a hypervisor-agnostic manner. Extreme Networks 5410-01
Direct Attach is enabled through a software feature pack
that implements this functionality across the entire data Figure 3
center product portfolio (Summit X450a, Summit X480,
Summit X650 and BlackDiamond 8800 series chassis with
8900-series modules).

© 2010 Extreme Networks, Inc. All rights reserved. The Four Pillar Strategy for Next Generation Data Center—6
 Extreme Networks White Paper

Pillar 4: Automated and Customized Data Conclusion

Center Networks
Today’s data center is transforming rapidly to take advantage
As data centers deploy at scale, with increased complexity
of the service and economic advantages of virtualization and
and computational density, it becomes increasingly important
consolidation. We believe that data centers require a phased
to be able to automate redundant tasks and simplify add/
solution that allows users to migrate from a traditional or
change/delete processes within the data center. In addition,
“physical” infrastructure to a virtual one and eventually to
as integration of data center storage, networking and server
cloud-based services without forcing technologies or an
resources increases, the network administrator needs the
operating methodology on the user.
ability to easily integrate and customize deployments via
common APIs. The ability to tightly couple all components of
Extreme Networks Four Pillars strategy provides a seamless
the infrastructure and orchestrate the management of the
migration path for next-generation data centers that allows
pieces via a common platform becomes essential to realizing
organizations to take advantage of the move from physical to
the benefits of highly virtualized and cloud environments.
virtual to cloud at their own pace without having to forklift
upgrade the infrastructure along the way.
The Extreme Networks switch portfolio is designed from the
ground up to support automation and customization in the
data center. The key elements to enabling automation and
customization within the data center include:

• A single modular operating system that runs across the

product portfolio
1 Widgets
to run convenient
programs on
2 EPICenter®
to manage databases
required for
• Support for standard XML API and SDK for easy integra- your switch virtualization
tion with other management and provisioning platforms

• Dynamic scripting capability that enables easy automa-

tion of routine tasks (a.k.a. widgets – see Figure 4)

• A powerful management platform for managing and

configuring the entire network
3 XML
to develop custom
• Support for loadable modules that enable various applications; interface
switch with external
functionality and capabilities CLI XML Widgets
applications and
databases
Scripts

HTTPS SDK CLEAR-

Flow
Extreme Networks automation and customization capabilities Universal
SNMP v3 Port
are built on the foundation of a single modular operating
Loadable
Loadable
Loadable Loadable

system, ExtremeXOS, and the EPICenter management Modules Modules Modules

Modules
platform. This foundation is the enabling element for the
ExtremeXOS®
open API-based automation tools which include software-
loadable modules such as Extreme Networks XNV, XML-
based Web services and dynamic scripting capabilities that
enable trigger-based automation capabilities in the network, 5409-01
such as virtual port profiles.
Figure 4

Corporate Europe, Middle East, Africa Asia Pacific Japan

and North America and South America Phone +65 6836 5437 Phone +81 3 5842 4011
Extreme Networks, Inc. Phone +31 30 800 5100
3585 Monroe Street
www.extremenetworks.com Santa Clara, CA 95051 USA
Phone +1 408 579 2800

© 2010 Extreme Networks, Inc. All rights reserved. Extreme Networks, the Extreme Networks logo, BlackDiamond, Direct Attach, EPICenter, ExtremeXOS, Summit and XNV are either
registered trademarks or trademarks of Extreme Networks, Inc. in the United States and/or other countries. All other trademarks are the trademarks of their respective owners.
Specifications are subject to change without notice. 1663_01 04/10