Savremena arhitektura

računarskih mreža u
Data centrima

Viktor Krstić,
Senior Solution Engineer
Clico
CLICO in Europe
• Poland: HQ Kraków, Offices:
Katowice, Rzeszów, and Warsaw
• Czech, Slovakia: Praha
• Hungary: Budapest
• Serbia: Belgrade
• Croatia: Zagreb
• Slovenia: Ljubljana
• Romania: Bucharest
• Bulgaria: Sofia
• Strong presence(VAR) in Baltics

© 1991 − 2018, CLICO sp. z o.o.

CLICO – Vendori

© 1991 − 2018, CLICO sp. z o.o.

Agenda
• Data Center network requirements
• Data Center network architectures
• Juniper Virtual Chassis
• Juniper Virtual Chassis Fabric
• Juniper Junos Fusion
• IP Fabric with EVPN VXLAN overlay
• IP Fabric with SDN-managed overlay

*This presentation uses materials from Juniper Networks website and documentation.

© 1991 − 2018, CLICO sp. z o.o.

Data Center network
requirements
Data Center network requirements
• Very high throughputs
• 10/25Gbps in access, 40/100Gbps in aggregation/core
• Layer 2 connectivity between servers (in most cases)
• Redundancy with quick failover
• High scalability with easy extendibility
• Rich QoS capabilities
• Low impact software upgrades
• Large buffers
• High automation capabilities (on-box scripts, DevOps, SDN)

© 1991 − 2018, CLICO sp. z o.o.

High throughput and multi-speed interfaces
• 10Gbps, 25Gbps, 40Gbps, 50Gbps, 100Gbps interfaces
• Breakout/Fanout cables

© 1991 − 2018, CLICO sp. z o.o.

Redundancy with quick failover
• Maximum permissible failover time depended on applications used in DC
• The network should converge before application/hypervisor HA features notice lack of
connection
• Proprietary solutions (i.e. Virtual Chassis Fabric, MC-LAG, vPC)
• Standardized solutions (Legacy STP, TRILL/SPB, OSPF/BGP, LFA, BFD, etc.)

© 1991 − 2018, CLICO sp. z o.o.

High scalability with easy extendibility

© 1991 − 2018, CLICO sp. z o.o.

Rich QoS capabilities
• Advanced queueing algorithms
• Lossless queue for iSCSI, FCoE, etc.
• SP, WRR, SDWRR, CBWFQ schedulers
• Separate queues for multicast
• Priority-based Flow Control (PFC)
• PAUSE Frame for specific priority
• Enhanced Transmission Selection (ETS)
• Groups apps based on Ethertype or
TCP/UDP ports
• Announce traffic priority for each group
to neighboring devices
• Allows to allocate a percentage of
bandwidth for each group
• DCBX for negotiation of PFC and ETS
• uses LLDP with DCB TLVs

© 1991 − 2018, CLICO sp. z o.o.

Large buffers
• Minimizes traffic loss during microbursts
• Important for IP storage, Hadoop, many-to-one applications
• Note - not required for all environments

© 1991 − 2018, CLICO sp. z o.o.

High automation capabilities
Python Ruby
Ansible SaltStack Puppet Chef JSNAPy ASAP
Scripts Scripts

JET PyEZ Framework RubyEZ Library Python / SLAX

THRIFT/
NETCONF Junoscript REST CLI
gRPC

JTI SNMP
JSD XML-RPC
Sensor RO

MQTT Ephemeral DB YANG JTI

Chassis Data Plane (PFE)

Junos Platform Automation Stack

© 1991 − 2018, CLICO sp. z o.o.

Data Center network
architectures
Multi Chassis Link Aggregation (MC-LAG)
• Also know as MLAG, vPC, Distributed Trunking, Multi-Chassis Trunking, etc.
• No standardized implementation
• Layer 3 and most Layer 2 mechanisms still run separately on each switch
• Often chosen because of low-impact upgrades

© 1991 − 2018, CLICO sp. z o.o.

 Juniper Virtual Chassis
and
Juniper Virtual Chassis Fabric

© 1991 − 2018, CLICO sp. z o.o.

Switch Fabric
• Many switches acting as a single logical device
• Local switching and routing in most implementations
• Mostly proprietary solutions
• Juniper Virtual Chassis and Virtual Chassis Fabric

© 1991 − 2018, CLICO sp. z o.o.

Juniper Virtual Chassis (VC)
• Up to 10 switches as single logical device
• Single management IP and console
• One protocol instance for whole stack
(STP, LLDP, LACP, OSPF, etc.) RE 0

• Redundant and backup routing engine (failover, NSSU)

LAG 1
RE 1

• Stack connection using standard interfaces

(can span up to 70km)
• Any topology of stack connections
• Link aggregation using interfaces from more than one switch
LAG 2
• Local switching and routing 10 RU

• Available on most Juniper switches

© 1991 − 2018, CLICO sp. z o.o.

Juniper VC – Sample topologies

© 1991 − 2018, CLICO sp. z o.o.

Juniper VC - Summary
• Used in Enterprises and small Data Centers
• Very easy configuration and management
• Layer 2 easily extended between server connected to the same Virtual Chassis
• Limited scalability

© 1991 − 2018, CLICO sp. z o.o.

Juniper Virtual Chassis Fabric (VCF)
• Up to 20 switches as single logical device
• More advanced compared to Virtual Chassis (not available on all switches)
• Can use only spine-and-leaf topology
• Constant, low latency (as low as ~1.8usec)
• Multi-path forwarding Physical Logical
• Elephant flow handling Active Backup

1 RU, 48 SFP+ & 1 QIC

Node #1 Node #2 Node #3 Node #4 Node #16

© 1991 − 2018, CLICO sp. z o.o.

Juniper VCF – Building blocks

QFX5110, QFX5100, EX4300

© 1991 − 2018, CLICO sp. z o.o.

Juniper VCF – Sample topologies

© 1991 − 2018, CLICO sp. z o.o.

Juniper VCF – Extending beyond 20 switch limit

© 1991 − 2018, CLICO sp. z o.o.

Juniper VCF – Forwarding plane

© 1991 − 2018, CLICO sp. z o.o.

Juniper VCF – Adaptive Flowlet Splicing

© 1991 − 2018, CLICO sp. z o.o.

Juniper VCF - Licensing

© 1991 − 2018, CLICO sp. z o.o.

Juniper VCF - Summary
• Used in small and medium Data Centers
• Up to 20 switches per VCF
• Very easy configuration and management
• Advanced load balancing capabilities
• FCoE support
• Inexpensive licensing model

© 1991 − 2018, CLICO sp. z o.o.

Juniper Junos Fusion

© 1991 − 2018, CLICO sp. z o.o.

Aggregation devices with satellites (extenders)
• Aggregation devices responsible for most/all forwarding decisions
• Satellite devices acting as simple port extenders
• Whole solution configured as a single device

© 1991 − 2018, CLICO sp. z o.o.

Juniper Junos Fusion
• Different underlaying mechanisms than VC and VCF
• Based on 802.1BR and EVPN/MC-LAG
• Up to 128 satellite devices

ICL or EVPN

© 1991 − 2018, CLICO sp. z o.o.

Junos Fusion – Building blocks
• Aggregation devices:
• QFX10k

• Satellite switches:
• EX4300
• QFX5100
• QFX5110
• QFX5200

• Satellite devices run simple Linux-based OS

• Aggregation devices control satellites
• Simple management similar to VC and VCF
• Extended ports provide features supported
by aggregation devices

© 1991 − 2018, CLICO sp. z o.o.

Junos Fusion - summary
• Large scale with simple management
• Advanced features on inexpensive devices
• Uses standard switches as satellite devices
• Sustain DC growth while using same device:
VC -> VCF -> Junos Fusion
• Reuse switches in other parts of network if Junos
Fusion is no longer needed
• Licensing based on the number of satellite
devices (pay-as-you-grow)

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric with EVPN overlay

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric with Ethernet VPN overlay
• Switches act as routers, L3 connections between switches (OSPF/ISIS)
• VXLAN/MPLS tunnels between switches to extend L2 domains between servers
• BGP advertises tunnel end-points, MAC and IP addresses

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric
• Based on standardized protocols
• Interoperable with other vendors
• No build-in limit on the number of spine and leaf devices
• Spine-and-leaf architecture most commonly used
• Highest scaling possibilities

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric – Layer 2 connectivity
• Layer 2 connectivity required between servers
• IPv4/IPv6 between switches, Layer 2 needs to be tunneled

• Tunneling mainly using VXLAN or MPLS

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric – EVPN
• BGP EVPN provides scalability and simplifies configuration
• Autodiscovery
• Signaling (VTEPs, MAC/IPs)
• Mobility
• Distributed default gateway
• Link aggregation
• Advanced routing policies possible
• Interoperability with L3VPN

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric with EVPN summary
• IP Fabric for redundant connectivity with no loops
• All links can be used thanks to ECMP
• Very quick failover based on routing protocol (OSPF/ISIS/BGP) and BFD
• Multivendor interoperability
• Very scalable
• Link aggregation from server to multiple switches (in MC-LAG two switches is the limit)
• All switches work as separate devices:
• More complicated configuration compared to VC/VCF, Junos Fusion and similar technologies
• Safer upgrades compared to VC/VCF, Junos Fusion and similar technologies
• Can use Junos Space management system to quickly configure IP Fabric on all switches

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric with SDN-managed
overlay

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric with SDN-managed overlay
• Switches act as routers, L3 connections between switches (OSPF/ISIS)
• VXLAN/MPLS tunnels between servers to extend L2 domains between servers
• SDN controller advertises tunnel end points, MAC and IP addresses

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric with SDN-managed overlay
• Based on proprietary SDN controller
• Juniper Contrail
• VMware NSX
• Nuage Networks by Nokia
• Switches from any vendor, basic feature requirements
• IP connectivity between servers
• MTU >= 1600 bytes
• Any connections architecture (spine-and-leaf still recommended)
• Scaling highly depended on server resources and controller software
• Highest automation and integration capabilities
• Additional services (firewall, load balancing, service chaining, etc.) as integral part of overlay
network

© 1991 − 2018, CLICO sp. z o.o.

Overlay network tunnels between hypervisors

VXLAN/MPLS tunnels

vRouter (Juniper Contrail)

NSX vSwitch (VMware NSX)
VRS (Nokia Nuage)

© 1991 − 2018, CLICO sp. z o.o.

Overlay network management
• SDN controller creates/removes tunnels as required
• Configuration of physical network does not change
• No networking knowledge needed to create overlay networks

© 1991 − 2018, CLICO sp. z o.o.

Connections Between VMs and the Physical Network

© 1991 − 2018, CLICO sp. z o.o.

Additional overlay features (based on Juniper Contrail)

© 1991 − 2018, CLICO sp. z o.o.

Juniper Contrail API integration example

© 1991 − 2018, CLICO sp. z o.o.

Orange Cloudwatt – Public Cloud based on Contrail

© 1991 − 2018, CLICO sp. z o.o.

IP Fabric with SDN overlay
• Communication within overlay networks
Easy configuration, traffic separation, service chaining
• Built-in routing, L4 firewall, load balancer, IPAM, etc.
Can potentially replace external solutions
• Standard DC switches from any vendor
• Highly decreased OPEX
Physical network configured once, easy overlay network
management, very high automation possibilities
• Investment protection
New overlay network features implemented in software,
no need to buy new networking devices/servers
• Unification of devices in data center due to NFV
Easier maintenance and support, lower cost of purchase,
pay-as-you-grow

© 1991 − 2018, CLICO sp. z o.o.

Data Center Summary

© 1991 − 2018, CLICO sp. z o.o.

Data Center architectures

© 1991 − 2018, CLICO sp. z o.o.

Summary of DC architectures

Scalability Ease of Ease of software Multivendor Bulit-in DCI features Automation Service chaining
management upgrade Interoperability capabilities features

MC-LAG Medium Low High Low None Medium None

Switch Fabric Low High Medium Low None Medium None

Switch Extenders Medium High Medium None None Medium None

Very High
IP Fabric with
(hardware Low Very High Very High Very Good Medium Medium
EVPN
dependent)
Depends on
Very High
controller (High
IP Fabric with SDN (software/hardware Very High Low Very Good Very High Very High
for Juniper
dependent)
Contrail)

© 1991 − 2018, CLICO sp. z o.o.

Juniper Data Center Switches – QFX10k

© 1991 − 2018, CLICO sp. z o.o.

Juniper Data Center Switches – QFX5k
• Based on Broadcom ASICs
• QFX5100 – 1/10G access, 40G core, L2 VXLAN GW, up to 2.56Tbps
• QFX5110 – 1/10G access, 40/100G core, L2/L3 VXLAN GW, up to 2.56Tbps
• QFX5120 - 10/25G access, 100G core, L2/L3 VXLAN GW, up to 2.56Tbps
• QFX5200 – 10/25G access, 100G core, L2 VXLAN GW, up to 3.2Tbps

© 1991 − 2018, CLICO sp. z o.o.

Juniper Data Center Devices – Comparison

© 1991 − 2018, CLICO sp. z o.o.

Data Center architectures – case studies

© 1991 − 2018, CLICO sp. z o.o.

Gartner and Forrester Wave

© 1991 − 2018, CLICO sp. z o.o.

Data Center architecture summary
• No universal solution
• Architecture chosen based on scale and feature requirements
• Fabric solutions often chosen in small and mid-size DCs
• Provide simple management
• Low learning curve
• IP Fabric with overlay chosen for large DCs
• When fabric solutions do not scale enough
• Interoperable, based on standardized protocols
• EVPN emerged as multivendor standard for DC architecture
• EVPN also a great option for Data Center Interconnection
• SDN-based solutions gain popularity and become mature
• Great automation capabilities and feature set

© 1991 − 2018, CLICO sp. z o.o.

Questions?

© 1991 − 2018, CLICO sp. z o.o.

Thank you
[email protected]
[email protected]
[email protected]