B Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide,
Release 6.x
First Published: December 23, 2013
Last Modified: April 19, 2016
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version
of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.
This product includes cryptographic software written by Eric Young ([email protected]).
This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/)
This product includes software written by Tim Hudson ([email protected]).
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://
www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership
relationship between Cisco and any other company. (1110R)
© 2013 - 2015 Cisco Systems, Inc. All rights reserved.

CONTENTS
Preface Preface xiii

Audience xiii
Document Conventions xiii
Related Documentation for Cisco Nexus 9000 Series Switches xiv
Documentation Feedback xiv
Obtaining Documentation and Submitting a Service Request xv
CHAPTER 1 New and Changed Information 1

New and Changed Information 1
CHAPTER 2 Overview 3
About Interfaces 3
Ethernet Interfaces 4
Access Ports 4
Routed Ports 4
Management Interface 4
Port-Channel Interfaces 4
Subinterfaces 4
Loopback Interfaces 4
Breakout Interfaces 5
Module Level Breakout 5
Notes About Breakout Interfaces 5
High Bandwidth Interfaces 5
Virtual Device Contexts 5
High Availability for Interfaces 5
CHAPTER 3 Configuring Basic Interface Parameters 7
Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 6.x
iii
Contents
About the Basic Interface Parameters 7

Description 7
Beacon 7
Error Disabled 8
Interface Status Error Policy 8
Port MTU Size 8
Bandwidth 9
Throughput Delay 9
Administrative Status 9
Unidirectional Link Detection Parameter 10
UDLD Overview 10
Default UDLD Configuration 11
UDLD Aggressive and Nonaggressive Modes 11
Port-Channel Parameters 12
Cisco QSFP+ to SFP+ Adapter Module Support 12
Licensing Requirements 13
Guidelines and Limitations 13
Default Settings 14
Configuring the Basic Interface Parameters 15
Specifying the Interfaces to Configure 15
Configuring the Description 16
Configuring the Beacon Mode 18
Configuring the Error-Disabled State 19
Enabling the Error-Disable Detection 19
Enabling the Error-Disabled Recovery 21
Configuring the Error-Disabled Recovery Interval 22
Configuring the MTU Size 22
Configuring the Interface MTU Size 23
Configuring the System Jumbo MTU Size 24
Configuring the Bandwidth 26
Configuring the Throughput Delay 27
Shutting Down and Activating the Interface 29
Configuring the UDLD Mode 30
Configuring the Debounce Timer 33
Verifying the Basic Interface Parameters 34
iv
Contents
Monitoring the Interface Counters 34

Displaying Interface Statistics 34
Clearing Interface Counters 36
Configuration Example for QSA 36
CHAPTER 4 Configuring Layer 2 Interfaces 39

Information About Access and Trunk Interfaces 40
About Access and Trunk Interfaces 40
IEEE 802.1Q Encapsulation 41
Access VLANs 42
Native VLAN IDs for Trunk Ports 42
Tagging Native VLAN Traffic 43
Allowed VLANs 43
Default Interfaces 44
Switch Virtual Interface and Autostate Behavior 44
SVI Autostate Enable/Disable 44
High Availability 44
Virtualization Support 44
Counter Values 44
Licensing Requirements for Layer 2 Port Modes 46
Prerequisites for Layer 2 Interfaces 46
Guidelines and Limitations for Layer 2 Interfaces 46
Default Settings for Layer 2 Interfaces 48
Configuring Access and Trunk Interfaces 48
Guidelines for Configuring Access and Trunk Interfaces 48
Configuring a VLAN Interface as a Layer 2 Access Port 48
Configuring Access Host Ports 50
Configuring Trunk Ports 52
Configuring the Native VLAN for 802.1Q Trunking Ports 54
Configuring the Allowed VLANs for Trunking Ports 55
Configuring a Default Interface 57
Configuring SVI Autostate Disable for the System 58
Configuring SVI Autostate Disable Per SVI 59
Configuring the Device to Tag Native VLAN Traffic 61
Changing the System Default Port Mode to Layer 2 62
v
Contents
Verifying the Interface Configuration 64

Monitoring the Layer 2 Interfaces 65
Configuration Examples for Access and Trunk Ports 65
Related Documents 65
CHAPTER 5 Configuring Layer 3 Interfaces 67

About Layer 3 Interfaces 67
Routed Interfaces 67
Subinterfaces 68
VLAN Interfaces 69
Loopback Interfaces 69
Licensing Requirements for Layer 3 Interfaces 70
Prerequisites for Layer 3 Interfaces 70
Default Settings 71
Configuring Layer 3 Interfaces 71
Configuring a Routed Interface 71
Configuring a Subinterface on a Routed Interface 73
Configuring a VLAN Interface 75
Configuring a Loopback Interface 77
Assigning an Interface to a VRF 78
Configuring a DHCP Client on an Interface 79
Verifying the Layer 3 Interfaces Configuration 80
Monitoring the Layer 3 Interfaces 81
Configuration Examples for Layer 3 Interfaces 82
CHAPTER 6 Configuring Bidirectional Forwarding Detection 85

About BFD 85
Asynchronous Mode 86
BFD Detection of Failures 86
Distributed Operation 87
BFD Echo Function 87
vi
Contents
Security 87
Licensing Requirements for BFD 88
Prerequisites for BFD 88
Default Settings 90
Configuring BFD 91
Configuration Hierarchy 91
Task Flow for Configuring BFD 91
Enabling the BFD Feature 91
Configuring Global BFD Parameters 92
Configuring BFD on an Interface 93
Configuring BFD on a Port Channel 95
Configuring the BFD Echo Function 96
Configuring BFD Support for Routing Protocols 98
Configuring BFD on BGP 98
Configuring BFD on EIGRP 99
Configuring BFD on OSPF 101
Configuring BFD on IS-IS 102
Configuring BFD on HSRP 104
Configuring BFD on VRRP 105
Configuring BFD on PIM 106
Configuring BFD on Static Routes 108
Disabling BFD on an Interface 109
Configuring BFD Interoperability 110
Configuring BFD Interoperability in Cisco NX-OS Devices in a Point-to-Point Link 110
Configuring BFD Interoperability in Cisco NX-OS Devices in a Switch Virtual Interface 111
Configuring BFD Interoperability in Cisco NX-OS Devices in Logical Mode 112
Verifying BFD Interoperability in a Cisco Nexus 9000 Series Device 113
Verifying the BFD Configuration 114
Monitoring BFD 114
Configuration Examples for BFD 115
Show Example for BFD 115
vii
Contents
RFCs 116
CHAPTER 7 Configuring Port Channels 117

About Port Channels 117
Port Channels 118
Port-Channel Interfaces 119
Basic Settings 119
Compatibility Requirements 120
Load Balancing Using Port Channels 122
Symmetric Hashing 123
Resilient Hashing 123
LACP 124
LACP Overview 124
Port-Channel Modes 125
LACP ID Parameters 126
LACP System Priority 126
LACP Port Priority 126
LACP Administrative Key 126
LACP Marker Responders 127
LACP-Enabled and Static Port Channels Differences 127
LACP Compatibility Enhancements 127
LACP Port-Channel Minimum Links and MaxBundle 128
LACP Fast Timers 128
Licensing Requirements for Port Channeling 129
Prerequisites for Port Channeling 129
Default Settings 130
Configuring Port Channels 131
Creating a Port Channel 131
Adding a Layer 2 Port to a Port Channel 133
Adding a Layer 3 Port to a Port Channel 135
Configuring the Bandwidth and Delay for Informational Purposes 137
Shutting Down and Restarting the Port-Channel Interface 139
viii
Contents
Configuring a Port-Channel Description 140

Configuring the Speed and Duplex Settings for a Port-Channel Interface 141
Configuring Load Balancing Using Port Channels 143
Enabling LACP 144
Configuring LACP Port-Channel Port Modes 145
Configuring LACP Port-Channel Minimum Links 146
Configuring the LACP Port-Channel MaxBundle 148
Configuring the LACP Fast Timer Rate 149
Configuring the LACP System Priority 150
Configuring the LACP Port Priority 151
Disabling LACP Graceful Convergence 152
Reenabling LACP Graceful Convergence 154
Disabling LACP Suspend Individual 155
Reenabling LACP Suspend Individual 156
Configuring Port Channel Hash Distribution 157
Configuring Port Channel Hash Distribution at the Global Level 158
Configuring Port Channel Hash Distribution at the Port Channel Level 158
Verifying the Port-Channel Configuration 159
Monitoring the Port-Channel Interface Configuration 160
Example Configurations for Port Channels 161
CHAPTER 8 Configuring vPCs 163

Information About vPCs 163
vPC Overview 163
vPC Terminology 166
vPC Peer Link Overview 167
Features That You Must Manually Configure on the Primary and Secondary Devices 169
Configuring Layer 3 Backup Routes on a vPC Peer Link 170
Peer-Keepalive Link and Messages 170
vPC Peer-Gateway 171
vPC Domain 172
vPC Topology 173
Compatibility Parameters for vPC Interfaces 174
Configuration Parameters That Must Be Identical 175
ix
Contents
Configuration Parameters That Should Be Identical 176

Consequences of Parameter Mismatches 177
vPC Number 177
Moving Other Port Channels into a vPC 178
Configuring vPC Peer Links and Links to the Core on a Single Module 178
vPC Interactions with Other Features 179
vPC and LACP 179
vPC Peer Links and STP 180
vPC Peer Switch 182
vPC and ARP or ND 182
vPC Multicast—PIM, IGMP, and IGMP Snooping 182
Multicast PIM Dual DR (Proxy DR ) 183
IP PIM PRE-BUILD SPT 184
vPC Peer Links and Routing 184
Best Practices for Layer 3 and vPC Configuration 185
Layer 3 and vPC Configuration Overview 185
Guidelines for Layer 3 and vPC Configurations 186
Example Topologies for Layer 3 and vPC 188
Peering Between Routers 188
Peering with an External Router Using Layer 3 Links 188
Peering Between vPC Devices for a Backup Routing Path 189
Peering Between Two Routers with vPC Devices as Transit Switches 190
Peering with an External Router on Parallel Interconnected Routed Ports 190
Peering Over a vPC Interconnection on Parallel Interconnected Routed
Ports 190
Peering Over a PC Interconnection and Dedicated Interswitch Link Using
non-vPC VLAN 191
CFSoE 192
vPC and Orphan Ports 192
vPC Recovery After an Outage 193
Autorecovery 193
vPC Peer Roles After a Recovery 193
vPC Forklift Upgrade Scenario 193
x
Contents
Licensing Requirements for vPCs 196

Configuring vPCs 198
Enabling vPCs 198
Disabling vPCs 199
Creating a vPC Domain and Entering vpc-domain Mode 200
Configuring a vPC Keepalive Link and Messages 202
Creating a vPC Peer Link 203
Configuring a vPC Peer-Gateway 205
Configuring a Graceful Consistency Check 206
Checking the Configuration Compatibility on a vPC Peer Link 208
Moving Other Port Channels into a vPC 208
Manually Configuring a vPC Domain MAC Address 210
Manually Configuring the System Priority 211
Manually Configuring the vPC Peer Device Role 212
Configuring the Tracking Feature on a Single-Module vPC 214
Configuring for Recovery After an Outage 215
Configuring Reload Restore 215
Configuring an Autorecovery 218
Configuring the Suspension of Orphan Ports 219
Configuring the vPC Peer Switch 221
Configuring a Pure vPC Peer Switch Topology 221
Configuring a Hybrid vPC Peer Switch Topology 222
Verifying the vPC Configuration 224
Monitoring vPCs 225
Configuration Examples for vPCs 225
CHAPTER 9 Configuring IP Tunnels 229

Information About IP Tunnels 229
IP Tunnel Overview 229
GRE Tunnels 230
Point-to-Point IP-in-IP Tunnel Encapsulation and Decapsulation 230
Multi-Point IP-in-IP Tunnel Decapsulation 230
xi
Contents
Path MTU Discovery 231

Licensing Requirements for IP Tunnels 231
Prerequisites for IP Tunnels 231
Configuring IP Tunnels 233
Enabling Tunneling 233
Creating a Tunnel Interface 234
Configuring a Tunnel Interface 236
Configuring a GRE Tunnel 238
Enabling Path MTU Discovery 239
Assigning VRF Membership to a Tunnel Interface 239
Verifying the IP Tunnel Configuration 241
Configuration Examples for IP Tunneling 241
APPENDIX A IETF RFCs supported by Cisco NX-OS Interfaces 243

IPv6 RFCs 243
APPENDIX B Configuration Limits for Cisco NX-OS Interfaces 245
xii
Preface
This preface includes the following sections:
• Audience, page xiii

• Document Conventions, page xiii
• Related Documentation for Cisco Nexus 9000 Series Switches, page xiv
• Documentation Feedback, page xiv
• Obtaining Documentation and Submitting a Service Request, page xv
Audience
This publication is for network administrators who install, configure, and maintain Cisco Nexus switches.
Document Conventions
Command descriptions use the following conventions:
Convention Description
bold Bold text indicates the commands and keywords that you enter literally
as shown.
Italic Italic text indicates arguments for which the user supplies the values.
[x] Square brackets enclose an optional element (keyword or argument).
[x | y] Square brackets enclosing keywords or arguments separated by a vertical

bar indicate an optional choice.
{x | y} Braces enclosing keywords or arguments separated by a vertical bar

indicate a required choice.
xiii
Preface
Related Documentation for Cisco Nexus 9000 Series Switches
[x {y | z}] Nested set of square brackets or braces indicate optional or required
choices within optional or required elements. Braces and a vertical bar
within square brackets indicate a required choice within an optional
element.
variable Indicates a variable for which you supply values, in context where italics
cannot be used.
string A nonquoted set of characters. Do not use quotation marks around the
string or the string will include the quotation marks.
Examples use the following conventions:
screen font Terminal sessions and information the switch displays are in screen font.
boldface screen font Information you must enter is in boldface screen font.
italic screen font Arguments for which you supply values are in italic screen font.
<> Nonprinting characters, such as passwords, are in angle brackets.
[] Default responses to system prompts are in square brackets.
!, # An exclamation point (!) or a pound sign (#) at the beginning of a line

of code indicates a comment line.
Related Documentation for Cisco Nexus 9000 Series Switches

The entire Cisco Nexus 9000 Series switch documentation set is available at the following URL:
http://www.cisco.com/en/US/products/ps13386/tsd_products_support_series_home.html
Documentation Feedback
To provide technical feedback on this document, or to report an error or omission, please send your comments
to [email protected]. We appreciate your feedback.
xiv
Preface
Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service
request, and gathering additional information, see What's New in Cisco Product Documentation at: http://
www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html.
Subscribe to What's New in Cisco Product Documentation, which lists all new and revised Cisco technical
documentation as an RSS feed and delivers content directly to your desktop using a reader application. The
RSS feeds are a free service.
xv
Preface
xvi
CHAPTER 1
New and Changed Information
This chapter provides release-specific information for each new and changed feature in the Cisco Nexus
9000 Series NX-OS Interfaces Configuration Guide.
• New and Changed Information, page 1

This table summarizes the new and changed features for the Cisco Nexus 9000 Series NX-OS Interfaces
Configuration Guide and where they are documented.
Table 1: New and Changed Features
Feature Description Changed Where Documented

in
Release
vPC forklift upgrade support Support for upgrading from a 6.1(2)I3(4) vPC Forklift Upgrade Scenario
pair of Nexus 9000 switches in
a vPC topology to a different
pair of Nexus 9000 series
switches.
IP-in-IP tunnel support Enables encapsulation and 6.1(2)I3(4) Point-to-Point IP-in-IP Tunnel
decapsulation of packets to Encapsulation and
create a tunnel. Decapsulation
Subinterface support on Support for one or more 6.1(2)I3(3) Configuring Layer 3

port-channel interfaces subinterfaces on a port-channel Interfaces, on page 67
interface.
GRE support for IP tunnels Support for the GRE carrier 6.1(2)I3(2) Configuring IP Tunnels, on
protocol to enable IP tunnels to page 229
enable IPV4 transport between
two devices.
1
Feature Description Changed Where Documented

in
Release
Cisco Nexus 9300 subinterface Added support of Cisco Nexus 6.1(2)I3(1) Configuring Layer 3
support 9300 subinterface support. Interfaces, on page 67
(Removed restriction of no
subinterface support for Cisco
Nexus 9300 platforms.)
FEX support Added Cisco Nexus 2000 6.1(2)I2(3) Configuring vPCs, on page 163
Fabric Extender(FEX) support.
Cisco QSFP+ to SFP+ Added the Cisco QSFP+ to 6.1(2)I2(2) Configuring Basic Interface
SFP+ Adapter (QSA) module Parameters, on page 7
Adapter (QSA) module
feature to provide 40G to 10G
conversion support.
2
CHAPTER 2
Overview
• About Interfaces, page 3
• Virtual Device Contexts, page 5
• High Availability for Interfaces, page 5
About Interfaces
Cisco NX-OS supports multiple configuration parameters for each of the interface types supported. Most of
these parameters are covered in this guide but some are described in other documents.
The following table shows where to get further information on the parameters you can configure for an
interface.
Table 2: Interface Parameters
Feature Parameters Further Information

Basic parameters description, duplex, error disable, “Configuring Basic Interface
flow control, MTU, beacon Parameters”
Layer 3 medium, IPv4 and IPv6 addresses “Configuring Layer 3 Interfaces”
Layer 3 bandwidth, delay, IP routing, VRFs Cisco Nexus 9000 Series NX-OS
Unicast Routing Configuration
Guide
Cisco Nexus 9000 Series NX-OS
Multicast Routing Configuration
Guide
Port Channels channel group, LACP “Configuring Port Channels”
Security EOU Cisco Nexus 9000 Series NX-OS

Security Configuration Guide
3
Overview
Ethernet Interfaces
Ethernet Interfaces
Ethernet interfaces include routed ports.
Access Ports
An access port carries traffic for one VLAN. This type of port is a Layer 2 interface only.
For more information on routed ports, see the “Information About Access and Trunk Interfaces” section.
Routed Ports
A routed port is a physical port that can route IP traffic to another device. A routed port is a Layer 3 interface
only.
For more information on routed ports, see the “Routed Interfaces” section.
Management Interface
You can use the management Ethernet interface to connect the device to a network for remote management
using a Telnet client, the Simple Network Management Protocol (SNMP), or other management agents. The
management port (mgmt0) is autosensing and operates in full-duplex mode at a speed of 10/100/1000 Mb/s.
For more information on the management interface, see the Cisco Nexus 9000 Series NX-OS Fundamentals
Configuration Guide. You will also find information on configuring the IP address and default IP routing for
the management interface in this document.
Port-Channel Interfaces
A port channel is a logical interface that is an aggregation of multiple physical interfaces. You can bundle up
to 32 individual links to physical ports into a port channel to improve bandwidth and redundancy. You can
also use port channeling to load balance traffic across these channeled physical interfaces. For more information
about port-channel interfaces, see the “Configuring Port Channels” section.
Subinterfaces
You can create virtual subinterfaces on a parent interface configured as a Layer 3 interface. A parent interface
can be a physical port. Subinterfaces divide the parent interface into two or more virtual interfaces on which
you can assign unique Layer 3 parameters such as IP addresses and dynamic routing protocols.
Loopback Interfaces
A virtual loopback interface is a virtual interface with a single endpoint that is always up. Any packet that is
transmitted over a virtual loopback interface is immediately received by that interface. Loopback interfaces
emulate a physical interface. For more information about subinterfaces, see the “Loopback Interfaces” section.
4
Overview
Breakout Interfaces
Breakout Interfaces
Cisco NX-OS supports the breakout of high bandwidth 40G interfaces at the module level or at the per-port
level.
Module Level Breakout

For module level breakout, the interface breakout command splits the high bandwidth 40G interface of a
module into four 10G interfaces. The module is reloaded and the configuration for the interface is removed
when the command is executed.
The following is an example of the command:
switch# configure terminal
switch(config)# interface breakout module 1
Module will be reloaded. Are you sure you want to continue(yes/no)? yes
The no interface breakout module module_number command undoes the breakout configuration. It puts all
interfaces of the module in 40G mode and deletes the configuration for the previous 10G interfaces.
Notes About Breakout Interfaces
High Bandwidth Interfaces

The breakout of high bandwidth interfaces are supported only on:
• The X9636PQ, X9432PQ, and X9536PQ line cards on a Cisco Nexus 9500 Series switch.
• The Cisco Nexus 9332PQ switch.
• The Cisco Nexus 3164Q switch.
Virtual Device Contexts

Cisco NX-OS can segment operating system and hardware resources into virtual device contexts (VDCs) that
emulate virtual devices. The Cisco Nexus 9000 Series switch does not support multiple VDCs. All switch
resources are managed in the default VDC.
High Availability for Interfaces

Interfaces support stateful and stateless restarts. A stateful restart occurs on a supervisor switchover. After
the switchover, Cisco NX-OS applies the runtime configuration.
5
Overview
High Availability for Interfaces
6
CHAPTER 3
Configuring Basic Interface Parameters
This chapter describes how to configure the basic interface parameters on Cisco NX-OS devices.
• About the Basic Interface Parameters, page 7

• Licensing Requirements, page 13
• Guidelines and Limitations, page 13
• Default Settings, page 14
• Configuring the Basic Interface Parameters, page 15
• Verifying the Basic Interface Parameters, page 34
• Monitoring the Interface Counters, page 34
• Configuration Example for QSA, page 36
About the Basic Interface Parameters
Description
For the Ethernet and management interfaces, you can configure the description parameter to provide a
recognizable name for the interface. Using a unique name for each interface allows you to quickly identify
the interface when you are looking at a listing of multiple interfaces.
For information about setting the description parameter for port-channel interfaces, see the “Configuring a
Port-Channel Description” section. For information about configuring this parameter for other interfaces, see
the “Configuring the Description” section.
Beacon
The beacon mode allows you to identify a physical port by flashing its link state LED with a green light. By
default, this mode is disabled. To identify the physical port for an interface, you can activate the beacon
parameter for the interface.
7
Error Disabled
For information about configuring the beacon parameter, see the “Configuring the Beacon Mode” section.
Error Disabled
A port is in the error-disabled (err-disabled) state when the port is enabled administratively (using the no
shutdown command) but disabled at runtime by any process. For example, if UDLD detects a unidirectional
link, the port is shut down at runtime. However, because the port is administratively enabled, the port status
displays as err-disable. Once a port goes into the err-disable state, you must manually reenable it or you can
configure a timeout value that provides an automatic recovery. By default, the automatic recovery is not
configured, and by default, the err-disable detection is enabled for all causes.
When an interface is in the err-disabled state, use the errdisable detect cause command to find information
about the error.
You can configure the automatic error-disabled recovery timeout for a particular error-disabled cause and
configure the recovery period.
The errdisable recovery cause command provides an automatic recovery after 300 seconds.
You can use the errdisable recovery interval command to change the recovery period within a range of 30
to 65535 seconds. You can also configure the recovery timeout for a particular err-disable cause.
If you do not enable the error-disabled recovery for the cause, the interface stays in the error-disabled state
until you enter the shutdown and no shutdown commands. If the recovery is enabled for a cause, the interface
is brought out of the error-disabled state and allowed to retry operation once all the causes have timed out.
Use the show interface status err-disabled command to display the reason behind the error.
Interface Status Error Policy

Cisco NX-OS policy servers such as Access Control List (ACL) Manager and Quality of Service (QoS)
Manager, maintain a policy database. A policy is defined through the command-line interface.
Policies are pushed when you configure a policy on an interface to ensure that policies that are pushed are
consistent with the hardware policies. To clear the errors and to allow the policy programming to proceed
with the running configuration, enter the no shutdown command. If the policy programming succeeds, the
port is allowed to come up. If the policy programming fails, the configuration is inconsistent with the hardware
policies and the port is placed in an error-disabled policy state. The error-disabled policy state remains and
the information is stored to prevent the same port from being brought up in the future. This process helps to
avoid unnecessary disruption to the system.
Port MTU Size

The maximum transmission unit (MTU) size specifies the maximum frame size that an Ethernet port can
process. For transmissions to occur between two ports, you must configure the same MTU size for both ports.
A port drops any frames that exceed its MTU size.
By default, each port has an MTU of 1500 bytes, which is the IEEE 802.3 standard for Ethernet frames. Larger
MTU sizes are possible for more efficient processing of data with less overhead. The larger frames, called
jumbo frames, can be up to 9216 bytes in size, which is also the default system jumbo MTU size.
On a Layer 3 interface, you can configure an MTU size between 576 and 9216 bytes.
8
Bandwidth
Note The global LAN port MTU size applies to the traffic through a Layer 3 Ethernet LAN port that is configured
with a nondefault MTU size.
For a Layer 2 port, you can configure an MTU size that is either the system default (1500 bytes) or the system
jumbo MTU size (initially 9216 bytes).
Note If you change the system jumbo MTU size, Layer 2 ports automatically use the system default MTU size
(1500 bytes) unless you specify the new system jumbo MTU size for some or all of those ports.
For information about setting the MTU size, see the “Configuring the MTU Size” section.
Bandwidth
Ethernet ports have a fixed bandwidth of 1,000,000 Kb at the physical layer. Layer 3 protocols use a bandwidth
value that you can set for calculating their internal metrics. The value that you set is used for informational
purposes only by the Layer 3 protocols—it does not change the fixed bandwidth at the physical layer. For
example, the Enhanced Interior Gateway Routing Protocol (EIGRP) uses the minimum path bandwidth to
determine a routing metric, but the bandwidth at the physical layer remains at 1,000,000 Kb.
For information about configuring the bandwidth parameter for port-channel interfaces, see the “Configuring
the Bandwidth and Delay for Informational Purposes” section. For information about configuring the bandwidth
parameter for other interfaces, see the “Configuring the Bandwidth” section.
Throughput Delay
Specifying a value for the throughput-delay parameter provides a value used by Layer 3 protocols; it does not
change the actual throughput delay of an interface. The Layer 3 protocols can use this value to make operating
decisions. For example, the Enhanced Interior Gateway Routing Protocol (EIGRP) can use the delay setting
to set a preference for one Ethernet link over another, if other parameters such as link speed are equal. The
delay value that you set is in the tens of microseconds.
For information about configuring the bandwidth parameter for port-channel interfaces, see the “Configuring
the Bandwidth and Delay for Informational Purposes” section. For information about configuring the
throughput-delay parameter for other interfaces, see the “Configuring the Throughput Delay” section.
Administrative Status
The administrative-status parameter determines whether an interface is up or down. When an interface is
administratively down, it is disabled and unable to transmit data. When an interface is administratively up, it
is enabled and able to transmit data.
For information about configuring the administrative status parameter for port-channel interfaces, see the
“Shutting Down and Restarting the Port-Channel Interface” section. For information about configuring the
administrative-status parameter for other interfaces, see the “Shutting Down and Activating the Interface”
section.
9
Unidirectional Link Detection Parameter
UDLD Overview
The Cisco-proprietary Unidirectional Link Detection (UDLD) protocol allows devices that are connected
through fiber-optic or copper (for example, Category 5 cabling) Ethernet cables to monitor the physical
configuration of the cables and detect when a unidirectional link exists. When a device detects a unidirectional
link, UDLD shuts down the affected LAN port and alerts the user. Unidirectional links can cause a variety of
problems.
UDLD performs tasks that autonegotiation cannot perform, such as detecting the identities of neighbors and
shutting down misconnected LAN ports. When you enable both autonegotiation and UDLD, Layer 1 detections
work to prevent physical and logical unidirectional connections and the malfunctioning of other protocols.
A unidirectional link occurs whenever traffic transmitted by the local device over a link is received by the
neighbor but traffic transmitted from the neighbor is not received by the local device. If one of the fiber strands
in a pair is disconnected, as long as autonegotiation is active, the link does not stay up. In this case, the logical
link is undetermined, and UDLD does not take any action. If both fibers are working normally at Layer 1,
UDLD determines whether those fibers are connected correctly and whether traffic is flowing bidirectionally
between the correct neighbors. This check cannot be performed by autonegotiation, because autonegotiation
operates at Layer 1.
The Cisco Nexus 9000 Series device periodically transmits UDLD frames to neighbor devices on LAN ports
with UDLD enabled. If the frames are echoed back within a specific time frame and they lack a specific
acknowledgment (echo), the link is flagged as unidirectional and the LAN port is shut down. Devices on both
ends of the link must support UDLD in order for the protocol to successfully identify and disable unidirectional
links. You can configure the transmission interval for the UDLD frames, either globally or for the specified
interfaces.
Note By default, UDLD is locally disabled on copper LAN ports to avoid sending unnecessary control traffic
on this type of media.
The figure shows an example of a unidirectional link condition. Device B successfully receives traffic from
device A on the port. However, device A does not receive traffic from device B on the same port. UDLD
detects the problem and disables the port.
Figure 1: Unidirectional Link
10
Default UDLD Configuration

The following table shows the default UDLD configuration.
Table 3: UDLD Default Configuration
Feature Default Value

UDLD global enable state Globally disabled
UDLD per-port enable state for fiber-optic media Enabled on all Ethernet fiber-optic LAN ports
UDLD per-port enable state for twisted-pair (copper) Disabled on all Ethernet 10/100 and 1000BASE-TX
media LAN ports
UDLD aggressive mode Disabled
UDLD message interval 15 seconds
For information about configuring the UDLD for the device and its port, see the “Configuring the UDLD
Mode” section.
UDLD Aggressive and Nonaggressive Modes

UDLD aggressive mode is disabled by default. You can configure UDLD aggressive mode only on
point-to-point links between network devices that support UDLD aggressive mode. If UDLD aggressive mode
is enabled, when a port on a bidirectional link that has a UDLD neighbor relationship established stops
receiving UDLD frame, UDLD tries to reestablish the connection with the neighbor. After eight failed retries,
the port is disabled.
When you enable the UDLD aggressive mode, the following occurs:
One side of a link has a port stuck (both transmission and receive)
One side of a link remains up while the other side of the link is down
In these cases, the UDLD aggressive mode disables one of the ports on the link, which prevents traffic from
being discarded.
Note You enable the UDLD aggressive mode globally to enable that mode on all the fiber ports. You must
enable the UDLD aggressive mode on copper ports on specified interfaces.
Tip When a line card upgrade is being performed during an in-service software upgrade (ISSU) and some of
the ports on the line card are members of a Layer 2 port channel and are configured with UDLD aggressive
mode, if you shut down one of the remote ports, UDLD puts the corresponding port on the local device
into an error-disabled state. This behavior is correct.
11
Port-Channel Parameters
To restore service after the ISSU has completed, enter the shutdown command followed by the no shutdown
command on the local port.
Port-Channel Parameters
A port channel is an aggregation of physical interfaces that comprise a logical interface. You can bundle up
to 32 individual interfaces into a port channel to provide increased bandwidth and redundancy. Port channeling
also load balances traffic across these physical interfaces. The port channel stays operational if at least one
physical interface within the port channel is operational.
You can create Layer 3 port channels by bundling compatible Layer 3 interfaces.
Any configuration changes that you apply to the port channel are applied to each interface member of that
port channel.
For information about port channels and for information about configuring port channels, see Chapter 6,
“Configuring Port Channels.”
Cisco QSFP+ to SFP+ Adapter Module Support

The Cisco QSFP+ to SFP+ Adapter (QSA) module provides 10G support for the 40G uplink ports that are a
part of the Cisco Nexus M6PQ and Cisco Nexus M12PQ uplink modules of specific Cisco Nexus 9300 devices
described below.
A group of six consecutive ports in the M6PQ or M12PQ uplink module must be operating at the same speed
(40G or 10G) to use the QSA/QSFP modules.
• For Cisco Nexus 9396PX devices, 2/1-6 ports form the first port speed group and the remaining 2/7-12
ports form the second port speed group.
• For Cisco Nexus 93128PX/TX devices, 2/1-6 ports form the first port speed group and the remaining
2/7-8 ports form the second port speed group.
• For Cisco Nexus 937xPX/TX devices, 1/49 - 54 ports form the only port speed group.
Use the speed-group 10000 command to configure the first port of a port speed group for the QSA. This
command specifies the administrator speed preference for the port group. (The default port speed is 40G.)
• The speed-group 10000 command specifies a speed of 10G.
• The no speed-group 10000 command specifies a speed of 40G.
After the speed has been configured, the compatible transceiver modules are enabled. The remaining transceiver
modules in the port group (incompatible transceiver modules) become error disabled with a reason of "check
speed-group config".
Note The Cisco QSFP+ to SFP+ Adapter (QSA) module does not provide 10G support for the 40G line cards
for Cisco Nexus 9500 devices.
12
Licensing Requirements
Licensing Requirements
The following table shows the licensing requirements for this feature:
Product License Requirement

Cisco NX-OS The basic interface parameters require no license.
Any feature not included in a license package is
bundled with the NX-OS image and is provided at no
extra charge to you.
Guidelines and Limitations

Basic interface parameters have the following configuration guidelines and limitations:
• show commands with the internal keyword are not supported.
• Fiber-optic Ethernet ports must use Cisco-supported transceivers. To verify that the ports are using
Cisco-supported transceivers, use the show interface transceivers command. Interfaces with
Cisco-supported transceivers are listed as functional interfaces.
• A port can be either a Layer 2 or a Layer 3 interface; it cannot be both simultaneously.
By default, each port is a Layer 3 interface.
You can change a Layer 3 interface into a Layer 2 interface by using the switchport command. You
can change a Layer 2 interface into a Layer 3 interface by using the no switchport command.
• You usually configure Ethernet port speed and duplex mode parameters to auto to allow the system to
negotiate the speed and duplex mode between ports. If you decide to configure the port speed and duplex
modes manually for these ports, consider the following:
◦Before you configure the speed and duplex mode for an Ethernet or management interface, see the
Default Settings section for the combinations of speeds and duplex modes that can be configured
at the same time.
◦If you set the Ethernet port speed to auto, the device automatically sets the duplex mode to auto.
◦If you enter the no speed command, the device automatically sets both the speed and duplex
parameters to auto (the no speed command produces the same results as the speed auto command).
◦If you configure an Ethernet port speed to a value other than auto (for example, 1G, 10G, or 40G),
you must configure the connecting port to match. Do not configure the connecting port to negotiate
the speed.
◦To configure speed, duplex, and automatic flow control for an Ethernet interface, you can use the
negotiate auto command. To disable automatic negotiation, use the no negotiate auto command.
Note The device cannot automatically negotiate the Ethernet port speed and duplex mode if
the connecting port is configured to a value other than auto.
13
Default Settings
Caution Changing the Ethernet port speed and duplex mode configuration might shut down and
reenable the interface.
• For BASE-T copper ports, auto-negotiation is enabled even when fixed speed is configured.
• The port profile feature is not supported.
Default Settings
The following lists the default settings for the basic interface parameters.
Parameter Default
Description Blank
Beacon Disabled
Bandwidth Data rate of interface
Throughput delay 100 microseconds
Administrative status Shutdown
MTU 1500 bytes
UDLD global Globally disabled
UDLD per-port enable state for fiber-optic media Enabled on all Ethernet fiber-optic LAN ports
UDLD per-port enable state for copper media Disabled on all Ethernet 1G, 10G, or 40G LAN ports
UDLD message interval Disabled
UDLD aggressive mode Disabled
Error disable Disabled
Error disable recovery Disabled
Error disable recovery interval 300 seconds
Buffer-boost Enabled
Note Feature available on N9K-X9564TX and
N9K-X9564PX line cards and Cisco Nexus
9300 series devices.
14
Configuring the Basic Interface Parameters
Configuring the Basic Interface Parameters

When you configure an interface, you must specify the interface before you can configure its parameters.
Specifying the Interfaces to Configure

Before You Begin
Before you can configure the parameters for one or more interfaces of the same type, you must specify the
type and the identities of the interfaces.
The following table shows the interface types and identities that you should use for specifying the Ethernet
and management interfaces.
Table 4: Information Needed to Identify an Interface for Configurations
Interface Type Identity

Ethernet I/O module slot numbers and port numbers on the
module
Management 0 (for port 0)
The interface range configuration mode allows you to configure multiple interfaces with the same configuration
parameters. After you enter the interface range configuration mode, all command parameters you enter are
attributed to all interfaces within that range until you exit out of the interface range configuration mode.
You enter a range of interfaces using dashes (-) and commas (,). Dashes separate contiguous interfaces and
commas separate noncontiguous interfaces. When you enter noncontiguous interfaces, you must enter the
media type for each interface.
This example shows how to configure a contiguous interface range:
switch(config)# interface ethernet 2/29-30
switch(config-if-range)#
This example shows how to configure a noncontiguous interface range:
switch(config)# interface ethernet 2/29, ethernet 2/33, ethernet 2/35
You can specify subinterfaces in a range only when the subinterfaces are on the same port, for example,
2/29.1-2. But you cannot specify the subinterfaces in a range of ports, for example, you cannot enter
2/29.2-2/30.2. You can specify two of the subinterfaces discretely, for example, you can enter 2/29.2, 2/30.2.
This example shows how to configure a a breakout cable:
switch(config)# interface ethernet 1/2/1
SUMMARY STEPS
1. configure terminal
2. interface interface
15
Configuring the Description
DETAILED STEPS
Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch(config)#
Step 2 interface interface Specifies the interface that you are configuring. You can specify the interface
type and identity. For an Ethernet port, use ethernet slot/port. For the
Example: management interface, use mgmt0.
switch(config)# interface ethernet
2/1 Examples:
switch(config-if)#
• The 1st example shows how to specify the slot 2, port 1 Ethernet interface.
Example: • The 2nd example shows how to specify the management interface.
switch(config)# interface mgmt0
switch(config-if)#
Note You do not need to add a space between the interface type and identity
(port or slot/port number) For example, for the Ethernet slot 4, port 5
interface, you can specify either “ethernet 4/5” or “ethernet4/5.” The
management interface is either “mgmt0” or “mgmt 0.”
When you are in the interface configuration mode, the commands that you enter
configure the interface that you specified for this mode.

You can provide textual interface descriptions for the Ethernet and management interfaces.
SUMMARY STEPS
3. description text
4. show interface interface
5. exit
6. copy running-config startup-config
16
DETAILED STEPS

Example:
switch(config)#
Step 2 interface interface Specifies the interface that you are configuring. You can
specify the interface type and identity. For an Ethernet port,
Example: use ethernet slot/port. For the management interface, use
switch(config)# interface ethernet 2/1 mgmt0.
switch(config-if)#
Examples:
Example: • The 1st example shows how to specify the slot 2, port
switch(config-if)#
1 Ethernet interface.
• The 2nd example shows how to specify the management
interface.
Step 3 description text Specifies the description for the interface.
Example:
switch(config-if)# description Ethernet port
3 on module 1
switch(config-if)#
Step 4 show interface interface (Optional) Displays the interface status, which includes the
description parameter.
Example:
switch(config)# show interface ethernet 2/1
Step 5 exit Exits the interface mode.
Example:
switch(config-if)# exit
switch(config)#
Step 6 copy running-config startup-config (Optional) Copies the running configuration to the startup
configuration.
Example:
switch(config)# copy running-config
startup-config
This example shows how to set the interface description to Ethernet port 24 on module 3:
switch(config)# interface ethernet 3/24
switch(config-if)# description server1
switch(config-if)#
The output of the show interface eth command is enhanced as shown in the following example:
Switch# show version
17
Configuring the Beacon Mode
Software
BIOS: version 06.26
NXOS: version 6.1(2)I2(1) [build 6.1(2)I2.1]
BIOS compile time: 01/15/2014
NXOS image file is: bootflash:///n9000-dk9.6.1.2.I2.1.bin
NXOS compile time: 2/25/2014 2:00:00 [02/25/2014 10:39:03]
switch# show interface ethernet 6/36

Ethernet6/36 is up
admin state is up, Dedicated Interface
Hardware: 40000 Ethernet, address: 0022.bdf6.bf91 (bia 0022.bdf8.2bf3)
Internet Address is 192.168.100.1/24
MTU 9216 bytes, BW 40000000 Kbit, DLY 10 usec
Configuring the Beacon Mode

You can enable the beacon mode for an Ethernet port to flash its LED to confirm its physical location.
SUMMARY STEPS
2. interface ethernet slot/port
3. [no] beacon
4. show interface ethernet slot/port
5. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 interface ethernet slot/port Specifies an interface to configure, and enters interface
configuration mode.
Example:
switch(config-if)#
Step 3 [no] beacon Enables the beacon mode or disables the beacon mode.
The default mode is disabled.
Example:
switch(config)# beacon
switch(config-if)#
Step 4 show interface ethernet slot/port (Optional) Displays the interface status, which includes
the beacon mode state.
Example:
switch(config-if)#
18
Configuring the Error-Disabled State

Example:
switch(config)#
Step 6 copy running-config startup-config (Optional) Copies the running configuration to the
startup configuration.
Example:
switch(config)# copy running-config startup-config
This example shows how to enable the beacon mode for the Ethernet port 3/1:
switch(config-if)# beacon
switch(config-if)#
This example shows how to disable the beacon mode for the Ethernet port 3/1:
switch(config-if)# no beacon
switch(config-if)#
This example shows how to configure the dedicated mode for Ethernet port 4/17 in the group that includes
ports 4/17, 4/19, 4/21, and 4/23:
switch(config)# interface ethernet 4/17, ethernet 4/19, ethernet 4/21, ethernet 4/23
switch(config-if)# shutdown
switch(config-if)# interface ethernet 4/17
switch(config-if)# no shutdown
switch(config-if)#

You can view the reason an interface moves to the error-disabled state and configure automatic recovery.
Enabling the Error-Disable Detection

You can enable error-disable detection in an application. As a result, when a cause is detected on an interface,
the interface is placed in an error-disabled state, which is an operational state that is similar to the link-down
state.
19
SUMMARY STEPS
2. errdisable detect cause {acl-exception | all | link-flap | loopback}
3. shutdown
4. no shutdown
5. show interface status err-disabled
DETAILED STEPS

Example:
switch(config)#
Step 2 errdisable detect cause {acl-exception | all | link-flap Specifies a condition under which to place the interface in
| loopback} an error-disabled state. The default is enabled.
Example:
switch(config)# errdisable detect cause all
switch(config-if)#
Step 3 shutdown Brings the interface down administratively. To manually

recover the interface from the error-disabled state, enter
Example: this command first.
switch(config)#
Step 4 no shutdown Brings the interface up administratively and enables the

interface to recover manually from the error-disabled state.
Example:
switch(config)#
Step 5 show interface status err-disabled (Optional) Displays information about error-disabled
interfaces.
Example:
switch(config)# show interface status
err-disabled
configuration.
Example:
startup-config
20
This example shows how to enable the error-disabled detection in all cases:
switch(config)# errdisable detect cause all
switch(config)#
Enabling the Error-Disabled Recovery

You can specify the application to bring the interface out of the error-disabled state and retry coming up. It
retries after 300 seconds, unless you configure the recovery timer (see the errdisable recovery interval
command).
SUMMARY STEPS
2. errdisable recovery cause {all | bpduguard | failed-port-state | link-flap | loopback | miscabling |
psecure-violation | security-violation | storm-control | udld | vpc-peerlink}
DETAILED STEPS

Example:
switch(config)#
Step 2 errdisable recovery cause {all | bpduguard | failed-port-state Specifies a condition under which the interface
| link-flap | loopback | miscabling | psecure-violation | automatically recovers from the error-disabled state,
security-violation | storm-control | udld | vpc-peerlink} and the device retries bringing the interface up. The
device waits 300 seconds to retry. The default is
Example: disabled.
switch(config)# errdisable recovery cause all
switch(config-if)#
interfaces.
Example:
switch(config)# show interface status err-disabled
switch(config-if)#
Example:
This example shows how to enable error-disabled recovery under all conditions:
switch(config)# errdisable recovery cause all
switch(config)#
21
Configuring the MTU Size
Configuring the Error-Disabled Recovery Interval

You can configure the error-disabled recovery timer value.
SUMMARY STEPS
2. errdisable recovery interval interval
DETAILED STEPS

Example:
switch(config)#
Step 2 errdisable recovery interval interval Specifies the interval for the interface to recover from
the error-disabled state. The range is from 30 to 65535
Example: seconds, and the default is 300 seconds.
switch(config)# errdisable recovery interval 32
switch(config-if)#
interfaces.
Example:
switch(config)# show interface status err-disabled
switch(config-if)#
Example:
This example shows how to configure the error-disabled recovery timer to set the interval for recovery to 32
seconds:
switch(config)# errdisable recovery interval 32
switch(config)#

You can configure the maximum transmission unit (MTU) size for Layer 2 and Layer 3 Ethernet interfaces.
For Layer 3 interfaces, you can configure the MTU to be between 576 and 9216 bytes (even values are
22
required). For Layer 2 interfaces, you can configure the MTU to be either the system default MTU (1500
bytes) or the system jumbo MTU size (which has the default size of 9216 bytes).
Note You can change the system jumbo MTU size, but if you change that value, the Layer 2 interfaces that use
that value automatically changes to the new system jumbo MTU value.
By default, Cisco NX-OS configures Layer 3 parameters. If you want to configure Layer 2 parameters, you
need to switch the port mode to Layer 2.
You can change the port mode by using the switchport command.
After changing the port mode to Layer 2, you can return to configuring Layer 3 interfaces by changing the
port mode again, by using the no switchport command.
Configuring the Interface MTU Size

For Layer 3 interfaces, you can configure an MTU size that is between 576 and 9216 bytes.
For Layer 2 interfaces, you can configure all Layer 2 interfaces to use either the default MTU size (1500
bytes) or the system jumbo MTU size (default size of 9216 bytes).
If you need to use a different system jumbo MTU size for Layer 2 interfaces, see the “Configuring the System
Jumbo MTU Size” section.
SUMMARY STEPS
3. [switchport | no switchport]
4. mtu size
6. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 interface ethernet slot/port Specifies an Ethernet interface to configure, and

enters interface configuration mode.
Example:
switch(config-if)#
23

Step 3 [switchport | no switchport] Specifies to use Layer 3.
Example:
switch(config-if)# no switchport
switch(config-if)#
Step 4 mtu size For a Layer 3 interface, specifies any even number
between 576 and 9216.
Example:
switch(config-if)# mtu 9216
switch(config-if)#
Step 5 show interface ethernet slot/port (Optional) Displays the interface status, which
includes the MTU size.
Example:
Example:
switch(config)#
Example:
This example shows how to configure the Layer 2 Ethernet port 3/1 with the default MTU size (1500):
switch(config-if)# switchport
switch(config-if)#
Configuring the System Jumbo MTU Size

You can configure the system jumbo MTU size, which can be used to specify the MTU size for Layer 2
interfaces. You can specify an even number between 1500 and 9216. If you do not configure the system jumbo
MTU size, it defaults to 9216 bytes.
Note To configure jumbo frames for FEX modules, configure the FEX fabric port-channel interface with the
required MTU size for the FEX module.
24
SUMMARY STEPS
2. system jumbomtu size
3. show running-config all
4. interface type slot/port
5. mtu size
6. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 system jumbomtu size Specifies the system jumbo MTU size. Use an even number
between 1500 and 9216.
Example: Note In general accepted practice, a jumbo frame is
switch(config)# system jumbomtu 8000
switch(config)# considered to have an MTU size greater than 9000
bytes.
Step 3 show running-config all (Optional) Displays the current operating configuration, which
includes the system jumbo MTU size.
Example:
switch(config)# show running-config all |
include jumbomtu
Step 4 interface type slot/port Specifies an interface to configure and enters interface
configuration mode.
Example:
switch(config-if)#
Step 5 mtu size For a Layer 2 interface, specifies either the default MTU size
(1500) or the system jumbo MTU size that you specified
Example: earlier.
switch(config-if)# For a Layer 3 interface, specifies any even size between 576
and 9216.
Example:
switch(config)#
25
Configuring the Bandwidth

configuration.
Example:
startup-config
This example shows how to configure the system jumbo MTU as 8000 bytes and how to change the MTU
specification for an interface that was configured with the previous jumbo MTU size:
switch(config)# system jumbomtu 8000
switch(config)# show running-config
switch(config-if)#
Configuring the Bandwidth

You can configure the bandwidth for Ethernet interfaces. The physical layer uses an unchangeable bandwidth
of 1G, 10G, or 40G, but you can configure a value of 1 to 100,000,000 KB for Level 3 protocols.
SUMMARY STEPS
3. bandwidth kbps
5. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 interface ethernet slot/port Specifies an Ethernet interface to configure, and enters
interface configuration mode.
Example:
switch(config-if)#
26
Configuring the Throughput Delay

Step 3 bandwidth kbps Specifies the bandwidth as an informational-only value
between 1 and 100,000,000.
Example:
switch(config-if)# bandwidth 1000000
switch(config-if)#
the bandwidth value.
Example:
Example:
switch(config)#
Example:
This example shows how to configure an informational value of 1,000,000 Kb for the Ethernet slot 3, port 1
interface bandwidth parameter:
switch(config-if)#

You can configure the interface throughput delay for Ethernet interfaces. The actual delay time does not
change, but you can set an informational value between 1 and 16777215, where the value represents the
number of tens of microseconds.
SUMMARY STEPS
3. delay value
5. exit
27
DETAILED STEPS

Example:
switch(config)#
Example:
switch(config-if)#
Step 3 delay value Specifies the delay time in tens of microseconds. You
can set an informational value range between 1 and
Example: 16777215 tens of microseconds.
switch(config-if)# delay 10000
switch(config-if)#
the throughput-delay time.
Example:
switch(config-if)#
Example:
switch(config)#
configuration.
Example:
startup-config
This example shows how to configure the throughput-delay time so that one interface is preferred over another.
A lower delay value is preferred over a higher value. In this example, Ethernet 7/48 is preferred over 7/47.
The default delay for 7/48 is less than the configured value on 7/47, which is set for the highest value
(16777215):
switch(config-if)# ip address 192.168.10.1/24
switch(config-if)# ip router eigrp 10
switch(config-if)# ip router eigrp 10
switch(config-if)#
28
Shutting Down and Activating the Interface
Note You must first ensure the EIGRP feature is enabled by running the feature eigrp command.
Shutting Down and Activating the Interface

You can shut down and restart Ethernet or management interfaces. When you shut down interfaces, they
become disabled and all monitoring displays show them as being down. This information is communicated
to other network servers through all dynamic routing protocols. When the interfaces are shut down, the interface
is not included in any routing updates. To activate the interface, you must restart the device.
SUMMARY STEPS
3. shutdown
5. no shutdown
7. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 interface interface Specifies the interface that you are configuring. You can
specify the interface type and identity. For an Ethernet port,
Example: use ethernet slot/port. For the management interface, use
switch(config)# interface ethernet 2/1 mgmt0.
switch(config-if)#
Examples:
switch(config-if)# • The 1st example shows how to specify the slot 2, port
1 Ethernet interface.
• The 2nd example shows how to specify the
management interface.
29
Configuring the UDLD Mode

Step 3 shutdown Disables the interface.
Example:
switch(config-if)#
administrative status.
Example:
switch(config-if)# show interface ethernet 2/1
switch(config-if)#
Step 5 no shutdown Reenables the interface.
Example:
switch(config-if)#
administrative status.
Example:
switch(config-if)# show interface ethernet 2/1
switch(config-if)#
Example:
switch(config)#
configuration.
Example:
startup-config
This example shows how to change the administrative status for Ethernet port 3/1 from disabled to enabled:
switch(config-if)#

You can configure normal unidirectional link detection (UDLD) modes for Ethernet interfaces on devices
configured to run UDLD.
Before you can enable a UDLD mode for an interface, you must make sure that UDLD is already enabled on
the device that includes the interface. UDLD must also be enabled on the other linked interface and its device.
The following table lists CLI details to enable and disable UDLD on different interfaces
30
Table 5: CLI Details to Enable or Disable UDLD on Different Interfaces
Description Fiber port Copper or Nonfiber port

Default setting Enabled Disabled
Enable UDLD command no udld disable udld enable
Disable UDLD command udld disable no udld enable
Before You Begin

You must enable UDLD for the other linked port and its device.
SUMMARY STEPS
2. [no] feature udld
3. udld message-time seconds
4. udld aggressive
6. udld [enable | disable]
7. show udld [ethernet slot/port | global | neighbors]
8. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 [no] feature udld Enables/Disables UDLD for the device.
Example:
switch(config)# feature udld
switch(config)#
switch(config)# no feature udld
switch(config)#
Step 3 udld message-time seconds (Optional) Specifies the interval between sending UDLD
messages. The range is from 7 to 90 seconds, and the default
Example: is 15 seconds.
switch(config)# udld message-time 30
switch(config)#
31

Step 4 udld aggressive Optional) Specifies UDLD mode to be aggressive.
Note For copper interfaces, you enter the interface
Example: command mode for those interfaces you want to
switch(config)# udld aggressive
switch(config)# configure for UDLD aggressive mode and issue this
command in interface command model.
Step 5 interface ethernet slot/port (Optional) Specifies an interface to configure, and enters
Example:
switch(config-if)#
Step 6 udld [enable | disable] (Optional) Enables UDLD on the specified copper port or
disables UDLD on the specified fiber port.
Example: To enable UDLD on copper ports, enter the udld enable
switch(config-if)# udld enable
switch(config-if)# command. To enable UDLD on fiber ports, enter the no udld
disable command.
Step 7 show udld [ethernet slot/port | global | neighbors] (Optional) Displays the UDLD status.
Example:
switch(config)# show udld
switch(config)#
Example:
switch(config-if-range)# exit
switch(config)#
configuration.
Example:
startup-config
This example shows how to enable the UDLD for the device:
switch(config)#
This example shows how to set the UDLD message interval to 30 seconds:
switch(config)# udld message-time 30
switch(config)#
This example shows how to disable UDLD for Ethernet port 3/1:
switch(config-if-range)# no udld enable
32
Configuring the Debounce Timer
This example shows how to disable UDLD for the device:

switch(config)# no feature udld
switch(config)# exit
Configuring the Debounce Timer

You can enable the debounce timer for Ethernet ports by specifying a debounce time (milliseconds) or disable
the timer by specifying a debounce time of 0.
Note The link debounce time command can only be applied to a physical Ethernet interface.
Note Use the show interface debounce command to display the debounce times for all Ethernet ports.
SUMMARY STEPS
3. link debounce time time
DETAILED STEPS

Example:
switch(config)#
Example:
switch(config-if)#
Step 3 link debounce time time Enables the debounce timer for the specified time (1 to
5000 milliseconds).
Example: If you specify 0 milliseconds, the debounce timer is
switch(config-if)# link debounce time 1000
switch(config-if)# disabled.
• The following example enables the debounce timer and sets the debounce time to 1000 milliseconds for
an Ethernet interface:
33
Verifying the Basic Interface Parameters

• The following example disables the debounce timer for an Ethernet interface:

Verifying the Basic Interface Parameters

You can verify the basic interface parameters by displaying their values. You can also clear the counters listed
when you display the parameter values.
To display basic interface configuration information, perform one of the following tasks:
Command Purpose
show cdp all Displays the CDP status.
show interface interface Displays the configured states of one or all interfaces.
show interface brief Displays a table of interface states.
show interface status err-disabled Displays information about error-disabled interfaces.
show udld interface Displays the UDLD status for the current interface
or all interfaces.
show udld global Displays the UDLD status for the current device.
Monitoring the Interface Counters

You can display and clear interface counters using Cisco NX-OS.
Displaying Interface Statistics

You can set up to three sampling intervals for statistics collections on interfaces.
SUMMARY STEPS
2. interface ether slot/port
3. load-interval counters [1 | 2 | 3] seconds
5. exit
34
Displaying Interface Statistics
DETAILED STEPS

Example:
switch(config)#
Step 2 interface ether slot/port Specifies interface.
Example:
switch(config)# interface ether 4/1
switch(config)#
Step 3 load-interval counters [1 | 2 | 3] seconds Sets up to three sampling intervals to collect bit-rate and
packet-rate statistics. The default values for each counter
Example: is as follows:
switch(config)# load-interval counters 1 100
switch(config)# 1—30 seconds (60 seconds for VLAN)
2—300 seconds
3—not configured
Step 4 show interface interface (Optional) Displays the interface status, which includes
the counters.
Example:
switch#
Example:
switch(config)#
Example:
This example shows how to set the three sample intervals for the Ethernet port 3/1:
switch(config-if)# load-interval counter 1 60
switch(config-if)#
35
Clearing Interface Counters
Clearing Interface Counters

You can clear the Ethernet and management interface counters by using the clear counters interface command.
You can perform this task from the configuration mode or interface configuration mode.
SUMMARY STEPS
1. clear counters interface [all | ethernet slot/port | loopback number | mgmt number | port channel
channel-number]
3. show interface [ethernet slot/port | port channel channel-number] counters
DETAILED STEPS

Step 1 clear counters interface [all | ethernet slot/port | loopback number | Clears the interface counters.
mgmt number | port channel channel-number]
Example:
switch# clear counters ethernet 2/1
switch#
Step 2 show interface interface (Optional) Displays the interface status.
Example:
switch#
Step 3 show interface [ethernet slot/port | port channel channel-number] (Optional) Displays the interface counters.
counters
Example:
switch# show interface ethernet 2/1 counters
switch#
This example shows how to clear the counters on Ethernet port 5/5:
switch# clear counters interface ethernet 5/5
switch#
Configuration Example for QSA

For a Cisco Nexus 9396PX:
• Using the default configuration on port 2/1, all the QSFPs in port group 2/1-6 are brought up with a
speed of 40G. If there are any QSA modules in port group 2/1-6, they are error disabled.
36
• Using the speed-group [ 10000 | 40000] command to configure port 2/7, all the QSAs in port group
2/7-12 are brought up with a speed of 10G or 40G. If there are any QSFP modules in port group 2/7-12,
they are error disabled.
This example shows how to configure QSA for the first port in the speed group for a Cisco Nexus 9396PX:
switch# conf t
switch(config-if)# speed-group 10000
37
38
CHAPTER 4
Configuring Layer 2 Interfaces
This chapter describes how to configure Layer 2 switching ports as access or trunk ports on Cisco NX-OS
devices.
Note A Layer 2 port can function as either one of the following:

• A trunk port
• An access port
Note See the Cisco Nexus 9000 Series NX-OS System Management Configuration Guide for information about
configuring a SPAN destination interface.
You can configure Layer 2 switching ports as access or trunk ports. Trunks carry the traffic of multiple
VLANs over a single link and allow you to extend VLANs across an entire network. All Layer 2 switching
ports maintain media access control (MAC) address tables.
Note See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for information about
VLANs, MAC address tables, private VLANs, and the Spanning Tree Protocol.
• Information About Access and Trunk Interfaces, page 40

• Licensing Requirements for Layer 2 Port Modes , page 46
• Prerequisites for Layer 2 Interfaces, page 46
• Guidelines and Limitations for Layer 2 Interfaces, page 46
• Default Settings for Layer 2 Interfaces, page 48
• Configuring Access and Trunk Interfaces , page 48
• Verifying the Interface Configuration, page 64
• Monitoring the Layer 2 Interfaces, page 65
39
Information About Access and Trunk Interfaces
• Configuration Examples for Access and Trunk Ports, page 65

• Related Documents, page 65
Information About Access and Trunk Interfaces
Note See the Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide for complete information
on high-availability features.
Note The device supports only IEEE 802.1Q-type VLAN trunk encapsulation.
About Access and Trunk Interfaces

A Layer 2 port can be configured as an access or a trunk port as follows:
• An access port can have only one VLAN configured on that port; it can carry traffic for only one VLAN.
• A trunk port can have two or more VLANs configured on that port; it can carry traffic for several VLANs
simultaneously.
By default, all ports on the device are Layer 3 ports.

You can make all ports Layer 2 ports using the setup script or by entering the system default
switchportcommand. See the Cisco Nexus 9000 Series NX-OS Fundamentals Configuration Guide for
information about using the setup script. To configure the port as a Layer 2 port using the CLI, use the
switchport command.
All ports in the same trunk must be in the same VDC, and trunk ports cannot carry VLANs from different
VDCs.
The following figure shows how you can use trunk ports in the network. The trunk port carries traffic for two
or more VLANs.
Figure 2: Trunk and Access Ports and VLAN Traffic
40
IEEE 802.1Q Encapsulation
Note See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for information about
VLANs.
In order to correctly deliver the traffic on a trunk port with several VLANs, the device uses the IEEE 802.1Q
encapsulation, or tagging, method (see the “IEEE 802.1Q Encapsulation” section for more information).
Note See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for information about
subinterfaces on Layer 3 interfaces.
To optimize the performance on access ports, you can configure the port as a host port. Once the port is
configured as a host port, it is automatically set as an access port, and channel grouping is disabled. Use the
host designation to decrease the time that it takes the designated port to begin to forward packets.
Only an end station can be set as a host port; you will receive an error message if you attempt to configure
other ports as hosts.
If an access port receives a packet with an 802.1Q tag in the header other than the access VLAN value, that
port drops the packet without learning its MAC source address.
A Layer 2 interface can function as either an access port or a trunk port; it cannot function as both port types
simultaneously.
When you change a Layer 2 interface back to a Layer 3 interface, that interface loses all the Layer 2
configuration and resumes the default VLAN configurations.
IEEE 802.1Q Encapsulation
Note For information about VLANs, see the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration
Guide.
A trunk is a point-to-point link between the switch and another networking device. Trunks carry the traffic
of multiple VLANs over a single link and allow you to extend VLANs across an entire network.
To correctly deliver the traffic on a trunk port with several VLANs, the device uses the IEEE 802.1Q
encapsulation, or tagging, method that uses a tag that is inserted into the frame header. This tag carries
information about the specific VLAN to which the frame and packet belong. This method allows packets that
are encapsulated for several different VLANs to traverse the same port and maintain traffic separation between
41
Access VLANs
the VLANs. Also, the encapsulated VLAN tag allows the trunk to move traffic end-to-end through the network
on the same VLAN.
Figure 3: Header Without and With 802.1Q Tag
Access VLANs
When you configure a port in access mode, you can specify which VLAN will carry the traffic for that interface.
If you do not configure the VLAN for a port in access mode, or an access port, the interface carries traffic for
the default VLAN (VLAN1).
You can change the access port membership in a VLAN by specifying the new VLAN. You must create the
VLAN before you can assign it as an access VLAN for an access port. If you change the access VLAN on an
access port to a VLAN that is not yet created, the system shuts that access port down.
If an access port receives a packet with an 802.1Q tag in the header other than the access VLAN value, that
port drops the packet without learning its MAC source address.
Native VLAN IDs for Trunk Ports

A trunk port can carry nontagged packets simultaneously with the 802.1Q tagged packets. When you assign
a default port VLAN ID to the trunk port, all untagged traffic travels on the default port VLAN ID for the
trunk port, and all untagged traffic is assumed to belong to this VLAN. This VLAN is referred to as the native
VLAN ID for a trunk port. That is, the native VLAN ID is the VLAN that carries untagged traffic on trunk
ports.
42
Tagging Native VLAN Traffic
Note Native VLAN ID numbers must match on both ends of the trunk.
The trunk port sends an egressing packet with a VLAN that is equal to the default port VLAN ID as untagged;
all the other egressing packets are tagged by the trunk port. If you do not configure a native VLAN ID, the
trunk port uses the default VLAN.
Note You cannot use a Fibre Channel over Ethernet (FCoE) VLAN as a native VLAN for an Ethernet trunk
switchport.
Tagging Native VLAN Traffic

The Cisco software supports the IEEE 802.1Q standard on trunk ports. In order to pass untagged traffic through
the trunk ports, you must create a VLAN that does not tag any packets (or you can use the default VLAN).
Untagged packets can pass through trunk ports and access ports.
However, all packets that enter the device with an 802.1Q tag that matches the value of the native VLAN on
the trunk are stripped of any tagging and egress the trunk port as untagged packets. This situation can cause
problems because you may want to retain the tagging on packets on the native VLAN for the trunk port.
You can configure the device to drop all untagged packets on the trunk ports and to retain the tagging of
packets entering the device with 802.1Q values that are equal to that of the native VLAN ID. All control traffic
still passes on the native VLAN. This configuration is global; trunk ports on the device either do or do not
retain the tagging for the native VLAN.
Allowed VLANs
By default, a trunk port sends traffic to and receives traffic from all VLANs. All VLAN IDs are allowed on
each trunk. However, you can remove VLANs from this inclusive list to prevent traffic from the specified
VLANs from passing over the trunk. Later, you can add any specific VLANs that you may want the trunk to
carry traffic for back to the list.
To partition the Spanning Tree Protocol (STP) topology for the default VLAN, you can remove VLAN1 from
the list of allowed VLANs. Otherwise, VLAN1, which is enabled on all ports by default, will have a very big
STP topology, which can result in problems during STP convergence. When you remove VLAN1, all data
traffic for VLAN1 on this port is blocked, but the control traffic continues to move on the port.
Note See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for more information
about STP.
Note You can change the block of VLANs reserved for internal use. See the Cisco Nexus 9000 Series NX-OS
Layer 2 Switching Configuration Guide for more information about changing the reserved VLANs.
43
Default Interfaces
Default Interfaces
You can use the default interface feature to clear the configured parameters for both physical and logical
interfaces such as the Ethernet, loopback, VLAN network, tunnel, and the port-channel interface.
Note A maximum of eight ports can be selected for the default interface. The default interfaces feature is not
supported for management interfaces because the device could go to an unreachable state.
Switch Virtual Interface and Autostate Behavior

In Cisco NX-OS, a switch virtual interface (SVI) represents a logical interface between the bridging function
and the routing function of a VLAN in the device.
The operational state of this interface is governed by the state of the various ports in its corresponding VLAN.
An SVI interface on a VLAN comes up when at least one port in that VLAN is in the Spanning Tree Protocol
(STP) forwarding state. Similarly, this interface goes down when the last STP forwarding port goes down or
goes to another STP state.
SVI Autostate Enable/Disable

You can also use the SVI for inband management of a device by enabling or disabling the SVI autostate
feature. Specifically, you can configure the autostate disable feature to keep an SVI up even if no interface is
up in the corresponding VLAN. You can configure this feature for the system (for all SVIs) or for an individual
SVI.
High Availability
The software supports high availability for Layer 2 ports.
about high availability features.
Virtualization Support
All ports in the same trunk must be in the same VDC, and trunk ports cannot carry VLANs from different
VDCs.
Counter Values
See the following information on the configuration, packet size, incremented counter values, and traffic.
44
Counter Values
Configuration Packet Size Incremented Counters Traffic

L2 port – without any 6400 and 10000 Jumbo, giant, and input Dropped
MTU configuration error
L2 port – with jumbo 6400 Jumbo Forwarded

MTU 9216 in
network-qos
configuration
L2 port – with jumbo 10000 Jumbo, giant, and input Dropped

MTU 9216 in error
network-qos
configuration
Layer 3 port with default 6400 Jumbo Packets are punted to the
Layer 3 MTU and jumbo CPU (subjected to CoPP
MTU 9216 in configs), get fragmented,
network-qos and then they are
configuration forwarded by the
software.
Layer 3 port with default 6400 Jumbo Packets are punted to the
Layer 3 MTU and jumbo CPU (subjected to CoPP
MTU 9216 in configs), get fragmented,
network-qos and then they are
configuration forwarded by the
software.
Layer 3 port with default 10000 Jumbo, giant, and input Dropped
Layer 3 MTU and jumbo error
MTU 9216 in
network-qos
configuration
Layer 3 port with jumbo 6400 Jumbo Forwarded without any

Layer 3 MTU and jumbo fragmentation.
MTU 9216 in
network-qos
configuration
Layer 3 port with jumbo 10000 Jumbo, giant, and input Dropped
Layer 3 MTU and jumbo error
MTU 9216 in
network-qos
configuration
Layer 3 port with jumbo 6400 and 10000 Jumbo, giant, and input Dropped
Layer 3 MTU and default error
L2 MTU configuration
45
Licensing Requirements for Layer 2 Port Modes
Note • Under 64 bytes packet with good CRC–The short frame counter increments.
• Under 64 bytes packet with bad CRC–The runts counter increments.
• Greater than 64 bytes packet with bad CRC–The CRC counter increments.
Licensing Requirements for Layer 2 Port Modes


Cisco NX-OS Layer 2 port modes require no license. Any feature not included in a license
package is bundled with the Cisco NX-OS system images and is provided at
no extra charge to you.
Prerequisites for Layer 2 Interfaces

Layer 2 interfaces have the following prerequisites:
• You are logged onto the device.
• You must configure the port as a Layer 2 port before you can use the switchport modecommand. By
default, all ports on the device are Layer 3 ports. By default, all ports on the Cisco Nexus 9504 and Cisco
Nexus 9508 devices are Layer 2 ports.
Guidelines and Limitations for Layer 2 Interfaces

VLAN trunking has the following configuration guidelines and limitations:
• A port can be either a Layer 2 or a Layer 3 interface; it cannot be both simultaneously.
• When you change a Layer 3 port to a Layer 2 port or a Layer 2 port to a Layer 3 port, all layer-dependent
configuration is lost. When you change an access or trunk port to a Layer 3 port, all information about
the access VLAN, native VLAN, allowed VLANs, and so forth, is lost.
• Do not connect devices with access links because access links may partition a VLAN.
• When connecting Cisco devices through an 802.1Q trunk, make sure that the native VLAN for an 802.1Q
trunk is the same on both ends of the trunk link. If the native VLAN on one end of the trunk is different
from the native VLAN on the other end, spanning tree loops might result.
46
Guidelines and Limitations for Layer 2 Interfaces
• Disabling spanning tree on the native VLAN of an 802.1Q trunk without disabling spanning tree on
every VLAN in the network can cause spanning tree loops. You must leave spanning tree enabled on
the native VLAN of an 802.1Q trunk. If you cannot leave spanning tree enabled, you must disable
spanning tree on every VLAN in the network. Make sure that your network has no physical loops before
you disable spanning tree.
• When you connect two Cisco devices through 802.1Q trunks, the devices exchange spanning tree bridge
protocol data units (BPDUs) on each VLAN allowed on the trunks. The BPDUs on the native VLAN
of the trunk are sent untagged to the reserved IEEE 802.1D spanning tree multicast MAC address
(01-80-C2-00-00-00). The BPDUs on all other VLANs on the trunk are sent tagged to the reserved Cisco
Shared Spanning Tree (SSTP) multicast MAC address (01-00-0c-cc-cc-cd).
• Non-Cisco 802.1Q devices maintain only a single instance of spanning tree (the Mono Spanning Tree)
that defines the spanning tree topology for all VLANs. When you connect a Cisco switch to a non-Cisco
switch through an 802.1Q trunk, the Mono Spanning Tree of the non-Cisco switch and the native VLAN
spanning tree of the Cisco switch combine to form a single spanning tree topology known as the Common
Spanning Tree (CST).
• Because Cisco devices transmit BPDUs to the SSTP multicast MAC address on VLANs other than the
native VLAN of the trunk, non-Cisco devices do not recognize these frames as BPDUs and flood them
on all ports in the corresponding VLAN. Other Cisco devices connected to the non-Cisco 802.1Q cloud
receive these flooded BPDUs. This BPDU reception allows Cisco switches to maintain a per-VLAN
spanning tree topology across a cloud of non-Cisco 802.1Q devices. The non-Cisco 802.1Q cloud that
separates the Cisco devices is treated as a single broadcast segment between all devices connected to
the non-Cisco 802.1Q cloud through 802.1Q trunks.
• Make certain that the native VLAN is the same on all of the 802.1Q trunks that connect the Cisco devices
to the non-Cisco 802.1Q cloud.
• If you are connecting multiple Cisco devices to a non-Cisco 802.1Q cloud, all of the connections must
be through 802.1Q trunks. You cannot connect Cisco devices to a non-Cisco 802.1Q cloud through
access ports because doing so places the access port on the Cisco device into the spanning tree “port
inconsistent” state and no traffic will pass through the port.
• You can group trunk ports into port-channel groups, but all trunks in the group must have the same
configuration. When a group is first created, all ports follow the parameters set for the first port to be
added to the group. If you change the configuration of one of these parameters, the device propagates
that setting to all ports in the group, such as the allowed VLANs and the trunk status. For example, if
one port in a port group ceases to be a trunk, all ports cease to be trunks.
• If you try to enable 802.1X on a trunk port, an error message appears, and 802.1X is not enabled. If you
try to change the mode of an 802.1X-enabled port to trunk, the port mode is not changed.
• Only ingress unicast packet counters are supported for SVI counters.
• When MAC addresses are cleared on a VLAN with the clear mac address-table dynamic command, the
dynamic ARP (Address Resolution Protocol) entries on that VLAN are refreshed.
• If a static ARP entry exists on the VLAN and no MAC address to port mapping is present, the supervisor
may generate an ARP request to learn the MAC address. Upon learning the MAC address, the adjacency
entry points to the correct physical port.
• Cisco NX-OS does not support transparent bridging between two VLANs when one of the SVIs is on
the Cisco Nexus 9000 using the BIA MAC (burned-in MAC address). This occurs when the BIA MAC
is shared between SVIs/VLANs.
47
Default Settings for Layer 2 Interfaces
A MAC, different from the BIA MAC, can be configured under the SVI for transparent bridging to work
properly.
• Port-local VLANs do not support Fabric Extenders (FEX).
Default Settings for Layer 2 Interfaces

The following table lists the default settings for device access and trunk port mode parameters.
Table 6: Default Access and Trunk Port Mode Parameters
Parameters Default
Switchport mode Access
Allowed VLANs 1 to 3967, 4048 to 4094
Access VLAN ID VLAN1
Native VLAN ID VLAN1
Native VLAN ID tagging Disabled
Administrative state Shut
Configuring Access and Trunk Interfaces
Note If you are familiar with the Cisco IOS CLI, be aware that the Cisco NX-OS commands for this feature
might differ from the Cisco IOS commands that you would use.
Guidelines for Configuring Access and Trunk Interfaces

All VLANs on a trunk must be in the same VDC.
Configuring a VLAN Interface as a Layer 2 Access Port

You can configure a Layer 2 port as an access port. An access port transmits packets on only one, untagged
VLAN. You specify which VLAN traffic that the interface carries, which becomes the access VLAN. If you
do not specify a VLAN for an access port, that interface carries traffic only on the default VLAN. The default
VLAN is VLAN1.
The VLAN must exist before you can specify that VLAN as an access VLAN. The system shuts down an
access port that is assigned to an access VLAN that does not exist.
48
Configuring a VLAN Interface as a Layer 2 Access Port
Before You Begin

Ensure that you are configuring a Layer 2 interface.
SUMMARY STEPS
2. interface ethernet {{type slot/port} | {port-channel number}}
3. switchport mode [access | trunk]
4. switchport access vlan vlan-id
5. exit
6. show interface
7. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface ethernet {{type slot/port} | Specifies an interface to configure, and enters interface
{port-channel number}} configuration mode.
Example:
switch(config-if)#
Step 3 switchport mode [access | trunk] Sets the interface as a nontrunking nontagged, single-VLAN Layer
2 interface. An access port can carry traffic in one VLAN only.
Example: By default, an access port carries traffic for VLAN1; to set the
switch(config-if)# switchport mode access access port to carry traffic for a different VLAN, use the
switchport access vlan command.
Step 4 switchport access vlan vlan-id Specifies the VLAN for which this access port will carry traffic.
If you do not enter this command, the access port carries traffic
Example: on VLAN1 only; use this command to change the VLAN for which
switch(config-if)# switchport access vlan the access port carries traffic.
5
Step 5 exit Exits the interface configuration mode.
Example:
switch(config)#
49
Configuring Access Host Ports

Step 6 show interface (Optional) Displays the interface status and information.
Example:
switch# show interface
Step 7 no shutdown (Optional) Clears the errors on the interfaces and VLANs where
policies correspond with hardware policies. This command allows
Example: policy programming to continue and the port to come up. If policies
switch# configure terminal do not correspond, the errors are placed in an error-disabled policy
switch(config)# int e3/1 state.
configuration.
Example:
startup-config
This example shows how to set Ethernet 3/1 as a Layer 2 access port that carries traffic for VLAN 5 only:
switch(config-if)# switchport mode access
switch(config-if)# switchport access vlan 5
switch(config-if)#
Note You should apply the switchport host command only to interfaces that are connected to an end station.
You can optimize the performance of access ports that are connected to end stations by simultaneously setting
that port as an access port. An access host port handles the STP like an edge port and immediately moves to
the forwarding state without passing through the blocking and learning states. Configuring an interface as an
access host port also disables port channeling on that interface.
Note See “Configuring Port Channels” section and the Cisco Nexus 9000 Series NX-OS Layer 2 Switching
Configuration Guide for information about port-channel interfaces
Before You Begin

Ensure that you are configuring the correct interface to an interface that is an end station.
50
SUMMARY STEPS
2. interface ethernet type slot/port
3. switchport host
4. exit
5. show interface
6. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface ethernet type slot/port Specifies an interface to configure, and enters interface
configuration mode.
Example:
switch(config-if)#
Step 3 switchport host Sets the interface to be an access host port, which immediately
moves to the spanning tree forwarding state and disables port
Example: channeling on this interface.
switch(config-if)# switchport host Note Apply this command only to end
stations.
Example:
switch(config)#
Example:
policies correspond with hardware policies. This command
Example: allows policy programming to continue and the port to come
switch# configure terminal up. If policies do not correspond, the errors are placed in an
switch(config)# int e3/1 error-disabled policy state.
51
Configuring Trunk Ports

configuration.
Example:
startup-config
This example shows how to set Ethernet 3/1 as a Layer 2 access port with PortFast enabled and port channel
disabled:
switch(config-if)# switchport host
switch(config-if)#

You can configure a Layer 2 port as a trunk port. A trunk port transmits untagged packets for one VLAN plus
encapsulated, tagged, packets for multiple VLANs. (See the “IEEE 802.1Q Encapsulation” section for
information about encapsulation.)
Note The device supports 802.1Q encapsulation only.
Before You Begin

Before you configure a trunk port, ensure that you are configuring a Layer 2 interface.
SUMMARY STEPS
2. interface {type slot/port | port-channel number}
3. switchport mode [access | trunk]
4. exit
5. show interface
6. no shutdown
52
DETAILED STEPS

Example:
switch(config)#
Step 2 interface {type slot/port | port-channel number} Specifies an interface to configure, and enters interface
configuration mode.
Example:
switch(config-if)#
Step 3 switchport mode [access | trunk] Sets the interface as a Layer 2 trunk port. A trunk port can carry
traffic in one or more VLANs on the same physical link (VLANs
Example: are based on the trunk-allowed VLANs list). By default, a trunk
switch(config-if)# switchport mode trunk interface can carry traffic for all VLANs. To specify that only
certain VLANs are allowed on the specified trunk, use the
switchport trunk allowed vlan command.
Example:
switch(config)#
Example:
configuration.
Example:
startup-config
This example shows how to set Ethernet 3/1 as a Layer 2 trunk port:
switch(config-if)# switchport mode trunk
switch(config-if)#
53
Configuring the Native VLAN for 802.1Q Trunking Ports
Configuring the Native VLAN for 802.1Q Trunking Ports

You can configure the native VLAN for 802.1Q trunk ports. If you do not configure this parameter, the trunk
port uses the default VLAN as the native VLAN ID.
Note You cannot configure an FCoE VLAN as a native VLAN for an Ethernet interface.
SUMMARY STEPS
2. interface {{type slot/port} | {port-channel number}}
3. switchport trunk native vlan vlan-id
4. exit
5. show vlan
6. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface {{type slot/port} | {port-channel number}} Specifies an interface to configure, and enters interface
configuration mode.
Example:
switch(config-if)#
Step 3 switchport trunk native vlan vlan-id Sets the native VLAN for the 802.1Q trunk. Valid values are
from 1 to 4094, except those VLANs reserved for internal
Example: use. The default value is VLAN1.
switch(config-if)# switchport trunk native
vlan 5
Step 4 exit Exits interface configuration mode.
Example:
switch(config)#
Step 5 show vlan (Optional) Displays the status and information of VLANs.
Example:
switch# show vlan
54
Configuring the Allowed VLANs for Trunking Ports

Step 6 no shutdown (Optional) Clears the errors on the interfaces and VLANs
where policies correspond with hardware policies. This
Example: command allows policy programming to continue and the port
switch# configure terminal to come up. If policies do not correspond, the errors are placed
switch(config)# int e3/1 in an error-disabled policy state.
configuration.
Example:
startup-config
This example shows how to set the native VLAN for the Ethernet 3/1, Layer 2 trunk port to VLAN 5:
switch(config-if)# switchport trunk native vlan 5
switch(config-if)#

You can specify the IDs for the VLANs that are allowed on the specific trunk port.
Note The switchport trunk allowed vlan vlan-list command replaces the current VLAN list on the specified
port with the new list. You are prompted for confirmation before the new list is applied.
If you are doing a copy and paste of a large configuration, you might see some failures because the CLI
is waiting for a confirmation before accepting other commands. To avoid this problem, you can disable
prompting by using the terminal dont-ask command before you paste the configuration.
Before You Begin

Before you configure the allowed VLANs for the specified trunk ports, ensure that you are configuring the
correct interfaces and that the interfaces are trunks.
Note You can change the block of VLANs reserved for internal use. See the Cisco Nexus 9000 Series NX-OS
Layer 2 Switching Configuration Guide for more information about changing the reserved VLANs.
55
SUMMARY STEPS
2. interface {ethernet slot/port | port-channel number}
3. switchport trunk allowed vlan {vlan-list add vlan-list | all | except vlan-list | none | remove vlan-list}
4. exit
5. show vlan
6. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface {ethernet slot/port | port-channel Specifies an interface to configure, and enters interface configuration
number} mode.
Example:
Step 3 switchport trunk allowed vlan {vlan-list add Sets the allowed VLANs for the trunk interface. The default is to
vlan-list | all | except vlan-list | none | remove allow all VLANs on the trunk interface: 1 to 3967 and 4048 to 4094.
vlan-list} VLANs 3968 to 4047 are the default VLANs reserved for internal
use by default. By default, all VLANs are allowed on all trunk
Example: interfaces.
switch(config-if)# switchport trunk
allowed vlan add 15-20# The default reserved VLANs are 3968 to 4094, and you can change
the block of reserved VLANs. See the Cisco Nexus 9000 Series NX-OS
Layer 2 Switching Configuration Guide for more information.
Note You cannot add internally allocated VLANs as allowed
VLANs on trunk ports. The system returns a message if you
attempt to list an internally allocated VLAN as an allowed
VLAN.
Example:
switch(config)#
Step 5 show vlan (Optional) Displays the status and information for VLANs.
Example:
switch# show vlan
56
Configuring a Default Interface

configuration.
Example:
startup-config
This example shows how to add VLANs 15 to 20 to the list of allowed VLANs on the Ethernet 3/1, Layer 2
trunk port:
switch(config-if)# switchport trunk allowed vlan 15-20
switch(config-if)#
Configuring a Default Interface

The default interface feature allows you to clear the existing configuration of multiple interfaces such as
Ethernet, loopback, VLAN network, port-channel, and tunnel interfaces. All user configuration under a
specified interface will be deleted. You can optionally create a checkpoint before clearing the interface
configuration so that you can later restore the deleted configuration.
Note The default interface feature is not supported for management interfaces because the device could go to
an unreachable state.
SUMMARY STEPS
2. default interface int-if [checkpoint name]
3. exit
4. show interface
5. no shutdown
57
Configuring SVI Autostate Disable for the System
DETAILED STEPS

Example:
switch(config)#
Step 2 default interface int-if [checkpoint name] Deletes the configuration of the interface and restores the default
configuration. Use the ? keyword to display the supported
switch(config)# default interface ethernet
3/1 checkpoint test8 Use the checkpoint keyword to store a copy of the running
configuration of the interface before clearing the configuration.
Step 3 exit Exits global configuration mode.
Example:
switch(config)#
Example:
This example shows how to delete the configuration of an Ethernet interface while saving a checkpoint of the
running configuration for rollback purposes:
switch(config)# default interface ethernet 3/1 checkpoint test8
.......Done
switch(config)#
Configuring SVI Autostate Disable for the System

You can manage an SVI with the SVI autostate feature. You can configure the SVI autostate disable feature
to keep an SVI up even if no interface is up in the corresponding VLAN. (Similarly, configure the SVI autostate
enable feature so an SVI goes down when no interface is up in the corresponding VLAN). Use this procedure
to configure this feature for the entire system.
Note The system default interface-vlan autostate command enables the SVI autostate feature.
58
Configuring SVI Autostate Disable Per SVI
SUMMARY STEPS
2. [no] system default interface-vlan autostate
3. no shutdown
4. show running-config [all]
DETAILED STEPS

Example:
switch(config)#
Step 2 [no] system default interface-vlan autostate Disables the default autostate behavior for the device.
Note Use the system default interface-vlan autostate
Example: command to enable the autostate behavior for the device.
switch(config)# no system default
interface-vlan autostate
Step 4 show running-config [all] (Optional) Displays the running configuration.

To display the default and configured information, use the all
Example: keyword.
This example shows how to disable the default autostate behavior on the Cisco NX-OS device:
switch(config)# no system default interface-vlan autostate

You can configure SVI autostate enable or disable on individual SVIs. The SVI-level setting overrides the
system-level SVI autostate configuration for that particular SVI.
59
SUMMARY STEPS
2. feature interface-vlan
3. interface vlan vlan-id
4. [no] autostate
5. exit
6. show running-config interface vlan vlan-id
7. no shutdown
8. show startup-config interface vlan vlan-id
DETAILED STEPS

Example:
switch(config)#
Step 2 feature interface-vlan Enables VLAN interface mode.
Example:
switch(config)# feature interface-vlan
Step 3 interface vlan vlan-id Creates a VLAN interface and enters interface configuration
mode. The range is from 1 and 4094.
Example:
switch(config-if)# interface vlan10
switch(config)#
Step 4 [no] autostate By default, enables the SVI autostate feature on specified
interface.
Example: To disable the default settings, use the no form of this
switch(config-if)# no autostate
command.
Example:
switch(config)#
Step 6 show running-config interface vlan vlan-id (Optional) Displays the running configuration for the specified
VLAN interface.
Example:
switch(config)# show running-config interface
vlan10
60
Configuring the Device to Tag Native VLAN Traffic

Step 7 no shutdown (Optional) Clears the errors on the interfaces and VLANs
where policies correspond with hardware policies. This
Example: command allows policy programming to continue and the
switch# configure terminal port to come up. If policies do not correspond, the errors are
switch(config)# int e3/1 placed in an error-disabled policy state.
Step 8 show startup-config interface vlan vlan-id (Optional) Displays the VLAN configuration in the startup
configuration.
Example:
switch(config)# show startup-config interface
vlan10
This example shows how to disable the default autostate behavior on an individual SVI:
switch(config)# interface vlan10
witch(config-if)# no autostate
Configuring the Device to Tag Native VLAN Traffic

When you are working with 802.1Q trunked interfaces, you can maintain the tagging for all packets that enter
with a tag that matches the value of the native VLAN ID and drops all untagged traffic (you will still carry
control traffic on that interface). This feature applies to the entire device; you cannot apply it to selected
VLANs on a device.
The vlan dot1q tag native global command changes the behavior of all native VLAN ID interfaces on all
trunks on the device.
Note If you enable 802.1Q tagging on one device and disable it on another device, all traffic is dropped on the
device and this feature is disabled. You must configure this feature identically on each device.
SUMMARY STEPS
2. vlan dot1q tag native
3. exit
4. show vlan
5. no shutdown
61
Changing the System Default Port Mode to Layer 2
DETAILED STEPS

Example:
switch(config)#
Step 2 vlan dot1q tag native Modifies the behavior of a 802.1Q trunked native VLAN ID
interface. The interface maintains the taggings for all packets that
Example: enter with a tag that matches the value of the native VLAN ID and
switch(config)# vlan dot1q tag native drops all untagged traffic. The control traffic is still carried on the
native VLAN. The default is disabled.
Example:
switch(config)#
Step 4 show vlan (Optional) Displays the status and information for VLANs.
Example:
switch# show vlan
configuration.
Example:
startup-config
This example shows how to change the behavior of the native VLAN on an 802.1Q trunked interface to
maintain the tagged packets and drop all untagged traffic (except control traffic):
switch(config)# vlan dot1q tag native
switch#

You can set the system default port mode to Layer 2 access ports.
62
SUMMARY STEPS
2. system default switchport [shutdown]
3. exit
4. show interface brief
5. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 system default switchport [shutdown] Sets the default port mode for all interfaces on the system to Layer 2
access port mode and enters interface configuration mode. By default,
Example: all the interfaces are Layer 3.
switch(config-if)# system default switchport Note When the system default switchport shutdown command
is issued:
• Any FEX HIFs that are not configured with no
shutdown are shutdown. To avoid the shutdown,
configure the FEX HIFs with no shut
• Any Layer 2 port that is not specifically configured with
no shutdown are shutdown. To avoid the shutdown,
configure the Layer 2 port with no shut
Example:
switch(config)#
Step 4 show interface brief (Optional) Displays the status and information for interfaces.
Example:
switch# show interface brief
Step 5 no shutdown (Optional) Clears the errors on the interfaces and VLANs where policies
correspond with hardware policies. This command allows policy
Example: programming to continue and the port to come up. If policies do not
switch# configure terminal correspond, the errors are placed in an error-disabled policy state.
switch(config)# int e3/1
63
Verifying the Interface Configuration

Step 6 copy running-config startup-config (Optional) Copies the running configuration to the startup configuration.
Example:
startup-config
This example shows how to set the system ports to be Layer 2 access ports by default:
switch(config-if)# system default switchport
switch(config-if)#
Verifying the Interface Configuration

To display access and trunk interface configuration information, perform one of the following tasks.
Command Purpose
show interface ethernet slot/port [brief | | counters Displays the interface configuration.
| debounce | description | flowcontrol | mac-address
| status | transceiver]
show interface brief Displays interface configuration information,

including the mode.
show interface switchport Displays information, including access and trunk

interface, information for all Layer 2 interfaces.
show interface trunk [module module-number | vlan Displays trunk configuration information.
vlan-id]
show interface capabilities Displays information about the capabilities of the

interfaces.
show running-config [all] Displays information about the current configuration.

The all command displays the default and current
configurations.
show running-config interface ethernet slot/port Displays configuration information about the specified
interface.
show running-config interface port-channel Displays configuration information about the specified
slot/port port-channel interface.
show running-config interface vlan vlan-id Displays configuration information about the specified
VLAN interface.
64
Monitoring the Layer 2 Interfaces

Use the following commands to display Layer 2 interfaces:
Command Purpose
clear counters interface [interface] Clears the counters.
load- interval {interval seconds {1 | 2 | 3}} Cisco Nexus 9000 Series devices set three different
sampling intervals to bit-rate and packet-rate statistics.
show interface counters [module module] Displays input and output octets unicast packets,
multicast packets, and broadcast packets.
show interface counters detailed [all] Displays input packets, bytes, and multicast as well
as output packets and bytes.
show interface counters errors [module module] Displays information on the number of error packets.
Configuration Examples for Access and Trunk Ports

This example shows how to configure a Layer 2 access interface and assign the access VLAN mode for that
interface:
switch(config-if)# switchport mode access
switch(config-if)# switchport access vlan 5
switch(config-if)#
This example shows how to configure a Layer 2 trunk interface, assign the native VLAN and the allowed
VLANs, and configure the device to tag the native VLAN traffic on the trunk interface:
switch(config-if)# switchport trunk allowed vlan 5, 10
switch(config)# vlan dot1q tag native
switch(config)#
Related Documents
Related Documents Document Title
Configuring Layer 3 interfaces Configuring Layer 2 Interfaces section
65
Related Documents

Port Channels Configuring Port Channels section
VLANs, private VLANs, and STP Cisco Nexus 9000 Series NX-OS Layer 2 Switching
Configuration Guide
System management Cisco Nexus 9000 Series NX-OS System Management

Configuration Guide
High availability Cisco Nexus 9000 Series NX-OS High Availability

and Redundancy Guide
Licensing Cisco NX-OS Licensing Guide
Release Notes Cisco Nexus 9000 Series NX-OS Release Notes
66
CHAPTER 5
• About Layer 3 Interfaces, page 67
• Licensing Requirements for Layer 3 Interfaces, page 70
• Prerequisites for Layer 3 Interfaces, page 70
• Configuring Layer 3 Interfaces, page 71
• Verifying the Layer 3 Interfaces Configuration, page 80
• Monitoring the Layer 3 Interfaces, page 81
• Configuration Examples for Layer 3 Interfaces, page 82
About Layer 3 Interfaces

Layer 3 interfaces forward IPv4 and IPv6 packets to another device using static or dynamic routing protocols.
You can use Layer 3 interfaces for IP routing and inter-VLAN routing of Layer 2 traffic.
Routed Interfaces
You can configure a port as a Layer 2 interface or a Layer 3 interface. A routed interface is a physical port
that can route IP traffic to another device. A routed interface is a Layer 3 interface only and does not support
Layer 2 protocols, such as the Spanning Tree Protocol (STP).
All Ethernet ports are routed interfaces by default. You can change this default behavior with the CLI setup
script.
You can assign an IP address to the port, enable routing, and assign routing protocol characteristics to this
routed interface.
You can also create a Layer 3 port channel from routed interfaces. For more information about port channels,
see the “Configuring Port Channels” section.
67
Subinterfaces
Routed interfaces and subinterfaces support exponentially decayed rate counters. Cisco NX-OS tracks the
following statistics with these averaging counters:
• Input packets/sec
• Output packets/sec
• Input bytes/sec
• Output bytes/sec
Subinterfaces
You can create virtual subinterfaces on a parent interface configured as a Layer 3 interface. A parent interface
can be a physical port.
Subinterfaces divide the parent interface into two or more virtual interfaces on which you can assign unique
Layer 3 parameters such as IP addresses and dynamic routing protocols. The IP address for each subinterface
should be in a different subnet from any other subinterface on the parent interface.
You create a subinterface with a name that consists of the parent interface name (for example, Ethernet 2/1)
followed by a period and then by a number that is unique for that subinterface. For example, you could create
a subinterface for Ethernet interface 2/1 named Ethernet 2/1.1 where .1 indicates the subinterface.
Cisco NX-OS enables subinterfaces when the parent interface is enabled. You can shut down a subinterface
independent of shutting down the parent interface. If you shut down the parent interface, Cisco NX-OS shuts
down all associated subinterfaces as well.
One use of subinterfaces is to provide unique Layer 3 interfaces to each virtual local area network (VLAN)
supported by the parent interface. In this scenario, the parent interface connects to a Layer 2 trunking port on
another device. You configure a subinterface and associate the subinterface to a VLAN ID using 802.1Q
trunking.
The following figure shows a trunking port from a switch that connects to router B on interface E 2/1. This
interface contains three subinterfaces that are associated with each of the three VLANs carried by the trunking
port.
Figure 4: Subinterfaces for VLANs
For more information about VLANs, see the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration
Guide.
68
VLAN Interfaces
VLAN Interfaces
A VLAN interface, or switch virtual interface (SVI), is a virtual routed interface that connects a VLAN on
the device to the Layer 3 router engine on the same device. Only one VLAN interface can be associated with
a VLAN, but you need to configure a VLAN interface for a VLAN only when you want to route between
VLANs or to provide IP host connectivity to the device through a virtual routing and forwarding (VRF)
instance that is not the management VRF. When you enable VLAN interface creation, Cisco NX-OS creates
a VLAN interface for the default VLAN (VLAN 1) to permit remote switch administration.
You must enable the VLAN network interface feature before you can see configure it. The system automatically
takes a checkpoint prior to disabling the feature, and you can roll back to this checkpoint. See the Cisco Nexus
9000 Series NX-OS System Management Configuration Guide for information on rollbacks and checkpoints.
Note You cannot delete the VLAN interface for VLAN 1.
You can route across VLAN interfaces to provide Layer 3 inter-VLAN routing by configuring a VLAN
interface for each VLAN that you want to route traffic to and assigning an IP address on the VLAN interface.
For more information about IP addresses and IP routing, see the Cisco Nexus 9000 Series NX-OS Unicast
Routing Configuration Guide.
The following figure shows two hosts connected to two VLANs on a device. You can configure VLAN
interfaces for each VLAN that allows Host 1 to communicate with Host 2 using IP routing between the VLANs.
VLAN 1 communicates at Layer 3 over VLAN interface 1 and VLAN 10 communicates at Layer 3 over
VLAN interface 10.
Figure 5: Connecting Two VLANs with VLAN interfaces
Loopback Interfaces
A loopback interface is a virtual interface with a single endpoint that is always up. Any packet transmitted
over a loopback interface is immediately received by this interface. Loopback interfaces emulate a physical
interface. You can configure up to 1024 loopback interfaces, numbered 0 to 1023.
69
High Availability
You can use loopback interfaces for performance analysis, testing, and local communications. Loopback
interfaces can act as a termination address for routing protocol sessions. This loopback configuration allows
routing protocol sessions to stay up even if some of the outbound interfaces are down.
High Availability
Layer 3 interfaces support stateful and stateless restarts. After the switchover, Cisco NX-OS applies the runtime
configuration after the switchover.
See the Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide for complete information
about high availability.
Layer 3 interfaces support Virtual Routing and Forwarding instances (VRFs). VRFs exist within virtual device
contexts (VDCs). By default, Cisco NX-OS places you in the default VDC and default VRF .
You can configure up to 1024 loopback interfaces per VDC.
You can associate the interface with a VRF. For VLAN interfaces, you must configure the VLAN interface
in the same VDC as the VLAN.
See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for information about configuring
an interface in a VRF.
Note You must assign an interface to a VRF before you configure the IP address for that interface.
Licensing Requirements for Layer 3 Interfaces


Cisco NX-OS Layer 3 interfaces require no license. Any feature not included in
a license package is bundled with the Cisco NX-OS image and is
provided at no extra charge to you.
Prerequisites for Layer 3 Interfaces

Layer 3 interfaces have the following prerequisites:
• You are familiar with IP addressing and basic configuration. See the Cisco Nexus 9000 Series NX-OS
Unicast Routing Configuration Guide for more information about IP addressing.
70

Layer 3 interfaces have the following configuration guidelines and limitations:
• If you change a Layer 3 interface to a Layer 2 interface, Cisco NX-OS shuts down the interface, reenables
the interface, and removes all configuration specific to Layer 3. (For 6.1(2)I3(4) and earlier)
• If you change a Layer 2 interface to a Layer 3 interface, Cisco NX-OS shuts down the interface, reenables
the interface, and deletes all configuration specific to Layer 2. (For 6.1(2)I3(4) and earlier)
• Configuring a subinterface on a port-channel interface is not supported. (For 6.1(2)I3(4) and earlier)
• The Dynamic Host Configuration Protocol (DHCP) option is not supported when configuring a
subinterface on a port-channel interface.
Default Settings
The following table lists the default settings for Layer 3 interface parameters.
Table 7: Default Layer 3 Interface Parameters
Parameters Default
Admin state Shut
Configuring a Routed Interface

You can configure any Ethernet port as a routed interface.
71
Configuring a Routed Interface
SUMMARY STEPS
3. no switchport
4. [ip address ip-address/length | ipv6 address ipv6-address/length]
5. show interfaces
6. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface ethernet slot/port Enters interface configuration mode.
Example:
switch(config-if)#
Step 3 no switchport Configures the interface as a Layer 3 interface.
Example:
Step 4 [ip address ip-address/length | ipv6 address • Configures an IP address for this interface. See the Cisco
ipv6-address/length] Nexus 9000 Series NX-OS Unicast Routing Configuration
Guide for more information about IP addresses.
Example: • Configures an IPv6 address for this interface. See the
Cisco Nexus 9000 Series NX-OS Unicast Routing
Configuration Guide for more information about IPv6
Example: addresses.
switch(config-if)# ipv6 address 2001:0DB8::1/8
Step 5 show interfaces (Optional) Displays the Layer 3 interface statistics.
Example:
switch(config-if)# show interfaces ethernet
2/1
Step 6 no shutdown (Optional) Clears the errors on the interfaces where policies
correspond with hardware policies. This command allows
Example: policy programming to continue and the port to come up. If
switch# policies do not correspond, the errors are placed in an
switch(config-if)# int e2/1 error-disabled policy state.
72
Configuring a Subinterface on a Routed Interface

Step 7 copy running-config startup-config (Optional) Saves the configuration change.
Example:
startup-config
• Use the medium command to set the interface medium to either point to point or broadcast.
Command Purpose
medium {broadcast | p2p} Configures the interface medium as either point to
point or broadcast.
Example:
switch(config-if)# medium p2p medium p2p
Note The default setting is broadcast, and this setting does not appear in any of the show commands. However,
if you do change the setting to p2p, you will see this setting when you enter the show running config
command.
• Use the switchport command to convert a Layer 3 interface into a Layer 2 interface.
Command Purpose
switchport Configures the interface as a Layer 2 interface and
deletes any configuration specific to Layer 3 on this
Example:
interface.
switch(config-if)# switchportswitchport
• This example shows how to configure a routed interface:

switch(config-if)# copy running-config startup-config
The default setting for interfaces is routed. If you want to configure an interface for Layer 2, enter the
switchport command. Then, if you change a Layer 2 interface to a routed interface, enter the no
switchport command.

You can configure one or more subinterfaces on a routed interface made from routed interfaces.
73
Before You Begin

Configure the parent interface as a routed interface.
See the “Configuring a Routed Interface” section.
SUMMARY STEPS
2. interface ethernet slot/port.number
4. encapsulation dot1Q vlan-id
5. show interfaces
DETAILED STEPS

Example:
switch(config)#
Step 2 interface ethernet slot/port.number Creates a subinterface and enters subinterface configuration
mode.
Example:
switch(config)# interface ethernet 2/1.1
switch(config-subif)#
Step 3 [ip address ip-address/length | ipv6 address • Configures an IP address for this subinterface. See the
ipv6-address/length] Cisco Nexus 9000 Series NX-OS Unicast Routing
Configuration Guide for more information on IP
Example: addresses.
switch(config-subif)# ip address 192.0.2.1/8
• Configures an IPv6 address for this subinterface. See
the Cisco Nexus 9000 Series NX-OS Unicast Routing
Example: Configuration Guide for more information on IPv6
switch(config-subif)# ipv6 address 2001:0DB8::1/8
addresses.
Step 4 encapsulation dot1Q vlan-id Configures IEEE 802.1Q VLAN encapsulation on the
subinterface. The range is from 2 to 4093.
Example:
switch(config-subif)# encapsulation dot1Q 33
Step 5 show interfaces (Optional) Displays the Layer 3 interface statistics.
Example:
switch(config-subif)# show interfaces ethernet
2/1.1
74
Configuring a VLAN Interface

Example:
startup-config
• This example shows how to create a subinterface:

switch(config)# interface ethernet 2/1.1
switch(config-if)# encapsulation dot1Q 33
• The output of the show interface eth command is enhanced for the subinterfaces as shown in the
following :
switch# show interface ethernet 1/2.1
Ethernet1/2.1 is down (Parent Interface Admin down)
admin state is down, Dedicated Interface, [parent interface is Ethernet1/2]
Hardware: 40000 Ethernet, address: 0023.ac67.9bc1 (bia 4055.3926.61d4)
Internet Address is 10.10.10.1/24
MTU 1500 bytes, BW 40000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Auto-mdix is turned off
EtherType is 0x8100
L3 in Switched:
ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
L3 out Switched:
ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes

You can create VLAN interfaces to provide inter-VLAN routing.
SUMMARY STEPS
2. feature interface-vlan
3. interface vlan number
5. show interface vlan number
6. no shutdown
75
DETAILED STEPS

Step 1 configure terminal Enters configuration mode.
Example:
switch(config)#
Step 2 feature interface-vlan Enables VLAN interface mode.
Example:
Step 3 interface vlan number Creates a VLAN interface. The number range is from 1 to 4094.
Example:
switch(config)# interface vlan 10
switch(config-if)#
Step 4 [ip address ip-address/length | ipv6 address • Configures an IP address for this VLAN interface. See
ipv6-address/length] the Cisco Nexus 9000 Series NX-OS Unicast Routing
Configuration Guide for more information on IP
Example: addresses.
• Configures an IPv6 address for this VLAN interface. See
the Cisco Nexus 9000 Series NX-OS Unicast Routing
Example: Configuration Guide for more information on IPv6
addresses.
Step 5 show interface vlan number (Optional) Displays the Layer 3 interface statistics.
Example:
switch(config-if)# show interface vlan 10
Step 6 no shutdown (Optional) Clears the errors on the interfaces where policies
correspond with hardware policies. This command allows policy
Example: programming to continue and the port to come up. If policies
switch(config)# int e3/1 do not correspond, the errors are placed in an error-disabled
switch(config)# no shutdown policy state.
Example:
switch(config-if)# copy running-config
startup-config
This example shows how to create a VLAN interface:

76
Configuring a Loopback Interface

Configuring a Loopback Interface

You can configure a loopback interface to create a virtual interface that is always up.
Before You Begin

Ensure that the IP address of the loopback interface is unique across all routers on the network.
SUMMARY STEPS
2. interface loopback instance
4. show interface loopback instance
DETAILED STEPS

Example:
switch(config)#
Step 2 interface loopback instance Creates a loopback interface. The range is from 0 to 1023.
Example:
switch(config)# interface loopback 0
switch(config-if)#
Step 3 [ip address ip-address/length | ipv6 address • Configures an IP address for this interface. See the
ipv6-address/length] Cisco Nexus 9000 Series NX-OS Unicast Routing
Configuration Guide for more information about IP
Example: addresses.
• Configures an IPv6 address for this interface. See the
Cisco Nexus 9000 Series NX-OS Unicast Routing
Example: Configuration Guide for more information about IPv6
addresses.
Step 4 show interface loopback instance (Optional) Displays the loopback interface statistics.
Example:
switch(config-if)# show interface loopback 0
77
Assigning an Interface to a VRF

Example:
startup-config
This example shows how to create a loopback interface:

Assigning an Interface to a VRF

You can add a Layer 3 interface to a VRF.
SUMMARY STEPS
2. interface interface-type number
3. vrf member vrf-name
4. ip address ip-prefix/length
5. show vrf [vrf-name] interface interface-type number
DETAILED STEPS

Example:
switch(config)#
Step 2 interface interface-type number Enters interface configuration mode.
Example:
switch(config-if)#
Step 3 vrf member vrf-name Adds this interface to a VRF.
Example:
switch(config-if)# vrf member RemoteOfficeVRF
78
Configuring a DHCP Client on an Interface

Step 4 ip address ip-prefix/length Configures an IP address for this interface. You
must do this step after you assign this interface
Example: to a VRF.
Step 5 show vrf [vrf-name] interface interface-type number (Optional) Displays VRF information.
Example:
switch(config-vrf)# show vrf Enterprise interface
loopback 0
Example:
This example shows how to add a Layer 3 interface to the VRF:

Configuring a DHCP Client on an Interface

You can configure the DHCP client on an SVI, a management interface, or a physical Ethernet interface for
IPv4 or IPv6 address
SUMMARY STEPS
1. switch# configure terminal

2. switch(config)# interface ethernet type slot/port | mgmt mgmt-interface-number | vlan vlan id
3. switch(config-if)# [no] ipv6 address use-link-local-only
4. switch(config-if)# [no] [ip | ipv6] address dhcp
5. (Optional) switch(config)# copy running-config startup-config
DETAILED STEPS

Step 1 switch# configure terminal Enters global configuration mode.
Step 2 switch(config)# interface ethernet type slot/port Creates a physical Ethernet interface, a management interface, or
| mgmt mgmt-interface-number | vlan vlan id a VLAN interface.
The range of vlan id is from 1 to 4094.
79
Verifying the Layer 3 Interfaces Configuration

Step 3 switch(config-if)# [no] ipv6 address Prepares for request to the DHCP server.
use-link-local-only Note This command is only required for an IPv6
address.
Step 4 switch(config-if)# [no] [ip | ipv6] address dhcp Requests the DHCP server for an IPv4 or IPv6 address.
The no form of this command removes any address that was
acquired.
Step 5 switch(config)# copy running-config (Optional)

startup-config Saves the change persistently through reboots and restarts by
copying the running configuration to the startup configuration.
This example shows how to configure the IP address of a DHCP client on an SVI:
switch(config-if)# ip address dhcp
This example shows how to configure an IPv6 address of a DHCP client on a management interface:
switch(config)# interface mgmt 0
switch(config-if)# ipv6 address use-link-local-only
switch(config-if)# ipv6 address dhcp
Verifying the Layer 3 Interfaces Configuration

To display the Layer 3 configuration, perform one of the following tasks:
Command Purpose
show interface ethernet slot/port Displays the Layer 3 interface configuration, status,
and counters (including the 5-minute exponentially
decayed moving average of inbound and outbound
packet and byte rates).
show interface ethernet slot/port brief Displays the Layer 3 interface operational status.
show interface ethernet slot/port capabilities Displays the Layer 3 interface capabilities, including
port type, speed, and duplex.
show interface ethernet slot/port description Displays the Layer 3 interface description.
show interface ethernet slot/port status Displays the Layer 3 interface administrative status,
port mode, speed, and duplex.
80
Command Purpose
show interface ethernet slot/port.number Displays the subinterface configuration, status, and
counters (including the f-minute exponentially
decayed moving average of inbound and outbound
packet and byte rates).
show interface port-channel channel-id.number Displays the port-channel subinterface configuration,

status, and counters (including the 5-minute
exponentially decayed moving average of inbound
and outbound packet and byte rates).
show interface loopback number Displays the loopback interface configuration, status,
and counters.
show interface loopback number brief Displays the loopback interface operational status.
show interface loopback number description Displays the loopback interface description.
show interface loopback number status Displays the loopback interface administrative status
and protocol status.
show interface vlan number Displays the VLAN interface configuration, status,
and counters.
show interface vlan number brief Displays the VLAN interface operational status.
show interface vlan number description Displays the VLAN interface description.
show interface vlan number status Displays the VLAN interface administrative status
and protocol status.

Use the following commands to display Layer 3 statistics:
Command Purpose
load- interval {interval seconds {1 | 2 | 3}} Cisco Nexus 9000 Series devices set three different
sampling intervals to bit-rate and packet-rate statistics.
The range for VLAN network interface is 60 to 300
seconds, and the range for Layer interfaces is 30 to
300 seconds.
show interface ethernet slot/port counters Displays the Layer 3 interface statistics (unicast,
multicast, and broadcast).
81
Configuration Examples for Layer 3 Interfaces
Command Purpose
show interface ethernet slot/port counters brief Displays the Layer 3 interface input and output
counters.
show interface ethernet errors slot/port detailed Displays the Layer 3 interface statistics. You can
[all] optionally include all 32-bit and 64-bit packet and
byte counters (including errors).
show interface ethernet errors slot/port counters Displays the Layer 3 interface input and output errors.
errors
show interface ethernet errors slot/port counters Displays the Layer 3 interface counters reported by
snmp SNMP MIBs.
show interface ethernet slot/port.number counters Displays the subinterface statistics (unicast, multicast,
and broadcast).
show interface port-channel channel-id.number Displays the port-channel subinterface statistics

counters (unicast, multicast, and broadcast).
show interface loopback number counters Displays the loopback interface input and output
show interface loopback number detailed [all] Displays the loopback interface statistics. You can
optionally include all 32-bit and 64-bit packet and
byte counters (including errors).
show interface loopback number counters errors Displays the loopback interface input and output
errors.
show interface vlan number counters Displays the VLAN interface input and output
show interface vlan number counters detailed [all] Displays the VLAN interface statistics. You can
optionally include all Layer 3 packet and byte
counters (unicast and multicast).
show interface vlan number counters snmp Displays the VLAN interface counters reported by
SNMP MIBs.
Configuration Examples for Layer 3 Interfaces

This example shows how to configure Ethernet subinterfaces:
interface ethernet 2/1.10
description Layer 3
ip address 192.0.2.1/8
82
Related Documents
This example shows how to configure a loopback interface:

interface loopback 3
ip address 192.0.2.2/32
Related Documents
IP Cisco Nexus 9000 Series NX-OS Unicast Routing
Configuration Guide
VLANs Cisco Nexus 9000 Series NX-OS Layer 2 Switching

Configuration Guide
83
Related Documents
84
CHAPTER 6
Configuring Bidirectional Forwarding Detection
• About BFD, page 85
• Licensing Requirements for BFD, page 88
• Prerequisites for BFD, page 88
• Configuring BFD, page 91
• Configuring BFD Support for Routing Protocols, page 98
• Configuring BFD Interoperability, page 110
• Verifying the BFD Configuration, page 114
• Monitoring BFD, page 114
• Configuration Examples for BFD, page 115
• RFCs, page 116
About BFD
BFD is a detection protocol designed to provide fast forwarding-path failure detection times for media types,
encapsulations, topologies, and routing protocols. You can use BFD to detect forwarding path failures at a
uniform rate, rather than the variable rates for different protocol hello mechanisms. BFD makes network
profiling and planning easier and reconvergence time consistent and predictable.
BFD provides subsecond failure detection between two adjacent devices and can be less CPU-intensive than
protocol hello messages because some of the BFD load can be distributed onto the data plane on supported
modules.
85
Asynchronous Mode
Asynchronous Mode
Cisco NX-OS supports the BFD asynchronous mode, which sends BFD control packets between two adjacent
devices to activate and maintain BFD neighbor sessions between the devices. You configure BFD on both
devices (or BFD neighbors). Once BFD has been enabled on the interfaces and on the appropriate protocols,
Cisco NX-OS creates a BFD session, negotiates BFD session parameters, and begins to send BFD control
packets to each BFD neighbor at the negotiated interval. The BFD session parameters include the following:
• Desired minimum transmit interval—The interval at which this device wants to send BFD hello messages.
• Required minimum receive interval—The minimum interval at which this device can accept BFD hello
messages from another BFD device.
• Detect multiplier—The number of missing BFD hello messages from another BFD device before this
local device detects a fault in the forwarding path.
The following figure shows how a BFD session is established. The figure shows a simple network with two
routers running Open Shortest Path First (OSPF) and BFD. When OSPF discovers a neighbor (1), it sends a
request to the local BFD process to initiate a BFD neighbor session with the OSPF neighbor router (2). The
BFD neighbor session with the OSPF neighbor router is now established (3).
Figure 6: Establishing a BFD Neighbor Relationship
BFD Detection of Failures

Once a BFD session has been established and timer negotiations are complete, BFD neighbors send BFD
control packets that act in the same manner as an IGP hello protocol to detect liveliness, except at a more
accelerated rate. BFD detects a failure, but the protocol must take action to bypass a failed peer.
BFD sends a failure detection notice to the BFD-enabled protocols when it detects a failure in the forwarding
path. The local device can then initiate the protocol recalculation process and reduce the overall network
convergence time.
The following figure shows what happens when a failure occurs in the network (1). The BFD neighbor session
with the OSPF neighbor router is torn down (2). BFD notifies the local OSPF process that the BFD neighbor
is no longer reachable (3). The local OSPF process tears down the OSPF neighbor relationship (4). If an
alternative path is available, the routers immediately start converging on it.
86
Distributed Operation
Note Note The BFD failure detection occurs in less than a second, which is much faster than OSPF Hello
messages could detect the same failure.
Figure 7: Tearing Down an OSPF Neighbor Relationship
Distributed Operation
Cisco NX-OS can distribute the BFD operation to compatible modules that support BFD. This process offloads
the CPU load for BFD packet processing to the individual modules that connect to the BFD neighbors. All
BFD session traffic occurs on the module CPU. The module informs the supervisor when a BFD failure is
detected.
BFD Echo Function

The BFD echo function sends echo packets from the forwarding engine to the remote BFD neighbor. The
BFD neighbor forwards the echo packet back along the same path in order to perform detection; the BFD
neighbor does not participate in the actual forwarding of the echo packets. The echo function and the forwarding
engine are responsible for the detection process. BFD can use the slow timer to slow down the asynchronous
session when the echo function is enabled and reduce the number of BFD control packets that are sent between
two BFD neighbors. Also, the forwarding engine tests the forwarding path on the remote (neighbor) system
without involving the remote system, so there is less interpacket delay variability and faster failure detection
times.
The echo function is without asymmetry when both BFD neighbors are running echo function.
Security
Cisco NX-OS uses the packet Time to Live (TTL) value to verify that the BFD packets came from an adjacent
BFD peer. For all asynchronous and echo request packets, the BFD neighbor sets the TTL value to 255 and
the local BFD process verifies the TTL value as 255 before processing the incoming packet. For the echo
response packet, BFD sets the TTL value to 254.
You can configure SHA-1 authentication of BFD packets.
87
High Availability
High Availability
BFD supports stateless restarts. After a reboot or supervisor switchover, Cisco NX-OS applies the running
configuration and BFD immediately sends control packets to the BFD peers.
BFD supports virtual routing and forwarding instances (VRFs). VRFs exist within virtual device contexts
(VDCs). By default, Cisco NX-OS places you in the default VDC and default VRF.
Licensing Requirements for BFD


Cisco NX-OS BFD requires no license. Any feature not included in
a license package is bundled with the NX-OS image
and is provided at no extra charge to you.
Prerequisites for BFD

BFD has the following prerequisites:
• You must enable the BFD feature.
• Disable Internet Control Message Protocol (ICMP) redirect messages on BFD-enabled interfaces.
• Disable the IP packet verification check for identical IP source and destination addresses.
• See other detailed prerequisites that are listed with the configuration tasks.

BFD has the following configuration guidelines and limitations:
• BFD supports BFD version 1.
• BFD supports IPv4 and IPv6.
• BFD supports OSPFv3.
• BFD supports IS-ISv6.
• BFD supports BGPv6.
88
• BFD supports EIGRPv6.

• BFD supports only one session per address family, per interface.
• BFD supports single-hop BFD.
• BFD for BGP supports single-hop EBGP and iBGP peers.
• BFD supports keyed SHA-1 authentication beginning with Cisco NX-OS Release 5.2.
• BFD supports the following Layer 3 interfaces—physical interfaces, port channels, subinterfaces, and
VLAN interfaces.
• BFD depends on a Layer 3 adjacency information to discover topology changes, including Layer 2
topology changes. A BFD session on a VLAN interface (SVI) may not be up after the convergence of
the Layer 2 topology if there is no Layer 3 adjacency information available.
• For BFD on a static route between two devices, both devices must support BFD. If one or both of the
devices do not support BFD, the static routes are not programmed in the Routing Information Base
(RIB).
• Port channel configuration limitations:
◦For Layer 3 port channels used by BFD, you must enable LACP on the port channel.
◦For Layer 2 port channels used by SVI sessions, you must enable LACP on the port channel.
• SVI limitations:
◦An ASIC reset will cause traffic disruption for other ports. This event could possibly cause SVI
sessions on other ports to flap. Some triggers for an ASIC reset are port moves between VDCs,
reloading a VDC, or if the carrier interface is a virtual port channel (vPC), BFD is not supported
over the SVI interface.
◦When you change the topology (for example, add or delete a link into a VLAN, delete a member
from a Layer 2 port channel, and so on), the SVI session could be affected. It may go down first
and then come up after the topology discovery is finished.
◦When a BFD session is over SVI using virtual port-channel (vPC) peer-link (either BCM or GEM
based ports), the BFD echo function is not supported. You must disable the BFD echo function
for all sessions over SVI between vPC peer nodes using the no bfd echo command at the SVI
configuration level.
Tip If you do not want the SVI sessions to flap and you need to change the topology, you
can disable the BFD feature before making the changes and reenable BFD after the
changes have been made. You can also configure the BFD timer to be a large value (for
example, 5 seconds), and change it back to a fast timer after the above events complete.
• When you configure the BFD Echo function on the distributed Layer 3 port channels, reloading a member
module flaps the BFD session hosted on that module, which results in a packet loss.
If you connect the BFD peers directly without a Layer 2 switch in between, you can use the BFD per-link
mode as an alternative solution.
89
Default Settings
Note Using BFD per-link mode and subinterface optimization simultaneously on a Layer 3
port channel is not supported.
• When you specify a BFD neighbor prefix in the clear {ip | ipv6} route prefix command, the BFD echo
session will flap.
• The clear {ip | ipv6} route * command causes BFD echo sessions to flap.
• HSRP for IPv4 is supported with BFD.
• BFD packets generated by the Cisco NX-OS device linecards are sent with COS 6/DSCP CS6. The
DSCP/COS values for BFD packets are not user configurable.
• When configuring BFDv6 in no-bfd-echo mode, it is recommended to run with timers of 150 ms with
a multiplier of 3.
• BFDv6 is not supported for VRRPv3 and HSRP for v6.
Default Settings
The following table lists the default settings for BFD parameters.
Table 8: Default BFD Parameters
Parameters Default
BFD feature Disabled
Required minimum receive interval 50 milliseconds
Desired minimum transmit interval 50 milliseconds
Detect multiplier 3
Echo function Enabled
Mode Asynchronous
Port-channel Logical mode (one session per source-destination pair

address)
Slow timer 2000 milliseconds
90
Configuring BFD
Configuring BFD
Configuration Hierarchy
You can configure BFD at the global level and at the interface level. The interface configuration overrides
the global configuration.
For physical ports that are members of a port channel, the member port inherits the master port channel BFD
configuration.
Task Flow for Configuring BFD

Follow these steps in the following sections to configure BFD:
• Enabling the BFD Feature.
• Configuring Global BFD Parameters or Configuring BFD on an Interface.
Enabling the BFD Feature

You must enable the BFD feature before you can configure BFD on an interface and protocol.
Note Use the no feature bfd command to disable the BFD feature and remove all associated configuration.
Command Purpose
no feature bfd Disables the BFD feature and

removes all associated
Example:
switch(config)# no feature bfd configuration.
SUMMARY STEPS
2. feature bfd
3. show feature | include bfd
91
Configuring Global BFD Parameters
DETAILED STEPS

Example:
switch(config)#
Step 2 feature bfd Enables the BFD feature.
Example:
switch(config)# feature bfd
Step 3 show feature | include bfd (Optional) Displays enabled and disabled
features.
Example:
switch(config)# show feature | include
bfd
Example:
Configuring Global BFD Parameters

You can configure the BFD session parameters for all BFD sessions on the device. The BFD session parameters
are negotiated between the BFD peers in a three-way handshake.
See the Configuring BFD on an Interface section to override these global session parameters on an interface.
Before You Begin

Enable the BFD feature.
SUMMARY STEPS
2. bfd interval mintx min_rx msec multiplier value
3. bfd slow-timer [interval]
4. bfd echo-interface loopback interface number
5. show running-config bfd
92
Configuring BFD on an Interface
DETAILED STEPS

Example:
switch(config)#
Step 2 bfd interval mintx min_rx msec multiplier Configures the BFD session parameters for all BFD sessions on the
value device. This command overrides these values by configuring the BFD
session parameters on an interface. The mintx and msec range is from
Example: 50 to 999 milliseconds and the default is 50. The multiplier range is from
switch(config)# bfd interval 50 min_rx 1 to 50. The multiplier default is 3.
50 multiplier 3
Step 3 bfd slow-timer [interval] Configures the slow timer used in the echo function. This value
determines how fast BFD starts up a new session and at what speed the
Example: asynchronous sessions use for BFD control packets when the echo
switch(config)# bfd slow-timer 2000 function is enabled. The slow-timer value is used as the new control
packet interval, while the echo packets use the configured BFD intervals.
The echo packets are used for link failure detection, while the control
packets at the slower rate maintain the BFD session. The range is from
1000 to 30000 milliseconds. The default is 2000.
Step 4 bfd echo-interface loopback interface number Configures the interface used for Bidirectional Forwarding Detection
(BFD) echo frames. This command changes the source address for the
Example: echo packets to the one configured on the specified loopback interface.
switch(config)# bfd echo-interface The interface number range is from 0 to 1023.
loopback 1 3
Step 5 show running-config bfd (Optional) Displays the BFD running configuration.
Example:
switch(config)# show running-config bfd
Example:
startup-config

You can configure the BFD session parameters for all BFD sessions on an interface. The BFD session
parameters are negotiated between the BFD peers in a three-way handshake.
This configuration overrides the global session parameters for the configured interface.
93
Before You Begin

Ensure that Internet Control Message Protocol (ICMP) redirect messages are disabled on BFD-enabled
interfaces. Use the no ip redirects command or the no ipv6 redirects command on the interface.
Enable the BFD feature. See the Enabling the BFD Feature section.
SUMMARY STEPS
2. interface int-if
4. bfd authentication keyed-sha1 keyid id key ascii_key
DETAILED STEPS

Example:
switch(config)#
Step 2 interface int-if Enters interface configuration mode. Use the ? keyword to display
the supported interfaces.
Example:
switch(config-if)#
Step 3 bfd interval mintx min_rx msec multiplier value Configures the BFD session parameters for all BFD sessions on
the device. This command overrides these values by configuring
Example: the BFD session parameters on an interface. The mintx and msec
switch(config-if)# bfd interval 50 range is from 50 to 999 milliseconds and the default is 50. The
min_rx 50 multiplier 3 multiplier range is from 1 to 50. The multiplier default is 3.
Step 4 bfd authentication keyed-sha1 keyid id key (Optional) Configures SHA-1 authentication for all BFD sessions
ascii_key on the interface. The ascii_key string is a secret key shared among
BFD peers. The id value, a number between 0 and 255, is assigned
Example: to this particular ascii_key. BFD packets specify the key by id,
switch(config-if)# bfd authentication allowing the use of multiple active keys.
keyed-sha1 keyid 1 ascii_key cisco123
To disable SHA-1 authentication on the interface, use the no form
of the command.
Example:
switch(config-if)# show running-config bfd
94
Configuring BFD on a Port Channel

Example:
startup-config
Configuring BFD on a Port Channel

You can configure the BFD session parameters for all BFD sessions on a port channel. If per-link mode is
used for Layer 3 port channels, BFD creates a session for each link in the port channel and provides an
aggregate result to client protocols. For example, if the BFD session for one link on a port channel is up, BFD
informs client protocols, such as OSPF, that the port channel is up. The BFD session parameters are negotiated
between the BFD peers in a three-way handshake.
This configuration overrides the global session parameters for the configured port channel. The member ports
of the port channel inherit the port channel BFD session parameters.
Before You Begin

Ensure that you enable LACP on the port channel before you enable BFD.
interfaces. Use the no ip redirects command on the interface.
SUMMARY STEPS
2. interface port-channel number
3. bfd per-link
5. bfd authentication keyed-sha1 keyid id key ascii_key
DETAILED STEPS

Example:
switch(config)#
95
Configuring the BFD Echo Function

Step 2 interface port-channel number Enters port-channel configuration mode. Use the ? keyword to
display the supported number range.
Example:
switch(config)# interface port-channel 2
switch(config-if)#
Step 3 bfd per-link Configures the BFD sessions for each link in the port channel.
Example:
switch(config-if)# bfd per-link
Step 4 bfd interval mintx min_rx msec multiplier value (Optional) Configures the BFD session parameters for all BFD
sessions on the port channel. This command overrides these values
Example: by configuring the BFD session parameters. The mintx and msec
Step 5 bfd authentication keyed-sha1 keyid id key (Optional) Configures SHA-1 authentication for all BFD sessions
ascii_key on the interface. The ascii_key string is a secret key shared among
BFD peers. The id value, a number between 0 and 255, is assigned
Example: to this particular ascii_key. BFD packets specify the key by id,
switch(config-if)# bfd authentication allowing the use of multiple active keys.
keyed-sha1 keyid 1 ascii_key cisco123
To disable SHA-1 authentication on the interface, use the no form
of the command.
Example:
Example:
startup-config

You can configure the BFD echo function on one or both ends of a BFD-monitored link. The echo function
slows down the required minimum receive interval, based on the configured slow timer. The
RequiredMinEchoRx BFD session parameter is set to zero if the echo function is disabled. The slow timer
becomes the required minimum receive interval if the echo function is enabled.
Before You Begin

96
Configure the BFD session parameters. See the Configuring Global BFD Parameters section on or the
Configuring BFD on an Interface section.
interfaces. Use the no ip redirects command on the interface.
Ensure that the IP packet verification check for identical IP source and destination addresses is disabled. Use
the no hardware ip verify address identical command. See the Cisco Nexus 9000 Series NX-OS Unicast
Routing Configuration Guide for more information about this command.
SUMMARY STEPS
2. bfd slow-timer echo-interval
3. interface int-if
4. bfd echo
DETAILED STEPS

Example:
switch(config)#
Step 2 bfd slow-timer echo-interval Configures the slow timer used in the echo function. This value
determines how fast BFD starts up a new session and is used to
Example: slow down the asynchronous sessions when the BFD echo
switch(config)# bfd slow-timer 2000 function is enabled. This value overwrites the required minimum
receive interval when the echo function is enabled. The range
is from 1000 to 30000 milliseconds. The default is 2000.
Step 3 interface int-if Enters interface configuration mode. Use the ? keyword to
display the supported interfaces.
Example:
switch(config-if)#
Step 4 bfd echo Enables the echo function. The default is enabled.
Example:
switch(config-if)# bfd echo
Example:
97
Configuring BFD Support for Routing Protocols

Example:
startup-config
Configuring BFD Support for Routing Protocols
Configuring BFD on BGP

You can configure BFD for the Border Gateway Protocol (BGP).
Before You Begin

Configure the BFD session parameters. See the Configuring Global BFD Parameters section or the Configuring
BFD on an Interface section.
Enable the BGP feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.
SUMMARY STEPS
2. router bgp as-number
3. neighbor (ip-address | ipv6-address) remote-as as-number
4. bfd
5. show running-config bgp
DETAILED STEPS

Example:
switch(config)#
98
Configuring BFD on EIGRP

Step 2 router bgp as-number Enables BGP and assigns the AS number to the local BGP
speaker. The AS number can be a 16-bit integer or a 32-bit
Example: integer in the form of a higher 16-bit decimal number and
switch(config)# router bgp 64496 a lower 16-bit decimal number in xx.xx format.
switch(config-router)#
Step 3 neighbor (ip-address | ipv6-address) remote-as as-number Configures the IPv4 or IPv6 address and AS number for
a remote BGP peer. The ip-address format is x.x.x.x. The
Example: ipv6-address format is A:B::C:D.
switch(config-router)# neighbor
209.165.201.1 remote-as 64497
switch(config-router-neighbor)#
Step 4 bfd Enables BFD for this BGP peer.
Example:
switch(config-router-neighbor)# bfd
Step 5 show running-config bgp (Optional) Displays the BGP running configuration.
Example:
switch(config-router-neighbor)# show
running-config bgp
Example:
switch(config-router-neighbor)# copy
running-config startup-config

You can configure BFD for the Enhanced Interior Gateway Routing Protocol (EIGRP).
Before You Begin

Enable the EIGRP feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide
for more information.
99
SUMMARY STEPS
2. router eigrp instance-tag
3. bfd [ipv4 | ipv6]
4. interface int-if
5. ip eigrp instance-tag bfd
6. show ip eigrp [vrf vrf-name] [ interfaces if]
DETAILED STEPS

Example:
switch(config)#
Step 2 router eigrp instance-tag Creates a new EIGRP process with the configured instance tag.
The instance tag can be any case-sensitive, alphanumeric string
Example: up to 20 characters.
switch(config)# router eigrp Test1
switch(config-router)# If you configure an instance-tag that does not qualify as an AS
number, you must use the autonomous-system to configure the
AS number explicitly or this EIGRP instance will remain in the
shutdown state.
Step 3 bfd [ipv4 | ipv6] (Optional) Enables BFD for all EIGRP interfaces.
Example:
switch(config-router-neighbor)# bfd ipv4
Step 4 interface int-if Enters interface configuration mode. Use the ? keyword to display
Example:
switch(config-router-neighbor)# interface
ethernet 2/1
switch(config-if)#
Step 5 ip eigrp instance-tag bfd (Optional) Enables or disables BFD on an EIGRP interface. The
instance tag can be any case-sensitive, alphanumeric string up to
Example: 20 characters.
switch(config-if)# ip eigrp Test1 bfd
The default is disabled.
Step 6 show ip eigrp [vrf vrf-name] [ interfaces if] (Optional) Displays information about EIGRP. The vrf-name can
be any case-sensitive, alphanumeric string up to 32 characters.
Example:
switch(config-if)# show ip eigrp
100
Configuring BFD on OSPF

Example:
switch(config-if)# copy
Configuring BFD on OSPF

You can configure BFD for the Open Shortest Path First.
Before You Begin

Enable the OSPF feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.
SUMMARY STEPS
2. router ospf instance-tag
4. interface int-if
5. ip ospf bfd
6. show ip ospf [vrf vrf-name] [ interfaces if]
DETAILED STEPS

Example:
switch(config)#
Step 2 router ospf instance-tag Creates a new OSPF instance with the configured instance
tag. The instance tag can be any case-sensitive,
Example: alphanumeric string up to 20 characters.
switch(config)# router ospfv3 200
switch(config-router)#
101
Configuring BFD on IS-IS

Step 3 bfd [ipv4 | ipv6] (Optional) Enables BFD for all OSPF interfaces.
Example:
switch(config-router)# bfd
Step 4 interface int-if Enters interface configuration mode. Use the ? keyword to
display the supported interfaces.
Example:
switch(config-router)# interface
ethernet 2/1
switch(config-if)#
Step 5 ip ospf bfd (Optional) Enables or disables BFD on an OSPF interface.

The default is disabled.
Example:
switch(config-if)# ip ospf bfd
Step 6 show ip ospf [vrf vrf-name] [ interfaces if] (Optional) Displays information about OSPF. The vrf-name
can be any case-sensitive, alphanumeric string up to 32
Example: characters.
switch(config-if)# show ip ospf
Example:

You can configure BFD for the Intermediate System-to-Intermediate System (IS-IS) protocol.
Before You Begin

Enable the IS-IS feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.
102
SUMMARY STEPS
2. router isis instance-tag
4. interface int-if
5. isis bfd
6. show isis [vrf vrf-name] [ interface if]
DETAILED STEPS

Example:
switch(config)#
Step 2 router isis instance-tag Creates a new IS-IS instance with the configured
instance tag.
Example:
switch(config)# router isis 100
switch(config-router)# net
49.0001.1720.1600.1001.00
switch(config-router)# address-family ipv6
unicast
Step 3 bfd [ipv4 | ipv6] (Optional) Enables BFD for all OSPF interfaces.
Example:
switch(config-router)# bfd
Step 4 interface int-if Enters interface configuration mode. Use the ? keyword
to display the supported interfaces.
Example:
ethernet 2/1
switch(config-if)#
Step 5 isis bfd (Optional) Enables or disables BFD on an IS-IS

interface. The default is disabled.
Example:
switch(config-if)# isis bfd
Step 6 show isis [vrf vrf-name] [ interface if] (Optional) Displays information about IS-IS. The
vrf-name can be any case-sensitive, alphanumeric string
Example: up to 32 characters.
switch(config-if)# show isis
103
Configuring BFD on HSRP

Example:
Configuring BFD on HSRP

You can configure BFD for the Hot Standby Router Protocol (HSRP). The active and standby HSRP routers
track each other through BFD. If BFD on the standby HSRP router detects that the active HSRP router is
down, the standby HSRP router treats this event as an active time rexpiry and takes over as the active HSRP
router.
The show hsrp detail command shows this event as BFD@Act-down or BFD@Sby-down.
Before You Begin

Enable the HSRP feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.
SUMMARY STEPS
2. hsrp bfd all-interfaces
3. interface int-if
4. hsrp bfd
5. show running-config hsrp
DETAILED STEPS

Example:
switch(config)#
104
Configuring BFD on VRRP

Step 2 hsrp bfd all-interfaces (Optional) Enables or disables BFD on all HSRP
interfaces. The default is disabled.
Example:
switch# hsrp bfd all-interfaces
Step 3 interface int-if Enters interface configuration mode. Use the ? keyword
to display the supported interfaces.
Example:
ethernet 2/1
switch(config-if)#
Step 4 hsrp bfd (Optional) Enables or disables BFD on an HSRP

Example:
switch(config-if)# hsrp bfd
Step 5 show running-config hsrp (Optional) Displays the HSRP running configuration.
Example:
switch(config-if)# show running-config hsrp
Example:
Configuring BFD on VRRP

You can configure BFD for the Virtual Router Redundancy Protocol (VRRP). The active and standby VRRP
routers track each other through BFD. If BFD on the standby VRRP router detects that the active VRRP router
is down, the standby VRRP router treats this event as an active time rexpiry and takes over as the active VRRP
router.
The show vrrp detail command shows this event as BFD@Act-down or BFD@Sby-down.
Before You Begin

Enable the VRRP feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.
105
Configuring BFD on PIM
SUMMARY STEPS
2. interface int-if
3. vrrp group-no
4. vrrp bfd address
5. show running-config vrrp
DETAILED STEPS

Example:
switch(config)#
Step 2 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config)# interface
ethernet 2/1
switch(config-if)#
Step 3 vrrp group-no Specifies the VRRP group number.
Example:
switch(config-if)# vrrp 2
Step 4 vrrp bfd address Enables or disables BFD on a VRRP interface. The
default is disabled.
Example:
switch(config-if)# vrrp bfd
Step 5 show running-config vrrp (Optional) Displays the VRRP running configuration.
Example:
switch(config-if)# show running-config vrrp
Example:

You can configure BFD for the Protocol Independent Multicast (PIM) protocol.
106
Before You Begin

Enable the PIM feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.
SUMMARY STEPS
2. ip pim bfd
3. interface int-if
4. ip pim bfd-instance [disable]
5. show running-config pim
DETAILED STEPS

Example:
switch(config)#
Step 2 ip pim bfd Enables BFD for PIM.
Example:
switch(config)# ip pim bfd
Step 3 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config)# interface
ethernet 2/1
switch(config-if)#
Step 4 ip pim bfd-instance [disable] (Optional) Enables or disables BFD on a PIM

Example:
switch(config-if)# ip pim bfd-instance
Step 5 show running-config pim (Optional) Displays the PIM running configuration.
Example:
switch(config)# show running-config pim
Example:
switch(config)# copy
107
Configuring BFD on Static Routes
Configuring BFD on Static Routes

You can configure BFD for static routes on an interface. You can optionally configure BFD on a static route
within a virtual routing and forwarding (VRF) instance.
Before You Begin

SUMMARY STEPS
2. vrf context vrf-name
3. ip route route interface {nh-address | nh-prefix}
4. ip route static bfd interface {nh-address | nh-prefix}
5. show ip route static [vrf vrf-name]
DETAILED STEPS

Example:
switch(config)#
Step 2 vrf context vrf-name (Optional) Enters VRF configuration mode.
Example:
switch(config)# vrf context Red
switch(config-vrf)#
Step 3 ip route route interface {nh-address | nh-prefix} Creates a static route Use the ? keyword to display
Example:
switch(config-vrf)# ip route 192.0.2.1 ethernet 2/1
192.0.2.4
Step 4 ip route static bfd interface {nh-address | nh-prefix} Enables BFD for all static routes on an interface.
Use the? keyword to display the supported
switch(config-vrf)# ip route static bfd ethernet 2/1
192.0.2.4
108
Disabling BFD on an Interface

Step 5 show ip route static [vrf vrf-name] (Optional) Displays the static routes.
Example:
switch(config-vrf)# show ip route static vrf Red
Example:
switch(config-vrf)# copy
Disabling BFD on an Interface

You can selectively disable BFD on an interface for a routing protocol that has BFD enabled at the global or
VRF level.
To disable BFD on an interface, use one of the following commands in interface configuration mode:
Command Purpose
ip eigrp instance-tag bfd disable Disables BFD on an EIGRP interface. The instance
tag can be any case-sensitive, alphanumeric string up
Example:
switch(config-if)# ip eigrp Test1 bfd to 20 characters.
disable
ip ospf bfd disable Disables BFD on an OSPFv2 interface.

Example:
switch(config-if)# ip ospf bfd disable
isis bfd disable Disables BFD on an IS-IS interface.

Example:
switch(config-if)# isis bfd disable
109
Configuring BFD Interoperability
Configuring BFD Interoperability
Configuring BFD Interoperability in Cisco NX-OS Devices in a Point-to-Point

Link
SUMMARY STEPS
2. interface port-channel int-if
3. ip ospf bfd
4. no ip redirects
6. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel int-if Enters interface configuration mode. Use the ? keyword to display
Example:
Step 3 ip ospf bfd Enables BFD on an OSPFv2 interface. The default is disabled.
OSPF is used as an example. You can enable BFD of any of the
Example: supported protocols.
Step 4 no ip redirects Prevents the device from sending redirects.
Example:
switch(config-if)# no ip redirects
Step 5 bfd interval mintx min_rx msec multiplier value Configures the BFD session parameters for all BFD sessions on
the port channel. This command overrides these values by
Example: configuring the BFD session parameters. The mintx and msec
110
Configuring BFD Interoperability in Cisco NX-OS Devices in a Switch Virtual Interface

Step 6 exit Exits interface configuration mode and returns to EXEC mode.
Example:
Configuring BFD Interoperability in Cisco NX-OS Devices in a Switch Virtual

Interface
SUMMARY STEPS
2. interface port-channel vlan vlan-id
4. no ip redirects
5. ip address ip-address/length
6. ip ospf bfd
7. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel vlan vlan-id Creates a dynamic Switch Virtual Interface (SVI).
Example:
switch(config-if)#
Step 3 bfd interval mintx min_rx msec multiplier value Configures the BFD session parameters for all BFD
sessions on the device. The mintx and msec range is from
Example: 50 to 999 milliseconds and the default is 50. The multiplier
switch(config-if)# bfd interval 50 range is from 1 to 50. The multiplier default is 3.
min_rx 50 multiplier 3
Example:
111
Configuring BFD Interoperability in Cisco NX-OS Devices in Logical Mode

Step 5 ip address ip-address/length Configures an IP address for this interface.
Example:
Step 6 ip ospf bfd Enables BFD on an OSPFv2 interface. The default is

disabled.
Example:
Step 7 exit Exits interface configuration mode and returns to EXEC

mode.
Example:
Configuring BFD Interoperability in Cisco NX-OS Devices in Logical Mode

SUMMARY STEPS
2. interface port-channel type number.subinterface-id
4. no ip redirects
5. ip ospf bfd
6. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel type number.subinterface-id Enters port channel configuration mode. Use the ? keyword to
display the supported number range.
Example:
switch(config-if)# interface port-channel
50.2
112
Verifying BFD Interoperability in a Cisco Nexus 9000 Series Device

Step 3 bfd interval mintx min_rx msec multiplier value Configures the BFD session parameters for all BFD sessions
on the port channel. The mintx and msec range is from 50 to
Example: 999 milliseconds and the default is 50. The multiplier range is
switch(config-if)# bfd interval 50 from 1 to 50. The multiplier default is 3.
min_rx 50 multiplier 3
Example:
Step 5 ip ospf bfd Enables BFD on an OSPFv2 interface. The default is disabled.
OSPF is used as an example. You can enable BFD of any of
Example: the supported protocols.
Step 6 exit Exits interface configuration mode and returns to EXEC mode.
Example:
Verifying BFD Interoperability in a Cisco Nexus 9000 Series Device

The following example shows how to verify BFD interoperability in a Cisco Nexus 9000 Series device.
switch# show bfd neighbors details

OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int
Vrf
10.1.1.1 10.1.1.2 1140850707/2147418093 Up 6393(4) Up Vlan2121
default
Session state is Up and using echo function with 50 ms interval
Local Diag: 0, Demand mode: 0, Poll bit: 0, Authentication: None
MinTxInt: 50000 us, MinRxInt: 2000000 us, Multiplier: 3
Received MinRxInt: 2000000 us, Received Multiplier: 4
Holdown (hits): 8000 ms (0), Hello (hits): 2000 ms (108)
Rx Count: 92, Rx Interval (ms) min/max/avg: 347/1996/1776 last: 1606 ms ago
Tx Count: 108, Tx Interval (ms) min/max/avg: 1515/1515/1515 last: 1233 ms ago
Registered protocols: ospf
Uptime: 0 days 0 hrs 2 mins 44 secs
Last packet: Version: 1 - Diagnostic: 0
State bit: Up - Demand bit: 0
Poll bit: 0 - Final bit: 0
Multiplier: 4 - Length: 24
My Discr.: 2147418093 - Your Discr.: 1140850707
Min tx interval: 2000000 - Min rx interval: 2000000
Min Echo interval: 1000 - Authentication bit: 0
Hosting LC: 10, Down reason: None, Reason not-hosted: None
switch# show bfd neighbors details

OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int
Vrf
10.0.2.1 10.0.2.2 1140850695/131083 Up 270(3) Up Po14.121
default
Session state is Up and not using echo function
113
Verifying the BFD Configuration

Registered protocols: ospf
Verifying the BFD Configuration

To display BFD configuration information, perform one of the following:
Command Purpose
show running-config bfd Displays the running BFD configuration.
show startup-config bfd Displays the BFD configuration that will be applied
on the next system startup.
Monitoring BFD
Use the following commands to display BFD:
Command Purpose
show bfd neighbors [application name] [details] Displays information about BFD for a supported
application, such as BGP or OSPFv2.
show bfd neighbors [interface int-if] [details] Displays information about BGP sessions on an
interface.
show bfd neighbors [dest-ip ip-address] [src-ip Displays information about the specified BGP session
ip-address][details] on an interface.
show bfd neighbors [vrf vrf-name] [details] Displays information about BFD for a VRF.
show bfd [ipv4 | ipv6] [neighbors] Displays information about IPv4 neighbors or IPv6
neighbors.
114
Configuration Examples for BFD
Configuration Examples for BFD

This example shows how to configure BFD for OSPFv2 on Ethernet 2/1, using the default BFD session
parameters:
feature bfd
feature ospf
router ospf Test1
interface ethernet 2/1
ip ospf bfd
no shutdown
This example shows how to configure BFD for all EIGRP interfaces, using the default BFD session parameters:
feature bfd
feature eigrp
bfd interval 100 min_rx 100 multiplier 4
router eigrp Test2
bfd
This example shows how to configure BFDv6:
feature bfd
feature ospfv3
router ospfv3 Test1
interface Ethernet2/7
ipv6 router ospfv3 Test1 area 0.0.0.0
ospfv3 bfd
no shutdown
Show Example for BFD

This example shows results of the show bfd ipv6 neighbors details command.
#show bfd ipv6 neighbors details
OurAddr NeighAddr
LD/RD RH/RS Holdown(mult) State Int
Vrf
cc:10::2 cc:10::1
1090519335/1090519260 Up 5692(3) Up Po1
default
Session state is Up and using echo function with 250 ms interval

Registered protocols: bgp
115
Related Documents
Related Documents
Related Topic Document Title
BFD commands Cisco Nexus 9000 Series NX-OS Unicast Routing
Configuration Guide
RFCs
RFC Title
RFC 5880 Bidirectional Forwarding Detection (BFD)
RFC 5881 BFD for IPv4 and IPv6 (Single Hop)
116
CHAPTER 7
Configuring Port Channels
This chapter describes how to configure port channels and to apply and configure the Link Aggregation
Control Protocol (LACP) for more efficient use of port channels in the Cisco NX-OS devices.
On a single switch, the port-channel compatibility parameters must be the same among all the port-channel
members on the physical switch.
• About Port Channels, page 117

• Port Channels, page 118
• Port-Channel Interfaces, page 119
• Basic Settings, page 119
• Compatibility Requirements, page 120
• Load Balancing Using Port Channels, page 122
• Symmetric Hashing, page 123
• Resilient Hashing, page 123
• LACP, page 124
• Licensing Requirements for Port Channeling, page 129
• Prerequisites for Port Channeling, page 129
• Configuring Port Channels, page 131
About Port Channels

A port channel is an aggregation of multiple physical interfaces that creates a logical interface. You can bundle
up to 32 individual active links into a port channel to provide increased bandwidth and redundancy. Port
channeling also load balances traffic across these physical interfaces. The port channel stays operational as
long as at least one physical interface within the port channel is operational.
117
Port Channels
You can create a Layer 2 port channel by bundling compatible Layer 2 interfaces, or you can create Layer 3
port channels by bundling compatible Layer 3 interfaces. You cannot combine Layer 2 and Layer 3 interfaces
in the same port channel.
You can also change the port channel from Layer 3 to Layer 2. See the Configuring Layer 2 Interfaces chapter
for information about creating Layer 2 interfaces.
Any configuration changes that you apply to the port channel are applied to each member interface of that
port channel. For example, if you configure Spanning Tree Protocol (STP) parameters on the port channel,
the Cisco NX-OS software applies those parameters to each interface in the port channel.
Note After a Layer 2 port becomes part of a port channel, all switchport configurations must be done on the
port channel; you can no longer apply switchport configurations to individual port-channel members. You
cannot apply Layer 3 configurations to an individual port-channel member either; you must apply the
configuration to the entire port channel.
You can use static port channels, with no associated aggregation protocol, for a simplified configuration.
For more flexibility, you can use the Link Aggregation Control Protocol (LACP), which is defined in IEEE
802.3ad. When you use LACP, the link passes protocol packets. You cannot configure LACP on shared
interfaces.
See the LACP Overview section for information about LACP.
Port Channels
A port channel bundles physical links into a channel group to create a single logical link that provides the
aggregate bandwidth of up to 32 physical links. If a member port within a port channel fails, the traffic
previously carried over the failed link switches to the remaining member ports within the port channel.
However, you can enable the LACP to use port channels more flexibly. Configuring port channels with LACP
and static port channels require a slightly different procedure (see the “Configuring Port Channels” section).
Note The device does not support Port Aggregation Protocol (PAgP) for port channels.
Each port can be in only one port channel. All the ports in a port channel must be compatible; they must use
the same speed and duplex mode (see the “Compatibility Requirements” section). When you run static port
channels with no aggregation protocol, the physical links are all in the on channel mode; you cannot change
this mode without enabling LACP (see the “Port-Channel Modes” section).
You can create port channels directly by creating the port-channel interface, or you can create a channel group
that acts to aggregate individual ports into a bundle. When you associate an interface with a channel group,
the software creates a matching port channel automatically if the port channel does not already exist. In this
instance, the port channel assumes the Layer 2 or Layer 3 configuration of the first interface. You can also
create the port channel first. In this instance, the Cisco NX-OS software creates an empty channel group with
the same channel number as the port channel and takes the default Layer 2 or Layer 3 configuration, as well
as the compatibility configuration (see the “Compatibility Requirements” section).
118
Note The port channel is operationally up when at least one of the member ports is up and that port’s status is
channeling. The port channel is operationally down when all member ports are operationally down.
The following shows port-channel interfaces.
Figure 8: Port-Channel Interfaces
You can classify port-channel interfaces as Layer 2 or Layer 3 interfaces. In addition, you can configure Layer
2 port channels in either access or trunk mode. Layer 3 port-channel interfaces have routed ports as channel
members.
You can configure a Layer 3 port channel with a static MAC address. If you do not configure this value, the
Layer 3 port channel uses the router MAC of the first channel member to come up. See the Cisco Nexus 9000
Series NX-OS Layer 2 Switching Configuration Guide for information about configuring static MAC addresses
on Layer 3 port channels.
See the "Configuring Layer 2 Interfaces" chapter for information about configuring Layer 2 ports in access
or trunk mode and the "Configuring Layer 3 Interfaces" chapter for information about configuring Layer 3
interfaces and subinterfaces.
Basic Settings
You can configure the following basic settings for the port-channel interface:
119
Compatibility Requirements
• Bandwidth—Use this setting for informational purposes only; this setting is to be used by higher-level
protocols.
• Delay—Use this setting for informational purposes only; this setting is to be used by higher-level
protocols.
• Description
• Duplex
• IP addresses
• Maximum Transmission Unit (MTU)
• Shutdown
• Speed
When you add an interface to a channel group, the software checks certain interface attributes to ensure that
the interface is compatible with the channel group. For example, you cannot add a Layer 3 interface to a Layer
2 channel group. The Cisco NX-OS software also checks a number of operational attributes for an interface
before allowing that interface to participate in the port-channel aggregation.
The compatibility check includes the following operational attributes:
• Network layer
• (Link) speed capability
• Speed configuration
• Duplex capability
• Duplex configuration
• Port mode
• Access VLAN
• Trunk native VLAN
• Tagged or untagged
• Allowed VLAN list
• MTU size
• SPAN—Cannot be a SPAN source or a destination port
• Storm control
• Flow-control capability
• Flow-control configuration
• Media type, either copper or fiber
Use the show port-channel compatibility-parameters command to see the full list of compatibility checks
that the Cisco NX-OS uses.
120
You can only add interfaces configured with the channel mode set to on to static port channels, and you can
only add interfaces configured with the channel mode as active or passive to port channels that are running
LACP. You can configure these attributes on an individual member port. If you configure a member port with
an incompatible attribute, the software suspends that port in the port channel.
Alternatively, you can force ports with incompatible parameters to join the port channel if the following
parameters are the same:
• (Link) speed capability
• Speed configuration
• Duplex capability
• Duplex configuration
• Flow-control capability
• Flow-control configuration
When the interface joins a port channel, some of its individual parameters are removed and replaced with the
values on the port channel as follows:
• Bandwidth
• Delay
• Extended Authentication Protocol over UDP
• VRF
• IP address
• MAC address
• Spanning Tree Protocol
• NAC
• Service policy
• Access control lists (ACLs)
Many interface parameters remain unaffected when the interface joins or leaves a port channel as follows:
• Beacon
• Description
• CDP
• LACP port priority
• Debounce
• UDLD
• MDIX
• Rate mode
• Shutdown
• SNMP trap
121
Load Balancing Using Port Channels
Note When you delete the port channel, the software sets all member interfaces as if they were removed from
the port channel.
See the “LACP Marker Responders” section for information about port-channel modes.
Load Balancing Using Port Channels

The Cisco NX-OS software load balances traffic across all operational interfaces in a port channel by hashing
the addresses in the frame to a numerical value that selects one of the links in the channel. Port channels
provide load balancing by default. Port-channel load balancing uses MAC addresses, IP addresses, or Layer
4 port numbers to select the link. Port-channel load balancing uses either source or destination addresses or
ports, or both source and destination addresses or ports.
You can configure the load- balancing mode to apply to all port channels that are configured on the entire
device. You can configure one load-balancing mode for the entire device. You cannot configure the
load-balancing method per port channel.
You can configure the type of load-balancing algorithm used. You can choose the load-balancing algorithm
that determines which member port to select for egress traffic by looking at the fields in the frame.
The default load-balancing mode for Layer 3 interfaces is the source and destination IP address, and the default
load-balancing mode for non-IP traffic is the source and destination MAC address. Use the port-channel
load-balance command to set the load-balancing method among the interfaces in the channel-group bundle.
The default method for Layer 2 packets is src-dst-mac. The default method for Layer 3 packets is src-dst-ip.
You can configure the device to use one of the following methods to load balance across the port channel:
• Destination MAC address
• Source MAC address
• Source and destination MAC address
• Destination IP address
• Source IP address
• Source and destination IP address
• Source TCP/UDP port number
• Destination TCP/UDP port number
• Source and destination TCP/UDP port number
Non-IP and Layer 3 port channels both follow the configured load-balancing method, using the source,
destination, or source and destination parameters. For example, when you configure load balancing to use the
source IP address, all non-IP traffic uses the source MAC address to load balance the traffic while the Layer
3 traffic load balances the traffic using the source IP address. Similarly, when you configure the destination
MAC address as the load-balancing method, all Layer 3 traffic uses the destination IP address while the non-IP
traffic load balances using the destination MAC address.
The load-balancing algorithms that use port channels do not apply to multicast traffic. Regardless of the
load-balancing algorithm you have configured, multicast traffic uses the following methods for load balancing
with port channels:
122
Symmetric Hashing
• Multicast traffic with Layer 4 information—Source IP address, source port, destination IP address,
destination port
• Multicast traffic without Layer 4 information—Source IP address, destination IP address
• Non-IP multicast traffic—Source MAC address, destination MAC address
Note Devices that run Cisco IOS can optimize the behavior of the member ports ASICs if a failure of a single
member occurred by running the port-channel hash-distribution command. The Cisco Nexus 9000 Series
device performs this optimization by default and does not require or support this command. Cisco NX-OS
does support the customization of the load-balancing criteria on port channels through the port-channel
load-balance command for the entire device.
Symmetric Hashing
To be able to effectively monitor traffic on a port channel, it is essential that each interface connected to a
port channel receives both forward and reverse traffic flows. Normally, there is no guarantee that the forward
and reverse traffic flows will use the same physical interface. However, when you enable symmetric hashing
on the port channel, bidirectional traffic is forced to use the same physical interface and each physical interface
in the port channel is effectively mapped to a set of flows.
When symmetric hashing is enabled, the parameters used for hashing, such as the source and destination IP
address, are normalized before they are entered into the hashing algorithm. This process ensures that when
the parameters are reversed (the source on the forward traffic becomes the destination on the reverse traffic),
the hash output is the same. Therefore, the same interface is chosen.
Only the following load-balancing algorithms support symmetric hashing:
• src-dst ip
• src-dst ip-l4port
Resilient Hashing
With the exponential increase in the number of physical links used in data centers, there is also the potential
for an increase in the number of failed physical links. In static hashing systems that are used for load balancing
flows across members of port channels or Equal Cost Multipath (ECMP) groups, each flow is hashed to a
link. If a link fails, all flows are rehashed across the remaining working links. This rehashing of flows to links
results in some packets being delivered out of order even for those flows that were not hashed to the failed
link.
This rehashing also occurs when a link is added to the port channel or Equal Cost Multipath (ECMP) group.
All flows are rehashed across the new number of links, which results in some packets being delivered out of
order. Resilient hashing supports only unicast traffic.
Resilient hashing maps flows to physical ports. In case a link fails, the flows assigned to the failed link are
redistributed uniformly among the working links. The existing flows through the working links are not rehashed
and their packets are not delivered out of order.
123
LACP
Resilient hashing is supported only by ECMP groups and on port channel interfaces. When a link is added to
the port channel or ECMP group, some of the flows hashed to the existing links are rehashed to the new link,
but not across all existing links.
Resilient hashing supports IPv4 and IPv6 known unicast traffic, but it does not support IPv4 multicast traffic.
Note Resilient hashing is supported on Network Forwarding Engine (NFE) based Cisco Nexus 9300 Series
switches and Cisco Nexus 9500 Series switches. (NX-OS 7.0(3)I3(1) release and later).
LACP
LACP allows you to configure up to 16 interfaces into a port channel.
LACP Overview
Note You must enable LACP before you can use LACP. By default, LACP is disabled.
See the “Enabling LACP” section for information about enabling LACP.
The system automatically takes a checkpoint before disabling the feature, and you can roll back to this
checkpoint. See the Cisco Nexus 9000 Series NX-OS System Management Configuration Guide for information
about rollbacks and checkpoints.
The following figure shows how individual links can be combined into LACP port channels and channel
groups as well as function as individual links.
Figure 9: Individual Links Combined into a Port Channel
With LACP, you can bundle up to 16 interfaces in a channel group.
124
Port-Channel Modes
Note When you delete the port channel, the software automatically deletes the associated channel group. All
member interfaces revert to their original configuration.
You cannot disable LACP while any LACP configurations are present.
Port-Channel Modes
Individual interfaces in port channels are configured with channel modes. When you run static port channels
with no aggregation protocol, the channel mode is always set to on.
After you enable LACP globally on the device, you enable LACP for each channel by setting the channel
mode for each interface to active or passive. You can configure either channel mode for individual links in
the LACP channel group when you are adding the links to the channel group.
Note You must enable LACP globally before you can configure an interface in either the active or passive
channel mode.
The following table describes the channel modes.
Table 9: Channel Modes for Individual Links in a Port Channel
Channel Mode Description

passive LACP mode that places a port into a passive
negotiating state in which the port responds to LACP
packets that it receives but does not initiate LACP
negotiation.
active LACP mode that places a port into an active

negotiating state in which the port initiates
negotiations with other ports by sending LACP
packets.
on All static port channels (that are not running LACP)

remain in this mode. If you attempt to change the
channel mode to active or passive before enabling
LACP, the device displays an error message.
You enable LACP on each channel by configuring
the interface in that channel for the channel mode as
either active or passive. When an LACP attempts to
negotiate with an interface in the on state, it does not
receive any LACP packets and becomes an individual
link with that interface; it does not join the LACP
channel group.
The default port-channel mode is on.
125
LACP ID Parameters
Both the passive and active modes allow LACP to negotiate between ports to determine if they can form a
port channel based on criteria such as the port speed and the trunking state.The passive mode is useful when
you do not know whether the remote system, or partner, supports LACP.
Ports can form an LACP port channel when they are in different LACP modes if the modes are compatible
as in the following examples:
• A port in active mode can form a port channel successfully with another port that is in active mode.
• A port in active mode can form a port channel with another port in passive mode.
• A port in passive mode cannot form a port channel with another port that is also in passive mode, because
neither port will initiate negotiation.
• A port in on mode is not running LACP and cannot form a port channel with another port that is in
active or passive mode.
LACP ID Parameters
This section describes the LACP parameters.
LACP System Priority

Each system that runs LACP has an LACP system priority value. You can accept the default value of 32768
for this parameter, or you can configure a value between 1 and 65535. LACP uses the system priority with
the MAC address to form the system ID and also uses the system priority during negotiation with other devices.
A higher system priority value means a lower priority.
Note The LACP system ID is the combination of the LACP system priority value and the MAC address.
LACP Port Priority

Each port that is configured to use LACP has an LACP port priority. You can accept the default value of
32768 for the LACP port priority, or you can configure a value between 1 and 65535. LACP uses the port
priority with the port number to form the port identifier.
LACP uses the port priority to decide which ports should be put in standby mode when there is a limitation
that prevents all compatible ports from aggregating and which ports should be put into active mode. A higher
port priority value means a lower priority for LACP. You can configure the port priority so that specified
ports have a lower priority for LACP and are most likely to be chosen as active links, rather than hot-standby
links.
LACP Administrative Key

LACP automatically configures an administrative key value equal to the channel-group number on each port
configured to use LACP. The administrative key defines the ability of a port to aggregate with other ports. A
port’s ability to aggregate with other ports is determined by these factors:
• Port physical characteristics, such as the data rate and the duplex capability
126
LACP Marker Responders
• Configuration restrictions that you establish
LACP Marker Responders

You can dynamically redistribute the data traffic by using port channels. This redistribution might result from
a removed or added link or a change in the load-balancing scheme. Traffic redistribution that occurs in the
middle of a traffic flow can cause misordered frames.
LACP uses the Marker Protocol to ensure that frames are not duplicated or reordered due to this redistribution.
The Marker Protocol detects when all the frames of a given traffic flow are successfully received at the remote
end. LACP sends Marker PDUs on each of the port-channel links. The remote system responds to the Marker
PDU once it receives all the frames received on this link prior to the Marker PDU. The remote system then
sends a Marker Responder. Once the Marker Responders are received by the local system on all member links
of the port channel, the local system can redistribute the frames in the traffic flow with no chance of misordering.
The software supports only Marker Responders.
LACP-Enabled and Static Port Channels Differences

The following table summarizes the major differences between port channels with LACP enabled and static
port channels.
Table 10: Port Channels with LACP Enabled and Static Port Channels
Configurations Port Channels with LACP Enabled Static Port Channels

Protocol applied Enable globally Not applicable
Channel mode of links Can be either: Can only be On

• Active
• Passive
Maximum number of links in 32 32

channel
LACP Compatibility Enhancements

When a Cisco Nexus 9000 Series device is connected to a non-Nexus peer, its graceful failover defaults may
delay the time taken for a disabled port to be brought down or cause traffic from the peer to be lost. To address
these conditions, the lacp graceful-convergence command was added.
By default, LACP sets a port to the suspended state if it does not receive an LACP PDU from the peer. In
some cases, although this feature helps in preventing loops created due to misconfigurations, it can cause
servers to fail to boot up because they require LACP to logically bring up the port. You can put a port into an
individual state by using the lacp suspend-individual.
127
LACP Port-Channel Minimum Links and MaxBundle
LACP Port-Channel Minimum Links and MaxBundle

A port channel aggregates similar ports to provide increased bandwidth in a single manageable interface.
The introduction of the minimum links and maxbundle feature further refines LACP port-channel operation
and provides increased bandwidth in one manageable interface.
The LACP port-channel minimum links feature does the following:
• Configures the minimum number of ports that must be linked up and bundled in the LACP port channel.
• Prevents the low-bandwidth LACP port channel from becoming active.
• Causes the LACP port channel to become inactive if there are few active members ports to supply the
required minimum bandwidth.
The LACP MaxBundle defines the maximum number of bundled ports allowed in a LACP port channel.
The LACP MaxBundle feature does the following:
• Defines an upper limit on the number of bundled ports in an LACP port channel.
• Allows hot-standby ports with fewer bundled ports. (For example, in an LACP port channel with five
ports, you can designate two of those ports as hot-standby ports.)
Note The minimum links and maxbundle feature works only with LACP port channels. However, the device
allows you to configure this feature in non-LACP port channels, but the feature is not operational.
LACP Fast Timers

You can change the LACP timer rate to modify the duration of the LACP timeout. Use the lacp rate command
to set the rate at which LACP control packets are sent to an LACP-supported interface. You can change the
timeout rate from the default rate (30 seconds) to the fast rate (1 second). This command is supported only
on LACP-enabled interfaces. To configure the LACP fast time rate, see the “Configuring the LACP Fast Timer
Rate” section.
ISSU and stateful switchover cannot be guaranteed with LACP fast timers.
You must configure the member ports and other port channel-related configuration from the virtual device
context (VDC) that contains the port channel and member ports. You can use the numbers from 1 to 4096 in
each VDC to number the port channels.
All ports in one port channel must be in the same VDC. When you are using LACP, all possible 8 active ports
and all possible 8 standby ports must be in the same VDC.
128
High Availability
Note You must configure load balancing using port channels in the default VDC. See the “Load Balancing
Using Port Channels” section for more information about load balancing.
High Availability
Port channels provide high availability by load balancing traffic across multiple ports. If a physical port fails,
the port channel is still operational if there is an active member in the port channel. You can bundle ports
from different modules and create a port channel that remains operational even if a module fails because the
settings are common across the module.
Port channels support stateful and stateless restarts. A stateful restart occurs on a supervisor switchover. After
the switchover, the Cisco NX-OS software applies the runtime configuration after the switchover.
The port channel goes down if the operational ports fall below the configured minimum links number.
about high-availability features.
Licensing Requirements for Port Channeling


Cisco NX-OS Port channeling requires no license. Any feature not included in
a license package is bundled with the Cisco NX-OS image and is
provided at no extra charge to you.
Prerequisites for Port Channeling

Port channeling has the following prerequisites:
• You must be logged onto the device.
• All ports for a single port channel must be either Layer 2 or Layer 3 ports.
• All ports for a single port channel must meet the compatibility requirements. See the “Compatibility
Requirements” section for more information about the compatibility requirements.
• You must configure load balancing from the default VDC.
129

Port channeling has the following configuration guidelines and limitations:
• The LACP port-channel minimum links and maxbundle feature is not supported for host interface port
channels.
• You must enable LACP before you can use that feature.
• You can configure multiple port channels on a device.
• Do not put shared and dedicated ports into the same port channel. (See the “Configuring Basic Interface
Parameters” chapter for information about shared and dedicated ports.)
• For Layer 2 port channels, ports with different STP port path costs can form a port channel if they are
compatibly configured with each other. See the “Compatibility Requirements” section for more information
about the compatibility requirements.
• In STP, the port-channel cost is based on the aggregated bandwidth of the port members.
• After you configure a port channel, the configuration that you apply to the port channel interface affects
the port channel member ports. The configuration that you apply to the member ports affects only the
member port where you apply the configuration.
• LACP does not support half-duplex mode. Half-duplex ports in LACP port channels are put in the
suspended state.
• You must remove the port-security information from a port before you can add that port to a port channel.
Similarly, you cannot apply the port-security configuration to a port that is a member of a channel group.
• Do not configure ports that belong to a port channel group as private VLAN ports. While a port is part
of the private VLAN configuration, the port channel configuration becomes inactive.
• Channel member ports cannot be a source or destination SPAN port.
• The port channel might be affected by the limitations of the Application Leaf Engine (ALE) uplink ports
on Cisco Nexus 9300 and 9500 Series devices:Limitations for ALE Uplink Ports
Default Settings
The following table lists the default settings for port-channel parameters.
Table 11: Default Port-Channel Parameters
Parameters Default
Port channel Admin up
Load balancing method for Layer 3 interfaces Source and destination IP address
Load balancing method for Layer 2 interfaces Source and destination MAC address
130
Parameters Default
Load balancing per module Disabled
LACP Disabled
Channel mode on
LACP system priority 32768
LACP port priority 32768
Minimum links for LACP 1
Maxbundle 32
Minimum links for FEX fabric port channel 1
Note See the "Configuring Basic Interface Parameters” chapter for information about configuring the maximum
transmission unit (MTU) for the port-channel interface. See the “Configuring Layer 3 Interfaces” chapter
for information about configuring IPv4 and IPv6 addresses on the port-channel interface.
Creating a Port Channel

You can create a port channel before you create a channel group. The software automatically creates the
associated channel group.
Note When the port channel is created before the channel group, the port channel should be configured with
all of the interface attributes that the member interfaces are configured with. Use the switchport mode
trunk {allowed vlan vlan-id | native vlan-id} command to configure the members.
This is required only when the channel group members are Layer 2 ports (switchport) and trunks (switchport
mode trunk).
131
Creating a Port Channel
Note Use the no interface port-channel command to remove the port channel and delete the associated channel
group.
Command Purpose
no interface port-channel channel-number Removes the port channel and deletes the
associated channel group.
Example:
switch(config)# no interface port-channel 1
Before You Begin

Enable LACP if you want LACP-based port channels.
SUMMARY STEPS
2. interface port-channel channel-number
3. show port-channel summary
4. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel channel-number Specifies the port-channel interface to configure, and enters the
interface configuration mode. The range is from 1 to 4096. The
Example: Cisco NX-OS software automatically creates the channel group
switch(config)# interface port-channel 1 if it does not already exist.
switch(config-if)
Step 3 show port-channel summary (Optional) Displays information about the port channel.
Example:
switch(config-router)# show port-channel
summary
switch# configure terminal policies do not correspond, the errors are placed in an
132
Adding a Layer 2 Port to a Port Channel

configuration.
Example:
startup-config
This example shows how to create a port channel:

switch (config)# interface port-channel 1
See the “Compatibility Requirements” section for details on how the interface configuration changes when
you delete the port channel.

You can add a Layer 2 port to a new channel group or to a channel group that already contains Layer 2 ports.
The software creates the port channel associated with this channel group if the port channel does not already
exist.
Note Use the no channel-group command to remove the port from the channel group.
Command Purpose
no channel-group Removes the port from the channel group.
Example:
switch(config)# no channel-group
Before You Begin

All Layer 2 member ports must run in full-duplex mode and at the same speed
133
SUMMARY STEPS
3. switchport
4. switchport mode trunk
5. switchport trunk {allowed vlan vlan-id | native vlan-id}
6. channel-group channel-number [force] [mode {on | active | passive}]
7. show interface type slot/port
8. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface type slot/port Specifies the interface that you want to add to a channel group, and
enters the interface configuration mode.
Example:
switch(config-if)#
Step 3 switchport Configures the interface as a Layer 2 access port.
Example:
switch(config)# switchport
Step 4 switchport mode trunk (Optional) Configures the interface as a Layer 2 trunk port.
Example:
switch(config)# switchport mode trunk
Step 5 switchport trunk {allowed vlan vlan-id | native (Optional) Configures necessary parameters for a Layer 2 trunk
vlan-id} port.
Example:
switch(config)# switchport trunk native 3
switch(config-if)#
Step 6 channel-group channel-number [force] [mode Configures the port in a channel group and sets the mode. The
{on | active | passive}] channel-number range is from 1 to 4096. This command creates
the port channel associated with this channel group if the port
Example: channel does not already exist. All static port-channel interfaces
are set to mode on. You must set all LACP-enabled port-channel
interfaces to active or passive. The default mode is on.
134

• switch(config-if)# channel-group 5 (Optional) Forces an interface with some incompatible
switch(config-if)# channel-group 5 configurations to join the channel. The forced interface must have
•
force the same speed, duplex, and flow control settings as the channel
group.
Note The force option fails if the port has a QoS policy
mismatch with the other members of the port channel.
Step 7 show interface type slot/port (Optional) Displays interface information.
Example:
switch# show interface port channel 5
configuration.
Example:
startup-config
This example shows how to add a Layer 2 Ethernet interface 1/4 to channel group 5:
switch (config)# interface ethernet 1/4
switch(config-if)# channel-group 5

You can add a Layer 3 port to a new channel group or to a channel group that is already configured with Layer
3 ports. The software creates the port channel associated with this channel group if the port channel does not
already exist.
If the Layer 3 port that you are adding has a configured IP address, the system removes that IP address before
adding the port to the port channel. After you create a Layer 3 port channel, you can assign an IP address to
the port-channel interface.
135
Note Use the no channel-group command to remove the port from the channel group. The port reverts to its
original configuration. You must reconfigure the IP addresses for this port.
Command Purpose
no channel-group Removes the port from the channel group.
Example:
switch(config)# no channel-group
Before You Begin

Remove any IP addresses configured on the Layer 3 interface.
SUMMARY STEPS
3. no switchport
4. channel-group channel-number [force] [mode {on | active | passive}]
5. show interface type slot/port
6. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface type slot/port Specifies the interface that you want to add to a channel group,
and enters the interface configuration mode.
Example:
switch(config-if)#
Step 3 no switchport Configures the interface as a Layer 3 port.
Example:
Step 4 channel-group channel-number [force] [mode {on Configures the port in a channel group and sets the mode. The
| active | passive}] channel-number range is from 1 to 4096. The Cisco NX-OS
136
Configuring the Bandwidth and Delay for Informational Purposes

software creates the port channel associated with this channel
Example: group if the port channel does not already exist.
• switch(config-if)# channel-group 5 (Optional) Forces an interface with some incompatible

configurations to join the channel. The forced interface must have
• switch(config-if)# channel-group 5
force the same speed, duplex, and flow control settings as the channel
group.
Step 5 show interface type slot/port (Optional) Displays interface information.
Example:
configuration.
Example:
startup-config
This example shows how to add a Layer 3 Ethernet interface 1/5 to channel group 6 in on mode:
switch(config-if)# channel-group 6
This example shows how to create a Layer 3 port-channel interface and assign the IP address:

The bandwidth of the port channel is determined by the number of total active links in the channel.
You configure the bandwidth and delay on port-channel interfaces for informational purposes.
137
SUMMARY STEPS
3. bandwidth value
4. delay value
5. exit
6. show interface port-channel channel-number
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel channel-number Specifies the port-channel interface that you want to configure,
and enters the interface mode.
Example:
switch(config-if)#
Step 3 bandwidth value Specifies the bandwidth, which is used for informational
purposes. The range is from 1 to 3,200,000,000 kbs. The
Example: default value depends on the total active interfaces in the
switch(config-if)# bandwidth 60000000 channel group.
switch(config-if)#
Step 4 delay value Specifies the throughput delay, which is used for informational
purposes. The range is from 1 to 16,777,215 tens of
Example: microseconds. The default value is 10 microseconds.
switch(config-if)#
Step 5 exit Exits the interface mode and returns to the configuration mode.
Example:
switch(config)#
Step 6 show interface port-channel channel-number (Optional) Displays interface information for the specified
port channel.
Example:
switch# show interface port-channel 2
configuration.
Example:
startup-config
138
Shutting Down and Restarting the Port-Channel Interface
This example shows how to configure the informational parameters of the bandwidth and delay for port
channel 5:
switch(config-if)#
Shutting Down and Restarting the Port-Channel Interface

You can shut down and restart the port-channel interface. When you shut down a port-channel interface, no
traffic passes and the interface is administratively down.
SUMMARY STEPS
3. shutdown
4. exit
6. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel channel-number Specifies the port-channel interface that you want to configure,
and enters the interface mode.
Example:
switch(config-if)#
Step 3 shutdown Shuts down the interface. No traffic passes and the interface
displays as administratively down. The default is no shutdown.
Example: Note Use the no shutdown command to open the interface.
switch(config-if)# The interface displays as administratively up. If there are
no operational problems, traffic passes. The default is no
shutdown.
139
Configuring a Port-Channel Description

Step 4 exit Exits the interface mode and returns to the configuration mode.
Example:
switch(config)#
Step 5 show interface port-channel channel-number (Optional) Displays interface information for the specified port
channel.
Example:
switch(config-router)# show interface
port-channel 2
switch# configure terminal policies do not correspond, the errors are placed in an
configuration.
Example:
startup-config
This example shows how to bring up the interface for port channel 2:
Configuring a Port-Channel Description

You can configure a description for a port channel.
SUMMARY STEPS
3. description
4. exit
140
Configuring the Speed and Duplex Settings for a Port-Channel Interface
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel channel-number Specifies the port-channel interface that you want to
configure, and enters the interface mode.
Example:
switch(config-if)#
Step 3 description Allows you to add a description to the port-channel interface.

You can use up to 80 characters in the description. By default,
Example: the description does not display; you must configure this
switch(config-if)# description engineering parameter before the description displays in the output.
switch(config-if)#
Step 4 exit Exits the interface mode and returns to the configuration
mode.
Example:
switch(config)#
Step 5 show interface port-channel channel-number (Optional) Displays interface information for the specified
port channel.
Example:
configuration.
Example:
startup-config
This example shows how to add a description to port channel 2:

switch(config-if)# description engineering

You can configure the speed and duplex settings for a port-channel interface.
141
SUMMARY STEPS
3. speed {10 | 100 | 1000 | auto}
4. duplex {auto | full | half}
5. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel channel-number Specifies the port-channel interface that you want to
configure, and enters the interface mode.
Example:
switch(config-if)#
Step 3 speed {10 | 100 | 1000 | auto} Sets the speed for the port-channel interface. The default
is auto for autonegotiation.
Example:
switch(config-if)# speed auto
switch(config-if)#
Step 4 duplex {auto | full | half} Sets the duplex for the port-channel interface. The default
is auto for autonegotiation.
Example:
switch(config-if)# speed auto
switch(config-if)#
mode.
Example:
switch(config)#
Step 6 show interface port-channel channel-number (Optional) Displays interface information for the
specified port channel.
Example:
configuration.
Example:
142
Configuring Load Balancing Using Port Channels
This example shows how to set port channel 2 to 100 Mb/s:

switch(config-if)# speed 100
Configuring Load Balancing Using Port Channels

You can configure the load-balancing algorithm for port channels that applies to the entire device.
Note Use the no port-channel load-balance command to restore the default load-balancing algorithm of
source-dest-mac for non-IP traffic and source-dest-ip for IP traffic.
Command Purpose
no port-channel load-balance Restores the default load-balancing
algorithm.
Example:
switch(config)# no port-channel load-balance
Before You Begin

SUMMARY STEPS
2. port-channel load-balance method {dst ip | dst ip-port-vlan | dst ip-vlan | dst mac | dst port | src-dst
ip [symmetric] | src-dst ip-gre | source-dst mac | source-dst port | src-ip port | src-dst ip-l4port
[symmetric] | src-dst l4port | src-dst mac | src ip | src mac | src-port} [fex {fex-range | all}] [rotate
rotate]
3. show port-channel load-balance
DETAILED STEPS

Example:
switch(config)#
Step 2 port-channel load-balance method {dst ip | dst ip-port-vlan Specifies the load-balancing algorithm for the device. The
| dst ip-vlan | dst mac | dst port | src-dst ip [symmetric] | range depends on the device. The default for Layer 3 is
143
Enabling LACP

src-dst ip-gre | source-dst mac | source-dst port | src-ip port src-dst-ip for both IPv4 and IPv6, and the default for
| src-dst ip-l4port [symmetric] | src-dst l4port | src-dst non-IP is src-dest-mac.
mac | src ip | src mac | src-port} [fex {fex-range | all}] [rotate Use the no port-channel load-balance src-dst mac
rotate] asymmetric command to revert back to the default system
settings (symmetrical).
Example:
Note Only the following load-balancing algorithms
• switch(config)#port-channel load-balance support symmetric hashing:
src-dst mac asymmetric
switch(config)#
• src-dst ip
• switch(config)#no port-channel load-balance
src-dst mac asymmetric • src-dst ip-l4port
switch(config)#
Step 3 show port-channel load-balance (Optional) Displays the port-channel load-balancing

algorithm.
Example:
switch(config-router)# show port-channel
load-balance
configuration.
Example:
Enabling LACP
LACP is disabled by default; you must enable LACP before you begin LACP configuration. You cannot
disable LACP while any LACP configuration is present.
LACP learns the capabilities of LAN port groups dynamically and informs the other LAN ports. Once LACP
identifies correctly matched Ethernet links, it group the links into a port channel. The port channel is then
added to the spanning tree as a single bridge port.
To configure LACP, you must do the following:
• Enable LACP globally by using the feature lacp command.
• You can use different modes for different interfaces within the same LACP-enabled port channel. You
can change the mode between active and passive for an interface only if it is the only interface that is
designated to the specified channel group.
SUMMARY STEPS
2. feature lacp
144
Configuring LACP Port-Channel Port Modes
DETAILED STEPS

Example:
switch(config)#
Step 2 feature lacp Enables LACP on the device.
Example:
switch(config)# feature lacp
Example:
This example shows how to enable LACP:

switch (config)# feature lacp
Configuring LACP Port-Channel Port Modes

After you enable LACP, you can configure the channel mode for each individual link in the LACP port channel
as active or passive. This channel configuration mode allows the link to operate with LACP.
When you configure port channels with no associated aggregation protocol, all interfaces on both sides of the
link remain in the on channel mode.
SUMMARY STEPS
3. channel-group number mode {active | on | passive}
4. show port-channel summary
145
Configuring LACP Port-Channel Minimum Links
DETAILED STEPS

Example:
switch(config)#
Step 2 interface type slot/port Specifies the interface that you want to add to a channel group,
and enters the interface configuration mode.
Example:
switch(config-if)#
Step 3 channel-group number mode {active | on | passive} Specifies the port mode for the link in a port channel. After
LACP is enabled, you configure each link or the entire channel
Example: as active or passive.
switch(config-if)# channel-group 5 mode active
When you run port channels with no associated aggregation
protocol, the port-channel mode is always on.
The default port-channel mode is on.
Step 4 show port-channel summary (Optional) Displays summary information about the port
channels.
Example:
switch(config-if)# show port-channel summary
configuration.
Example:
startup-config
This example shows how to set the LACP-enabled interface to the active port-channel mode for Ethernet
interface 1/4 in channel group 5:

You can configure the LACP minimum links feature. Although minimum links and maxbundles work only
in LACP, you can enter the CLI commands for these features for non-LACP port channels, but these commands
are nonoperational.
146
Note Use the no lacp min-links command to restore the default port-channel minimum links configuration.
Command Purpose
no lacp min-links Restores the default port-channel minimum
links configuration.
Example:
switch(config)# no lacp min-links
Before You Begin

Ensure that you are in the correct port-channel interface.
SUMMARY STEPS
3. lacp min-links number
4. show running-config interface port-channel number
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel number Specifies the interface to configure, and enters the
Example:
switch(config-if)#
Step 3 lacp min-links number Specifies the port-channel interface to configure the
number of minimum links. The range is from 1 to 16.
Example:
switch(config-if)# lacp min-links 3
Step 4 show running-config interface port-channel number (Optional) Displays the port-channel minimum links
configuration.
Example:
switch(config-if)# show running-config interface
port-channel 3
147
Configuring the LACP Port-Channel MaxBundle
This example shows how to configure the minimum number of port-channel member interfaces to be up/active
for the port-channel to be up/active:
switch(config-if)# lacp min-links 3
Configuring the LACP Port-Channel MaxBundle

You can configure the LACP maxbundle feature. Although minimum links and maxbundles work only in
LACP, you can enter the CLI commands for these features for non-LACP port channels, but these commands
are nonoperational.
Note Use the no lacp max-bundle command to restore the default port-channel max-bundle configuration.
Command Purpose
no lacp max-bundle Restores the default port-channel max-bundle
configuration.
Example:
switch(config)# no lacp max-bundle
Before You Begin

Ensure that you are in the correct port-channel interface.
SUMMARY STEPS
3. lacp max-bundle number
4. show running-config interface port-channel number
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel number Specifies the interface to configure, and enters the interface
configuration mode.
Example:
switch(config-if)#
Step 3 lacp max-bundle number Specifies the port-channel interface to configure max-bundle.
148
Configuring the LACP Fast Timer Rate

The default value for the port-channel max-bundle is 16. The
Example: allowed range is from 1 to 32.
switch(config-if)# lacp max-bundle Note Even if the default value is 16, the number of active
members in a port channel is the minimum of the
pc_max_links_config and pc_max_active_members that
is allowed in the port channel.
Step 4 show running-config interface port-channel (Optional) Displays the port-channel max-bundle configuration.
number
Example:
switch(config-if)# show running-config
interface port-channel 3
This example shows how to configure the port channel interface max-bundle:
switch(config-if)# lacp max-bundle 3
Configuring the LACP Fast Timer Rate

You can change the LACP timer rate to modify the duration of the LACP timeout. Use the lacp rate command
to set the rate at which LACP control packets are sent to an LACP-supported interface. You can change the
timeout rate from the default rate (30 seconds) to the fast rate (1 second). This command is supported only
on LACP-enabled interfaces.
Note We do not recommend changing the LACP timer rate. HA and SSO are not supported when the LACP
fast rate timer is configured.
Before You Begin

Ensure that you have enabled the LACP feature.
SUMMARY STEPS
3. lacp rate fast
149
Configuring the LACP System Priority
DETAILED STEPS

Example:
switch(config)#
Step 2 interface type slot/port Specifies the interface to configure and enters the interface
configuration mode.
Example:
switch(config-if)#
Step 3 lacp rate fast Configures the fast rate (one second) at which LACP control
packets are sent to an LACP-supported interface.
Example: To reset the timeout rate to its default, use the no form of the
switch(config-if)# lacp rate fast
command.
This example shows how to configure the LACP fast rate on Ethernet interface 1/4:
switch(config-if)# lacp rate fast
This example shows how to restore the LACP default rate (30 seconds) on Ethernet interface 1/4.
switch(config-if)# no lacp rate fast
Configuring the LACP System Priority

The LACP system ID is the combination of the LACP system priority value and the MAC address.
Before You Begin

Enable LACP.
SUMMARY STEPS
2. lacp system-priority priority
3. show lacp system-identifier
150
Configuring the LACP Port Priority
DETAILED STEPS

Example:
switch(config)#
Step 2 lacp system-priority priority Configures the system priority for use with LACP. Valid values
are from 1 through 65535, and higher numbers have a lower
Example: priority. The default value is 32768.
switch(config)# lacp system-priority 40000
Note Each VDC has a different LACP system ID because
the software adds the MAC address to this configured
value.
Step 3 show lacp system-identifier (Optional) Displays the LACP system identifier.
Example:
switch(config-if)# show lacp system-identifier
configuration.
Example:
startup-config
This example shows how to set the LACP system priority to 2500:
switch(config)# lacp system-priority 2500
Configuring the LACP Port Priority

When you enable LACP, you can configure each link in the LACP port channel for the port priority.
Before You Begin

Enable LACP.
SUMMARY STEPS
3. lacp port-priority priority
151
Disabling LACP Graceful Convergence
DETAILED STEPS

Example:
switch(config)#
Step 2 interface type slot/port Specifies the interface that you want to add to a channel
group, and enters the interface configuration mode.
Example:
switch(config-if)#
Step 3 lacp port-priority priority Configures the port priority for use with LACP. Valid values
are from 1 through 65535, and higher numbers have a lower
Example: priority. The default value is 32768.
switch(config-if)# lacp port-priority
40000
configuration.
Example:
startup-config
This example shows how to set the LACP port priority for Ethernet interface 1/4 to 40000:
switch(config-if)# lacp port-priority 40000

By default, LACP graceful convergence is enabled. In situations where you need to support LACP
interoperability with devices where the graceful failover defaults may delay the time taken for a disabled port
to be brought down or cause traffic from the peer to be lost, you can disable convergence. If the downstream
access switch is not a Cisco Nexus device, disable the LACP graceful convergence option.
Note The port channel has to be in the administratively down state before the command can be run.
Before You Begin

Enable LACP.
152
SUMMARY STEPS
3. shutdown
4. no lacp graceful-convergence
5. no shutdown
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel number Specifies the port channel interface to configure and
Example:
switch(config-if)#
Step 3 shutdown Administratively shuts down the port channel.
Example:
switch(config-if) shutdown
Step 4 no lacp graceful-convergence Disables LACP graceful convergence on the port

channel.
Example:
switch(config-if)# no lacp graceful-convergence
Step 5 no shutdown Brings the port channel administratively up.
Example:
switch(config-if) no shutdown
Example:
This example shows how to disable LACP graceful convergence on a port channel:
switch(config-if)# no lacp graceful-convergence
153
Reenabling LACP Graceful Convergence

If the default LACP graceful convergence is once again required, you can reenable convergence.
SUMMARY STEPS
3. shutdown
4. lacp graceful-convergence
5. no shutdown
DETAILED STEPS

Example:
switch(config)#
Example:
switch(config-if)#
Example:
Step 4 lacp graceful-convergence Enables LACP graceful convergence on the port

channel.
Example:
switch(config-if)# lacp graceful-convergence
Example:
Example:
154
Disabling LACP Suspend Individual
This example shows how to enable LACP graceful convergence on a port channel:
switch(config-if)# lacp graceful-convergence
Disabling LACP Suspend Individual

LACP sets a port to the suspended state if it does not receive an LACP PDU from the peer. This process can
cause some servers to fail to boot up as they require LACP to logically bring up the port.
Note You should only enter the lacp suspend-individual command on edge ports. The port channel has to be
in the administratively down state before you can use this command.
Before You Begin

Enable LACP.
SUMMARY STEPS
3. shutdown
4. no lacp suspend-individual
5. no shutdown
DETAILED STEPS

Example:
switch(config)#
Example:
switch(config-if)#
Example:
155
Reenabling LACP Suspend Individual

Step 4 no lacp suspend-individual Disables LACP individual port suspension behavior on
the port channel.
Example:
switch(config-if)# no lacp suspend-individual
Example:
Example:
This example shows how to disable LACP individual port suspension on a port channel:
switch(config-if)# no lacp suspend-individual
Reenabling LACP Suspend Individual

You can reenable the default LACP individual port suspension.
SUMMARY STEPS
3. shutdown
4. lacp suspend-individual
5. no shutdown
DETAILED STEPS

Example:
switch(config)#
156
Configuring Port Channel Hash Distribution

Example:
switch(config-if)#
Example:
Step 4 lacp suspend-individual Enables LACP individual port suspension behavior on

the port channel.
Example:
switch(config-if)# lacp suspend-individual
Example:
Example:
This example shows how to reenable the LACP individual port suspension on a port channel:
switch(config-if)# lacp suspend-individual

Cisco NX-OS supports the adaptive and fixed hash distribution configuration for both global and port-channel
levels. This option minimizes traffic disruption by minimizing Result Bundle Hash (RBH) distribution changes
when members come up or go down so that flows that are mapped to unchange RBH values continue to flow
through the same links. The port-channel level configuration overrules the global configuration. The default
configuration is adaptive globally, and there is no configuration for each port channel, so there is no change
during an ISSU. No ports are flapped when the command is applied, and the configuration takes effect at the
next member link change event. Both modes work with RBH module or non-module schemes.
During an ISSD to a lower version that does not support this feature, you must disable this feature if the fixed
mode command is being used globally or if there is a port-channel level configuration.
157
Configuring Port Channel Hash Distribution at the Global Level
SUMMARY STEPS
2. no port-channel hash-distribution {adaptive | fixed}
DETAILED STEPS

Example:
switch(config)#
Step 2 no port-channel hash-distribution {adaptive | fixed} Specifies the port-channel hash distribution at the global level.
The default is adaptive mode.
Example:
switch(config)# port-channel hash-distribution The command does not take effect until the next member link
adaptive event (link down/up/no shutdown/shutdown). (Do you still
switch(config)#
want to continue(y/n)? [yes])
configuration.
Example:
startup-config
This example shows how to configure hash distribution at the global level:
switch(config)# no port-channel hash-distribution fixed
Configuring Port Channel Hash Distribution at the Port Channel Level
SUMMARY STEPS
2. interface port-channel {channel-number | range}
3. no port-channel port hash-distribution {adaptive | fixed}
158
Verifying the Port-Channel Configuration
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel {channel-number | range} Specifies the interface to configure, and enters the interface
configuration mode.
Example:
switch# interface port-channel 4
switch(config-if)#
Step 3 no port-channel port hash-distribution {adaptive | Specifies the port-channel hash distribution at the port
fixed} channel level.
There is no default.
Example:
switch(config-if)# port-channel port The command does not take effect until the next member
hash-distribution adaptive link event (link down/up/no shutdown/shutdown). (Do you
switch(config-if)
still want to continue(y/n)? [yes])
configuration.
Example:
startup-config
This example shows how to configure hash distribution as a global-level command:

switch(config)# no port-channel hash-distribution fixed
Verifying the Port-Channel Configuration

To display port-channel configuration information, perform one of the following tasks:
Command Purpose
show interface port-channel channel-number Displays the status of a port-channel interface.
show feature Displays enabled features.
load- interval {interval seconds {1 | 2 | 3}} Sets three different sampling intervals to bit-rate and
packet-rate statistics.
show port-channel compatibility-parameters Displays the parameters that must be the same among
the member ports in order to join a port channel.
159
Monitoring the Port-Channel Interface Configuration
Command Purpose
show port-channel database [interface Displays the aggregation state for one or more
port-channel channel-number] port-channel interfaces.
show port-channel load-balance Displays the type of load balancing in use for port
channels.
show port-channel summary Displays a summary for the port-channel interfaces.
show port-channel traffic Displays the traffic statistics for port channels.
show port-channel usage Displays the range of used and unused channel
numbers.
show lacp {counters [interface port-channel Displays information about LACP.

channel-number] | [interface type/slot] | neighbor
[interface port-channel channel-number] |
port-channel [interface port-channel
channel-number] | system-identifier]]}
show running-config interface port-channel Displays information about the running configuration
channel-number of the port-channel.
Monitoring the Port-Channel Interface Configuration

Use the following commands to display port-channel interface configuration information.
Command Purpose
clear counters interface port-channel Clears the counters.
channel-number
clear lacp counters [interface port-channel Clears the LACP counters.

channel-number]
load- interval {interval seconds {1 | 2 | 3}} Sets three different sampling intervals to bit-rate and
packet-rate statistics.
show interface counters [module module] Displays input and output octets unicast packets,
multicast packets, and broadcast packets.
show interface counters detailed [all] Displays input packets, bytes, and multicast and
output packets and bytes.
show interface counters errors [module module] Displays information about the number of error
packets.
160
Example Configurations for Port Channels
Command Purpose
show lacp counters Displays statistics for LACP.
Example Configurations for Port Channels

This example shows how to create an LACP port channel and add two Layer 2 interfaces to that port channel:

switch (config)# feature lacp
switch (config-if)# interface ethernet 1/4
switch(config-if)# lacp port priority 40000
switch(config-if)# channel-group 5 mode
This example shows how to add two Layer 3 interfaces to a channel group. The Cisco NX-OS software
automatically creates the port channel:

switch(config-if)# no ip address
switch(config-if)# no ip address
Related Documents
System management Cisco Nexus 9000 Series NX-OS System Management
Configuration Guide
High availability Cisco Nexus 9000 Series NX-OS High Availability

and Redundancy Guide
Licensing Cisco NX-OS Licensing Guide
161
Related Documents
162
CHAPTER 8
Configuring vPCs
This chapter describes how to configure virtual port channels (vPCs) on Cisco NX-OS devices.
You can use any of the interfaces of the Nexus 9000 device for the vPC peer link.
The port channel compatibility parameters must be the same for all the port channel members on the physical
switch.
You cannot configure shared interfaces to be part of a vPC.
Note The port channel compatibility parameters must also be the same for all vPC member ports on both peers
and therefore you must use the same type of module in each chassis.
• Information About vPCs, page 163

• Licensing Requirements for vPCs, page 196
• Configuring vPCs, page 198
• Verifying the vPC Configuration, page 224
• Monitoring vPCs, page 225
• Configuration Examples for vPCs, page 225
Information About vPCs
vPC Overview
A virtual port channel (vPC) allows links that are physically connected to two different Cisco Nexus 9000
Series devices to appear as a single port channel by a third device (see figure). The third device can be a
163
Configuring vPCs
vPC Overview
switch, server, or any other networking device that supports port channels. A vPC can provide Layer 2
multipathing, which allows you to create redundancy and increase the bisectional bandwidth by enabling
multiple parallel paths between nodes and allowing load balancing traffic.
Figure 10: vPC Architecture
You can use only Layer 2 port channels in the vPC. A vPC domain is associated to a single Virtual Device
Context (VDC), so all vPC interfaces belonging to a given vPC domain must be defined in the same VDC.
You configure the port channels by using one of the following:
• No protocol
• Link Aggregation Control Protocol (LACP)
When you configure the port channels in a vPC—including the vPC peer link channel—without using LACP,
each device can have up to eight active links in a single port channel. When you configure the port channels
in a vPC—including the vPC peer link channels—using LACP, each device can have eight active links and
eight standby links in a single port channel. (See the “vPC Interactions with Other Features” section for more
information on using LACP and vPCs.)
Note You must enable the vPC feature before you can configure or run the vPC functionality.
The system automatically takes a checkpoint prior to disabling the feature, and you can roll back to this
checkpoint.
After you enable the vPC functionality, you create the peer-keepalive link, which sends heartbeat messages
between the two vPC peer devices.
You can create a vPC peer link by configuring a port channel on one Cisco Nexus 9000 Series chassis by
using two or more 10-Gigabit Ethernet ports or 40-Gigabit Ethernet ports. To ensure that you have the correct
hardware to enable and run a vPC, enter the show hardware feature-capability command. If you see an X
across from the vPC in your command output, your hardware cannot enable the vPC feature.
We recommend that you configure the vPC peer link Layer 2 port channels as trunks. On another Cisco Nexus
9000 Series chassis, you configure another port channel again using two or more 10-Gigabit Ethernet ports
or 40-Gigabit Ethernet ports in the dedicated port mode. Connecting these two port channels creates a vPC
164
Configuring vPCs
vPC Overview
peer link in which the two linked Cisco Nexus devices appear as one device to a third device. The third device,
or downstream device, can be a switch, server, or any other networking device that uses a regular port channel
to connect to the vPC. If you are not using the correct module, the system displays an error message.
We recommend that you configure the vPC peer links on dedicated ports of different modules to reduce the
possibility of a failure. For the best resiliency scenario, use at least two modules.
If you must configure all the vPC peer links and core-facing interfaces on a single module, you should configure
a track object that is associated with the Layer 3 link to the core and on all the links on the vPC peer link on
both vPC peer devices. Once you configure this feature and if the primary vPC peer device fails, the system
automatically suspends all the vPC links on the primary vPC peer device. This action forces all the vPC traffic
to the secondary vPC peer device until the system stabilizes.
You can create a track object and apply that object to all links on the primary vPC peer device that connect
to the core and to the vPC peer link. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration
Guide for information about the track interface command.
The vPC domain includes both vPC peer devices, the vPC peer-keepalive link, the vPC peer link, and all of
the port channels in the vPC domain connected to the downstream device. You can have only one vPC domain
ID on each device.
In this version, you can connect each downstream device to a single vPC domain ID using a single port channel.
Note Always attach all vPC devices using port channels to both vPC peer devices.
A vPC (see figure) provides the following benefits:

• Allows a single device to use a port channel across two upstream devices
• Eliminates Spanning Tree Protocol (STP) blocked ports
• Provides a loop-free topology
• Uses all available uplink bandwidth
• Provides fast convergence if either the link or a device fails
• Provides link-level resiliency
165
Configuring vPCs
vPC Terminology
• Assures high availability
Figure 11: vPC Interfaces in One VDC
vPC Terminology
The terminology used in vPCs is as follows:
• vPC—The combined port channel between the vPC peer devices and the downstream device.
• vPC peer device—One of a pair of devices that are connected with the special port channel known as
the vPC peer link.
• vPC peer link—The link used to synchronize states between the vPC peer devices. Both ends must be
on 10-Gigabit Ethernet or 40-Gigabit Ethernet interfaces.
• vPC member port—An interface that belongs to a vPC.
• Host vPC port—A Fabric Extender host interfaces that belongs to a vPC.
• vPC domain—This domain includes both vPC peer devices, the vPC peer-keepalive link, and all of the
port channels in the vPC connected to the downstream devices. It is also associated to the configuration
mode that you must use to assign vPC global parameters.
• vPC peer-keepalive link—The peer-keepalive link monitors the vitality of a vPC peer Cisco Nexus 9000
Series device. The peer-keepalive link sends configurable, periodic keepalive messages between vPC
peer devices.
We recommend that you associate a peer-keepalive link to a separate virtual routing and forwarding
(VRF) instance that is mapped to a Layer 3 interface in each vPC peer device. If you do not configure
a separate VRF, the system uses the management VRF by default. However, if you use the management
166
Configuring vPCs
vPC Terminology
interfaces for the peer-keepalive link, you must put a management switch connected to both the active
and standby management ports on each vPC peer device (see figure).
Figure 12: Separate Switch Required to Connect Management Ports for vPC Peer-Keepalive Link
No data or synchronization traffic moves over the vPC peer-keepalive link; the only traffic on this link
is a message that indicates that the originating switch is operating and running a vPC.
• vPC member port—Interfaces that belong to the vPCs.
• Dual-active— Both vPC peers act as primary. This situation occurs when the peer-keepalive and peer-link
go down when both the peers are still active. In this case, the secondary vPC assumes that the primary
vPC is inactive and acts as the primary vPC.
• Recovery—When the peer-keepalive and the peer-link come up, one switch becomes the secondary
vPC. On the switch that becomes the secondary vPC, the vPC links go down and come back up.
vPC Peer Link Overview

You can have only two devices as vPC peers; each device can serve as a vPC peer to only one other vPC peer.
The vPC peer devices can also have non-vPC links to other devices.
167
Configuring vPCs
vPC Terminology
See the following figure for invalid vPC peer configurations.
Figure 13: vPC Peer Configurations That Are Not Allowed
To make a valid configuration, you first configure a port channel on each device and then configure the vPC
domain. You assign the port channel on each device as a peer link, using the same vPC domain ID. For
redundancy, we recommend that you should configure at least two of the dedicated ports into the port channel
because if one of the interfaces in the vPC peer link fails, the device automatically falls back to use another
interface in the peer link.
Note We recommend that you configure the Layer 2 port channels in trunk mode.
Many operational parameters and configuration parameters must be the same in each device connected by a
vPC peer link (see the “Compatibility Parameters for vPC Interfaces” section). Because each device is completely
independent on the management plane, you must ensure that the devices are compatible on the critical
parameters. vPC peer devices have separate control planes. After configuring the vPC peer link, you should
display the configuration on each vPC peer device to ensure that the configurations are compatible.
Note You must ensure that the two devices connected by the vPC peer link have certain identical operational
and configuration parameters. For more information on required configuration consistency, see the
“Compatibility Parameters for vPC Interfaces” section.
When you configure the vPC peer link, the vPC peer devices negotiate that one of the connected devices is
the primary device and the other connected device is the secondary device (see the “Configuring vPCs” section).
The Cisco NX-OS software uses the lowest MAC address to elect the primary device. The software takes
different actions on each device—that is, the primary and secondary—only in certain failover conditions. If
the primary device fails, the secondary device becomes the new primary device when the system recovers,
and the previously primary device is now the secondary device.
You can also configure which of the vPC devices is the primary device. Changing the priority of the vPC peer
devices can cause the interfaces in your network to go up and down. If you want to configure the role priority
again to make one vPC device the primary device, configure the role priority on both the primary vPC device
with a lower priority value and the secondary vPC device with the higher value. Then, shut down the port
168
Configuring vPCs
vPC Terminology
channel that is the vPC peer link on both devices by entering the shutdown command, and finally reenable
the port channel on both devices by entering the no shutdown command.
Note We recommend that you use two different modules for redundancy on each vPC peer device on each vPC
peer link.
The software keeps all traffic that forwards across the vPC peer devices as local traffic. A packet that ingresses
the port channel uses one of the local links rather than moving across the vPC peer link. Unknown unicast,
multicast, and broadcast traffic (including STP BPDUs) are flooded across the vPC peer link. The software
keeps the multicast forwarding state synchronized on both of the vPC peer devices.
You can configure any of the standard load-balancing schemes on both the vPC peer link devices and the
downstream device (see the “Configuring Port Channels” chapter for information about load balancing).
Configuration information flows across the vPC peer links using the Cisco Fabric Services over Ethernet
(CFSoE) protocol. (See the “vPC and Orphan Ports” section for more information about CFSoE.)
All MAC addresses for those VLANs configured on both devices are synchronized between vPC peer devices.
The software uses CFSoE for this synchronization. (See the “vPC and Orphan Ports” section for information
about CFSoE.)
If the vPC peer link fails, the software checks the status of the remote vPC peer device using the peer-keepalive
link, which is a link between vPC peer devices that ensures that both devices are up. If the vPC peer device
is up, the secondary vPC device disables all vPC ports on its device, to prevent loops and disappearing or
flooding traffic. The data then forwards down the remaining active links of the port channel.
Note We recommend that you create and configure a separate VRF and configure a Layer 3 port on each vPC
peer device in that VRF for the vPC peer-keepalive link. The default ports and VRF for the peer-keepalive
are the management ports and VRF.
The software learns of a vPC peer device failure when the keepalive messages are not returned over the
peer-keepalive link.
Use a separate link (vPC peer-keepalive link) to send configurable keepalive messages between the vPC peer
devices. The keepalive messages on the vPC peer-keepalive link determines whether a failure is on the vPC
peer link only or on the vPC peer device. The keepalive messages are used only when all the links in the peer
link fail. See the “Peer-Keepalive Link and Messages” section for information about the keepalive message.
Features That You Must Manually Configure on the Primary and Secondary Devices
You must manually configure the following features to conform to the primary/secondary mapping of each
of the vPC peer devices:
• STP root—Configure the primary vPC peer device as the STP primary root device and configure the
vPC secondary device to be the STP secondary root device. See the “vPC Peer Links and STP” section
for more information about vPCs and STP.
◦We recommend that you configure the vPC peer link interfaces as STP network ports so that Bridge
Assurance is enabled on all vPC peer links
169
Configuring vPCs
Peer-Keepalive Link and Messages
◦We recommend that you configure Rapid per VLAN Spanning Tree plus (PVST+) so that the
primary device is the root for all VLANs and configure Multiple Spanning Tree (MST) so that the
primary device is the root for all instances.
• Layer 3 VLAN network interface—Configure Layer 3 connectivity from each vPC peer device by
configuring a VLAN network interface for the same VLAN from both devices.
• HSRP active—If you want to use Hot Standby Router Protocol (HSRP) and VLAN interfaces on the
vPC peer devices, configure the primary vPC peer device with the HSRP active highest priority. Configure
the secondary device to be the HSRP standby and ensure that you have VLAN interfaces on each vPC
device that are in the same administrative and operational mode. (See the “vPC Peer Links and Routing”
section for more information on vPC and HSRP.)
We recommend that you configure Unidirectional Link Detection (UDLD) on both sides of the vPC peer link.
See the “Configuring the UDLD Mode” section for information about configuring UDLD.
Configuring Layer 3 Backup Routes on a vPC Peer Link

You can use VLAN network interfaces on the vPC peer devices to link to Layer 3 of the network for such
applications as HSRP and PIM. However, we recommend that you configure a separate Layer 3 link for
routing from the vPC peer devices, rather than using a VLAN network interface for this purpose.
Ensure that you have a VLAN network interface configured on each peer device and that the interface is
connected to the same VLAN on each device. Also, each VLAN interface must be in the same administrative
and operational mode. For more information about configuring VLAN network interfaces, see the “Configuring
Layer 3 Interfaces” chapter.
If a failover occurs on the vPC peer link, the VLAN interfaces on the vPC peer devices are also affected. If
a vPC peer link fails, the system brings down associated VLAN interfaces on the secondary vPC peer device.
You can ensure that specified VLAN interfaces do not go down on the vPC secondary device when the vPC
peer link fails.
Use the dual-active exclude interface-vlan command to configure this feature.
Note When you attach a Layer 3 device to a vPC domain, the peering of routing protocols using a VLAN also
carried on the vPC peer link is not supported. If routing protocol adjacencies are needed between vPC
peer devices and a generic Layer 3 device, you must use physical routed interfaces for the interconnection.
Use of the vPC peer-gateway feature does not change this requirement.
Peer-Keepalive Link and Messages

The Cisco NX-OS software uses the peer-keepalive link between the vPC peers to transmit periodic,
configurable keepalive messages. You must have Layer 3 connectivity between the peer devices to transmit
these messages; the system cannot bring up the vPC peer link unless the peer-keepalive link is already up and
running.
170
Configuring vPCs
vPC Peer-Gateway
Note We recommend that you associate the vPC peer-keepalive link to a separate VRF mapped to a Layer 3
interface in each vPC peer device. If you do not configure a separate VRF, the system uses the management
VRF and management ports by default. Do not use the peer link itself to send and receive vPC
peer-keepalive messages.
If one of the vPC peer devices fails, the vPC peer device on the other side of the vPC peer link senses the
failure by not receiving any peer-keepalive messages. The default interval time for the vPC peer-keepalive
message is 1 second, and you can configure the interval between 400 milliseconds and 10 seconds.
You can configure a hold-timeout value with a range of 3 to 10 seconds; the default hold-timeout value is 3
seconds. This timer starts when the vPC peer link goes down. During this hold-timeout period, the secondary
vPC peer device ignores vPC peer-keepalive messages, which ensures that network convergence occurs before
a vPC action takes place. The purpose of the hold-timeout period is to prevent false-positive cases.
You can also configure a timeout value with a range of 3 to 20 seconds; the default timeout value is 5 seconds.
This timer starts at the end of the hold-timeout interval. During the timeout period, the secondary vPC peer
device checks for vPC peer-keepalive hello messages from the primary vPC peer device. If the secondary
vPC peer device receives a single hello message, that device disables all vPC interfaces on the secondary vPC
peer device.
The difference between the hold-timeout and the timeout parameters is as follows:
• During the hold-timeout, the vPC secondary device does not take any action based on any keepalive
messages received, which prevents the system taking action when the keepalive might be received just
temporarily, such as if a supervisor fails a few seconds after the peer link goes down.
• During the timeout, the vPC secondary device takes action to become the vPC primary device if no
keepalive message is received by the end of the configured interval.
See the “Configuring vPCs” section for information about configuring the timer for the keepalive messages.
Note Ensure that both the source and destination IP addresses used for the peer-keepalive messages are unique
in your network and these IP addresses are reachable from the VRF associated with the vPC peer-keepalive
link.
Use the command-line interface (CLI) to configure the interfaces you are using the vPC peer-keepalive
messages as trusted ports. Leave the precedence at the default (6) or configure it higher. The following is an
example of configuring an interface as a trusted port:
(config)# class-map type qos match-all trust-map

(config-cmap-qos)# match cos 4-7
(config)# policy-map type qos ingresspolicy
(config-pmap-qos)# class trust-map
(config)# interface Ethernet 8/11
(config-if)# service-policy type qos input ingresspolicy
vPC Peer-Gateway
You can configure vPC peer devices to act as the gateway even for packets that are destined to the vPC peer
device’s MAC address.
171
Configuring vPCs
vPC Domain
Use the peer-gateway command to configure this feature.
Note The peer-gateway exclude-vlan command that is used when configuring a VLAN interface for Layer 3
backup routing on vPC peer devices is not supported.
Some network-attached storage (NAS) devices or load balancers might have features that help to optimize
the performances of particular applications. These features enable the device to avoid a routing-table lookup
when responding to a request that originated from a host that is not locally attached to the same subnet. Such
devices might reply to traffic using the MAC address of the sender Cisco Nexus 9000 Series device rather
than the common HSRP gateway. This behavior is noncomplaint with some basic Ethernet RFC standards.
Packets that reach a vPC device for the nonlocal router MAC address are sent across the peer link and could
be dropped by the built in vPC loop avoidance mechanism if the final destination is behind another vPC.
The vPC peer-gateway capability allows a vPC switch to act as the active gateway for packets that are addressed
to the router MAC address of the vPC peer. This feature enables local forwarding of packets without the need
to cross the vPC peer link. In this scenario, the feature optimizes use of the peer link and avoids potential
traffic loss.
Configuring the peer-gateway feature must be done on both primary and secondary vPC peers and is
nondisruptive to the operations of the device or to the vPC traffic. The vPC peer-gateway feature can be
configured globally under the vPC domain submode.
When you enable this feature, Cisco NX-OS automatically disables IP redirects on all interface VLANs
mapped over a vPC VLAN to avoid generation of IP redirect messages for packets switched through the peer
gateway router.
Packets that arrive at the peer-gateway vPC device have their Time to Live (TTL) decremented, so that packets
carrying a TTL of 1 might get dropped in transit due to TTL expiration. You should take this situation into
account when the peer-gateway feature is enabled and particular network protocols that source packets with
a TTL of 1 operate on a vPC VLAN.
vPC Domain
You can use the vPC domain ID to identify the vPC peer links and the ports that are connected to the vPC
downstream devices.
The vPC domain is also a configuration mode that you use to configure the keepalive messages and other
vPC peer link parameters rather than accept the default values. See the “Configuring vPCs” section for more
information about configuring these parameters.
To create a vPC domain, you must first create a vPC domain ID on each vPC peer device using a number
from 1 to 1000. You can have only one vPC domain per VDC.
You must explicitly configure the port channel that you want to act as the peer link on each device. You
associate the port channel that you made a peer link on each device with the same vPC domain ID to form a
single vPC domain. Within this domain, the system provides a loop-free topology and Layer 2 multipathing.
You can only configure these port channels and vPC peer links statically. All ports in the vPC on each of the
vPC peer devices must be in the same VDC. You can configure the port channels and vPC peer links either
using LACP or no protocol. We recommend that you use LACP with the interfaces in active mode to configure
port channels in each vPC, which ensures an optimized, graceful recovery in a port-channel failover scenario
and provides configuration checks against configuration mismatches among the port channels themselves.
172
Configuring vPCs
vPC Topology
The vPC peer devices use the vPC domain ID that you configure to automatically assign a unique vPC system
MAC address. Each vPC domain has a unique MAC address that is used as a unique identifier for the specific
vPC-related operations, although the devices use the vPC system MAC addresses only for link-scope operations,
such as LACP. We recommend that you create each vPC domain within the contiguous Layer 2 network with
a unique domain ID. You can also configure a specific MAC address for the vPC domain, rather than having
the Cisco NX-OS software assign the address.
See the “vPC and Orphan Ports” section for more information about displaying the vPC MAC table.
After you create a vPC domain, the Cisco NX-OS software creates a system priority for the vPC domain. You
can also configure a specific system priority for the vPC domain.
Note When manually configuring the system priority, you must ensure that you assign the same priority value
on both vPC peer devices. If the vPC peer devices have different system priority values, vPC does not
come up.
vPC Topology
The following figure shows a basic configuration in which the Cisco Nexus 9000 Series device ports are
directly connected to another switch or host and are configured as part of a port channel that becomes part of
a vPC.
Figure 14: Switch vPC Topology
In the figure, vPC 20 is configured on port channel 20, which has Eth1/10 on the first device and Eth2/1 on
the second as member ports.
173
Configuring vPCs
Compatibility Parameters for vPC Interfaces
You can configure a vPC from the peer devices through Fabric Extenders (FEXs) as shown in the figure.
Figure 15: FEX Straight-Through Topology (Host vPC)
In the figure, each FEX is single-homed (straight-through FEX topology) with a Cisco Nexus 9000 Series
device. The host interfaces on this FEX are configured as port channels and those port channels are configured
as vPCs. Eth101/1/1 and Eth102/1/5 are configured as members of PO200, and PO200 is configured for vPC
200.
In both topologies, port channels P020 and P0200 must be configured identically on the peer switches and
configuration synchronization is used to synchronize the configurations of the vPC switches.
See the Cisco Nexus 2000 Series NX-OS Fabric Extender Configuration Guide for Cisco Nexus 9000 Series
Switches for more information about configuring FEX ports.

Many configuration and operational parameters must be identical on all interfaces in the vPC. We recommend
that you configure the Layer 2 port channels that you use for the vPC peer link in trunk mode.
After you enable the vPC feature and configure the peer link on both vPC peer devices, Cisco Fabric Services
(CFS) messages provide a copy of the configuration on the local vPC peer device configuration to the remote
vPC peer device. The system then determines whether any of the crucial configuration parameters differ on
the two devices. (See the “vPC and Orphan Ports” section for more information about CFS.)
174
Configuring vPCs
Note Enter the show vpc consistency-parameterscommand to display the configured values on all interfaces
in the vPC. The displayed configurations are only those configurations that would limit the vPC peer link
and vPC from coming up.
The compatibility check process for vPCs differs from the compatibility check for regular port channels.
See the “Configuring Port Channels” chapter for information about regular port channels.
Configuration Parameters That Must Be Identical

The configuration parameters in this section must be configured identically on both devices of the vPC peer
link; otherwise, the vPC moves fully or partially into a suspended mode.
Note You must ensure that all interfaces in the vPC have the identical operational and configuration parameters
listed in this section.
Note Enter the show vpc consistency-parameters command to display the configured values on all interfaces
in the vPC. The displayed configurations are only those configurations that would limit the vPC peer link
and vPC from coming up.
The devices automatically check for compatibility for some of these parameters on the vPC interfaces. The
per-interface parameters must be consistent per interface, and the global parameters must be consistent globally:
• Port-channel mode: on, off, or active (port-channel mode can, however, be active/passive on each side
of the vPC peer)
• Link speed per channel
• Duplex mode per channel
• Trunk mode per channel:
◦Native VLAN
◦VLANs allowed on trunk
◦Tagging of native VLAN traffic
• Spanning Tree Protocol (STP) mode

• STP region configuration for Multiple Spanning Tree
• Enable/disable state per VLAN
• STP global settings:
◦Bridge Assurance setting
◦Port type setting
◦Loop Guard settings
175
Configuring vPCs
• STP interface settings:

◦Loop Guard
◦Root Guard
• Maximum Transmission Unit (MTU)
If any of these parameters are not enabled or defined on either device, the vPC consistency check ignores
those parameters.
Note To ensure that none of the vPC interfaces are in the suspend mode, enter the show vpc brief and show
vpc consistency-parameters commands and check the syslog messages.
Configuration Parameters That Should Be Identical

When any of the following parameters are not configured identically on both vPC peer devices, a
misconfiguration might cause undesirable behavior in the traffic flow:
• MAC aging timers
• Static MAC entries
• VLAN interface—Each device on the end of the vPC peer link must have a VLAN interface configured
for the same VLAN on both ends and they must be in the same administrative and operational mode.
Those VLANs configured on only one device of the peer link do not pass traffic using the vPC or peer
link. You must create all VLANs on both the primary and secondary vPC devices, or the VLAN will
be suspended.
• All ACL configurations and parameters
• Quality of Service (QoS) configuration and parameters
◦BPDU Filter
◦BPDU Guard
◦Cost
◦Link type
◦Priority
◦VLANs (Rapid PVST+)
• Port security
• Cisco Trusted Security (CTS)
• Dynamic Host Configuration Protocol (DHCP) snooping
• Network Access Control (NAC)
176
Configuring vPCs
vPC Number
• Dynamic ARP Inspection (DAI)

• IP source guard (IPSG)
• Internet Group Management Protocol (IGMP) snooping
• Hot Standby Routing Protocol (HSRP)
• Protocol Independent Multicast (PIM)
• All routing protocol configurations
To ensure that all the configuration parameters are compatible, we recommend that you display the
configurations for each vPC peer device once you configure the vPC.
Consequences of Parameter Mismatches

You can configure the graceful consistency check feature, which suspends only the links on the secondary
peer device when a mismatch is introduced in a working vPC. This feature is configurable only in the CLI
and is enabled by default.
Use the graceful consistency-check command to configure this feature.
As part of the consistency check of all parameters from the list of parameters that must be identical, the system
checks the consistency of all VLANs.
The vPC remains operational, and only the inconsistent VLANs are brought down. This per-VLAN consistency
check feature cannot be disabled and does not apply to Multiple Spanning Tree (MST) VLANs.
vPC Number
Once you have created the vPC domain ID and the vPC peer link, you create port channels to attach the
downstream device to each vPC peer device. That is, you create one port channel to the downstream device
from the primary vPC peer device and you create another port channel to the downstream device from the
secondary peer device.
Note We recommend that you configure the ports on the downstream devices that connect to a host or a network
device that is not functioning as a switch or a bridge as STP edge ports.
On each vPC peer device, you assign a vPC number to the port channel that connects to the downstream
device. You will experience minimal traffic disruption when you are creating vPCs. To simplify the
configuration, you can assign the vPC ID number to every port channel to be the same as the port channel
itself (that is, vPC ID 10 for port channel 10).
Note The vPC number that you assign to the port channel that connects to the downstream device from the vPC
peer device must be identical on both vPC peer devices.
177
Configuring vPCs
Moving Other Port Channels into a vPC
Note You must attach a downstream device using a port channel to both vPC peer devices.
To connect to the downstream device, you create a port channel to the downstream device from the primary
vPC peer device and you create another port channel to the downstream device from the secondary peer device.
device. You will experience minimal traffic disruption when you are creating vPCs.
Configuring vPC Peer Links and Links to the Core on a Single Module
Note We recommend that you configure the vPC peer links on dedicated ports of different modules to reduce
the possibility of a failure. For the best resiliency scenario, use at least two modules.
If you must configure all the vPC peer links and core-facing interfaces on a single module, you should configure,
using the command-line interface, a track object and a track list that is associated with the Layer 3 link to the
core and on all vPC peer links on both vPC peer devices. You use this configuration to avoid dropping traffic
if that particular module goes down because when all the tracked objects on the track list go down, the system
does the following:
• Stops the vPC primary peer device sending peer-keepalive messages, which forces the vPC secondary
peer device to take over.
• Brings down all the downstream vPCs on that vPC peer device, which forces all the traffic to be rerouted
in the access switch toward the other vPC peer device.
Once you configure this feature and if the module fails, the system automatically suspends all the vPC links
on the primary vPC peer device and stops the peer-keepalive messages. This action forces the vPC secondary
device to take over the primary role and all the vPC traffic to go to this new vPC primary device until the
system stabilizes.
You should create a track list that contains all the links to the core and all the vPC peer links as its object.
Enable tracking for the specified vPC domain for this track list. Apply this same configuration to the other
vPC peer device. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for information
about configuring object tracking and track lists.
Note This example uses Boolean OR in the track list and forces all traffic to the vPC peer device only for a
complete module failure. If you want to trigger a switchover when any core interface or peer link goes
down, use a Boolean AND in the torack list below.
To configure a track list to switch over a vPC to the remote peer when all related interfaces on a single module
fail, follow these steps:
1 Configure track objects on an interface (Layer 3 to core) and on a port channel (vPC peer link).
switch(config-if)# track 35 interface ethernet 8/35 line-protocol
178
Configuring vPCs
vPC Interactions with Other Features
switch(config-track)# track 23 interface ethernet 8/33 line-protocol

switch(config)# track 55 interface port-channel 100 line-protocol
2 Create a track list that contains all the interfaces in the track list using the Boolean OR to trigger when all
objects fail.
switch(config)# track 44 list boolean OR

switch(config-track)# object 23
switch(config-track)# end
3 Add this track object to the vPC domain:
switch(config)# vpc domain 1

switch(config-vpc-domain)# track 44
4 Display the track object:
switch# show vpc brief

Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
vPC role : secondary
Number of vPCs configured : 52
Track object : 44
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po100 up 1-5,140
vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- -------------------------- ------------
1 Po1 up success success 1-5,140
This example shows how to display information about the track objects:
switch# show track brief

Track Type Instance Parameter State Last
Change
23 Interface Ethernet8/33 Line Protocol UP 00:03:05
35 Interface Ethernet8/35 Line Protocol UP 00:03:15
44 List ----- Boolean
or UP 00:01:19
55 Interface port-channel100 Line Protocol UP 00:00:34
vPC and LACP

LACP uses the system MAC address of the vPC domain to form the LACP Aggregation Group (LAG) ID
for the vPC. (See the “Configuring Port Channels” chapter for information about LAG-ID and LACP.)
179
Configuring vPCs
You can use LACP on all the vPC port channels, including those channels from the downstream device. We
recommend that you configure LACP with active mode on the interfaces on each port channel on the vPC
peer devices. This configuration allows you to more easily detect compatibility between devices, unidirectional
links, and multihop connection, and provides dynamic reaction to run-time changes and link failures.
We recommend that you manually configure the system priority on the vPC peer link devices to ensure that
the vPC peer link devices have a higher LACP priority than the downstream connected devices. A lower
numerical value system priority means a higher LACP priority.
Note When manually configuring the system priority, you must ensure that you assign the same priority value
on both vPC peer devices. If the vPC peer devices have different system priority values, vPC does not
come up.
vPC Peer Links and STP

Although vPCs provide a loop-free Layer 2 topology, STP is still required to provide a fail-safe mechanism
to protect against any incorrect or defective cabling or possible misconfiguration. When you first bring up a
vPC, STP reconverges. STP treats the vPC peer link as a special link and always includes the vPC peer link
in the STP active topology.
We recommend that you set all the vPC peer link interfaces to the STP network port type so that Bridge
Assurance is automatically enabled on all vPC peer links. We also recommend that you do not enable any of
the STP enhancement features on vPC peer links. If the STP enhancements are already configured, they do
not cause any problems for the vPC peer links..
When you are running both MST and Rapid PVST+, ensure that the PVST simulation feature is correctly
configured.
See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for information about STP
enhancement features and PVST simulation.
Note You must configure a list of parameters to be identical on the vPC peer devices on both sides of the vPC
peer link. See the “Compatibility Parameters for vPC Interfaces” section for information about these
required matched settings.
STP is distributed; that is, the protocol continues running on both vPC peer devices. However, the configuration
on the vPC peer device elected as the primary device controls the STP process for the vPC interfaces on the
secondary vPC peer device.
The primary vPC device synchronizes the STP state on the vPC secondary peer device using Cisco Fabric
Services over Ethernet (CFSoE). See the “vPC and Orphan Ports” section for information about CFSoE.
The STP process for vPC also relies on the periodic keepalive messages to determine when one of the connected
devices on the peer link fails. See the “Peer-Keepalive Link and Messages” section for information about these
messages.
The vPC manager performs a proposal/handshake agreement between the vPC peer devices that set the primary
and secondary devices and coordinates the two devices for STP. The primary vPC peer device then controls
the STP protocol on both the primary and secondary devices. We recommend that you configure the primary
vPC peer device as the STP primary root device and configure the secondary VPC device to be the STP
secondary root device.
180
Configuring vPCs
If the primary vPC peer device fails over to the secondary vPC peer device, there is no change in the STP
topology.
The BPDUs uses the MAC address set for the vPC for the STP bridge ID in the designated bridge ID field.
The vPC primary device sends these BPDUs on the vPC interfaces.
You must configure both ends of vPC peer link with the identical STP configuration for the following
parameters:
• STP global settings:
◦STP mode
◦STP region configuration for MST
◦Enable/disable state per VLAN
◦Bridge Assurance setting
◦Loop Guard settings

◦Loop Guard
◦Root Guard
Note If any of these parameters are misconfigured, the Cisco NX-OS software suspends all interfaces in the
vPC. Check the syslog and enter the show vpc brief command to see if the vPC interfaces are suspended.
Ensure that the following STP interface configurations are identical on both sides of the vPC peer links or
you may see unpredictable behavior in the traffic flow:
• BPDU Filter
• BPDU Guard
• Cost
• Link type
• Priority
• VLANs (PVRST+)
Note Display the configuration on both sides of the vPC peer link to ensure that the settings are identical.
You can use the show spanning-tree command to display information about the vPC when that feature is
enabled. See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for an example.
181
Configuring vPCs
Note We recommend that you configure the ports on the downstream devices as STP edge ports. You should
configure all host ports connected to a switch as STP edge ports. See the Cisco Nexus 9000 Series NX-OS
Layer 2 Switching Configuration Guide for more information about STP port types.
vPC Peer Switch

The vPC peer switch feature was added to Cisco NX-OS to address performance concerns around STP
convergence. This feature allows a pair of Cisco Nexus 9000 Series devices to appear as a single STP root in
the Layer 2 topology. This feature eliminates the need to pin the STP root to the vPC primary switch and
improves vPC convergence if the vPC primary switch fails.
To avoid loops, the vPC peer link is excluded from the STP computation. In vPC peer switch mode, STP
BPDUs are sent from both vPC peer devices to avoid issues related to STP BPDU timeout on the downstream
switches, which can cause traffic disruption.
This feature can be used with the pure peer switch topology in which the devices all belong to the vPC.
Note Peer-switch feature is supported on networks that use vPC and STP-based redundancy is not supported.
If the vPC peer-link fail in a hybrid peer-switch configuration, you can lose traffic. In this scenario, the
vPC peers use the same STP root ID as well as the same bridge ID. The access switch traffic is split in
two with half going to the first vPC peer and the other half to the second vPC peer. With peer link failure,
there is no impact to the north/south traffic but the east/west traffic is lost.
See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for information about STP
enhancement features and Rapid PVST+.
vPC and ARP or ND

A feature was added to Cisco NX-OS to address table synchronization across vPC peers using the reliable
transport mechanism of the Cisco Fabric Service over Ethernet (CFSoE) protocol. You must enable the ip
arp synchronize and ipv6 nd synchronize commands to support faster convergence of address tables between
the vPC peers. This convergence overcomes the delay that occurs in ARP table restoration for IPv4 or ND
table restoration for IPv6 when the peer link port channel flaps or when a vPC peer comes back online.
vPC Multicast—PIM, IGMP, and IGMP Snooping

The Cisco NX-OS software for the Nexus 9000 Series devices does not support Source-Specific Multicast
(SSM) or Bidirectional (BIDR) on a vPC. The Cisco NX-OS software fully supports PIM Any Source Multicast
(ASM) on a vPC.
The software keeps the multicast forwarding state synchronized on both of the vPC peer devices. The IGMP
snooping process on a vPC peer device shares the learned group information with the other vPC peer device
through the vPC peer link; the multicast states are always synchronized on both vPC peer devices. The PIM
process in vPC mode ensures that only one of the vPC peer devices forwards the multicast traffic to the
receivers.
Each vPC peer is a Layer 2 or Layer 3 device. Multicast traffic flows from only one of the vPC peer devices.
You might see duplicate packets in the following scenarios:
182
Configuring vPCs
• Orphan hosts
• When the source and receivers are in the Layer 2 vPC cloud in different VLANs with multicast routing
enabled and a vPC member link goes down.
You might see negligible traffic loss in the following scenarios:

• When you reload the vPC peer device that is forwarding the traffic.
• When you restart PIM on the vPC peer device that is forwarding the traffic.
Ensure that you dual-attach all Layer 3 devices to both vPC peer devices. If one vPC peer device goes down,
the other vPC peer device continues to forward all multicast traffic normally.
The following outlines vPC PIM and vPC IGMP/IGMP snooping:
• vPC PIM—The PIM process in vPC mode ensures that only one vPC peer device forwards multicast
traffic. The PIM process in vPC mode synchronizes the source state with both vPC peer devices and
elects which vPC peer device forwards the traffic.
• vPC IGMP/IGMP snooping—The IGMP process in vPC mode synchronizes the designated router (DR)
information on both vPC peer devices. Dual DRs are available for IGMP when you are in vPC mode.
Dual DRs are not available when you are not in vPC mode, because both vPC peer devices maintain the
multicast group information between the peers.
Note A PIM neighbor relationship between a vPC VLAN (a VLAN that is carried on a vPC peer link) and a
downstream vPC-attached Layer 3 device is not supported, which can result in dropped multicast packets.
If a PIM neighbor relationship is required with a downstream Layer 3 device, a physical Layer 3 interface
must be used instead of a vPC interface.
You should enable or disable IGMP snooping identically on both vPC peer devices, and all the feature
configurations should be identical. IGMP snooping is on by default.
Note The following commands are not supported in vPC mode:

• ip pim spt-threshold infinity
• ip pim use-shared-tree-only
See the Cisco Nexus 9000 Series NX-OS Multicast Routing Configuration Guide for more information about
multicasting.
Multicast PIM Dual DR (Proxy DR )

By default, a multicast router sends PIM joins upstream only if it has interested receivers. These interested
receivers can either be IGMP hosts (they communicate through IGMP reports) or other multicast routers (they
communicate through PIM joins).
In the Cisco NX-OS vPC implementation, PIM works in dual designated router (DR) mode. That is, if a vPC
device is a DR on a vPC SVI outgoing interface (OIF), its peer automatically assumes the proxy DR role.
183
Configuring vPCs
IGMP adds an OIF (the report is learned on that OIF) to the forwarding if the OIF is a DR. With dual DRs,
both vPC devices have an identical (*,G) entry with respect to the vPC SVI OIFs as shown in this example:
VPC Device1:
------------
(*,G)
oif1 (igmp)
VPC Device2:
------------
(*,G)
oif1 (igmp)
IP PIM PRE-BUILD SPT

When the multicast source is in a Layer 3 cloud (outside the vPC domain), one vPC peer is elected as the
forwarder for the source. This forwarder election is based on the metrics to reach the source. If there is a tie,
the vPC primary is chosen as the forwarder. Only the forwarder has the vPC OIFs in its associated (S,G) and
the nonforwarder (S,G) has 0 OIFs. Therefore, only the forwarder sends PIM (S,G) joins toward the source
as shown in this example:
VPC Device1 (say this is Forwarder for Source 'S'):

------------
(*,G)
oif1 (igmp)
(S,G)
oif1 (mrib)
VPC Device2:
------------
(*,G)
oif1 (igmp)
(S,G)
NULL
In the case of a failure (for example, a Layer 3 Reverse Path Forwarding (RPF) link on the forwarder becomes
inoperable or the forwarder gets reloaded), if the current nonforwarder ends up becoming the forwarder, it
has to start sending PIM joins for (S,G) toward the source to pull the traffic. Depending upon the number of
hops to reach the source, this operation might take some time (PIM is a hop-by-hop protocol).
To eliminate this issue and get better convergence, use the ip pim pre-build-spt command. This command
enables PIM send joins even if the multicast route has 0 OIFs. In a vPC device, the nonforwarder sends PIM
(S,G) joins upstream toward the source. The downside is that the link bandwidth upstream from the
nonforwarder gets used for the traffic that is ultimately dropped by it. The benefits that result with better
convergence far outweigh the link bandwidth usage. Therefore, we recommend that you use this command
if you use vPCs.
vPC Peer Links and Routing

The First Hop Redundancy Protocols (FHRPs) interoperate with vPCs. The Hot Standby Routing Protocol
(HSRP), and Virtual Router Redundancy Protocol (VRRP) all interoperate with vPCs. We recommend that
you dual-attach all Layer 3 devices to both vPC peer devices.
The primary FHRP device responds to ARP requests, even though the secondary vPC device forwards the
data traffic.
To simplify initial configuration verification and vPC/HSRP troubleshooting, you can configure the primary
vPC peer device with the FHRP active router highest priority.
184
Configuring vPCs
In addition, you can use the priority command in the if-hsrp configuration mode to configure failover thresholds
for when a group state enabled on a vPC peer link is in standby or in listen state. You can configure lower
and upper thresholds to prevent the interface from going up and down.
VRRP acts similarly to HSRP when running on vPC peer devices. You should configure VRRP the same way
that you configure HSRP.
When the primary vPC peer device fails over to the secondary vPC peer device, the FHRP traffic continues
to flow seamlessly.
We recommend that you configure routing adjacency between the two vPC peer devices to act as a backup
routing path. If one vPC peer device loses Layer 3 uplinks, the vPC can redirect the routed traffic to the other
vPC peer device and leverage its active Layer 3 uplinks.
You can configure the inter-switch link for a backup routing path in the following ways:
• Create a Layer 3 link between the two vPC peer devices.
• Use the non-VPC VLAN trunk with a dedicated VLAN interface.
• Use a vPC peer link with a dedicated VLAN interface.
We do not recommend that you configure the burnt-in MAC address option (use-bia) for HSRP or manually
configure virtual MAC addresses for any FHRP protocol in a vPC environment because these configurations
can adversely affect vPC load balancing. The HSRP use-bia option is not supported on vPCs. When you are
configuring custom MAC addresses, you must configure the same MAC address on both vPC peer devices.
You can use the delay restore command to configure a restore timer that delays the vPC coming back up
until after the peer adjacency forms and the VLAN interfaces are back up. This feature enables you to avoid
packet drops when the routing tables might not be converged before the vPC is once again passing traffic.
Use the delay restore command to configure this feature.
To delay the VLAN interfaces on the restored vPC peer device from coming up, use the interfaces-vlan
option of the delay restore command.
See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for more information about
FHRPs and routing.
Best Practices for Layer 3 and vPC Configuration

This section describes best practices for using and configuring Layer 3 with vPC.
Layer 3 and vPC Configuration Overview

When a Layer 3 device is connected to a vPC domain through a vPC, it has the following views:
• At Layer 2, the Layer 3 device sees a unique Layer 2 switch presented by the vPC peer devices.
• At Layer 3, the Layer 3 device sees two distinct Layer 3 devices (one for each vPC peer device).
vPC is a Layer 2 virtualization technology, so at Layer 2, both vPC peer devices present themselves as a
unique logical device to the rest of the network.
There is no virtualization technology at Layer 3, so each vPC peer device is seen as a distinct Layer 3 device
by the rest of the network.
185
Configuring vPCs
The following figure illustrates the two different Layer 2 and Layer 3 views with vPC.
Figure 16: Different Views for vPC Peer Devices
Guidelines for Layer 3 and vPC Configurations

To connect Layer 3 devices to a vPC domain, use Layer 3 links from Layer 3 devices to connect each vPC
peer device.
Note The vPC loop avoidance rule does not allow the attachment of a Layer 3 device to a vPC domain using a
vPC.
Layer 3 devices are able to initiate Layer 3 routing protocol adjacencies with both vPC peer devices.
One or multiple Layer 3 links can be used to connect a Layer 3 device to each vPC peer device. Cisco Nexus
9000 series devices support Layer 3 Equal Cost Multipathing (ECMP) with up to 16 hardware load-sharing
paths per prefix. Traffic from a vPC peer device to a Layer 3 device can be load-balanced across all the Layer
3 links interconnecting the two devices together.
Using Layer 3 ECMP on the Layer 3 device can effectively use all Layer 3 links from the device to the vPC
domain. Traffic from a Layer 3 device to the vPC domain can be load-balanced across all the Layer 3 links
interconnecting the two entities together.
186
Configuring vPCs
The supported connection model for a Layer 3 device to the vPC domain is illustrated in the following figure.
Figure 17: Using Separate Layer 3 Links to Connect L3 Device to a vPC Domain
Follow these guidelines when connecting a Layer 3 device to the vPC domain:
• Use separate Layer 3 links to connect Layer 3 devices to the vPC domain.
• Do not use a Layer 2 vPC to attach a Layer 3 device to a vPC domain unless the Layer 3 device can
statically route to the HSRP address configured on the vPC peer devices.
• When both routed and bridged traffic are required, use individual Layer 3 links for routed traffic and a
separate Layer 2 port-channel for bridged traffic when both routed and bridged traffic are required.
• Enable Layer 3 connectivity between vPC peer device by configuring a VLAN network interface for
the same VLAN from both devices or by using a dedicated Layer 3 link between the two peer devices
(for Layer 3 backup routing path purposes).
187
Configuring vPCs
Example Topologies for Layer 3 and vPC

This section contains examples of network topologies for Layer 3 and vPC.
Figure 18: Legend
Peering Between Routers

In this example, vPC is used as a Layer 2 transit path. Because there is no direct routing protocol peering
adjacency from the Layer 3 device to any vPC peer device, this topology is supported.
Figure 19: Peering Between Routers
Peering with an External Router Using Layer 3 Links

This example shows a topology that uses Layer 3 links to connect a Layer 3 device to the vPC domain.
188
Configuring vPCs
Note Interconnecting the 2 entities together in this way is a best practice.
Figure 20: Peering with an External Router Using Layer 3 Links
Peering Between vPC Devices for a Backup Routing Path

This example shows peering between the two vPC peer devices with a Layer 3 backup routed path. If the
Layer 3 uplinks on vPC peer device 1 or vPC peer device 2 fail, the path between the two peer devices is used
to redirect traffic to the switch that has the Layer 3 uplinks in the up state.
The Layer 3 backup routing path can be implemented using a dedicated interface VLAN (such as SVI) over
the vPC peer-link or by using dedicated Layer 2 or Layer 3 links across the two vPC peer devices.
Figure 21: Peering Between vPC Devices for a Backup Routing Path
189
Configuring vPCs
Peering Between Two Routers with vPC Devices as Transit Switches

This example is similar to the Peering between Routers topology. The difference here is that the vPC domains
are only used as Layer 2 transit paths.
Figure 22: Peering Between Two Routers with vPC Devices as Transit Switches
Peering with an External Router on Parallel Interconnected Routed Ports

This example shows the Layer 3 device attached to the vPC domain through two different types of links,
Layer 2 links and Layer 3 links.
The Layer 2 links are used for bridged traffic (traffic staying in the same VLAN) or inter-VLAN traffic
(assuming vPC domain hosts the interface VLAN and associated HSRP configuration).
The Layer 3 links are used for routing protocol peering adjacency with each vPC peer device.
The purpose of this topology is to attract specific traffic to go through the Layer 3 device. Layer 3 links are
also used to carry routed traffic from a Layer 3 device to the vPC domain.
Figure 23: Peering with an External Router on Parallel Interconnected Routed Ports
Peering Over a vPC Interconnection on Parallel Interconnected Routed Ports

When routing protocol peering adjacency is required to be established between the two data centers, a best
practice is to add dedicated Layer 3 links between the two sites as shown in this example.
190
Configuring vPCs
The vPC link between the two data centers carry bridged traffic or inter-VLAN traffic while the dedicated
Layer 3 links carry the routed traffic across the two sites.
Figure 24: Peering Over a vPC Interconnection on Parallel Interconnected Routed Ports
Peering Over a PC Interconnection and Dedicated Interswitch Link Using non-vPC VLAN
This example shows when the Layer 3 device is single-attached to the vPC domain, you can use a non-vPC
VLAN with a dedicated inter-switch link to establish the routing protocol peering adjacency between the
Layer 3 device and each vPC peer device. However, the non-vPC VLAN must be configured to use a static
MAC that is different than the vPC VLAN.
Note Configuring the vPC VLAN (and vPC peer-link) for this purpose is not supported.
Figure 25: Peering Over a PC Interconnection and Dedicated Interswitch Link Using non-vPC VLAN
191
Configuring vPCs
CFSoE
The Cisco Fabric Services over Ethernet (CFSoE) is a reliable state transport mechanism that is used to
synchronize the actions of the vPC peer devices. CFSoE carries messages and packets for many features linked
with vPC, such as STP and IGMP. Information is carried in CFS/CFSoE protocol data units (PDUs).
When you enable the vPC feature, the device automatically enables CFSoE, and you do not have to configure
anything. CFSoE distributions for vPCs do not need the capabilities to distribute over IP or the CFS regions.
You do not need to configure anything for the CFSoE feature to work correctly on vPCs.
The CFSoE transport is local to each VDC.
You can use the show mac address-table command to display the MAC addresses that CFSoE synchronizes
for the vPC peer link.
Note Do not enter the no cfs eth distribute or the no cfs distribute command. You must enable CFSoE for
vPC functionality. If you do enter either of these commands with vPC enabled, the system displays an
error message.
When you enter the show cfs application command, the output displays “Physical-eth,” which shows the
applications that are using CFSoE.
CFS also transports data over TCP/IP. See the Cisco Nexus 9000 Series NX-OS System Management
Configuration Guide for more information about CFS over IP.
Note The software does not support CFS regions.
vPC and Orphan Ports

When a device that is not vPC-capable connects to each peer, the connected ports are known as orphan ports
because they are not members of a vPC. The device’s link to one peer will be active (forwarding) and the
other link will be standby (blocking) due to STP.
If a peer link failure or restoration occurs, an orphan port’s connectivity might be bound to the vPC failure or
restoration process. For example, if a device’s active orphan port connects to the secondary vPC peer, the
device loses any connections through the primary peer if a peer link failure occurs and the vPC ports are
suspended by the secondary peer. If the secondary peer were to also suspend the active orphan port, the device’s
standby port becomes active, provides a connection to the primary peer, and restores connectivity. You can
configure in the CLI that specific orphan ports are suspended by the secondary peer when it suspends its vPC
ports and are restored when the vPC is restored.
All ports in a given vPC must be in the same VDC. This version of the software supports only one vPC domain
per VDC. You can use the numbers from 1 to 4096 in each VDC to number the vPC.
192
Configuring vPCs
vPC Recovery After an Outage
vPC Recovery After an Outage

In a data center outage, both of the Cisco Nexus 9000 Series devices that include a vPC get reloaded.
Occasionally only one peer can be restored. With no functioning peer-keepalive or peer link, the vPC cannot
function normally, but depending on your Cisco NX-OS release, a method might be available to allow vPC
services to use only the local ports of the functional peer.
Autorecovery
You can configure the Cisco Nexus 9000 Series device to restore vPC services when its peer fails to come
online by using the auto-recovery command. You must save this setting in the startup configuration. On
reload, if the peer link is down and three consecutive peer-keepalive messages are lost, the secondary device
assumes the primary STP role and the primary LACP role. The software reinitializes the vPCs, bringing up
its local ports. Because there are no peers, the consistency check is bypassed for the local vPC ports. The
device elects itself to be the STP primary regardless of its role priority and also acts as the master for LACP
port roles.
vPC Peer Roles After a Recovery

When the other peer device completes its reload and adjacency forms, the following process occurs:
1 The first vPC peer maintains its current role to avoid any transition reset to other protocols. The peer
accepts the other available role.
2 When an adjacency forms, consistency checks are performed and appropriate actions are taken.
High Availability
During an In-Service Software Upgrade (ISSU), the software reload process on the first vPC device locks its
vPC peer device by using CFS messaging over the vPC communications channel. Only one device at a time
is upgraded. When the first device completes its upgrade, it unlocks its peer device. The second device then
performs the upgrade process, locking the first device as it does so. During the upgrade, the two vPC devices
temporarily run different releases of Cisco NX-OS, however the system functions correctly because of its
backward compatibility support.
about high-availability features.
vPC Forklift Upgrade Scenario

The following describes a scenario for migrating from a pair of Cisco Nexus 9000 Series switches in a vPC
topology to a different pair of Cisco Nexus 9000 Series switches. Typically the scenario might be migrating
from a pair of Cisco Nexus 9508 vPC peer nodes to a pair of Cisco Nexus 9516 switches.
Considerations for a vPC forklift upgrade:
193
Configuring vPCs
• vPC Role Election and Sticky-bit

When the two vPC systems are joined to form a vPC domain, priority decides which device is the vPC
primary and which is the vPC secondary. When the primary device is reloaded, the system comes back
online and connectivity to the vPC secondary device (now the operational primary) is restored. The
operational role of the secondary device (operational primary) does not change (to avoid unnecessary
disruptions). This behavior is achieved with a sticky-bit, where the sticky information is not saved in
the startup configuration. This method makes the device that is up and running win over the reloaded
device. Hence, the vPC primary becomes the vPC operational secondary. Sticky-bit is also set when a
vPC node comes up with peer-link and peer-keepalive down and it becomes primary after the auto
recovery period.
• vPC Delay Restore
The delay restore timer is used to delay the vPC from coming up on the restored vPC peer device after
a reload when the peer adjacency is already established.
To delay the VLAN interfaces on the restored vPC peer device from coming up, use the interfaces-vlan
option of the delay restore command.
• vPC Auto-Recovery
During a data center power outage when both vPC peer switches go down, if only one switch is restored,
the auto-recovery feature allows that switch to assume the role of the primary switch and the vPC links
come up after the auto-recovery time period. The default auto-recovery period is 240 seconds.
The following example is a migration scenario that replaces vPC peer nodes Node1 and Node2 with New_Node1
and New_Node2.
Migration Step Expected Behavior Node1 Node1 Node2 Node2

Configured Operational Configured Operational
role ( Ex: role role (Ex: role
role priority role priority
100) 200)
1 Initial state Traffic is primary Primary secondary Secondary
forwarded by both Sticky bit: Sticky bit:
vPC peers – False False
Node1 and Node2.
Node1 is primary
and Node2 is
secondary.
2 Node2 replacement – Traffic converged primary Primary secondary Secondary

Shut all vPCs and on Primary vPC Sticky bit: Sticky bit:
uplinks on Node2. peer Node1. False False
Peer-link and vPC
peer-keepalive are in
administrative up
state.
3 Remove Node2. Node1 will primary Primary n/a n/a

continue to Sticky bit:
forward traffic. False
194
Configuring vPCs
Migration Step Expected Behavior Node1 Node1 Node2 Node2

Configured Operational Configured Operational
role ( Ex: role role (Ex: role
role priority role priority
100) 200)
4 Configure New_Node2 will primary Primary secondary Secondary
New_Node2. Copy come up as Sticky bit: Sticky bit:
the configuration to secondary. Node1 False False
startup config. vPC continue to be
peer-link and primary.
peer-keepalive in Traffic will
administrative up continue to be
state. forwarded on
Power off Node01.
New_Node2.
Make all connections.
Power on
New_Node2.
5 Bring up all vPCs Traffic will be primary Primary secondary Secondary

and uplink ports on forwarded by both Sticky bit: Sticky bit:
New_Node2. Node 1 and False False
New_Node2.
6 Node1 replacement - Traffic will primary Primary secondary Secondary

converge on
Shut vPCs and Sticky bit: Sticky bit:
New_Node2.
uplinks on Node1. False False
7 Remove Node1. New_Node2 will n/a n/a secondary Primary

become secondary, Sticky bit:
operational True
primary and sticky
bit will be set to
True.
8 Configure New_Node1 will primary Secondary secondary Primary

New_Node1. Copy come up as Sticky bit: Sticky bit:
running to startup. primary, False True
operational
Power off the new
secondary.
Node1. Make all
connections. Power
on New_Node1.
9 Bring up all vPCs Traffic will be primary Secondary secondary Primary

and uplink ports on forwarded by both Sticky bit: Sticky bit:
New_Node1. New Node1 and False True
new Node2.
195
Configuring vPCs
Licensing Requirements for vPCs
Note If you prefer to have the configured secondary node as the operational secondary and the configured
primary as the operational primary, then Node2 can be reloaded at the end of the migration. This is optional
and does not have any functional impact.
Licensing Requirements for vPCs


Cisco NX-OS Cisco NX-OS vPCs require no license. Any feature not included in a license
package is bundled with the Cisco NX-OS system images and is provided at
no extra charge to you.

vPCs have the following configuration guidelines and limitations:
• vPC peers can only operate dissimilar versions of NX-OS software during the upgrade or downgrade
process.
• vPC peers that run different versions out of the upgrade/downgrade period are not supported.
• All ports for a given vPC must be in the same VDC.
• You must enable vPCs before you can configure them.
• You must configure the peer-keepalive link and messages before the system can form the vPC peer link.
• Only Layer 2 port channels can be in vPCs.
• You must configure both vPC peer devices; the configuration is not sent from one device to the other.
• To configure multilayer (back-to-back) vPCs, you must assign unique vPC domain ID for each respective
vPC.
• Check that the necessary configuration parameters are compatible on both sides of the vPC peer link.
See the “Compatibility Parameters for vPC Interfaces” section for information about compatibility
recommendations.
• You might experience minimal traffic disruption while configuring vPCs.
• The software does not support BIDR PIM or SSM on vPCs.
• The software does not support DHCP snooping, DAI, or IPSG in a vPC environment; DHCP Relay is
supported.
196
Configuring vPCs
Default Settings
• The software does not support CFS regions.

• Port security is not supported on port channels.
• We recommend that you configure all the port channels in the vPC using LACP with the interfaces in
active mode.
• Configure a separate Layer 3 link for routing from the vPC peer devices, rather than using a VLAN
network interface for this purpose.
• Back-to-back, multilayer vPC topologies require unique domain IDs on each respective vPC.
• Layer 3 over vPC configurations are not supported.
• Having the same Hot Standby Router Protocol (HSRP)/Virtual Router Redundancy Protocol (VRRP)
group on all nodes on a double sided vPC is supported on Cisco NX-OS 7.0(3)I2(1) and later releases.
• When migrating from a pair of spine nodes to a pair of Cisco Nexus 9000 devices, the HSRP priority
should be configured so that the Cisco Nexus 9000 vPC peers are in Active/Standby state. There is no
support for Cisco Nexus 9000 vPC peers in HSRP state to be in Active/Listen state, or Standby/Listen
state. (7.(0)I2(2) or later)
• When using vPCs, we recommend that you use default timers for FHRP (HSRP, VRRP), and PIM
configurations. There is no advantage in convergence times when using aggressive timers in vPC
configurations.
• If you configure open shortest path first (OSPF) in a vPC environment, use the following timer commands
in router configuration mode on the core switch to ensure fast OSPF convergence when a vPC peer link
is shut down:
switch (config-router)# timers throttle spf 1 50 50

switch (config-router)# timers lsa-arrival 10
See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for further details about
OSPF.
• BFD for HSRP is not supported in a vPC environment.
• The STP port cost is fixed to 200 in a vPC environment.
• Jumbo frames are enabled by default on the vPC peer link.
• To accommodate increased traffic when the vPC goes down and traffic needs to cross the peer-link, it
is a best practice to use multiple high bandwidth interfaces (such as the 40G interfaces for the Cisco
Nexus 9000) across linecards for the peer-link.
• The vpc orphan-ports suspend command also applies to ports in non-vPC VLANs and Layer 3 ports.
However, it is recommended to be used with ports in VPC vlans.
• FEX vPC is not supported between FEX (any model) and the Cisco Nexus 9300 (TOR) and Cisco Nexus
9500 (EOR) Series switches as the parent switches.
Default Settings
The following table lists the default settings for vPC parameters.
197
Configuring vPCs
Configuring vPCs
Table 12: Default vPC Parameters
Parameters Default
vPC system priority 32667
vPC peer-keepalive message Disabled
vPC peer-keepalive interval 1 second
vPC peer-keepalive timeout 5 seconds
vPC peer-keepalive UDP port 3200
Configuring vPCs
Note You must use these procedures on both devices on both sides of the vPC peer link. You configure both
of the vPC peer devices using these procedures.
This section describes how to configure vPCs using the command-line interface (CLI).
Enabling vPCs
You must enable the vPC functionality before you can configure and use vPCs.
SUMMARY STEPS
2. feature vpc
3. exit
4. show feature
198
Configuring vPCs
Disabling vPCs
DETAILED STEPS

Example:
switch(config)#
Step 2 feature vpc Enables vPCs on the device.
Example:
switch(config)# feature vpc
Example:
switch#
Step 4 show feature (Optional) Displays which features are enabled on the
device.
Example:
switch# show feature
Example:
switch# copy running-config startup-config
This example shows how to enable the vPC feature:

switch(config)# feature vpc
switch(config)#
Disabling vPCs
Note When you disable the vPC functionality, the device clears all the vPC configurations.
SUMMARY STEPS
2. no feature vpc
3. exit
4. show feature
199
Configuring vPCs
Creating a vPC Domain and Entering vpc-domain Mode
DETAILED STEPS

Example:
switch(config)#
Step 2 no feature vpc Disables vPCs on the device.
Example:
switch(config)# no feature vpc
Example:
switch#
Step 4 show feature (Optional) Displays which features are enabled on

the device.
Example:
switch# show feature
Example:
This example shows how to disable the vPC feature:

switch(config)# no feature vpc
switch#

You can create a vPC domain and put the vPC peer link port channels into the identical vPC domain on both
vPC peer devices. Use a unique vPC domain number throughout a single VDC. This domain ID is used to
automatically to form the vPC system MAC address.
You can also use this command to enter vpc-domain command mode.
200
Configuring vPCs
SUMMARY STEPS
2. vpc domain domain-id
3. exit
4. show vpc brief
DETAILED STEPS

Example:
switch(config)#
Step 2 vpc domain domain-id Creates a vPC domain on the device, and enters vpc-domain
configuration mode for configuration purposes. There is
Example: no default; the range is from 1 to 1000.
switch(config-vpc-domain)#
Step 3 exit Exits vpc-domain configuration mode.
Example:
switch#
Step 4 show vpc brief (Optional) Displays brief information about each vPC
domain.
Example:
configuration.
Example:
This example shows how to enter the vpc-domain command mode to configure an existing vPC domain:

switch(config-vpc-domain)# exit
switch(config)#
201
Configuring vPCs
Configuring a vPC Keepalive Link and Messages
Configuring a vPC Keepalive Link and Messages
Note You must configure the vPC peer-keepalive link before the system can form the vPC peer link.
You can configure the destination IP for the peer-keepalive link that carries the keepalive messages. Optionally,
you can configure other parameters for the keepalive messages.
Note We recommend that you configure a separate VRF instance and put a Layer 3 port from each vPC peer
device into that VRF for the vPC peer-keepalive link. Do not use the peer link itself to send vPC
peer-keepalive messages. For information about creating and configuring VRFs, see the Cisco Nexus
9000 Series NX-OS Unicast Routing Configuration Guide. Ensure that both the source and destination
IP addresses use for the peer-keepalive message are unique in your network. The management port and
management VRF are the defaults for these keepalive messages.
Before You Begin

Ensure that you have enabled the vPC feature.
SUMMARY STEPS
3. peer-keepalive destination ipaddress [hold-timeout secs | interval msecs {timeout secs} | {precedence
{prec-value | network | internet | critical | flash-override | flash | immediate priority | routine}} | tos
{tos-value | max-reliability | max-throughput | min-delay | min-monetary-cost | normal}} |tos-byte
tos-byte-value} | source ipaddress | vrf {name | management vpc-keepalive}]
4. exit
5. show vpc statistics
DETAILED STEPS

Example:
switch(config)#
Step 2 vpc domain domain-id Creates a vPC domain on the device, and enters vpc-domain
configuration mode.
Example:
202
Configuring vPCs
Creating a vPC Peer Link

Step 3 peer-keepalive destination ipaddress [hold-timeout secs Configures the IPv4 address for the remote end of the vPC
| interval msecs {timeout secs} | {precedence {prec-value peer-keepalive link.
| network | internet | critical | flash-override | flash | Note The system does not form the vPC peer link until
immediate priority | routine}} | tos {tos-value | you configure a vPC peer-keepalive link.
max-reliability | max-throughput | min-delay | The management ports and VRF are the defaults.
min-monetary-cost | normal}} |tos-byte tos-byte-value}
| source ipaddress | vrf {name | management Note We recommend that you configure a separate VRF
vpc-keepalive}] and use a Layer 3 port from each vPC peer device
in that VRF for the vPC peer-keepalive link. For
more information about creating and configuring
Example:
switch(config-vpc-domain)# peer-keepalive VRFs, see the Cisco Nexus 9000 Series NX-OS
destination 172.28.230.85 Unicast Routing Configuration Guide.
Example:
switch#
Step 5 show vpc statistics (Optional) Displays information about the configuration for
the keepalive messages.
Example:
switch# show vpc statistics
configuration.
Example:
For more information about configuring VRFs, see the Cisco Nexus 9000 Series NX-OS Unicast Routing
Configuration Guide.
This example shows how to configure the destination and source IP address and VRF for the vPC-peer-keepalive
link:
switch(config-vpc-domain)# peer-keepalive destination 172.168.1.2 source 172.168.1.1 vrf
vpc-keepalive
switch#

You create the vPC peer link by designating the port channel that you want on each device as the peer link
for the specified vPC domain. We recommend that you configure the Layer 2 port channels that you are
designating as the vPC peer link in trunk mode and that you use two ports on separate modules on each vPC
peer device for redundancy.
203
Configuring vPCs
Before You Begin

SUMMARY STEPS
3. switchport mode trunk
4. switchport trunk allowed vlan vlan-list
5. vpc peer-link
6. exit
7. show vpc brief
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel channel-number Selects the port channel that you want to use as the vPC
peer link for this device, and enters interface
Example: configuration mode.
switch(config-if)#
Step 3 switchport mode trunk (Optional) Configures this interface in trunk mode.
Example:
Step 4 switchport trunk allowed vlan vlan-list (Optional) Configures the permitted VLAN list.
Example:
switch(config-if)# switchport trunk
allowed vlan 1-120,201-3967
Step 5 vpc peer-link Configures the selected port channel as the vPC peer link,
and enters vpc-domain configuration mode.
Example:
switch(config-if)# vpc peer-link
Example:
switch#
204
Configuring vPCs
Configuring a vPC Peer-Gateway

Step 7 show vpc brief (Optional) Displays information about each vPC,
including information about the vPC peer link.
Example:
configuration.
Example:
This example shows how to configure a vPC peer link:

switch(config-if)# switchport mode
switch(config-if)# switchport trunk allowed vlan 1-120,201-3967
switch(config)#
Configuring a vPC Peer-Gateway

You can configure vPC peer devices to act as the gateway for packets that are destined to the vPC peer device's
MAC address.
Note When you attach a Layer 3 device to a vPC domain, the peering of routing protocols using a VLAN also
carried on the vPC peer-link is not supported. If routing protocol adjacencies are needed between vPC
peer devices and a generic Layer 3 device, you must use physical routed interfaces for the interconnection.
Use of the vPC peer-gateway feature does not change this requirement.
Before You Begin

SUMMARY STEPS
3. peer-gateway
4. exit
5. show vpc brief
205
Configuring vPCs
Configuring a Graceful Consistency Check
DETAILED STEPS

Example:
switch(config)#
Step 2 vpc domain domain-id Creates a vPC domain if it does not already exist, and
enters vpc-domain configuration mode.
Example:
switch(config-if)# vpc domain 5
Step 3 peer-gateway Enables Layer 3 forwarding for packets destined to the

peer's gateway MAC address.
Example:
switch(config-vpc-domain)# peer-gateway
Note Disable IP redirects on all interface-vlans of this
vPC domain for correct operation of this feature.
Example:
switch#
Step 5 show vpc brief (Optional) Displays information about each vPC,
including information about the vPC peer link.
Example:
Example:

You can configure the graceful consistency check feature, which is enabled by default. Unless this feature is
enabled, the vPC is completely suspended when a mismatch in a mandatory compatibility parameter is
introduced in a working vPC. When this feature is enabled, only the links on the secondary peer device are
suspended. See the “Compatibility Parameters for vPC Interfaces” section for information about consistent
configurations on the vPCs.
206
Configuring vPCs
SUMMARY STEPS
3. graceful consistency-check
4. exit
5. show vpc brief
DETAILED STEPS

Example:
switch(config)#
Step 2 vpc domain domain-id Creates a vPC domain if it does not already exist, and enters
vpc-domain configuration mode.
Example:
switch(config-if)# vpc domain 5
Step 3 graceful consistency-check Specifies that only the links on the secondary peer device are
suspended when a mismatch is detected in a mandatory
Example: compatibility parameter.
switch(config-vpc-domain)# graceful
consistency-check
Use the no form of this command to disable the feature.
Example:
switch#
Step 5 show vpc brief (Optional) Displays information on the vPCs.
Example:
This example shows how to enable the graceful consistency check feature:
switch(config-vpc-domain)# graceful consistency-check
switch(config)#
207
Configuring vPCs
Checking the Configuration Compatibility on a vPC Peer Link
Checking the Configuration Compatibility on a vPC Peer Link

After you have configured the vPC peer link on both vPC peer devices, check that the configurations are
consistent on all vPC interfaces. See the “Compatibility Parameters for vPC Interfaces” section for information
about consistent configurations on the vPCs.
SUMMARY STEPS
2. show vpc consistency-parameters {global | interface port-channel channel-number}
DETAILED STEPS

Example:
switch(config)#
Step 2 show vpc consistency-parameters {global | interface (Optional) Displays the status of those parameters
port-channel channel-number} that must be consistent across all vPC interfaces.
Example:
switch(config)# show vpc consistency-parameters global
switch(config)#
This example shows how to check that the required configurations are compatible across all the vPC interfaces:
switch(config)# show vpc consistency-parameters global
switch(config)#
Note Messages regarding the vPC interface configuration compatibility are also logged to the syslog.

We recommend that you attach the vPC domain downstream port channel to two devices for redundancy.
To connect to the downstream device, you create a port channel from the downstream device to the primary
vPC peer device and you create another port channel from the downstream device to the secondary peer device.
device. You will experience minimal traffic disruption when you are creating vPCs.
Before You Begin

208
Configuring vPCs
Ensure that you are using a Layer 2 port channel.
SUMMARY STEPS
3. vpc number
4. exit
5. show vpc brief
DETAILED STEPS

Example:
switch(config)#
Step 2 interface port-channel channel-number Selects the port channel that you want to put into the vPC to
connect to the downstream device, and enters interface
Example: configuration mode.
switch(config-if)#
Step 3 vpc number Configures the selected port channel into the vPC to connect to
the downstream device. You can use any module in the device
Example: for these port channels. The range is from 1 and 4096.
switch(config-if)# vpc 5
switch(config-vpc-domain)# Note The vPC number that you assign to the port channel
connecting to the downstream device from the vPC peer
device must be identical on both vPC peer devices.
Example:
switch#
Step 5 show vpc brief (Optional) Displays information on the vPCs.
Example:
configuration.
Example:
209
Configuring vPCs
Manually Configuring a vPC Domain MAC Address
This example shows how to configure a port channel to connect to the downstream device:
switch(config)#
Manually Configuring a vPC Domain MAC Address

When you create a vPC domain, the Cisco NX-OS software automatically creates a vPC system MAC address,
which is used for operations that are confined to the link-scope, such as LACP. However, you might choose
to configure the vPC domain MAC address manually.
Before You Begin

SUMMARY STEPS
3. system-mac mac-address
4. exit
5. show vpc role
DETAILED STEPS

Example:
switch(config)#
Step 2 vpc domain domain-id Enters the vPC domain number that you want to
configure. The system enters vpc-domain configuration
Example: mode.
Step 3 system-mac mac-address Enters the MAC address that you want for the specified
vPC domain in the following format: aaaa.bbbb.cccc.
Example:
switch(config-vpc-domain)# system-mac
23fb.4ab5.4c4e
210
Configuring vPCs
Manually Configuring the System Priority

Example:
switch#
Step 5 show vpc role (Optional) Displays the vPC system MAC address.
Example:
configuration.
Example:
This example shows how to manually configure a vPC domain MAC address:
switch(config-vpc-domain)# system-mac 13gb.4ab5.4c4e
switch(config)#
Manually Configuring the System Priority

When you create a vPC domain, the system automatically creates a vPC system priority. However, you can
also manually configure a system priority for the vPC domain.
Note We recommend that you manually configure the vPC system priority when you are running LACP to
ensure that the vPC peer devices are the primary devices on LACP. When you manually configure the
system priority, ensure that you configure the same priority value on both vPC peer devices. If these values
do not match, vPC does not come up.
Before You Begin

SUMMARY STEPS
3. system-priority priority
4. exit
5. show vpc role
211
Configuring vPCs
Manually Configuring the vPC Peer Device Role
DETAILED STEPS

Example:
switch(config)#
Step 2 vpc domain domain-id Enters the vPC domain number that you want to configure.
The system enters vpc-domain configuration mode.
Example:
Step 3 system-priority priority Enters the system priority that you want for the specified
vPC domain. The range of values is from 1 to 65535. The
Example: default value is 32667.
switch(config-vpc-domain)# system-priority 4000
Example:
switch#
Step 5 show vpc role (Optional) Displays the vPC system priority.
Example:
switch# show vpc role
configuration.
Example:
This example shows how to manually configure the vPC domain system priority:
switch(config-vpc-domain)# system-priority 4000
switch(config)#

By default, the Cisco NX-OS software elects a primary and secondary vPC peer device after you configure
the vPC domain and both sides of the vPC peer link. However, you might want to elect a specific vPC peer
device as the primary device for the vPC. Then, you would manually configure the role value for the vPC
peer device that you want as the primary device to be lower than the other vPC peer device.
212
Configuring vPCs
vPCs do not support role preemption. If the primary vPC peer device fails, the secondary vPC peer device
takes over to become operationally the vPC primary device. However, the original operational roles are not
restored if the formerly primary vPC comes up again.
Before You Begin

SUMMARY STEPS
3. role priority priority
4. exit
5. show vpc role
DETAILED STEPS

Example:
switch(config)#
Step 2 vpc domain domain-id Enters the vPC domain number that you want to configure.
The system enters vpc-domain configuration mode.
Example:
Step 3 role priority priority Enters the role priority that you want for the vPC system
priority. The range of values is from 1 to 65636, and the
Example: default value is 32667. A lower value means that this switch
switch(config-vpc-domain)# role priority 4 has a better chance of being the primary vPC.
Example:
switch#
Step 5 show vpc role (Optional) Displays the vPC system priority.
Example:
switch# show vpc role
configuration.
Example:
213
Configuring vPCs
Configuring the Tracking Feature on a Single-Module vPC
This example shows how to manually configure the role priority of the vPC peer device:
switch(config-vpc-domain)# role priority 4
switch(config)#
Configuring the Tracking Feature on a Single-Module vPC

If you must configure all the vPC peer links and core-facing interfaces on a single module, you should configure
a track object and a track list that is associated with the Layer 3 link to the core and on all the links on the
vPC peer link on both primary vPC peer devices. Once you configure this feature and if the primary vPC peer
device fails, the system automatically suspends all the vPC links on the primary vPC peer device. This action
forces all the vPC traffic to the secondary vPC peer device until the system stabilizes.
You must put this configuration on both vPC peer devices. Additionally, you should put the identical
configuration on both vPC peer devices because either device can become the operationally primary vPC peer
device.
Before You Begin

Ensure that you have configured the track object and the track list. Ensue that you assign all interfaces that
connect to the core and to the vPC peer link to the track-list object on both vPC peer devices.
SUMMARY STEPS
3. track track-object-id
4. exit
5. show vpc brief
DETAILED STEPS

Example:
switch(config)#
214
Configuring vPCs
Configuring for Recovery After an Outage

Step 2 vpc domain domain-id Enters the vPC domain number that you want to configure,
Example:
Step 3 track track-object-id Adds the previously configured track-list object with its
associated interfaces to the vPC domain. See the Cisco Nexus
Example: 9000 Series NX-OS Unicast Routing Configuration Guide for
switch(config-vpc-domain)# track object 23 information about configuring object tracking and track lists.
Example:
switch#
Step 5 show vpc brief (Optional) Displays information about the tracked objects.
Example:
configuration.
Example:
This example shows how to put the previously configured track-list object into the vPC domain on the vPC
peer device:
switch(config-vpc-domain)# track object 5
switch(config)#

If an outage occurs, the vPC waits for a peer adjacency to form on a switch reload. This situation can result
in an unacceptably long service disruption. You can configure the Cisco Nexus 9000 Series device to restore
vPC services when its peer fails to come on line.
Configuring Reload Restore

The reload restore command and procedure described in this section is deprecated. We recommend that you
use the auto-recovery command and procedure described in the “Configuring an Autorecovery” section.
online by using the reload restore command.
215
Configuring vPCs
Before You Begin

SUMMARY STEPS
3. reload restore [delay time-out]
4. exit
5. show running-config vpc
6. show vpc consistency-parameters interface port-channel number
DETAILED STEPS

Example:
switch(config)#
Example:
Step 3 reload restore [delay time-out] Configures the vPC to assume its peer is not functional and to
bring up the vPC. The default delay is 240 seconds. You can
Example: configure a time-out delay from 240 to 3600 seconds.
switch(config-vpc-domain)# reload restore
Use the no form of the command to reset the vPC to its default
settings.
Example:
switch#
Step 5 show running-config vpc (Optional) Displays information about the vPC, specifically
the reload status.
Example:
switch# show running-config vpc
216
Configuring vPCs

Step 6 show vpc consistency-parameters interface (Optional) Displays information about the vPC consistency
port-channel number parameters for the specified interface.
Example:
switch# show vpc consistency-parameters
configuration.
Example: Note To ensure the reload feature is enabled, you should
perform this step.
This example shows how to set the vPC reload restore feature and save it in the switch startup configuration:
Enter configuration commands, one per line. End with CNTL/Z.
switch(config-vpc-domain)# reload restore
Warning:
Enables restoring of vPCs in a peer-detached state after reload, will wait for 240
seconds (by default) to determine if peer is un-reachable
!Command: show running-config vpc

!Time: Wed Mar 24 18:43:54 2010
version 5.0(2)
feature vpc
logging level vpc 6
vpc domain 5
reload restore
This example shows how to examine the consistency parameters:
switch# show vpc consistency-parameters interface port-channel 1
Legend:
Type 1 : vPC will be suspended in case of mismatch
Name Type Local Value Peer Value
------------- ---- ----------- ---------------
STP Port Type 1 Default -
STP Port Guard 1 None -
STP MST Simulate PVST 1 Default -
mode 1 on -
Speed 1 1000 Mb/s -
Duplex 1 full -
Port Mode 1 trunk -
Native Vlan 1 1 -
MTU 1 1500 -
Allowed VLANs - 1-3967,4048-4093
Local suspended VLANs
217
Configuring vPCs
Configuring an Autorecovery
online by using the auto-recovery command.
Before You Begin

SUMMARY STEPS
3. auto-recovery [reload-delay time]
4. exit
5. show running-config vpc
6. show vpc consistency-parameters interface port-channel number
DETAILED STEPS

Example:
switch(config)#
Step 2 vpc domain domain-id Enters the vPC domain number that you want to configure, and
enters vpc-domain configuration mode.
Example:
Step 3 auto-recovery [reload-delay time] Configures the vPC to assume its peer is not functional and to
bring up the vPC, and specifies the time to wait after a reload
Example: to restore the vPC. The default delay is 240 seconds. You can
switch(config-vpc-domain)# auto-recovery configure a delay from 240 to 3600 seconds.
Use the no form of the command to reset the vPC to its default
settings.
Example:
switch#
218
Configuring vPCs
Configuring the Suspension of Orphan Ports

Step 5 show running-config vpc (Optional) Displays information about the vPC, specifically the
reload status.
Example:
Step 6 show vpc consistency-parameters interface (Optional) Displays information about the vPC consistency
port-channel number parameters for the specified interface.
Example:
switch# show vpc consistency-parameters
configuration.
Example: Note To ensure the autorecovery feature is enabled, you
should perform this step.
This example shows how to set the vPC autorecovery feature and save it in the switch startup configuration:
switch(config-vpc-domain)# auto-recovery
Warning:
Enables restoring of vPCs in a peer-detached state after reload, will wait for 240
seconds to determine if peer is un-reachable

When a device that is not vPC-capable connects to each peer, the connected ports are known as orphan ports
because they are not members of a vPC. You can explicitly declare physical interfaces as orphan ports to be
suspended (shut down) by the secondary peer when it suspends its vPC ports in response to a peer link or
peer-keepalive failure. The orphan ports are restored when the vPC is restored.
Note You can configure vPC orphan port suspension only on physical ports, not on port channel member ports.
Before You Begin

219
Configuring vPCs
SUMMARY STEPS
2. show vpc orphan-ports
4. vpc orphan-ports suspend
5. exit
DETAILED STEPS

Example:
switch(config)#
Step 2 show vpc orphan-ports (Optional) Displays a list of the orphan ports.
Example:
switch# show vpc orphan-ports
Step 3 interface type slot/port Specifies an interface to configure, and enters interface
configuration mode.
Example:
switch(config-if)#
Step 4 vpc orphan-ports suspend Configures the selected interface as a vPC orphan port
to be suspended by the secondary peer in the case of a
Example: vPC failure.
switch(config-if)# vpc orphan-ports suspend
Step 5 exit Exits interface configuration mode.
Example:
switch#
Example:
This example shows how to configure an interface as a vPC orphan port to be suspended by the secondary
peer in the case of a vPC failure:
switch(config-if)# vpc orphan-ports suspend
220
Configuring vPCs
Configuring the vPC Peer Switch
switch(config)#

You can configure the Cisco Nexus 9000 Series device to make a pair of vPC devices appear as a single STP
root in the Layer 2 topology.
Configuring a Pure vPC Peer Switch Topology

You can configure a pure vPC peer switch topology by using the peer-switch command and then setting the
best possible (lowest) spanning tree bridge priority value.
Before You Begin

Note When using a non-VPC dedicated trunk link between the VPC peers, the non-VPC VLANs should have
a different global priority on the peers to prevent STP from blocking the VLANs.
SUMMARY STEPS
3. peer-switch
4. spanning-tree vlan vlan-range priority value
5. exit
6. show spanning-tree summary
DETAILED STEPS

Example:
switch(config)#
Example:
221
Configuring vPCs

Step 3 peer-switch Enables the vPC switch pair to appear as a single STP root
in the Layer 2 topology.
Example: Use the no form of the command to disable the peer switch
switch(config-vpc-domain)# peer-switch
vPC topology.
Step 4 spanning-tree vlan vlan-range priority value Configures the bridge priority of the VLAN. Valid values
are multiples of 4096. The default value is 32768.
Example:
switch(config)# spanning-tree vlan 1
priority 8192
Example:
switch#
Step 6 show spanning-tree summary (Optional) Displays a summary of the spanning tree port
states including the vPC peer switch.
Example:
switch# show spanning-tree summary
configuration.
Example:
This example shows how to configure a pure vPC peer switch topology:
2010 Apr 28 14:44:44 switch %STP-2-VPC_PEERSWITCH_CONFIG_ENABLED: vPC peer-switch

configuration is enabled. Please make sure to configure spanning tree "bridge" priority as
per recommended guidelines to make vPC peer-switch operational.
switch(config-vpc-domain)# spanning-tree vlan 1 priority 8192

switch(config)#
Configuring a Hybrid vPC Peer Switch Topology

You can configure a hybrid vPC and non-vPC peer switch topology by using the spanning-tree
pseudo-information command to change the designated bridge ID so that it meets the STP VLAN-based
load-balancing criteria and then change the root bridge ID priority to a value that is better than the best bridge
priority. You then enable the peer switch.
Before You Begin

222
Configuring vPCs
When using a non-VPC dedicated trunk link between the VPC peers, the non-VPC VLANs should have a
different pseudo root priority on the peers to prevent STP from blocking the VLANs.
SUMMARY STEPS
2. spanning-tree pseudo-information
3. vlan vlan-id designated priority priority
4. vlan vlan-id root priority priority
6. peer-switch
7. exit
8. show spanning-tree summary
DETAILED STEPS

Example:
switch(config)#
Step 2 spanning-tree pseudo-information Configures the spanning tree pseudo information.
Example:
switch(config)# spanning-tree pseudo-information
switch(config-pseudo)#
Step 3 vlan vlan-id designated priority priority Configures the designated bridge priority of the VLAN.
Valid values are multiples of 4096 from 0 to 61440.
Example:
switch(config-pseudo)# vlan 1 designated
priority 8192
Step 4 vlan vlan-id root priority priority Configures the root bridge priority of the VLAN. Valid
values are multiples of 4096 from 0 to 61440.
Example:
switch(config-pseudo)# vlan 1 root
priority 4096
Example:
Step 6 peer-switch Enables the vPC switch pair to appear as a single STP root
in the Layer 2 topology.
Example: Use the no form of the command to disable the peer switch
vPC topology.
223
Configuring vPCs
Verifying the vPC Configuration

Example:
switch#
Step 8 show spanning-tree summary (Optional) Displays a summary of the spanning tree port
states including the vPC peer switch.
Example:
switch# show spanning-tree summary
configuration.
Example:
This example shows how to configure a hybrid vPC peer switch topology:
switch(config)# spanning-tree pseudo-information
switch(config-pseudo)# vlan 1 designated priority 8192
switch(config-pseudo)# vlan 1 root priority 4096
switch(config-pseudo)# vpc domain 5
switch(config)#
Verifying the vPC Configuration

To display vPC configuration information, perform one of the following tasks:
Command Purpose
show feature Displays whether the vPC is enabled or not.
show vpc brief Displays brief information about the vPCs.
show vpc consistency-parameters Displays the status of those parameters that must be
consistent across all vPC interfaces.
show running-config vpc Displays running configuration information for vPCs.
show port-channel capacity Displays how many port channels are configured and
how many are still available on the device.
show vpc statistics Displays statistics about the vPCs.
show vpc peer-keepalive Displays information about the peer-keepalive

messages.
224
Configuring vPCs
Monitoring vPCs
Command Purpose
show vpc role Displays the peer status, the role of the local device,
the vPC system MAC address and system priority,
and the MAC address and priority for the local vPC
device.
Monitoring vPCs
Use the show vpc statisticscommand to display vPC statistics.
Note This command displays the vPC statistics only for the vPC peer device that you are working on.
Configuration Examples for vPCs

The following example shows how to configure vPC on device A as shown in the figure:
Figure 26: vPC Configuration Example
1 Enable vPC and LACP.

switch(config)# feature vPC
switch(config)# feature lacp
225
Configuring vPCs
Configuration Examples for vPCs
2 (Optional) Configure one of the interfaces that you want to be a peer link in the dedicated port mode.
switch(config)# interface ethernet 7/1,
ethernet 7/3, ethernet 7/5. ethernet 7/7
switch(config-if)# rate-mode dedicated
switch(config)#
3 (Optional) Configure the second, redundant interface that you want to be a peer link in the dedicated port
mode.
switch(config)# interface ethernet 7/2, ethernet 7/4,
ethernet 7/6. ethernet 7/8
switch(config-if)# rate-mode dedicated
switch(config)#
4 Configure the two interfaces (for redundancy) that you want to be in the peer link to be an active Layer 2
LACP port channel.
switch(config)# interface ethernet 7/1-2
5 Create and enable the VLANs.
switch(config)# vlan 1-50
switch(config-vlan)# no shutdown
switch(config-vlan)# exit
6 Create a separate VRF for the vPC peer-keepalive link and add a Layer 3 interface to that VRF.
switch(config)# vrf context pkal
switch(config-vrf)# exit
switch(config-if)# vrf member pkal
7 Create the vPC domain and add the vPC peer-keepalive link.
switch(config-vpc-domain)# peer-keepalive
destination 172.23.145.217 source 172.23.145.218 vrf pkal
8 Configure the vPC peer link.
switch(config)#
9 Configure the interface for the port channel to the downstream device of the vPC.
switch(config-if)# allowed vlan 1-50
switch(config-if)# native vlan 20
226
Configuring vPCs
Related Documents

switch(config)#
10 Save the configuration.
Note If you configure the port channel first, ensure that it is a Layer 2 port channel.
Related Documents
Related Topic Related Topic
System management System management
High availability High availability
Release Notes Release Notes
227
Configuring vPCs
Related Documents
228
CHAPTER 9
Configuring IP Tunnels
This chapter describes how to configure IP tunnels using Generic Route Encapsulation (GRE) on Cisco
NX-OS devices.
• Information About IP Tunnels, page 229

• Licensing Requirements for IP Tunnels, page 231
• Prerequisites for IP Tunnels, page 231
• Configuring IP Tunnels, page 233
• Verifying the IP Tunnel Configuration, page 241
• Configuration Examples for IP Tunneling, page 241
Information About IP Tunnels

IP tunnels can encapsulate a same-layer or higher layer protocol and transport the result over IP through a
tunnel created between two devices.
IP Tunnel Overview
IP tunnels consists of the following three main components:
• Passenger protocol—The protocol that needs to be encapsulated. IPv4 is an example of a passenger
protocol.
• Carrier protocol—The protocol that is used to encapsulate the passenger protocol. Cisco NX-OS supports
GRE as a carrier protocol.
• Transport protocol—The protocol that is used to carry the encapsulated protocol. IPv4 is an example of
a transport protocol. An IP tunnel takes a passenger protocol, such as IPv4, and encapsulates that protocol
229
GRE Tunnels
within a carrier protocol, such as GRE. The device then transmits this carrier protocol over a transport
protocol, such as IPv4.
You configure a tunnel interface with matching characteristics on each end of the tunnel.
You must enable the tunnel feature before you can configure it. The system automatically takes a checkpoint
prior to disabling the feature, and you can roll back to this checkpoint. See the Cisco Nexus 9000 Series NX-OS
System Management Configuration Guide for information about rollbacks and checkpoints.
GRE Tunnels
You can use generic routing encapsulation (GRE) as the carrier protocol for a variety of passenger protocols.
The following figure shows the IP tunnel components for a GRE tunnel. The original passenger protocol
packet becomes the GRE payload and the device adds a GRE header to the packet. The device then adds the
transport protocol header to the packet and transmits it.
Figure 27: GRE PDU
Point-to-Point IP-in-IP Tunnel Encapsulation and Decapsulation

For 6.1(2)I3(4), 7.0(3)I1(2), and later, point-to-point IP-in-IP encapsulation and decapsulation is a type of
tunnel that you can create to send encapsulated packets from a source tunnel interface to a destination tunnel
interface. This type of tunnel will carry both inbound and outbound traffic.
Note The selection of IP-in-IP tunnel based on the PBR policy is not supported.
Multi-Point IP-in-IP Tunnel Decapsulation

For 6.1(2)I3(4), 7.0(3)I1(2), and later, multi-point IP-in-IP decapsulate-any is a type of tunnel that you can
create to decapsulate packets from any number of IP-in-IP tunnels to one tunnel interface. This tunnel will
not carry any outbound traffic. However, any number of remote tunnel endpoints can use a tunnel configured
this way as their destination.
230
Path MTU Discovery
Path MTU Discovery

Path maximum transmission unit (MTU) discovery (PMTUD) prevents fragmentation in the path between
two endpoints by dynamically determining the lowest MTU along the path from the packet's source to its
destination. PMTUD reduces the send MTU value for the connection if the interface receives information that
the packet would require fragmentation.
When you enable PMTUD, the interface sets the Don't Fragment (DF) bit on all packets that traverse the
tunnel. If a packet that enters the tunnel encounters a link with a smaller MTU than the MTU value for the
packet, the remote link drops the packet and sends an ICMP message back to the sender of the packet. This
message indicates that fragmentation was required (but not permitted) and provides the MTU of the link that
dropped the packet.
Note PMTUD on a tunnel interface requires that the tunnel endpoint can receive ICMP messages generated by
devices in the path of the tunnel. Check that ICMP messages can be received before using PMTUD over
firewall connections.
High Availability
IP tunnels support stateful restarts. A stateful restart occurs on a supervisor switchover. After the switchover,
Cisco NX-OS applies the runtime configuration after the switchover.
Licensing Requirements for IP Tunnels


Cisco NX-OS IP tunnels require an Enterprise Services license. For
a complete explanation of the Cisco NX-OS licensing
scheme and how to obtain and apply licenses, see the
Cisco NX-OS Licensing Guide.
Prerequisites for IP Tunnels

IP tunnels have the following prerequisites:
• You must be familiar with TCP/IP fundamentals to configure IP tunnels.
• You are logged on to the switch.
• You must enable the tunneling feature in a device before you can configure and enable any IP tunnels.
231

IP tunnels have the following configuration guidelines and limitations:
• Cisco NX-OS supports only the following protocols:
◦IPv4 passenger protocol.
◦GRE carrier protocol.
• Cisco NX-OS supports the following maximum number of tunnels:

◦IP tunnels - 8 tunnels.
◦GRE and IP-in-IP regular tunnels - 8 tunnels. (6.1(2)I3(4) and later)
• IP tunnels do not support access control lists (ACLs) or QoS policies.

• Cisco NX-OS supports the GRE header defined in IETF RFC 2784. Cisco NX-OS does not support
tunnel keys and other options from IETF RFC 1701.
• Cisco NX-OS does not support GRE tunnel keepalives.
• All unicast routing protocols are supported by IP tunnels.
• The IP tunnel interface cannot be configured to be a span source or destination.
• IP tunnels do not support PIM or other Multicast features and protocols. (6.1(2)I3(4) and later)
• The selection of IP-in-IP tunnel based on the PBR policy is not supported. (6.1(2)I3(4) and later)
• IP tunnels are supported only in the default system routing mode and not in other modes. (6.1(2)I3(4)
and later)
Default Settings
The following table lists the default settings for IP tunnel parameters.
Table 13: Default IP Tunnel Parameters
Parameters Default
Path MTU discovery age timer 10 minutes
Path MTU discovery minimum MTU 64
Tunnel feature Disabled
232
Enabling Tunneling
You must enable the tunneling feature before you can configure any IP tunnels.
SUMMARY STEPS
2. feature tunnel
3. exit
4. show feature
DETAILED STEPS

Example:
switch(config)#
Step 2 feature tunnel Allows the creation of a new tunnel interface.

To disable the tunnel interface feature, use the no form
Example: of this command.
switch(config)# feature tunnel
switch(config-if)#
mode.
Example:
switch#
Step 4 show feature (Optional) Displays information about the features enabled
on the device.
Example:
switch(config-if)# show feature
233
Creating a Tunnel Interface

Step 5 copy running-config startup-config (Optional) Saves this configuration change.
Example:
startup-config

You can create a tunnel interface and then configure this logical interface for your IP tunnel.
Note Cisco NX-OS supports a maximum of 8 IP tunnels.
Note Use the no interface tunnel command to remove the tunnel interface and all associated configuration.
Command Purpose
no interface tunnel number Deletes the tunnel interface and the
associated configuration.
Example:
switch(config)# no interface tunnel 1
description string Configures a description for the

tunnel.
Example:
switch(config-if)# description GRE tunnel
mtu value Sets the MTU of IP packets sent on

an interface.
Example:
tunnel ttl value Sets the tunnel time-to-live value.

The range is from 1 to 255.
Example:
switch(config-if)# tunnel ttl 100
Note Configuring a GRE tunnel or an IP-in-IP tunnel that uses a tunnel interface VRF that is different than the
use-vrf for the tunnel destination is not supported. You need to use the same VRF for a tunnel interface
and the tunnel destination.
234
Before You Begin

You can configure the tunnel source and the tunnel destination in different VRFs. Ensure that you have enabled
the tunneling feature.
SUMMARY STEPS
2. interface tunnel number
3. tunnel mode {gre ip | ipip {ip | decapsulate-any}}
4. tunnel source {ip-address |interface-name}
5. tunnel destination {ip-address |host-name}
6. tunnel use-vrf vrf-name
7. show interfaces tunnel number
DETAILED STEPS

Example:
switch(config)#
Step 2 interface tunnel number Creates a new tunnel interface.
Example:
switch(config)# interface tunnel 1
switch(config-if)#
Step 3 tunnel mode {gre ip | ipip {ip | decapsulate-any}} Sets this tunnel mode to GRE, ipip, or ipip decapsulate-only.
The gre and ip keywords specify that GRE encapsulation over
IP will be used.
The ipip keyword specifies that IP-in-IP encapsulation will be
used. The optional decapsulate-any keyword terminates IP-in-IP
tunnels at one tunnel interface. This keyword creates a tunnel
that will not carry any outbound traffic. However, remote tunnel
endpoints can use a tunnel configured as their destination.
Step 4 tunnel source {ip-address |interface-name} Configures the source address for this IP tunnel. The source can
be specified by IP address or logical interface name.
Example:
switch(config-if)# tunnel source
ethernet 1/2
235
Configuring a Tunnel Interface

Step 5 tunnel destination {ip-address |host-name} Configures the destination address for this IP tunnel. The
destination can be specified by IP address or logical host name.
Example:
switch(config-if)# tunnel destination
192.0.2.1
Step 6 tunnel use-vrf vrf-name (Optional) Uses the configured VRF to look up the tunnel IP
destination address.
Example:
switch(config-if)# tunnel use-vrf blue
Step 7 show interfaces tunnel number (Optional) Displays the tunnel interface statistics.
Example:
switch# show interfaces tunnel 1
Example:
startup-config
This example shows how to create a tunnel interface

switch(config-if)# tunnel source ethenet 1/2
switch(config-if)# tunnel destination 192.0.2.1

You can set a tunnel interface to GRE tunnel mode, ipip mode, or ipip decapsulate-only mode. GRE mode is
the default tunnel mode. (7.0(3)I1(2) and later)
Before You Begin

Ensure that you have enabled the tunneling feature.
SUMMARY STEPS
3. tunnel mode {gre ip | ipip {ip | decapsulate-any}}
5. mtu value
236
DETAILED STEPS

Example:
switch(config)#
Example:
switch(config-if)#
Step 3 tunnel mode {gre ip | ipip {ip | Sets this tunnel mode to GRE, ipip, or ipip decapsulate-only.
decapsulate-any}} The gre and ip keywords specify that GRE encapsulation over IP
will be used.
The ipip keyword specifies that IP-in-IP encapsulation will be used.
The optional decapsulate-any keyword terminates IP-in-IP tunnels
at one tunnel interface. This keyword creates a tunnel that will not
carry any outbound traffic. However, remote tunnel endpoints can
use a tunnel configured as their destination.
Step 4 show interfaces tunnel number (Optional) Displays the tunnel interface statistics.
Example:
switch(config-if)# show interfaces tunnel
1
Step 5 mtu value Sets the maximum transmission unit (MTU) of IP packets sent on
an interface.
The range is from 64 to 9192 units.
Example:
startup-config
This example shows how to create the tunnel interface to GRE:

switch(config-if)# tunnel mode gre ip
This example shows how to create an ipip tunnel:

switch(config-if)# tunnel mode ipip
237
Configuring a GRE Tunnel

switch(config-if)# no shut
Configuring a GRE Tunnel

You can set a tunnel interface to GRE tunnel mode. (6.1(2)I3(3) and 7.0(3)I1(1))
Note Cisco NX-OS supports only the GRE protocol for IPV4 over IPV4.
Before You Begin

SUMMARY STEPS
3. tunnel mode gre ip
DETAILED STEPS

Example:
switch(config)#
Example:
switch(config-if)#
Step 3 tunnel mode gre ip Sets this tunnel mode to GRE.
Example:
switch(config-if)# tunnel mode gre ip
Step 4 show interfaces tunnel number (Optional) Displays the tunnel interface
statistics.
Example:
switch(config-if)# show interfaces tunnel 1
238
Enabling Path MTU Discovery

Example:
startup-config
Enabling Path MTU Discovery

Use the tunnel path-mtu discovery command to enable path MTU discovery on a tunnel.
SUMMARY STEPS
1. tunnel path-mtu-discovery age-timer min

2. tunnel path-mtu-discovery min-mtu bytes
DETAILED STEPS

Step 1 tunnel path-mtu-discovery age-timer min Enables Path MTU Discovery (PMTUD) on a tunnel interface.
• min—Number of minutes. The range is from 10 to 30.
Example: The default is 10.
switch(config-if)# tunnel path-mtu-discovery
age-timer 25
Step 2 tunnel path-mtu-discovery min-mtu bytes Enables Path MTU Discovery (PMTUD) on a tunnel interface.
• bytes—Minimum MTU recognized.
Example:
switch(config-if)# tunnel path-mtu-discovery The range is from 64 to 9192. The default is 64.
min-mtu 1500
Assigning VRF Membership to a Tunnel Interface

You can add a tunnel interface to a VRF.
Before You Begin

Assign the IP address for a tunnel interface after you have configured the interface for a VRF.
239
Assigning VRF Membership to a Tunnel Interface
SUMMARY STEPS
3. vrf member vrf-name
4. ip address ip-prefix/length
5. show vrf [vrf-name] interface interface-type number
DETAILED STEPS

Example:
switch(config)#
Step 2 interface tunnel number Enters interface configuration mode.
Example:
switch(config-if)#
Step 3 vrf member vrf-name Adds this interface to a VRF.
Example:
Step 4 ip address ip-prefix/length Configures an IP address for this interface. You

must do this step after you assign this interface to
Example: a VRF.
Step 5 show vrf [vrf-name] interface interface-type number (Optional) Displays VRF information.
Example:
switch(config-vrf)# show vrf Enterprise
interface tunnel 0
Example:
This example shows how to add a tunnel interface to the VRF:

240
Verifying the IP Tunnel Configuration
Verifying the IP Tunnel Configuration

To verify the IP tunnel configuration information, perform one of the following tasks:
Command Purpose
show interface tunnel number Displays the configuration for the tunnel interface
(MTU, protocol, transport, and VRF). Displays input
and output packets, bytes, and packet rates.
show interface tunnel number brief Displays the operational status, IP address,
encapsulation type, and MTU of the tunnel interface.
show interface tunnel number counters Displays interface counters of input/output packets.
Note The byte count displayed with the interface
counters include the internal header size.
show interface tunnel number description Displays the configured description of the tunnel
interface.
show interface tunnel number status Displays the operational status of the tunnel interface.
show interface tunnel number status err-disabled Displays the error disabled status of the tunnel
interface.
Configuration Examples for IP Tunneling

The following example shows a simple GRE tunnel. Ethernet 1/2 is the tunnel source for router A and the
tunnel destination for router B. Ethernet interface 2/1 is the tunnel source for router B and the tunnel destination
for router A.
Router A:
feature tunnel
interface tunnel 0
ip address 209.165.20.2/8
tunnel source ethernet 1/2
tunnel destination 192.0.2.2
tunnel mode gre ip
tunnel path-mtu-discovery 25 1500

ip address 192.0.2.55/8
Router B:
feature tunnel
interface tunnel 0
ip address 209.165.20.1/8
tunnel source ethernet 2/1
tunnel destination 192.0.2.55
tunnel mode gre ip
241
Related Documents

ip address 192.0.2.2/8
Related Documents
IP Tunnel commands Cisco Nexus 9000 Series NX-OS Interfaces Command
Reference
242
APPENDIX A
IETF RFCs supported by Cisco NX-OS Interfaces
This appendix lists the IETF RFCs for interfaces supported by Cisco NX-OS.
• IPv6 RFCs, page 243
IPv6 RFCs
RFCs Title
RFC 2373 IP Version 6 Addressing Architecture
RFC 2374 An Aggregatable Global Unicast Address Format
RFC 2460 Internet Protocol, Version 6 (IPv6) Specification
RFC 2462 IPv6 Stateless Address Autoconfiguration
RFC 2464 Transmission of IPv6 Packets over Ethernet Networks
RFC 2467 Transmission of IPv6 Packets over FDDI Networks
RFC 2472 IP Version 6 over PPP
RFC 2492 IPv6 over ATM Networks
RFC 2590 Transmission of IPv6 Packets over Frame Relay Networks Specification
RFC 3021 Using 31-Bit Prefixes on IPv4 Point-to-Point Links
RFC 3152 Delegation of IP6.ARPA
RFC 3162 RADIUS and IPv6
RFC 3513 Internet Protocol Version 6 (IPv6) Addressing Architecture
243
IETF RFCs supported by Cisco NX-OS Interfaces
IPv6 RFCs
RFCs Title
RFC 3596 DNS Extensions to Support IP version 6
RFC 4193 Unique Local IPv6 Unicast Addresses
244
APPENDIX B
Configuration Limits for Cisco NX-OS Interfaces
The configuration limits are documented in the Cisco Nexus 9000 Series NX-OS Verified Scalability Guide.
245
Configuration Limits for Cisco NX-OS Interfaces
246

B Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide

Uploaded by

Copyright:

Available Formats

B Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

B Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide

Uploaded by

Copyright:

Available Formats

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide,

This product includes cryptographic software written by Eric Young ([email protected]).

This product includes software written by Tim Hudson ([email protected]).

© 2013 - 2015 Cisco Systems, Inc. All rights reserved.

Preface Preface xiii

CHAPTER 1 New and Changed Information 1

CHAPTER 3 Configuring Basic Interface Parameters 7

About the Basic Interface Parameters 7

Monitoring the Interface Counters 34

CHAPTER 4 Configuring Layer 2 Interfaces 39

Verifying the Interface Configuration 64

CHAPTER 5 Configuring Layer 3 Interfaces 67

CHAPTER 6 Configuring Bidirectional Forwarding Detection 85

CHAPTER 7 Configuring Port Channels 117

Configuring a Port-Channel Description 140

CHAPTER 8 Configuring vPCs 163

Configuration Parameters That Should Be Identical 176

Licensing Requirements for vPCs 196

CHAPTER 9 Configuring IP Tunnels 229

Path MTU Discovery 231

APPENDIX A IETF RFCs supported by Cisco NX-OS Interfaces 243

APPENDIX B Configuration Limits for Cisco NX-OS Interfaces 245

• Audience, page xiii

[x] Square brackets enclose an optional element (keyword or argument).

[x | y] Square brackets enclosing keywords or arguments separated by a vertical

{x | y} Braces enclosing keywords or arguments separated by a vertical bar

Examples use the following conventions:

<> Nonprinting characters, such as passwords, are in angle brackets.

[] Default responses to system prompts are in square brackets.

!, # An exclamation point (!) or a pound sign (#) at the beginning of a line

Related Documentation for Cisco Nexus 9000 Series Switches

Obtaining Documentation and Submitting a Service Request

• New and Changed Information, page 1

New and Changed Information

Table 1: New and Changed Features

Feature Description Changed Where Documented

Subinterface support on Support for one or more 6.1(2)I3(3) Configuring Layer 3

Feature Description Changed Where Documented

Table 2: Interface Parameters

Feature Parameters Further Information

Layer 3 medium, IPv4 and IPv6 addresses “Configuring Layer 3 Interfaces”

Port Channels channel group, LACP “Configuring Port Channels”

Security EOU Cisco Nexus 9000 Series NX-OS

Module Level Breakout

Notes About Breakout Interfaces

High Bandwidth Interfaces

Virtual Device Contexts

High Availability for Interfaces

• About the Basic Interface Parameters, page 7

About the Basic Interface Parameters

Interface Status Error Policy

Port MTU Size

Unidirectional Link Detection Parameter

Figure 1: Unidirectional Link

Default UDLD Configuration

Table 3: UDLD Default Configuration

Feature Default Value

UDLD aggressive mode Disabled

UDLD message interval 15 seconds