B APIC NXOS CLI User Guide Chapter 0111
B APIC NXOS CLI User Guide Chapter 0111
B APIC NXOS CLI User Guide Chapter 0111
• Implicit mode, a simpler mode, is not compatible with the APIC GUI or the REST API.
• Named (or Explicit) mode is compatible with the APIC GUI and the REST API.
In either case, the configuration should be considered read-only in the incompatible UI.
Note Except for the procedures in the Configuring Layer 3 External Connectivity Using the Named Mode section,
this guide describes Implicit mode procedures.
• Layer 3 external network objects (l3extOut) created using the Implicit mode CLI procedures are identified
by names starting with “__ui_” and are marked as read-only in the GUI. The CLI partitions these
external-l3 networks by function, such as interfaces, protocols, route-map, and EPG. Configuration
modifications performed through the REST API can break this structure, preventing further modification
through the CLI.
For the steps to remove such objects, see Troubleshooting Unwanted _ui_ Objects in the APIC Troubleshooting
Guide.
The steps for configuring layer 3 external connectivity can be summarized as follows:
1. Create a VRF under a tenant.
2. Configure and deploy the VRF on the border leaf switch.
3. Configure layer 3 interfaces on the border leaf Interfaces.
4. Configure route-maps on the leaf switch.
5. Configure routing protocols (BGP, OSPF, EIGRP) under leaf and leaf-interface.
6. Create and configure an external-L3 EPG under a tenant.
7. Deploy the external-L3 EPG on the border leaf switch.
Note For guidelines and cautions for configuring and maintaining Layer 3 outside connections, see Guidelines for
Routed Connectivity to Outside Networks, on page 5.
For information about the types of L3Outs, see External Layer 3 Outside Connection Types, on page 6.
A Layer 3 external outside network (l3extOut object) includes the routing protocol options (BGP, OSPF, or
EIGRP or supported combinations) and the switch-specific and interface-specific configurations. While the
l3extOut contains the routing protocol (for example, OSPF with its related Virtual Routing and Forwarding
(VRF) and area ID), the Layer 3 external interface profile contains the necessary OSPF interface details. Both
are needed to enable OSPF.
The l3extInstP EPG exposes the external network to tenant EPGs through a contract. For example, a tenant
EPG that contains a group of web servers could communicate through a contract with the l3extInstP EPG
according to the network configuration contained in the l3extOut. The outside network configuration can
easily be reused for multiple nodes by associating the nodes with the L3 external node profile. Multiple nodes
that use the same profile can be configured for fail-over or load balancing. Also, a node can be added to
multiple l3extOuts resulting in VRFs that are associated with the l3extOuts also being deployed on that node.
For scalability information, refer to the current Verified Scalability Guide for Cisco ACI.
Ingress-based policy enforcement Starting with Cisco APIC release 1.2(1), ingress-based policy
enforcement enables defining policy enforcement for Layer 3 Outside
(L3Out) traffic for both egress and ingress directions. The default is
ingress. During an upgrade to release 1.2(1) or higher, existing L3Out
configurations are set to egress so that the behavior is consistent with
the existing configuration. You do not need any special upgrade
sequence. After the upgrade, you change the global property value
to ingress. When it has been changed, the system reprograms the
rules and prefix entries. Rules are removed from the egress leaf and
installed on the ingress leaf, if not already present. If not already
configured, an Actrl prefix entry is installed on the ingress leaf.
Direct server return (DSR), and attribute EPGs require ingress based
policy enforcement. vzAny and taboo contracts ignore ingress based
policy enforcement. Transit rules are applied at ingress.
Bridge Domains with L3Outs A bridge domain in a tenant can contain a public subnet that is
advertised through an l3extOut provisioned in the common tenant.
Bridge domain route advertisement For When both OSPF and EIGRP are enabled on the same VRF on a
OSPF and EIGRP node and if the bridge domain subnets are advertised out of one of
the L3Outs, it will also get advertised out of the protocol enabled on
the other L3Out.
For OSPF and EIGRP, the bridge domain route advertisement is per
VRF and not per L3Out. The same behavior is expected when
multiple OSPF L3Outs (for multiple areas) are enabled on the same
VRF and node. In this case, the bridge domain route will be
advertised out of all the areas, if it is enabled on one of them.
BGP Maximum Prefix Limit Starting with Cisco APIC release 1.2(1x), tenant policies for BGP
l3extOut connections can be configured with a maximum prefix
limit, that enables monitoring and restricting the number of route
prefixes received from a peer. Once the maximum prefix limit has
been exceeded, a log entry is recorded, and further prefixes are
rejected. The connection can be restarted if the count drops below
the threshold in a fixed interval, or the connection is shut down. Only
one option can be used at a time. The default setting is a limit of
20,000 prefixes, after which new prefixes are rejected. When the
reject option is deployed, BGP accepts one more prefix beyond the
configured limit, before the APIC raises a fault.
MTU Cisco ACI does not support IP fragmentation. Therefore, when you
configure Layer 3 Outside (L3Out) connections to external routers,
or multipod connections through an Inter-Pod Network (IPN), it is
critical that the MTU is set appropriately on both sides. On some
platforms, such as Cisco ACI, Cisco NX-OS, and Cisco IOS, the
configurable MTU value takes into account the IP headers (resulting
in a max packet size to be set as 9000 for Cisco ACI, NX-OS and
IOS). However, other platforms such as IOS-XR configure the MTU
value exclusive of packet headers (resulting in a max packet size of
8986 bytes).
For the appropriate MTU values for each platform, see the relevant
configuration guides.
Cisco highly recommends that you test the MTU with CLI-based
commands. For example, on the Cisco NX-OS CLI, use a command
such as ping 1.1.1.1 df-bit packet-size 9000
source-interface ethernet 1/1.
Layer 4 to Layer 7 When you are using a multinode service graph, you must have the
two EPGs in separate VRF instances. For these functions, the system
must do a Layer 3 lookup, so the EPGs must be in separate VRFs.
This limitation follows legacy service insertion, based on Layer 2
and Layer 3 lookups.
QoS for L3Outs To configure QoS policies for an L3Out and enable the policies to
be enforced on the BL switch where the L3Out is located, use the
following guidelines:
• The VRF Policy Control Enforcement Direction must be set
toEgress.
• The VRF Policy Control Enforcement Preference must be set
to Enabled.
• When configuring the contract that controls communication
between the EPGs using the L3Out, include the QoS class or
Target DSCP in the contract or subject of the contract.
The External Layer 3 Outside connections are supported on the following interfaces:
• Layer 3 Routed Interface
• Sub-interface with 802.1Q tagging - With sub-interface, the same physical interface can be used to
provide a Layer 2 outside connection for multiple private networks.
• Switched Virtual Interface (SVI) - With an SVI interface, the same physical interface that supports Layer
2 and Layer 3 and the same physical interface can be used for a Layer 2 outside connection and a Layer
3 outside connection.
The managed objects that are used for the L3Outside connections are:
• External Layer 3 Outside (L3ext): Routing protocol options (OSPF area type, area, EIGRP AS, BGP),
private network, External Physical domain.
• Logical Node Profile: Profile where one or more nodes are defined for the External Layer 3 Outside
connections. The router-IDs and the loopback interface configuration is defined in the profile.
Note Use the same router-ID for the same node across multiple External Layer 3 Outside
connections.
Note Within a single L3Out, a node can only be part of one Logical Node Profile.
Configuring the node to be a part of multiple Logical Node Profiles in a single
L3Out might result in unpredictable behavior, such as a loopback address being
pushed from one Logical Node Profile but not from the other. Use additional path
bindings under the existing Logical Interface Profiles or create a new Logical
Interface Profile under the existing Logical Node Profile instead.
• Logical Interface Profile: IP interface configuration for IPv4 and IPv6 interfaces. It is supported on the
Route Interfaces, Routed Sub-Interfaces, and SVIs. The SVIs can be configured on physical ports,
port-channels or VPCs.
• OSPF Interface Policy: Includes details such as OSPF Network Type and priority.
• EIGRP Interface Policy: Includes details such as Timers and split horizon.
• BGP Peer Connectivity Profile: The profile where most BGP peer settings, remote-as, local-as, and BGP
peer connection options are configured. The BGP peer connectivity profile can be associated with the
logical interface profile or the loopback interface under the node profile. This determines the update-source
configuration for the BGP peering session.
• External Network Instance Profile (EPG) (l3extInstP): The external EPG is also referred to as the prefix
based EPG or InstP. The import and export route control policies, security import polices, and contract
associations are defined in this profile. Multiple external EPGs can be configured under a single L3Out.
Multiple external EPGs may be used when a different route or a security policy is defined on a single
External Layer 3 Outside connections. An external EPG or multiple external EPGs combine into a
route-map. The import/export subnets defined under the external EPG associate to the IP prefix-list match
clauses in the route-map. The external EPG is also where the import security subnets and contracts are
associated. This is used to permit or drop traffic for this L3out.
• Action Rules Profile: The action rules profile is used to define the route-map set clauses for the L3Out.
The supported set clauses are the BGP communities (standard and extended), Tags, Preference, Metric,
and Metric type.
• Route Control Profile: The route-control profile is used to reference the action rules profile(s). This can
be an ordered list of action rules profiles. The Route Control Profile can be referenced by a tenant BD,
BD subnet, external EPG, or external EPG subnet.
There are additional protocol settings for BGP, OSPF, and EIGRP L3Outs. These settings are configured per
tenant in the ACI Protocol Policies section in the GUI.
Note When configuring policy enforcement between external EPGs (transit routing case), you must configure the
second external EPG (InstP) with the default prefix 0/0 for export route control, aggregate export, and external
security. In addition, the preferred group must be excluded, and you must use an any contract (or desired
contract) between the transit InstPs.
Configuring a Layer 3 Outside for Tenant Networks Using the NX-OS Style CLI
These steps describe how to configure a Layer 3 outside network for tenant networks. This example shows
how to deploy a node and L3 port for tenant VRF external L3 connectivity using the NX-OS CLI.
This example is broken into steps for clarity. For a merged example, see NX-OS Style CLI Example: L3Out,
on page 12.
For an example using the commands for these prerequisites, see NX-OS Style CLI Example: L3Out
Prerequisites, on page 12.
Procedure
Example:
apic1(config)# leaf 103
apic1(config-leaf)# vrf context tenant t1 vrf v1
apic1(config-leaf-vrf)# router-id 11.11.11.103
apic1(config-leaf-vrf)# exit
apic1(config-leaf)# interface ethernet 1/3
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member tenant t1 vrf v1
apic1(config-leaf-if)# ip address 12.12.12.3/24
apic1(config-leaf-if)# exit
apic1(config-leaf)# exit
apic1(config-leaf-bgp-vrf-neighbor)# exit
apic1(config-leaf-bgp-vrf)#exit
apic1(config-leaf-bgp)# exit
apic1(config-leaf)# exit
apic1(config)# bgp-fabric
apic1(config-bgp-fabric)# asn 100
apic1(config-bgp-fabric)# route-reflector spine 104,105
Note Layer 3 routed and sub-interface port channels on border leaf switches are supported only on new generation
switches, which are switch models with "EX", "FX" or "FX2" at the end of the switch name.
Procedure
Step 3 interface port-channel channel-name Enters the interface configuration mode for the
specified port channel.
Example:
apic1(config-leaf)# interface
port-channel po1
Step 5 vrf member vrf-name tenant tenant-name Associates this port channel to this virtual
routing and forwarding (VRF) instance and L3
Example:
outside policy, where:
apic1(config-leaf-if)# vrf member v1
tenant t1 • vrf-name is the VRF name. The name can
be any case-sensitive, alphanumeric string
up to 32 characters.
• tenant-name is the tenant name. The name
can be any case-sensitive, alphanumeric
string up to 32 characters.
Step 6 vlan-domain member vlan-domain-name Associates the port channel template with the
previously configured VLAN domain.
Example:
apic1(config-leaf-if)# vlan-domain
member dom1
Step 7 ip address ip-address / subnet-mask Sets the IP address and subnet mask for the
specified interface.
Example:
apic1(config-leaf-if)# ip address
10.1.1.1/24
Step 8 ipv6 address sub-bits/prefix-length preferred Configures an IPv6 address based on an IPv6
general prefix and enables IPv6 processing on
Example:
an interface, where:
apic1(config-leaf-if)# ipv6 address
2001::1/64 preferred • sub-bits is the subprefix bits and host bits
of the address to be concatenated with the
Step 11 mtu mtu-value Sets the MTU for this class of service.
Example:
apic1(config-leaf-if)# mtu 1500
Example
This example shows how to configure a basic Layer 3 port channel.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface port-channel po1
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member v1 tenant t1
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# ip address 10.1.1.1/24
apic1(config-leaf-if)# ipv6 address 2001::1/64 preferred
apic1(config-leaf-if)# ipv6 link-local fe80::1
apic1(config-leaf-if)# mac-address 00:44:55:66:55::01
apic1(config-leaf-if)# mtu 1500
Procedure
Step 3 vrf member vrf-name tenant tenant-name Associates this port channel to this virtual
routing and forwarding (VRF) instance and L3
Example:
outside policy, where:, where:
apic1(config-leaf-if)# vrf member v1
tenant t1 • vrf-name is the VRF name. The name can
be any case-sensitive, alphanumeric string
up to 32 characters.
• tenant-name is the tenant name. The name
can be any case-sensitive, alphanumeric
string up to 32 characters.
Step 4 vlan-domain member vlan-domain-name Associates the port channel template with the
previously configured VLAN domain.
Example:
apic1(config-leaf-if)# vlan-domain
member dom1
Step 5 ip address ip-address / subnet-mask Sets the IP address and subnet mask for the
specified interface.
Example:
apic1(config-leaf-if)# ip address
10.1.1.1/24
Step 6 ipv6 address sub-bits/prefix-length preferred Configures an IPv6 address based on an IPv6
general prefix and enables IPv6 processing on
Example:
an interface, where:
apic1(config-leaf-if)# ipv6 address
2001::1/64 preferred • sub-bits is the subprefix bits and host bits
of the address to be concatenated with the
prefixes provided by the general prefix
specified with the prefix-name argument.
The sub-bits argument must be in the
form documented in RFC 2373 where the
address is specified in hexadecimal using
16-bit values between colons.
Step 9 mtu mtu-value Sets the MTU for this class of service.
Example:
apic1(config-leaf-if)# mtu 1500
Step 11 interface port-channel channel-name Enters the interface configuration mode for the
specified port channel.
Example:
apic1(config-leaf)# interface
port-channel po1
Step 12 vlan-domain member vlan-domain-name Associates the port channel template with the
previously configured VLAN domain.
Example:
apic1(config-leaf-if)# vlan-domain
member dom1
Step 14 interface port-channel channel-name.number Enters the interface configuration mode for the
specified sub-interface port channel.
Example:
apic1(config-leaf)# interface
port-channel po1.2001
Example
This example shows how to configure a basic Layer 3 sub-interface port-channel.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface vlan 2001
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member v1 tenant t1
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# ip address 10.1.1.1/24
apic1(config-leaf-if)# ipv6 address 2001::1/64 preferred
apic1(config-leaf-if)# ipv6 link-local fe80::1
apic1(config-leaf-if)# mac-address 00:44:55:66:55::01
apic1(config-leaf-if)# mtu 1500
apic1(config-leaf-if)# exit
apic1(config-leaf)# interface port-channel po1
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# exit
apic1(config-leaf)# interface port-channel po1.2001
apic1(config-leaf-if)# vrf member v1 tenant t1
apic1(config-leaf-if)# exit
Procedure
Step 3 interface Ethernet slot/port Enters interface configuration mode for the
interface you want to configure.
Example:
apic1(config-leaf)# interface Ethernet
1/1-2
Example
This example shows how to add ports to a Layer 3 port-channel.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface Ethernet 1/1-2
apic1(config-leaf-if)# channel-group p01
• There is no associated L3Out required for the bridge domain. When an Inter-VRF shared L3Out is used,
it is not necessary to associate the user tenant bridge domains with the L3Out in tenant common. If you
had a tenant-specific L3Out, it would still be associated to your bridge domains in your respective tenants.
• Two Layer 3 Outs can be in two different VRFs, and they can successfully exchange routes.
• This enhancement is similar to the Application EPG to Layer 3 Out inter-VRF communications. The
only difference is that instead of an Application EPG there is another Layer 3 Out. Therefore, in this
case, the contract is between two Layer 3 Outs.
In the following figure, there are two Layer 3 Outs with a shared subnet. There is a contract between the Layer
3 external instance profile (l3extInstP) in both the VRFs. In this case, the Shared Layer 3 Out for VRF1 can
communicate with the Shared Layer 3 Out for VRF2.
Figure 4: Shared Layer 3 Outs Communicating Between Two VRFs
Configuring Shared Layer 3 Out Inter-VRF Leaking Using the NX-OS Style CLI
- Named Example
Procedure
Configuring Shared Layer 3 Out Inter-VRF Leaking Using the NX-OS Style CLI
- Implicit Example
Procedure
(VXLAN VNI) across all the nodes in the fabric where the Layer 3 Out SVI is deployed as long as all SVI
interfaces use the same external encapsulation (SVI) as shown in the figure.
However, when different Layer 3 Outs are deployed, the ACI fabric uses different bridge domains even if
they use the same external encapsulation (SVI) as shown in the figure:
Figure 5: Local Scope Encapsulation and One Layer 3 Out
Starting with Cisco APIC release 2.3, it is now possible to choose the behavior when deploying two (or more)
Layer 3 Outs using the same external encapsulation (SVI).
The encapsulation scope can now be configured as Local or VRF:
• Local scope (default): The example behavior is displayed in the figure titled Local Scope Encapsulation
and Two Layer 3 Outs.
• VRF scope: The ACI fabric configures the same bridge domain (VXLAN VNI) across all the nodes and
Layer 3 Out where the same external encapsulation (SVI) is deployed. See the example in the figure
titled VRF Scope Encapsulation and Two Layer 3 Outs.
The mapping among the CLI, API, and GUI syntax is as follows:
Note The CLI commands to configure encapsulation scope are only supported when the VRF is configured through
a named Layer 3 Out configuration.
Procedure
Step 3 Create the VLAN interface. Creates the VLAN interface. The VLAN range
is 1-4094.
Example:
apic1(config-leaf)# interface vlan 2001
Note This feature is available in the APIC Release 2.2(3x) release and going forward with APIC Release 3.1(1). It
is not supported in APIC Release 3.0(x).
The Switch Virtual Interface (SVI) represents a logical interface between the bridging function and the routing
function of a VLAN in the device. SVI can have members that are physical ports, direct port channels, or
virtual port channels. The SVI logical interface is associated with VLANs, and the VLANs have port
membership.
The SVI state does not depend on the members. The default auto state behavior for SVI in Cisco APIC is that
it remains in the up state when the auto state value is disabled. This means that the SVI remains active even
if no interfaces are operational in the corresponding VLAN/s.
If the SVI auto state value is changed to enabled, then it depends on the port members in the associated VLANs.
When a VLAN interface has multiple ports in the VLAN, the SVI goes to the down state when all the ports
in the VLAN go down.
Procedure
Step 3 Create the VLAN interface. Creates the VLAN interface. The VLAN range
is 1-4094.
Example:
apic1(config-leaf)# interface vlan 2001
Procedure
Step 3 [no] vrf context tenant tenant-name vrf Configures a tenant VRF on the node.
vrf-name
Example:
apic1(config-leaf)# vrf context tenant
exampleCorp vrf v1
Step 4 (Optional) [no] router-id ipv4-address Assigns a router ID for routing protocols
running on the VRF. If you do not assign a
Example:
router ID, an ID is generated internally that is
apic1(config-leaf-vrf)# router-id unique to each leaf switch.
1.2.3.4
Step 5 [no] {ip | ipv6} route ip-prefix/masklen Configures static route information for the
next-hop-address [preferred] VRF.
Example:
apic1(config-leaf-vrf)# ip route
21.1.1.1/32 32.1.1.1
apic1(config-leaf-vrf)# ipv6 route
5001::1/128 6002::1
Step 8 vlan-domain member domain-name Assign a VLAN domain to the interface. The
VLAN domain must have already been created
Example:
using the vlan-domain command in the global
apic1(config-leaf-if)# vlan-domain configuration mode.
member dom1
Step 10 vrf member tenant tenant-name vrf vrf-name Attaches the interface to the tenant VRF.
Example:
apic1(config-leaf-if)# vrf member tenant
exampleCorp vrf v1
Step 11 [no] {ip | ipv6} address ip-prefix/masklen Configures IP addresses on the interface. The
[eui64] [secondary] [preferred] specified address can be declared as either:
Example: • preferred—The default source address
for traffic from the interface.
apic1(config-leaf-if)# ip address
10.1.1.1/24 • secondary—The secondary address of
apic1(config-leaf-if)# ipv6 address the interface.
2001::1/64 preferred
With the optional eui64 keyword, the host can
assign itself a 64-bit Extended Unique
Identifier (EUI).
In this mode, you can also configure ipv6
link-local, mac address, mtu, and other layer
3 properties on the interface.
Step 12 [[no]]ip dhcprelayaddress tenanttenant-name Sets or removes a DHCP relay address for the
dhcp-address{applicationapp-name external interface along with any supported
epgepg-name|external-1212-epg-name|external-1313-epg-name} options.
Example:
Examples
This example shows how to deploy a layer 3 port for external connectivity.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant exampleCorp vrf v1
apic1(config-leaf-vrf)# router-id 1.2.3.4
apic1(config-leaf-vrf)# ip route 21.1.1.1/32 32.1.1.1
apic1(config-leaf-vrf)# ipv6 route 5001::1/128 6002::1 preferred
apic1(config-leaf-vrf)# exit
This example shows how to configure a layer 3 subinterface port for external connectivity. In this
example, the subinterface ID (the "100" in 1/2.100) is actually the VLAN encapsulation instead of
an ID. All properties supported on a layer 3 port are available on the subinterface as well.
apic1# configure
apic1(config)# leaf 101
# SAME VRF CONTEXT CONFIGURATION AS PREVIOUS EXAMPLE
This example shows the methods to configure a switched virtual interface (SVI) for external
connectivity. Each external SVI is uniquely identified by its encap VLAN denoted in the SVI ID.
apic1# configure
apic1(config)# leaf 101
# SAME VRF CONTEXT CONFIGURATION AS PREVIOUS EXAMPLE
Note An external SVI must be configured on each of the participating nodes. This allows you to configure
different IP addresses on each of the nodes for the same SVI. If the vPC is part of an external SVI,
you must individually create an SVI on each of the participating vPC peers and you can provide
different IP addresses on each SVI.
OSPF Configuration
Configuring OSPF
OSPF can operate in one of the following modes in an area:
• When OSPF is used as the main routing protocol for the tenant VRF in the node, OSPF will import and
export routes defined in the route-map configured in the OSPF area. The route-map contains the export
routes.
• When OSPF is used as a connectivity protocol for BGP, OSPF advertises the loopback address which is
used as the source of the BGP session. Note that the loopback IP address and not the loopback ID is used.
In this case, a BGP session relying on OSPF will use the same loopback IP address in its update-source
command.
There is no need for separate configuration of OSPF and OSPFv3. The router OSPF mode handles both
OSPFv2 and OSPFv3 implicitly for the areas running under OSPF.
OSPF sessions are supported on all types of layer 3 Interfaces in the leaf, including:
• Layer 3 ports
• Subinterfaces
• External SVI
Procedure
Step 3 router ospf default Creates an OSPF routing process and enters
OSPF policy configuration.
Example:
apic1(config-leaf)# router ospf default
Step 4 vrf member tenant tenant-name vrf vrf-name Enables a VRF in the OSPF session.
Example:
apic1(config-leaf-ospf)# vrf member
tenant exampleCorp vrf v100
Step 5 (Optional) default-information originate Causes the switch to generate a default route.
[always]
Example:
apic1(config-leaf-ospf-vrf)#
default-information originate
Step 8 area area-id default-cost cost Sets OSPF default area cost to a value between
0 and 16777215.
Example:
apic1(config-leaf-ospf-vrf)# area 17
default-cost 20
Step 9 area area-id route-map map-name out Specifies a route-map for outbound filtering.
Example:
apic1(config-leaf-ospf-vrf)# area 17
route-map ospf-to-eigrp out
Step 10 area area-id loopback loopback-address When OSPF is used as a connectivity protocol
for BGP, OSPF advertises the loopback
Example:
address which is used as the source of the BGP
apic1(config-leaf-ospf-vrf)# area 17 session. Note that the loopback IP address and
loopback 192.0.20.11/32
not the loopback ID is used. In this case, a
BGP session relying on OSPF will use the
same loopback IP address in its update-source
command.
Step 13 area area-id range address-range cost cost Configures inter-area route summarization,
which summarizes the networks between areas.
Example:
apic1(config-leaf-ospf-vrf)# area 17 The range is the summary route to be
range 192.0.20.0/24 cost 20 advertised in areas. The cost is a value between
0 and 16777215.
Step 16 interface slot/port Specifies a port for the OSPF interface. The
interface could also be specified as interface
Example:
slot/port.vlan-id or interface vlanvlan-id.
apic1(config-leaf)# interface eth 1/2
Step 17 {ip | ipv6} router ospf default area area-id Creates an OSPF routing process and enters
OSPF policy configuration.
Example:
apic1(config-leaf-if)# ip router ospf
default area 17
Step 18 {ip | ipv6} ospf inherit interface-policy Inherits the OSPF interface template policy
if-policy-name tenant tenant-name under this tenant.
Example:
apic1(config-leaf-if)# ip ospf inherit
interface-policy ifPolicy3 tenant
exampleCorp
Step 19 [no] {ip | ipv6} ospf prefix-suppression Prevents OSPF from advertising all IP prefixes
{enable | disable | inherit} that belong to a specific interface, except for
prefixes that are associated with secondary IP
Example:
addresses.
apic1(config-leaf-if)# ip ospf
prefix-suppression enable
Step 21 [no] ip ospf authentication {md5 | none | Specifies the authentication type.
simple}
Example:
apic1(config-leaf-if)# ip ospf
authentication md5
Examples
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router ospf default
apic1(config-leaf-ospf)# vrf member tenant exampleCorp vrf v100
apic1(config-leaf-ospf-vrf)# area 0 nssa
apic1(config-leaf-ospf-vrf)# area 17 stub
apic1(config-leaf-ospf-vrf)# area 17 default-cost 20
apic1(config-leaf-ospf-vrf)# area 17 route-map ospf-to-eigrp out
apic1(config-leaf-ospf-vrf)# area 17 loopback 192.0.20.11/32
apic1(config-leaf-ospf-vrf)# inherit ipv4 ospf vrf-policy vrfTemplate2
apic1(config-leaf-ospf-vrf)# summary-address 182.1.20.0/24
apic1(config-leaf-ospf-vrf)# area 17 range 192.0.20.0/24 cost 20
apic1(config-leaf-ospf-vrf)# exit
apic1(config-leaf-ospf)# exit
apic1(config-leaf)# interface eth 1/3
apic1(config-leaf-if)# ip router ospf default area 17
apic1(config-leaf-if)# ip ospf inherit interface-policy ifPolicy3 tenant exampleCorp
apic1(config-leaf-if)# ip ospf prefix-suppression enable
apic1(config-leaf-if)# ip ospf passive-interface
apic1(config-leaf-if)# ip ospf authentication md5
apic1(config-leaf-if)# ip ospf authentication-key c1$c0123
Step 3 template ospf vrf-policy vrf-policy-name Creates the OSPF VRF policy template under
tenant tenant-name the specified tenant.
Example:
apic1(config-leaf)# template ospf
vrf-policy vrfTemplate3 tenant
exampleCorp
Step 4 timers throttle lsa start-time hold-interval Sets the start-interval, hold-interval, and
max-time max-interval for link-state advertisements
(LSA).
Example:
apic1(config-vrf-policy)# timers
throttle lsa 200 10000 45000
Step 5 timers lsa-group-pacing seconds Sets the interval in which LSAs are grouped
and refreshed, checksummed, or aged.
Example:
apic1(config-vrf-policy)# timers
lsa-group-pacing 240
Step 6 timers lsa-arrival milliseconds Sets the minimum interval between the arrival
of each LSA.
Example:
apic1(config-vrf-policy)# timers
lsa-arrival 1000
Step 7 timers throttle spf spf-start spf-hold Sets the SPF init-interval, hold-interval, and
spf-max-wait max-interval for LSA.
Example:
apic1(config-vrf-policy)# timers
throttle spf 5 1000 90000
Step 15 template ospf interface-policy if-policy-name Creates the OSPF interface policy template
tenant tenant-name under the specified tenant.
Example:
apic1(config-leaf)# template ospf
interface-policy ifTemplate5 tenant
exampleCorp
Step 17 [no] cost if-cost Sets the OSPF cost for the interface. The range
is 0 to 65535.
Example:
apic1(config-interface-policy)# cost
300
Step 18 [no] dead-interval seconds Sets the interval in seconds at which hello
packets must not be seen before neighbors
Example:
Step 19 [no] hello-interval seconds Specifies the interval between hello packets in
seconds. The range is 1 to 65535 seconds.
Example:
apic1(config-interface-policy)#
hello-interval 10
Step 21 [no] network {bcast | p2p | unspecified} Sets the OSPF interface policy network type,
which can be broadcast or point-to-point.
Example:
apic1(config-interface-policy)# network
p2p
Step 23 [no] priority priority Sets OSPF interface priority, which is used to
determine the designated router (DR) on a
Example:
specific network. The range is 0 to 255.
apic1(config-interface-policy)# priority
4
Step 25 [no] transmit-delay seconds Sets the estimated time required to send a
link-state update packet on the interface. The
Example:
range is from 1 to 450 seconds.
apic1(config-interface-policy)#
transmit-delay 2
Examples
This example shows how to configure a VRF template and an interface template.
apic1# configure
apic1(config)# leaf 101
BGP Configuration
Configuring BGP
Procedure
Examples
apic1# configure
apic1(config)# bgp-fabric
apic1(config-bgp-fabric)# asn 100
apic1(config-bgp-fabric)# route-reflector spine 105
What to do next
Configure BGP address family and counters.
Step 3 template bgp timers timer-policy-name Creates the BGP timers policy template under
tenant tenant-name the specified tenant.
Example:
apic1(config-leaf)# template bgp timers
bgpTimers tenant exampleCorp
This template will be available on all
leaves
where tenant exampleCorp has a VRF
deployment
Step 5 graceful-restart stalepath-time seconds Sets the maximum time that BGP keeps stale
routes from the restarting BGP peer. The range
is 1 to 3600 seconds.
apic1(config-bgp-timers)#
graceful-restart stalepath-time 3600
Step 6 timers bgp keep-alive-seconds hold-seconds Sets the keep-alive timer and hold timer values.
The range for both is 1 to 3600 seconds.
Step 8 template bgp address-family family-name Creates the BGP address family template under
tenant tenant-name the specified tenant.
Example:
apic1(config-leaf)# template bgp
address-family bgpAf1 tenant exampleCorp
This template will be available on all
leaves
where tenant exampleCorp has a VRF
deployment
Step 9 distance ebgp-distance ibgp-distance Sets the administrative distance for eBGP
local-distance routes, iBGP routes, and local routes. The
range is 1 to 255.
apic1(config-bgp-af)# distance 250 240
230
Examples
This example shows how to create a BGP timer template and an address family template.
apic1# configure
apic1(config)# leaf 101
Procedure
Step 4 vrf member tenant tenant-name vrf vrf-name Specifies the VRF instance to associate with
subsequent address family configuration mode
Example:
commands.
apic1(config-bgp)# vrf member tenant
exampleCorp vrf v100
Step 6 address-family {ipv4 | ipv6} unicast Declares neighbors with whom we want to
exchange normal IPv4 unicast routes.
Example:
apic1(config-leaf-bgp-vrf)#
address-family ipv4 unicast
Step 7 inherit bgp address-family family-name Adds the specified address family to this
address family.
Example:
apic1(config-leaf-bgp-vrf-af)# inherit
bgp address-family ipv4-af-pol
This template will be inherited on all
leaves where VRF v100 has been deployed
Step 8 exit
Example:
apic1(config-leaf-bgp-vrf-af)# exit
Examples
This example shows how to inherit a BGP timer configuration and IPv4 and IPv6 address families.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router bgp 100
apic1(config-bgp)# vrf member tenant exampleCorp vrf v100
apic1(config-leaf-bgp-vrf)# inherit bgp timer bgpTimers
This template will be inherited on all leaves where VRF v100 has been deployed
apic1(config-leaf-bgp-vrf)# address-family ipv4 unicast
apic1(config-leaf-bgp-vrf-af)# inherit bgp address-family ipv4-af-pol
This template will be inherited on all leaves where VRF v100 has been deployed
apic1(config-leaf-bgp-vrf-af)# exit
apic1(config-leaf-bgp-vrf)# address-family ipv6 unicast
apic1(config-leaf-bgp-vrf-af)# inherit bgp address-family ipv6-af-pol
This template will be inherited on all leaves where VRF v100 has been deployed
apic1(config-leaf-bgp-vrf-af)# exit
apic1(config-leaf-bgp-vrf)# exit
apic1(config-leaf)# exit
Step 4 vrf member tenant tenant-name vrf vrf-name Specifies the VRF instance to associate with
subsequent policy configuration mode
Example:
commands.
apic1(config-bgp)# vrf member tenant
exampleCorp vrf v100
Step 7 address-family {ipv4 | ipv6} unicast Declares neighbors with whom we want to
exchange normal IPv4 unicast routes.
Example:
apic1(config-leaf-bgp-vrf-neighbor)#
address-family ipv4 unicast
Step 8 [no] maximum-prefix count [action {log | Sets the maximum number of prefixes from
shut | restart [restart-timeminutes]}] this neighbor. the range is 1 to 300000
[threshold percent] prefixes. Other optional settings are:
Example: • action— The action to be performed
apic1(config-leaf-bgp-vrf-neighbor-af)# when the maximum prefix limit is
maximum-prefix 10 threshold 10 action reached. If the action is restart, you can
restart restart-time 10 optionally specify the restart-time, which
is the period of time in minutes before
restarting the peer when the maximum
prefix limit is reached. The range is 1 to
65535 minutes.
• threshold— The threshold percentage of
the maximum number of prefixes before
a warning is issued. The range is 1 to 100
percent.
Step 9 exit
Example:
apic1(config-leaf-bgp-vrf-neighbor-af)#
exit
Step 10 update-source {loopback ip-address | if the neighbor address is being learned through
ethernet ip-address | vlan vlan-id} OSPF, specify the same loopback address as
being used under OSPF.
Example:
apic1(config-leaf-bgp-vrf-neighbor)#
update-source loopback 192.0.2.230
The following table shows the interface settings that can be configured at this point.
Command Purpose
Examples
This example shows how to configure an IPv4 BGP neighbor.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router bgp 100
apic1(config-bgp)# vrf member tenant exampleCorp vrf v100
apic1(config-leaf-bgp-vrf)# aggregate-address 192.0.10.0/24 as-set
apic1(config-leaf-bgp-vrf)# neighbor 192.0.2.229/32
apic1(config-leaf-bgp-vrf-neighbor)# address-family ipv4 unicast
apic1(config-leaf-bgp-vrf-neighbor-af)# maximum-prefix 10 threshold 10 action restart
restart-time 10
apic1(config-leaf-bgp-vrf-neighbor-af)# exit
apic1(config-leaf-bgp-vrf-neighbor)# allow-self-as
apic1(config-leaf-bgp-vrf-neighbor)# allowed-self-as-count 2
apic1(config-leaf-bgp-vrf-neighbor)# disable-connected-check
apic1(config-leaf-bgp-vrf-neighbor)# disable-peer-as-check
apic1(config-leaf-bgp-vrf-neighbor)# ebgp-multihop 4
apic1(config-leaf-bgp-vrf-neighbor)# local-as 100
apic1(config-leaf-bgp-vrf-neighbor)# next-hop-self
apic1(config-leaf-bgp-vrf-neighbor)# password abcdef
apic1(config-leaf-bgp-vrf-neighbor)# remote-as 200
apic1(config-leaf-bgp-vrf-neighbor)# send-community extended
apic1(config-leaf-bgp-vrf-neighbor)# update-source vlan 601
apic1(config-leaf-bgp-vrf-neighbor)# update-source ethernet 1/15
apic1(config-leaf-bgp-vrf-neighbor)# update-source loopback 192.0.2.230
Warning: BGP Configuration changed. Please re-configure BGP Password if it was enabled
apic1(config-leaf-bgp-vrf-neighbor)# local-as 100 no-prepend replace-as dual-as
apic1(config-leaf-bgp-vrf-neighbor)# route-map rMapT3 out
apic1(config-leaf-bgp-vrf-neighbor)# weight 2000
apic1(config-leaf-bgp-vrf-neighbor)# private-as-control
apic1(config-leaf-bgp-vrf-neighbor)# exit
apic1(config-leaf-bgp-vrf)# exit
apic1(config-leaf)# exit
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router bgp 100
apic1(config-bgp)# vrf member tenant exampleCorp vrf v100
apic1(config-leaf-bgp-vrf)# neighbor 2001:80:1:2::229
apic1(config-leaf-bgp-vrf-neighbor)# address-family ipv6 unicast
apic1(config-leaf-bgp-vrf-neighbor-af)# maximum-prefix 100
apic1(config-leaf-bgp-vrf-neighbor-af)# exit
apic1(config-leaf-bgp-vrf-neighbor)# allow-self-as
apic1(config-leaf-bgp-vrf-neighbor)# allowed-self-as-count 2
apic1(config-leaf-bgp-vrf-neighbor)# disable-connected-check
apic1(config-leaf-bgp-vrf-neighbor)# disable-peer-as-check
apic1(config-leaf-bgp-vrf-neighbor)# ebgp-multihop 4
apic1(config-leaf-bgp-vrf-neighbor)# local-as 100
apic1(config-leaf-bgp-vrf-neighbor)# next-hop-self
apic1(config-leaf-bgp-vrf-neighbor)# password abcdef
apic1(config-leaf-bgp-vrf-neighbor)# remote-as 200
apic1(config-leaf-bgp-vrf-neighbor)# send-community extended
apic1(config-leaf-bgp-vrf-neighbor)# update-source vlan 601
apic1(config-leaf-bgp-vrf-neighbor)# update-source ethernet 1/15
apic1(config-leaf-bgp-vrf-neighbor)# update-source loopback 2001:80:1:2::230/128
Warning: BGP Configuration changed. Please re-configure BGP Password if it was enabled
apic1(config-leaf-bgp-vrf-neighbor)# local-as 100 no-prepend replace-as dual-as
apic1(config-leaf-bgp-vrf-neighbor)# route-map rMapT3 out
apic1(config-leaf-bgp-vrf-neighbor)# weight 2000
apic1(config-leaf-bgp-vrf-neighbor)# private-as-control
apic1(config-leaf-bgp-vrf-neighbor)# exit
apic1(config-leaf-bgp-vrf-af)# exit
apic1(config-leaf-bgp-vrf)# exit
apic1(config-leaf)# exit
Configuring a Per VRF Per Node BGP Timer Policy Using the NX-OS Style CLI
Procedure
Step 2 Create a timer policy. The specific values are provided as examples
only.
Example:
apic1# config
apic1(config)# leaf 101
apic1(config-leaf)# template bgp timers
pol7 tenant tn1
This template will be available on all
nodes where tenant tn1 has a VRF
deployment
apic1(config-bgp-timers)# timers bgp 120
240
apic1(config-bgp-timers)#
graceful-restart stalepath-time 500
apic1(config-bgp-timers)# maxas-limit
300
apic1(config-bgp-timers)# exit
apic1(config-leaf)# exit
apic1(config)# exit
apic1#
exit
exit
exit
apic1#
The two properties which enable you to configure more paths are maxEcmp and maxEcmpIbgp in the
bgpCtxAfPol object. After you configure these two properties, they are propagated to the rest of your
implementation.
Use the following commands when logged in to BGP:
maximum-paths [ibgp]
no maximum-paths [ibgp]
Example Configuration:
Procedure
Example:
apic1(config)# leaf 101
apic1(config-leaf)# template bgp address-family newAf tenant t1
This template will be available on all nodes where tenant t1 has a VRF deployment
apic1(config-bgp-af)# maximum-paths ?
<1-16> Maximum number of equal-cost paths for load sharing. The default is 16.
ibgp Configure multipath for IBGP paths
apic1(config-bgp-af)# maximum-paths 10
apic1(config-bgp-af)# maximum-paths ibpg 8
apic1(config-bgp-af)# end
apic1#
no maximum-paths [ibgp]
Prepend Appends the specified AS number to the AS path of the route matched by
the route map.
Note • You can configure more than one AS number.
• 4 byte AS numbers are supported.
• You can prepend a total 32 AS numbers. You must specify
the order in which the AS Number is inserted into the AS
Path attribute.
Prepend-last-as Prepends the last AS numbers to the AS path with a range between 1 and 10.
The following table describes the selection criteria for implementation of AS Path Prepend:
Procedure
To modify the autonomous system path (AS Path) for Border Gateway Protocol (BGP) routes, you can use
the set as-path command. The set as-path command takes the form of
apic1(config-leaf-vrf-template-route-profile)# set as-path {'prepend as-num [ ,... as-num ]
| prepend-last-as num}
Example:
apic1(config)# leaf 103
apic1(config-leaf)# vrf context tenant t1 vrf v1
apic1(config-leaf-vrf)# template route-profile rp1
apic1(config-leaf-vrf-template-route-profile)# set as-path ?
prepend Prepend to the AS-Path
prepend-last-as Prepend last AS to the as-path
apic1(config-leaf-vrf-template-route-profile)# set as-path prepend 100, 101, 102, 103
apic1(config-leaf-vrf-template-route-profile)# set as-path prepend-last-as 8
apic1(config-leaf-vrf-template-route-profile)# exit
apic1(config-leaf-vrf)# exit
apic1(config-leaf)# exit
What to do next
To disable AS Path prepend, use the no form of the shown command:
apic1(config-leaf-vrf-template-route-profile)# [no] set
as-path { prepend as-num [ ,... as-num ] | prepend-last-as num}
Procedure
Step 3 template route-profile profile-name tenant Creates a route-profile template under tenant
tenant-name for BGP dampening and route redistribution.
Example:
apic1(config-leaf)# template
route-profile map_eigrp tenant
exampleCorp
Step 4 Required: [no] set tag name Sets the tag value. The name parameter is an
unsigned integer.
Example:
apic1(config-leaf-template-route-profile)#
set tag 200
Step 6 template route-profile profile-name tenant Creates a route-profile template under tenant
tenant-name for BGP dampening and route redistribution.
Example:
apic1(config-leaf)# template
route-profile map_ospf tenant exampleCorp
Step 7 Required: [no] set tag name Sets the tag value. The name parameter is an
unsigned integer.
Example:
apic1(config-leaf-template-route-profile)#
set tag 100
Example
apic1# configure
What to do next
Configure a redistribute route-profile under BGP for OSPF and EIGRP using one of the route-profiles created
in this procedure.
Procedure
Step 4 vrf member tenant tenant-name vrf vrf-name Specifies the VRF instance to associate with
subsequent policy configuration mode
Example:
commands.
apic1(config-bgp)# vrf member tenant
exampleCorp vrf v100
Example
This example configures a redistribute route-profile under BGP for OSPF and EIGRP using the
route-profiles created in the example in Creating a Route-Profile with Tenant Scope. The redistribute
route-map allows (permits) all routes and applies the route-profile for the route-control actions. In
this example, all EIGRP learned routes will be redistributed into BGP with tag 200 and OSPF routes
will be redistributed into BGP with tag 100.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router bgp 100
apic1(config-bgp)# vrf member tenant exampleCorp vrf v1
apic1(config-leaf-bgp-vrf)# redistribute eigrp route-map map_eigrp
apic1(config-leaf-bgp-vrf)# redistribute ospf route-map map_ospf
Procedure
Step 3 template route-profile profile-name tenant Creates a route-profile template under tenant
tenant-name for BGP dampening and route redistribution.
Example:
apic1(config-leaf)# template
route-profile damp_rp tenant exampleCorp
Step 4 Required: [no] set dampening half-life reuse Configures route flap dampening behavior.
suppress max-suppress-time The parameters are:
Example:
Step 7 vrf member tenant tenant-name vrf vrf-name Specifies the VRF instance to associate with
subsequent policy configuration mode
Example:
commands.
apic1(config-bgp)# vrf member tenant
exampleCorp vrf v100
Step 9 address-family {ipv4 | ipv6} unicast Declares neighbors with whom we want to
exchange normal IPv4 unicast routes.
Example:
apic1(config-leaf-bgp-vrf-neighbor)#
address-family ipv4 unicast
Step 11 exit
Example:
apic1(config-leaf-bgp-vrf-neighbor-af)#
exit
Step 12 exit
Example:
apic1(config-leaf-bgp-vrf-neighbor)#
exit
Step 13 address-family {ipv4 | ipv6} unicast Declares neighbors with whom we want to
exchange normal IPv4 unicast routes.
Example:
apic1(config-leaf-bgp-vrf)#
address-family ipv4 unicast
Step 15 exit
Example:
apic1(config-leaf-bgp-vrf-af)# exit
Example
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# template route-profile damp_rp tenant exampleCorp
apic1(config-leaf-template-route-profile)# set dampening 15 750 2000 60
apic1(config-leaf-template-route-profile)# exit
apic1(config-leaf)# router bgp 100
apic1(config-bgp)# vrf member tenant exampleCorp vrf v100
apic1(config-leaf-bgp-vrf)# neighbor 192.0.2.229/32
apic1(config-leaf-bgp-vrf-neighbor)# address-family ipv4 unicast
apic1(config-leaf-bgp-vrf-neighbor-af)# inherit bgp dampening damp_rp
apic1(config-leaf-bgp-vrf-neighbor-af)# exit
apic1(config-leaf-bgp-vrf)# address-family ipv6 unicast
apic1(config-leaf-bgp-vrf-af)# inherit bgp dampening damp_rp
apic1(config-leaf-bgp-vrf-af)# exit
EIGRP Configuration
Creating EIGRP VRF and Interface Templates
Procedure
Step 3 template eigrp vrf-policy vrf-policy-name Creates the EIGRP VRF policy template under
tenant tenant-name the specified tenant.
Example:
apic1(config-leaf)# template eigrp
vrf-policy vrfTemplate3 tenant
exampleCorp
This template will be available on all
leaves where tenant exampleCorp has a
VRF deployment
Step 5 maximum-paths limit Sets EIGRP Maximum Path Limit for the VRF
policy template. The limit can be 1 to 32.
Example:
apic1(config-template-eigrp-vrf-pol)#
maximum-paths 8
Step 6 metric version 64bit Sets EIGRP metric style to wide metric (64
bits).
Example:
apic1(config-template-eigrp-vrf-pol)#
metric version 64bit
Step 7 timers active-time minutes Sets EIGRP active timer interval. The range
is 1 to 65535 minutes.
Example:
apic1(config-template-eigrp-vrf-pol)#
timers active-time 1
Step 9 ip hello-interval eigrp default seconds Sets EIGRP hello interval time. The range is
1 to 65535 seconds.
Example:
apic1(config-template-eigrp-if-pol)# ip
hello-interval eigrp default 10
Step 10 ip hold-interval eigrp default seconds Sets EIGRP hold interval time. The range is 1
to 65535 seconds.
Example:
apic1(config-template-eigrp-if-pol)# ip
hold-interval eigrp default 10
Examples
apic1# configure
apic1(config)# leaf 101
This template will be available on all leaves where tenant exampleCorp has a VRF deployment
apic1(config-template-eigrp-vrf-pol)# distance 2 5
apic1(config-template-eigrp-vrf-pol)# maximum-paths 8
apic1(config-template-eigrp-vrf-pol)# metric version 64bit
apic1(config-template-eigrp-vrf-pol)# timers active-time 1
apic1(config-template-eigrp-vrf-pol)# exit
What to do next
Configure EIGRP address family and counters.
Step 4 vrf member tenant tenant-name vrf vrf-name Specifies the VRF instance to associate with
subsequent address family configuration mode
Example:
commands.
apic1(config-eigrp)# vrf member tenant
exampleCorp vrf v100
Step 8 maximum-paths limit Sets EIGRP Maximum Path Limit for the VRF
policy template. The limit can be 1 to 32.
Example:
apic1(config-address-family)#
maximum-paths 8
Step 9 metric version 64bit Sets EIGRP metric style to wide metric (64
bits).
Example:
apic1(config-address-family)# metric
version 64bit
Step 10 timers active-time minutes Sets EIGRP active timer interval. The range
is 1 to 65535 minutes.
Example:
apic1(config-address-family)# timers
active-time 1
Step 11 inherit eigrp vrf-policy vrf-policy-name Applies an EIGRP VRF policy to this address
family.
Example:
apic1(config-address-family)# inherit
eigrp vrf-policy vrfTemplate3
Examples
This example shows how to configure an EIGRP address family and inherit an EIGRP VRF policy.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router eigrp default
apic1(config-eigrp)# vrf member tenant exampleCorp vrf v100
apic1(config-eigrp-vrf)# autonomous-system 300
apic1(config-eigrp-vrf)# address-family ipv4 unicast
This configuration will affect all leaves where VRF v100 has been deployed
apic1(config-address-family)# distance 2 5
This configuration will affect all leaves where VRF v100 has been deployed
apic1(config-address-family)# maximum-paths 8
This configuration will affect all leaves where VRF v100 has been deployed
apic1(config-address-family)# metric version 64bit
This configuration will affect all leaves where VRF v100 has been deployed
apic1(config-address-family)# timers active-time 1
This configuration will affect all leaves where VRF v100 has been deployed
Step 5 [no] vlan-domain member vlan-id Creates and enters the configuration mode for
the VLAN domain.
Example:
apic1(config-leaf-if)# vlan-domain
member dom1
Step 6 [no] vrf member tenant exampleCorp vrf Associates the interface with a VRF.
vrf-name
Example:
apic1(config-leaf-if)# vrf member tenant
exampleCorp vrf v100
Step 7 [no] {ip | ipv6} address Sets an IP address for the interface.
ip-address/mask-length
Example:
apic1(config-leaf-if)# ip address
181.12.12.1/24
Step 9 [no] {ip | ipv6} distribute-list eigrp default EIGRP advertises routes that are matched in
route-map map-name out the route-map specified in the distribute-list
command. The route prefixes mentioned in the
Example:
prefix-list in the route-map can be learned from
apic1(config-leaf-if)# ip other protocol sources like BGP, OSPF, Static,
distribute-list eigrp default route-map
rMapT5 out
Connected. Redistribute route-maps are
automatically created based on the
distribute-list command. Note that prefixes
learned from an EIGRP session running on an
another interface on the same switch will not
be filtered by the distribute-list and will always
be advertised out.
Step 10 [no] {ip | ipv6} hello-interval eigrp default Sets EIGRP hello interval time. The range is
seconds 1 to 65535 seconds.
Example:
apic1(config-leaf-if)# ip hello-interval
eigrp default 10
Step 11 [no] {ip | ipv6} hold-interval eigrp default Sets EIGRP hold interval time. The range is 1
seconds to 65535 seconds.
Example:
apic1(config-leaf-if)# ip hold-interval
eigrp default 10
Step 12 [no] {ip | ipv6} next-hop-self eigrp default Sets EIGRP next-hop-self flag.
Example:
apic1(config-leaf-if)# ip next-hop-self
eigrp default
Step 13 [no] {ip | ipv6} passive-interface eigrp Set EIGRP passive-interface flag.
default
Example:
apic1(config-leaf-if)# ip
passive-interface eigrp default
Step 14 [no] {ip | ipv6} split-horizon eigrp default Sets EIGRP split-horizon flag.
Example:
apic1(config-leaf-if)# ip split-horizon
eigrp default
Step 16 [no] ip summary-address eigrp default Configures route summarization for EIGRP.
ip-prefix A summary address can be configured to
advertise an aggregated prefix on an EIGRP
Example:
session.
apic1(config-leaf-if)# ip
summary-address eigrp default Note A summary address enabled on one
172.10.1.0/24 interface will also be applied on
apic1(config-leaf-if)# ip other EIGRP enabled interfaces on
summary-address eigrp default 2001::/64 the same VRF on the switch.
Examples
This example shows how to configure an EIGRP interface.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface ethernet 1/21
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf v100
apic1(config-leaf-if)# ip address 181.12.12.1/24
apic1(config-leaf-if)# ip router eigrp default
apic1(config-leaf-if)# ip distribute-list eigrp default route-map rMapT5 out
distribute list will be updated on all EIGRP interfaces on node 1021 VRF exampleCorp/v100
apic1(config-leaf-if)# ip hello-interval eigrp default 5
apic1(config-leaf-if)# ip hold-interval eigrp default 10
apic1(config-leaf-if)# ip next-hop-self eigrp default
apic1(config-leaf-if)# ip passive-interface eigrp default
apic1(config-leaf-if)# ip split-horizon eigrp default
apic1(config-leaf-if)# inherit eigrp ip interface-policy ifTemplate5
apic1(config-leaf-if)# ip summary-address eigrp default 172.10.1.0/24
apic1(config-leaf-if)# exit
apic1(config-leaf)# exit
apic1(config)# exit
Configuring Route-Maps
Configuring Templates
About Route Profiles
A route profile specifies the route-control set actions used in import, export, and redistribute route-maps.
Route profile templates can be defined either under the tenant or under the tenant VRF.
Procedure
Step 4 Required: [no] set community {regular | Sets the BGP community attribute.
extended} value {none | replace | additive}
Example:
apic1(config-leaf-template-route-profile)#
set community extended 20:22 additive
Step 5 Required: [no] set dampening half-life reuse Configures route flap dampening behavior.
suppress max-suppress-time The parameters are:
Example:
Step 6 Required: [no] set local-preference value Sets the BGP local preference value. The range
is from 0 to 4294967295.
Example:
apic1(config-leaf-template-route-profile)#
set local-preference 64
Step 7 Required: [no] set metric value Sets the metric for the destination routing
protocol.
Example:
apic1(config-leaf-template-route-profile)#
set metric 128
Step 8 Required: [no] set metric-type {type-1 | The options are as follows:
type-2}
• type-1—OSPF external type 1 metric
Example:
• type-2—OSPF external type 2 metric
apic1(config-leaf-template-route-profile)#
set metric-type type-2
Step 9 Required: [no] set tag name Sets the tag value for the destination routing
protocol. The name parameter is an unsigned
Example:
integer.
apic1(config-leaf-template-route-profile)#
set tag 1111
Step 10 Required: [no] set weight weight Sets the tag value for the destination routing
protocol. The weight parameter is an unsigned
Example:
integer.
apic1(config-leaf-template-route-profile)#
set weight 20
Examples
This example shows how to configure a tenant-scoped route profile.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# template route-profile rp1 tenant exampleCorp
This template will be available on all leaves where tenant exampleCorp has a VRF deployment
apic1(config-leaf-template-route-profile)# set community extended 20:22 additive
apic1(config-leaf-template-route-profile)# set dampening 15 750 2000 60
apic1(config-leaf-template-route-profile)# set local-preference 64
apic1(config-leaf-template-route-profile)# set metric 128
apic1(config-leaf-template-route-profile)# set metric-type type-2
apic1(config-leaf-template-route-profile)# set tag 1111
apic1(config-leaf-template-route-profile)# set weight 20
Note VRF-scoped route profiles name default-export and default-import values, which are automatically applied
on the match statements on the respective export/import route-maps used in the same tenant VRF.
Procedure
Step 3 [no] vrf context tenant tenant-name vrf Enables VRF on the leaf and enters VRF
vrf-name configuration mode.
Example:
apic1(config-leaf)# vrf context tenant
exampleCorp vrf vrf1
Step 5 Required: [no] set community {regular | Sets the BGP community attribute.
extended} {no-advertise| no-export|value
{none | replace | additive}
Example:
apic1(config-leaf-vrf-template-route-profile)#
set community extended 20:22 additive
Step 6 Required: [no] set local-preference value Sets the BGP local preference value. The range
is from 0 to 4294967295.
Example:
apic1(config-tenant-vrf-route-profile)#
set local-preference 64
Step 7 Required: [no] set metric value Sets the metric for the destination routing
protocol.
Example:
apic1(config-tenant-vrf-route-profile)#
set metric 128
Step 8 Required: [no] set metric-type {type-1 | The options are as follows:
type-2}
• type-1—OSPF external type 1 metric
Example:
• type-2—OSPF external type 2 metric
apic1(config-tenant-vrf-route-profile)#
set metric-type type-2
Step 9 Required: [no] set tag name Sets the tag value for the destination routing
protocol. The name parameter is an unsigned
Example:
integer.
apic1(config-tenant-vrf-route-profile)#
set tag 1111
Step 10 Required: [no] set weight weight Sets the tag value for the destination routing
protocol. The weight parameter is an unsigned
Example:
integer.
apic1(config-tenant-vrf-route-profile)#
set weight 20
Examples
This example shows how to configure a VRF-scoped route profile.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant exampleCorp vrf vrf1
apic1(config-leaf-vrf)# template route-profile default-export
apic1(config-leaf-vrf-template-route-profile)# set community extended 20:22 additive
Creating a Route-Map
Route-maps are created with a prefix-list on a per-tenant basis to indicate the bridge domain public subnets
to be advertised to external routers. In addition, a prefix-list must be created to allow all transit routes to be
advertised to an external router. The prefix-list for transit routes are configured by an administrator. The
default behavior is to deny all transit route advertisement to an external router.
Procedure
Step 3 [no] vrf context tenant tenant-name vrf Configures a tenant VRF on the node.
vrf-name
Example:
apic1(config-leaf)# vrf context tenant
exampleCorp vrf v1
Step 4 (Optional) [no] router-id ipv4-address Assigns a router ID for routing protocols
running on the VRF. If you do not assign a
Example:
router ID, an ID is generated internally that is
apic1(config-leaf-vrf)# router-id unique to each leaf switch.
1.2.3.4
Step 5 Required: [no] route-map name Creates a route-map and enters route-map
configuration.
Example:
apic1(config-leaf-vrf)# route-map bgpMap
Step 6 Required: [no] ip prefix-list list-name permit Creates a prefix-list under the route-map.
prefix/masklen [le {32 | 128}]
Example:
Step 7 Required: [no] match prefix-list list-name Matches a prefix-list that has already been
created and enters the match mode to configure
Example:
the route-control profile for the prefix-list.
apic1(config-leaf-vrf-route-map)# match
prefix-list list1
Step 8 Required: [no] set metric value Sets the metric for the destination routing
protocol.
Example:
apic1(config-leaf-vrf-route-map-match)#
set metric 128
Step 9 Required: [no] set metric-type {type-1 | The options are as follows:
type-2}
• type-1—OSPF external type 1 metric
Example:
• type-2—OSPF external type 2 metric
apic1(config-leaf-vrf-route-map-match)#
set metric-type type-2
Step 10 Required: [no] set local-preference value Sets the BGP local preference value. The range
is from 0 to 4294967295.
Example:
apic1(config-leaf-vrf-route-map-match)#
set local-preference 64
Step 11 Required: [no] set community {regular | Sets the community attribute for a BGP route
extended} value {none | replace | additive} update. Specify the community-value in aa:nn
format. Specify the action as one of the
Example:
following:
apic1(config-leaf-vrf-route-map-match)#
set community extended 20:22 additive • additive—Add to existing community
• replace—Replace existing community
• none—Do not change community
Step 12 Required: [no] set tag name Sets the tag value for the destination routing
protocol. The name parameter is an unsigned
Example:
integer.
apic1(config-leaf-vrf-route-map-match)#
set tag 1111
Step 13 Required: [no] set weight value Specifies the BGP weight for the routing table.
Example:
apic1(config-leaf-vrf-route-map-match)#
set weight 32
Step 14 Required: [no] contract {provider| consumer Add contract, required to leak routes (matching
} contract-name [imported] this prefix list) from the VRF.
Example:
Step 15 Required: [no] match route group Matches a route group that has already been
group-name [order number ] created and enters the match mode to configure
the route-map.
Example:
apic1(config-leaf-vrf-route-map)# match Repeat the steps 8-13 or only step 18 to
route group g1 order 1 configure the route map for the route group.
See step 17 to inherit the route map instead of
inline set actions.
Step 16 Required: [no] match bridge-domain Matches a bridge domain in order to export its
bd-name public subnets through the protocol.
Example:
apic1(config-leaf-vrf-route-map)#
bridge-domain bd1
Step 17 Required: [no] inherit route-profile Configures route map for bridge domain.
profile-name
Note The route map was already created
Example: using the command template
apic1(config-leaf-vrf-route-map-match)# route-profile.
inherit route-profile rp1
Step 18 Required: [no] bridge-domain-match Configures route map for bridge domain.
Example: Note Disables the bridge domain (BD)
apic1(config-leaf-vrf-route-map)# no match in a route map, eliminating
bridge-domain-match the need to delete the BD
configuration from the route map.
This is required if there are BDs
matched in a route map, and the
route map is used to filter out the
BD subnets using route
group/explicit prefix list.
Examples
This example shows how to create a route-map and add/match a prefix-list, a community-list, and a
bridge-domain.
# CREATE A ROUTE-MAP
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant exampleCorp vrf v1
apic1(config-leaf-vrf)# route-map bgpMap
# CREATE A PREFIX-LIST
apic1(config-leaf-vrf-route-map)# ip prefix-list list1 permit 13.13.13.0/24
apic1(config-leaf-vrf-route-map)# ip prefix-list list1 permit 14.14.14.0/24
# CREATE A BRIDGE-DOMAIN
apic1# configure
apic1(config)# tenant exampleCorp
apic1(config-tenant)# vrf context v1
apic1(config-tenant-vrf)# exit
apic1(config-tenant)# bridge-domain bd1
apic1(config-tenant-bd)# vrf member v1
apic1(config-tenant-bd)# exit
apic1(config-tenant)# interface bridge-domain bd1
apic1(config-tenant-interface)# ip address 13.13.13.1/24 scope public
apic1(config-tenant-interface)# exit
apic1(config-tenant)# exit
Examples
This example shows how to configure a route-map in BGP, OSPF and EIGRP.
# BGP
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router bgp 100
apic1(config-bgp)# vrf member tenant exampleCorp vrf v1
# OSPF
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router ospf default
apic1(config-leaf-ospf)# vrf member tenant exampleCorp vrf v1
apic1(config-leaf-ospf-vrf)# area 0.0.0.1 route-map map1 out
apic1(config-leaf-ospf-vrf)# area 0.0.0.1 route-map map2 in
apic1(config-leaf-ospf-vrf)# exit
apic1(config-leaf-ospf)# exit
apic1(config-leaf)# exit
#EIGRP
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface ethernet 1/3
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf v1
apic1(config-leaf-if)# ip address 13.13.13.13/24
apic1(config-leaf-if)# ip router eigrp default
apic1(config-leaf-if)# ip distribute-list eigrp default route-map map1 out
apic1(config-leaf-if)# exit
apic1(config-leaf)# exit
Procedure
Step 4 [no] export map map-name Configures route-map in this VRF to export
(leak) routes from this VRF into consumer
Example:
VRFs.
apic1(config-leaf-vrf)# export map
shared-route-map1
Examples
This example shows how to create and export a route-map.
# CREATE A ROUTE-MAP
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant exampleCorp vrf v1
apic1(config-leaf-vrf)# router-id 1.2.3.4
apic1(config-leaf-vrf)# route-map shared-route-map1
apic1(config-leaf-vrf-route-map)# ip prefix-list list1 permit 13.13.13.0/24
apic1(config-leaf-vrf-route-map)# match prefix-list list1
apic1(config-leaf-vrf-route-map-match)# contract provider prov1
apic1(config-leaf-vrf-route-map-match)# exit
apic1(config-leaf-vrf-route-map)# exit
apic1(config-leaf-vrf)# exit
apic1(config-leaf)# exit
Procedure
Step 2 [no] template bfd {ip | ipv6} Creates a BFD policy template.
global-policy-name
Example:
apic1(config)# template bfd ip
bfd_global
Step 3 [no] echo-address ip-address Specifies the IP address to use as the source
address for BFD echo packets.
Example:
apic1(config-bfd)# echo-address
192.0.20.123
apic1(config-bfd)# echo-address 34::1/64
Step 4 [no] slow-timer milliseconds Configures the slow timer used in the echo
function. This value determines how fast BFD
Example:
starts up a new sessions and at what speed the
apic1(config-bfd)# slow-timer 2000 asynchrounous sessions use for BFD control
packets when the echo function is enabled. The
slow-timer value is used as the new control
packet interval, while the echo packets use the
configured BFD intervals. The echo packets
are used for link failure detection, while the
control packets at the slower rate maintain the
BFD session. The range is from 1000 to 30000
milliseconds.
Step 5 [no] min-tx milliseconds Specifies the interval at which this device
wants to send BFD hello messages. The range
Example:
is 50 to 999 milliseconds.
apic1(config-bfd)# min-tx 100
Step 7 [no] multiplier policy-name Specifies the number of missing BFD hello
messages from another BFD device before this
Example:
local device detects a fault in the forwarding
apic1(config-bfd)# multiplier 3 path. The range is 1 to 50.
Step 11 [no] inherit bfd {ip | ipv6} Inherits the previously created BFD global
global-policy-name policies.
Example:
apic1(config-leaf-policy-group)# inherit
bfd ip bfd_global
Step 15 [no] leaf-policy-group leaf-policy-name Specifies the previously created leaf policy
group to be associated to the leaf switches.
Example:
Step 16 [no] leaf leaf-range Adds one or more leaf switches to the leaf
switch group.
Example:
apic1(config-leaf-group)# leaf 101-102
Examples
This example shows how to configure BFD globally and apply it to a group of leaf switches.
# CONFIGURE AN ACCESS LEAF POLICY GROUP AND INHERIT BFD GLOBAL POLICIES
apic1(config)# template leaf-policy-group leaf_pg1
apic1(config-leaf-policy-group)# inherit bfd ip bfd_global
apic1(config-leaf-policy-group)# exit
Configuring BFD Globally on Leaf Switch Using the NX-OS Style CLI
Procedure
Step 1 To configure the BFD IPV4 global configuration (bfdIpv4InstPol) using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# template bfd ip bfd_ipv4_global_policy
apic1(config-bfd)# [no] echo-address 1.2.3.4
apic1(config-bfd)# [no] slow-timer 2500
apic1(config-bfd)# [no] min-tx 100
apic1(config-bfd)# [no] min-rx 70
apic1(config-bfd)# [no] multiplier 3
apic1(config-bfd)# [no] echo-rx-interval 500
apic1(config-bfd)# exit
Step 2 To configure the BFD IPV6 global configuration (bfdIpv6InstPol) using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# template bfd ipv6 bfd_ipv6_global_policy
apic1(config-bfd)# [no] echo-address 34::1/64
apic1(config-bfd)# [no] slow-timer 2500
apic1(config-bfd)# [no] min-tx 100
apic1(config-bfd)# [no] min-rx 70
apic1(config-bfd)# [no] multiplier 3
apic1(config-bfd)# [no] echo-rx-interval 500
apic1(config-bfd)# exit
Step 3 To configure access leaf policy group (infraAccNodePGrp) and inherit the previously created BFD global
policies using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# template leaf-policy-group test_leaf_policy_group
apic1(config-leaf-policy-group)# [no] inherit bfd ip bfd_ipv4_global_policy
apic1(config-leaf-policy-group)# [no] inherit bfd ipv6 bfd_ipv6_global_policy
apic1(config-leaf-policy-group)# exit
Step 4 To associate the previously created leaf policy group onto a leaf using the NX-OS CLI:
Example:
Configuring BFD Globally on Spine Switch Using the NX-OS Style CLI
Use this procedure to configure BFD globally on spine switch using the NX-OS style CLI.
Procedure
Step 1 To configure the BFD IPV4 global configuration (bfdIpv4InstPol) using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# template bfd ip bfd_ipv4_global_policy
apic1(config-bfd)# [no] echo-address 1.2.3.4
apic1(config-bfd)# [no] slow-timer 2500
apic1(config-bfd)# [no] min-tx 100
apic1(config-bfd)# [no] min-rx 70
apic1(config-bfd)# [no] multiplier 3
apic1(config-bfd)# [no] echo-rx-interval 500
apic1(config-bfd)# exit
Step 2 To configure the BFD IPV6 global configuration (bfdIpv6InstPol) using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# template bfd ipv6 bfd_ipv6_global_policy
apic1(config-bfd)# [no] echo-address 34::1/64
apic1(config-bfd)# [no] slow-timer 2500
apic1(config-bfd)# [no] min-tx 100
apic1(config-bfd)# [no] min-rx 70
apic1(config-bfd)# [no] multiplier 3
apic1(config-bfd)# [no] echo-rx-interval 500
apic1(config-bfd)# exit
Step 3 To configure spine policy group and inherit the previously created BFD global policies using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# template spine-policy-group test_spine_policy_group
apic1(config-spine-policy-group)# [no] inherit bfd ip bfd_ipv4_global_policy
apic1(config-spine-policy-group)# [no] inherit bfd ipv6 bfd_ipv6_global_policy
apic1(config-spine-policy-group)# exit
Step 4 To associate the previously created spine policy group onto a spine switch using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# spine-profile test_spine_profile
apic1(config-spine-profile)# spine-group test_spine_group
apic1(config-spine-group)# spine-policy-group test_spine_policy_group
apic1(config-spine-group)# spine 103-104
apic1(config-leaf-group)# exit
Procedure
Step 7 [no] vrf context tenant tenant-name vrf Configures a tenant VRF on the node.
vrf-name
Example:
apic1(config-leaf)# vrf context tenant
exampleCorp vrf vrf1
Step 12 [no] template bfd template-name tenant Configures a BFD interface policy.
tenant-name
Example:
apic1(config-leaf)# template bfd
bfdIfPol1 tenant exampleCorp
Step 13 [no] echo-mode enable Enables or disables the sending of BFD echo
packets in addition to BFD control packets.
Example:
apic1(config-template-bfd-pol)#
echo-mode enable
Step 15 [no] min-tx milliseconds Specifies the interval at which this device
sends BFD hello messages. The range is 50 to
Example:
999 milliseconds.
apic1(config-template-bfd-pol)# min-tx
100
Step 16 [no] min-rx milliseconds Specifies the minimum interval at which this
device can accept BFD hello messages from
Example:
another BFD device. The range is 50 to 999
apic1(config-template-bfd-pol)# min-rx milliseconds.
70
Step 17 [no] multiplier policy-name Specifies the number of missing BFD hello
messages from another BFD device before this
Example:
local device detects a fault in the forwarding
apic1(config-template-bfd-pol)# path. The range is 1 to 50.
multiplier 5
Examples
This example shows how to configure a BFD override policy and apply it to an interface.
apic1# configure
apic1(config)# tenant exampleCorp
apic1(config-tenant)# vrf context vrf1
apic1(config-tenant-vrf)# exit
apic1(config-tenant)# exit
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant exampleCorp vrf vrf1
apic1(config-leaf-vrf)# exit
apic1(config-leaf)# interface eth 1/18
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf vrf1
apic1(config-leaf-if)# exit
Procedure
Step 5 [no] vrf member tenant tenant-name vrf Attaches the interface to the tenant VRF.
vrf-name
Note This command is used only if the
Example: interface is a VLAN interface.
apic1(config-leaf-if)# vrf member tenant
exampleCorp vrf vrf1
Step 6 bfd {ip | ipv6} tenant mode Enables BFD tenant mode.
Example:
apic1(config-leaf-if)# bfd ip tenant mode
Step 7 bfd {ip | ipv6} inherit interface-policy Inherits the specified BFD interface template
policy-name policy.
Example:
apic1(config-leaf-if)# bfd ip inherit
interface-policy bfdIfPol1
Step 8 bfd {ip | ipv6} authentication keyed-sha1 Configures BFD authentication as keyed
keyid keyid key key SHA-1.
Example:
apic1(config-leaf-if)# bfd ip
authentication keyed-sha1 key 10 key
password
Examples
This example shows how to inherit the previously created BFD interface policy onto a L3 interface
with an IPv4 address.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface eth 1/15
apic1(config-leaf-if)# bfd ip tenant mode
apic1(config-leaf-if)# bfd ip inherit interface-policy bfdIfPol1
apic1(config-leaf-if)# bfd ip authentication keyed-sha1 key 10 key password
This example shows how to inherit the previously created BFD interface policy onto a L3 interface
with an IPv6 address.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface eth 1/15
apic1(config-leaf-if)# ipv6 address 2001::10:1/64 preferred
This example shows how to configure BFD on a VLAN interface with an IPv4 address.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface vlan 15
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf vrf1
apic1(config-leaf-if)# bfd ip tenant mode
apic1(config-leaf-if)# bfd ip inherit interface-policy bfdIfPol1
apic1(config-leaf-if)# bfd ip authentication keyed-sha1 key 10 key password
This example shows how to configure BFD on a VLAN interface with an IPv6 address.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface vlan 15
apic1(config-leaf-if)# ipv6 address 2001::10:1/64 preferred
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf vrf1
apic1(config-leaf-if)# bfd ip tenant mode
apic1(config-leaf-if)# bfd ip inherit interface-policy bfdIfPol1
apic1(config-leaf-if)# bfd ip authentication keyed-sha1 key 10 key password
Procedure
Step 7 vrf member tenant tenant-name vrf vrf-name Specifies the VRF instance to associate with
subsequent policy configuration mode
Example:
commands.
apic1(config-bgp)# vrf member tenant
exampleCorp vrf v100
Step 9 [no] bfd enable Enables or disables BFD on the BGP consumer
protocol.
Example:
apic1(config-leaf-bgp-vrf-neighbor)# bfd
enable
Examples
This example shows how to enable BFD on the BGP consumer protocol.
apic1# configure
apic1(config)# bgp-fabric
apic1(config-bgp-fabric)# asn 200
apic1(config-bgp-fabric)# exit
apic1(config)# leaf 101
apic1(config-leaf)# router bgp 200
apic1(config-bgp)# vrf member tenant exampleCorp vrf v100
apic1(config-leaf-bgp-vrf)# neighbor 1.2.3.4
apic1(config-leaf-bgp-vrf-neighbor)# bfd enable
Procedure
Step 4 [no] {ip | ipv6} bfd eigrp enable Enables or disables BFD on the EIGRP
consumer protocol.
Example:
apic1(config-leaf-if)# ip bfd eigrp
enable
Examples
This example shows how to enable BFD on the EIGRP consumer protocol.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface eth 1/15
apic1(config-leaf-if)# ip bfd eigrp enable
Procedure
Step 4 [no] ip ospf bfd enable Enables or disables BFD on the OSPF consumer
protocol.
Example:
apic1(config-leaf-if)# ip ospf bfd enable
Examples
This example shows how to enable BFD on the OSPF consumer protocol.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface vlan 123
apic1(config-leaf-if)# ip ospf bfd enable
Procedure
Step 3 [no] vrf context tenant tenant-name vrf Configures a tenant VRF on the node.
vrf-name
Example:
apic1(config-leaf)# vrf context tenant
exampleCorp vrf vrf1
Step 4 [no] {ip | ipv6} route ip-prefix/masklen Enables or disables BFD on the static route
next-hop-address bfd consumer protocol.
Example:
apic1(config-leaf-vrf)# ip route
10.0.0.1/16 10.0.0.5 bfd
Examples
This example shows how to enable BFD on the static route consumer protocol.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant exampleCorp vrf vrf1
apic1(config-leaf-vrf)# ip route 10.0.0.1/16 10.0.0.5 bfd
Step 1 To enable BFD on the BGP consumer protocol using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# bgp-fabric
apic1(config-bgp-fabric)# asn 200
apic1(config-bgp-fabric)# exit
apic1(config)# leaf 101
apic1(config-leaf)# router bgp 200
apic1(config-bgp)# vrf member tenant t0 vrf v0
apic1(config-leaf-bgp-vrf)# neighbor 1.2.3.4
apic1(config-leaf-bgp-vrf-neighbor)# [no] bfd enable
Step 2 To enable BFD on the EIGRP consumer protocol using the NX-OS CLI:
Example:
Step 3 To enable BFD on the OSPF consumer protocol using the NX-OS CLI:
Example:
apic1# configure
apic1(config)# spine 103
apic1(config-spine)# interface ethernet 5/3.4
apic1(config-spine-if)# [no] ip ospf bfd enable
Step 4 To enable BFD on the Static Route consumer protocol using the NX-OS CLI:
Example:
Step 5 To enable BFD on IS-IS consumer protocol using the NX-OS CLI:
Example:
• Bidirectional PIM, Rendezvous Point (RP) within the ACI fabric, and PIM IPv6 are currently not
supported.
• IGMP snooping cannot be disabled on pervasive bridge domains with multicast routing enabled.
• Multicast routers are not supported in pervasive bridge domains.
• The Layer 3 multicast feature is supported on the following -EX model leaf switches:
• N9K-93180YC-EX
• N9K-93108TC-EX
• N9K-93180LC-EX
• Layer 3 Out ports and sub-interfaces are supported while external SVIs are not supported. Since external
SVIs are not supported, PIM cannot be enabled in L3-VPC.
• Enabling PIM on an L3Out causes an implicit external network to be configured. This results in the
L3Out being deployed and protocols potentially coming up even if you have not defined an external
network.
• For Layer 3 multicast support for multipod, when the ingress leaf switch receives a packet from a source
attached on a bridge domain that is enabled for multicast routing, the ingress leaf switch sends only a
routed VRF copy to the fabric (routed implies that the TTL is decremented by 1, and the source-mac is
rewritten with a pervasive subnet MAC). The egress leaf switch also routes the packet into receivers in
all the relevant bridge domains. Therefore, if a receiver is on the same bridge domain as the source, but
on a different leaf switch than the source, that receiver continues to get a routed copy, even though it is
in the same bridge domain.
For more information, see details about layer 3 multicast support for multipod that leverages existing
Layer 2 design, at the following link Adding Pods.
• Starting with Release 3.1(1x), Layer 3 multicast is supported with FEX. Multicast sources or receivers
connected to FEX ports are supported. For further details about how to add FEX in your testbed, see
Configure a Fabric Extender with Application Centric Infrastructure at this URL: https://www.cisco.com/
c/en/us/support/docs/cloud-systems-management/application-policy-infrastructure-controller-apic/
200529-Configure-a-Fabric-Extender-with-Applica.html. For releases preceeding Release 3.1(1x), Layer
3 multicast is not supported with FEX. Multicast sources or receivers connected to FEX ports are not
supported.
Note Cisco ACI does not support IP fragmentation. Therefore, when you configure Layer 3 Outside (L3Out)
connections to external routers, or multipod connections through an Inter-Pod Network (IPN), it is critical
that the interface MTU is set appropriately on both ends of a link. On some platforms, such as Cisco ACI,
Cisco NX-OS, and Cisco IOS, the configurable MTU value does not take into account the ethernet headers
(matching IP MTU, and excluding the 14-18 ethernet header size), while other platforms, such as IOS-XR,
include the ethernet header in the configured MTU value. A configured value of 9000 results in a max IP
packet size of 9000 bytes in Cisco ACI, Cisco NX-OS, and Cisco IOS, but results in a max IP packet size of
8986 bytes for an IOS-XR untagged interface.
For the appropriate MTU values for each platform, see the relevant configuration guides.
We highly recommend that you test the MTU using CLI-based commands. For example, on the Cisco NX-OS
CLI, use a command such as ping 1.1.1.1 df-bit packet-size 9000 source-interface ethernet 1/1.
Step 5 (Optional) [no] ip pim auto-rp {forward Configures PIM auto-RP (Rendezvous Point)
[listen] | listen | mapping-agent-policy options. Auto-RP automates the distribution
mapping-agent-policy-name} of group-to-RP mappings in a PIM network.
You can choose to forward auto-RP messages,
Example:
listen to auto-RP messages, or associate a
apic1(config-tenant-vrf)# ip pim auto-rp route-map policy for filtering mapping agent
forward listen
messages.
Step 6 (Optional) [no] ip pim bsr {forward [listen] Configures PIM bootstrap router (BSR)
| listen | bsr-policy options. BSR performs similarly to auto-RP
mapping-agent-policy-name} in that it uses candidate routers for the RP
function and for relaying the RP information
Example:
for a group. RP information is distributed
apic1(config-tenant-vrf)# ip pim bsr through BSR messages, which are carried
forward listen
within PIM messages. You can choose to
Step 7 (Optional) [no] ip pim fast-convergence Enables the PIM fast convergence feature,
which allows the switch to discover
Example:
unresponsive neighbors more quickly.
apic1(config-tenant-vrf)# ip pim
fast-convergence
Step 8 (Optional) [no] ip pim mtu mtu-size Configures the maximum size of a PIM
message. The range is 1500 to 65536 bytes.
Example:
apic1(config-tenant-vrf)# ip pim mtu
1500
Step 9 (Optional) [no] ip pim register-policy Specifies the name of a policy for filtering
register-policy-name register messages.
Example:
apic1(config-tenant-vrf)# ip pim
register-policy regPolicy1
Step 10 (Optional) [no] ip pim register-rate-limit Specifies a rate limit for PIM data registers.
mtu-size The range is 0 to 65535 packets per second.
Example:
apic1(config-tenant-vrf)# ip pim
register-rate-limit 1024
Step 11 (Optional) [no] ip pim register-source Configures a source IP address for PIM
ip-address messages.
Example:
apic1(config-tenant-vrf)# ip pim
register-source 192.0.20.123
Step 12 (Optional) [no] ip pim rp-address ip-address Configures a static route processor (RP)
[route-map route-map-name] address for a multicast group range.
Example:
apic1(config-tenant-vrf)# ip pim
rp-address 192.0.20.99
Step 13 (Optional) [no] ip pim sg-expiry-timer Configures the (S, G) expiry timer interval for
ip-address [sg-list route-map-name] PIM sparse mode (PIM-SM) (S, G) multicast
routes. The range is 180 to 604801 seconds.
Example:
The optional sg-list parameter specifies S,G
apic1(config-tenant-vrf)# ip pim values to which the timer applies. The default
sg-expiry-timer 4096
is 4096.
Step 14 (Optional) [no] ip pim ssm route-map Configures Source Specific Multicast (SSM),
route-map-name which is an extension of IP multicast in which
Step 15 (Optional) [no] ip pim state-limit max-entries Configures a maximum number of PIM state
[reserved route-map-name entries in the current VRF instance. The range
[maximum-reserve-state-entries]] is 0 to 4294967295 maximum state entries.
You can optionally specify a number of state
Example:
entries to be reserved for the routes specified
apic1(config-tenant-vrf)# ip pim in a policy map and you can specify the
state-limit 100000 reserved
myReservedPolicy 40000
maximum reserved (*, G) and (S, G) entries
allowed in this VRF. This number must be less
than or equal to the maximum states allowed.
The range is from 1 to 4294967295.
Step 16 (Optional) [no] ip pim use-shared-tree-only Creates the PIM (*, G) state only (where no
group-list policy-name source state is created). The policy defines the
group prefixes where this feature is applied.
Example:
apic1(config-tenant-vrf)# ip pim
use-shared-tree-only group-list myGroup1
What to do next
Configure IGMP options for the VRF.
Procedure
Step 10 [no] ip igmp group-timeout seconds Sets the group membership timeout for
IGMPv2. The range is 3 to 65535 seconds. The
Example:
default is 260 seconds.
apic1(config-tenant-interface)# ip igmp
group-timeout 260
Step 11 [no] ip igmp inherit interface-policy Associates a IGMP interface policy to this
policy-name interface.
Example:
apic1(config-tenant-interface)# ip igmp
inherit interface-policy MyIfPolicy
Step 13 [no] ip igmp last-member-query-count count Sets the number of times that the software
sends an IGMP query in response to a host
Example:
leave message. The range is 1 to 5 queries. The
apic1(config-tenant-interface)# ip igmp default is 2 queries.
last-member-query-count 2
Step 14 [no] ip igmp Sets the query interval waited after sending
last-member-query-response-time seconds membership reports before the software deletes
the group state. The range is 1 to 25 seconds.
Example:
The default is 1 second.
apic1(config-tenant-interface)# ip igmp
last-member-query-response-time 1
Step 15 [no] ip igmp querier-timeout seconds Sets the number of seconds that the software
waits after the previous querier has stopped
Example:
querying and before it takes over as the querier.
apic1(config-tenant-interface)# ip igmp The range is 1 to 65535 seconds. The default
querier-timeout 255
is 255 seconds.
Step 16 [no] ip igmp query-interval seconds Sets the frequency at which the software sends
IGMP host query messages. You can tune the
Example:
number of IGMP messages on the network by
apic1(config-tenant-interface)# ip igmp setting a larger value so that the software sends
query-interval 125
IGMP queries less often. The range is 1 to
18000 seconds. The default is 125 seconds.
Step 17 [no] ip igmp query-max-response-time Sets the response time advertised in IGMP
seconds queries. You can tune the burstiness of IGMP
messages on the network by setting a larger
Example:
value so that host responses are spread out over
apic1(config-tenant-interface)# ip igmp a longer time. This value must be less than the
query-max-response-time 10
query interval. The range is 1 to 25 seconds.
The default is 10 seconds.
Step 19 [no] ip igmp report-policy policy-name Configures an access policy for IGMP reports
that is based on a route-map policy.
Example:
apic1(config-tenant-interface)# ip igmp
report-policy MyReportPolicy
Step 21 [no] ip igmp snooping Enables IGMP snooping for the interface.
Example:
apic1(config-tenant-interface)# ip igmp
snooping
Step 22 [no] ip igmp snooping fast-leave Enables the software to remove the group state
when it receives an IGMP Leave report without
Example:
sending an IGMP query message. This
apic1(config-tenant-interface)# ip igmp parameter is used for IGMPv2 hosts when no
snooping fast-leave
more than one host is present on each port.
Step 23 [no] ip igmp snooping Sets a time interval in seconds after which the
last-member-query-interval group is removed from the associated port if
no hosts respond to an IGMP query message.
Example:
The range is 1 to 25 seconds. The default is 5
apic1(config-tenant-interface)# ip igmp seconds.
snooping last-member-query-interval 5
Step 24 [no] ip igmp snooping policy policy-name Associates the bridge domain with an IGMP
snooping policy.
Example:
apic1(config-tenant-interface)# ip igmp
snooping policy MySnoopingPolicy
Step 25 [no] ip igmp snooping querier Enables an IP IGMP snooping querier, which
sends out periodic IGMP queries that trigger
Example:
IGMP report messages from hosts who want
apic1(config-tenant-interface)# ip igmp to receive IP multicast traffic. IGMP snooping
snooping querier
listens to these IGMP reports to establish
appropriate forwarding.
Step 26 [no] ip igmp snooping query-interval Configures a snooping query interval when
seconds you do not enable PIM because multicast
traffic does not need to be routed. The range
Example:
is 1 to 18000 seconds. The default is 125
apic1(config-tenant-interface)# ip igmp seconds.
snooping query-interval 125
Step 30 [no] ip igmp startup-query-count count Sets the number of queries sent at startup that
are separated by the startup query interval. The
Example:
range is 1 to 10 queries. The default is 2
apic1(config-tenant-interface)# ip igmp queries.
startup-query-count 2
Step 31 [no] ip igmp startup-query-interval seconds Sets the query interval used when the software
starts up. By default, this interval is shorter
Example:
than the query interval so that the software can
apic1(config-tenant-interface)# ip igmp establish the group state as quickly as possible.
startup-query-interval 31
The range is 1 to 18000 seconds. The default
is 260 seconds. The default is 31 seconds.
Step 32 [no] ip igmp state-limit max-states [reserved Configures a per interface limit on the number
route-map-name [max-reserved-gsg-entries]] of mroutes states created as a result of IGMP
membership reports (IGMP joins). The range
Example:
of states allowed is 1 to 4294967295 states.
apic1(config-tenant-interface)# ip igmp You can optionally specify a number of state
state-limit 100000 reserved
myReservedPolicy 40000
entries to be reserved for the routes specified
in a policy map and you can specify the
maximum reserved (*, G) and (S, G) entries
allowed on the interface. The number of
reserved states must be less than or equal to
the maximum states allowed. The range is from
1 to 4294967295.
Step 33 [no] ip igmp static-oif route-map Statically binds a multicast group to the
route-map-name outgoing interface (OIF), which is handled by
the device hardware. The route map defines
Example:
the group prefixes where this feature is applied.
apic1(config-tenant-interface)# ip igmp
static-oif route-map MyOifMap
Step 34 [no] ip igmp version {v1 | v2 | v3} Configures the IGMP version number for the
interface. The default version is v2.
Example:
apic1(config-tenant-interface)# ip igmp
version v3
What to do next
Configure an L3 Out for the tenant, enable PIM, and configure the leaf interface.
Procedure
Step 11 [no] ip igmp group-timeout seconds Sets the group membership timeout for
IGMPv2. The range is 3 to 65535 seconds. The
Example:
default is 260 seconds.
apic1(config-leaf-if)# ip igmp
group-timeout 260
Step 12 [no] ip igmp inherit interface-policy Associates a IGMP interface policy to this
policy-name interface.
Example:
apic1(config-leaf-if)# ip igmp inherit
interface-policy MyIfPolicy
Step 13 [no] ip igmp join-group route-map Statically binds one or more multicast groups
route-map-name to the interface. The route-map policy lists the
group prefixes, group ranges, and source
Example:
prefixes.
apic1(config-leaf-if)# ip igmp
join-group route-map MyGroupsRMap
Step 14 [no] ip igmp last-member-query-count count Sets the number of times that the software
sends an IGMP query in response to a host
Example:
leave message. The range is 1 to 5 queries. The
apic1(config-leaf-if)# ip igmp default is 2 queries.
last-member-query-count 2
Step 15 [no] ip igmp Sets the query interval waited after sending
last-member-query-response-time seconds membership reports before the software deletes
Step 16 [no] ip igmp querier-timeout seconds Sets the number of seconds that the software
waits after the previous querier has stopped
Example:
querying and before it takes over as the querier.
apic1(config-leaf-if)# ip igmp The range is 1 to 65535 seconds. The default
querier-timeout 255
is 255 seconds.
Step 17 [no] ip igmp query-interval seconds Sets the frequency at which the software sends
IGMP host query messages. You can tune the
Example:
number of IGMP messages on the network by
apic1(config-leaf-if)# ip igmp setting a larger value so that the software sends
query-interval 125
IGMP queries less often. The range is 1 to
18000 seconds. The default is 125 seconds.
Step 18 [no] ip igmp query-max-response-time Sets the response time advertised in IGMP
seconds queries. You can tune the burstiness of IGMP
messages on the network by setting a larger
Example:
value so that host responses are spread out over
apic1(config-leaf-if)# ip igmp a longer time. This value must be less than the
query-max-response-time 10
query interval. The range is 1 to 25 seconds.
The default is 10 seconds.
Step 20 [no] ip igmp report-policy policy-name Configures an access policy for IGMP reports
that is based on a route-map policy.
Example:
apic1(config-leaf-if)# ip igmp
report-policy MyReportPolicy
Step 21 [no] ip igmp robustness-variable value Sets the robustness variable to compensate for
packet loss on a congested network. The
Example:
robustness value is used by the IGMP software
apic1(config-leaf-if)# ip igmp to determine the number of times to send
robustness-variable 2
messages. You can use a larger value for a
lossy network. The range is 1 to 7. The default
is 2.
Step 22 [no] ip igmp startup-query-count count Sets the number of queries sent at startup that
are separated by the startup query interval. The
Example:
range is 1 to 10 queries. The default is 2
apic1(config-leaf-if)# ip igmp queries.
startup-query-count 2
Step 24 [no] ip igmp state-limit max-states [reserved Configures a per interface limit on the number
route-map-name [max-reserved-gsg-entries]] of mroutes states created as a result of IGMP
membership reports (IGMP joins). The range
Example:
of states allowed is 1 to 4294967295 states.
apic1(config-leaf-if)# ip igmp You can optionally specify a number of state
state-limit 100000 reserved
myReservedPolicy 40000
entries to be reserved for the routes specified
in a policy map and you can specify the
maximum reserved (*, G) and (S, G) entries
allowed on the interface. The number of
reserved states must be less than or equal to
the maximum states allowed. The range is from
1 to 4294967295.
Step 25 [no] ip igmp static-oif route-map Statically binds a multicast group to the
route-map-name outgoing interface (OIF), which is handled by
the device hardware. The route map defines
Example:
the group prefixes where this feature is applied.
apic1(config-leaf-if)# ip igmp
static-oif route-map MyOifMap
Step 26 [no] ip igmp version {v1 | v2 | v3} Configures the IGMP version number for the
interface. The default version is v2.
Example:
apic1(config-leaf-if)# ip igmp version
v3
apic1# configure
apic1(config)# tenant exampleCorp
apic1(config-tenant)# vrf context exampleCorp_vrf1
apic1(config-tenant-vrf)# ip pim
apic1(config-tenant-vrf)# ip pim fast-convergence
apic1(config-tenant-vrf)# ip pim bsr forward
# ENABLE AND CONFIGURE IGMP ON THE TENANT VRF AND BRIDGE DOMAIN
apic1(config-tenant-vrf)# ip igmp
apic1(config-tenant-vrf)# exit
apic1(config-tenant)# interface bridge-domain exampleCorp_bd
apic1(config-tenant-interface)# ip multicast
apic1(config-tenant-interface)# ip igmp allow-v3-asm
apic1(config-tenant-interface)# ip igmp fast-leave
apic1(config-tenant-interface)# exit
Procedure
apic1(config-tenant-l3ext-epg)# match
ip 192.0.20.0/24
apic1(config-tenant-l3ext-epg)# match
ipv6 2001::1/64
Step 6 set qos-class class Specifies the QOS level for the EPG.
Example:
apic1(config-tenant-l3ext-epg)# set
qos-class level1
Step 7 set dscp dscp-value Specifies the DSCP value for the EPG.
Example:
apic1(config-tenant-l3ext-epg)# set dscp
af31
Step 8 contract consumer contract-name Specifies the consumer contract for the EPG.
Example:
apic1(config-tenant-l3ext-epg)# contract
consumer cConsumer1
Step 9 contract provider contract-name Specifies the provider contract for the EPG.
Example:
apic1(config-tenant-l3ext-epg)# contract
provider cProvider1
Step 10 contract deny contract-name Specifies a deny contract for the EPG.
Example:
apic1(config-tenant-l3ext-epg)# contract
deny cDeny1
Step 11 exit
Example:
apic1(config-tenant-l3ext-epg)# exit
Step 12 exit
Example:
apic1(config-tenant)# exit
Step 15 external-l3 epg epg-name Associates the external layer 3 EPG on the
VRF.
Example:
apic1(config-leaf-vrf)# external-l3 epg
epgExtern1
Examples
This example shows how to configure an external layer 3 EPG and to deploy the EPG on a leaf.
apic1# configure
apic1(config)# tenant exampleCorp
Step 5 vrf member vrf-name Associates the L3Out with the tenant VRF.
Example:
apic1(config-tenant-l3out)# vrf member
v1
Step 9 vrf context tenant tenant-name vrf vrf-name Configures a tenant VRF on the node.
l3out l3out-name
Step 10 Required: [no] router-id ipv4-address Assigns a router ID for routing protocols
running on the VRF.
Example:
apic1(config-leaf-vrf)# router-id
1.2.3.4
Step 11 [no] {ip | ipv6} route ip-prefix/masklen Configures static route information for the
next-hop-address [preferred] VRF.
Example:
apic1(config-leaf-vrf)# ip route
21.1.1.1/32 32.1.1.1
apic1(config-leaf-vrf)# ipv6 route
5001::1/128 6002::1
Examples
This example shows how to create a named L3Out under the tenant, assign it to the tenant VRF, and
deploy it on the border leaf switch.
apic1# configure
apic1(config)# tenant exampleCorp
apic1(config-tenant)# vrf context v1
apic1(config-tenant)# l3out out1
apic1(config-tenant-l3out)# vrf member v1
apic1(config-tenant-l3out)# exit
apic1(config-tenant)# exit
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant exampleCorp vrf v1 l3out out1
apic1(config-leaf-vrf)# router-id 1.2.3.4
apic1(config-leaf-vrf)# ip route 21.1.1.1/32 32.1.1.1
What to do next
Configure layer 3 interfaces for the named L3Out.
• interface Ethernet
• interface port-channel
• interface vpc
Procedure
Step 5 vrf member tenant tenant-name vrf vrf-name Attaches the interface to the tenant VRF.
l3out l3out-name
Example:
apic1(config-leaf-if)# vrf member tenant
exampleCorp vrf v1 l3out out1
Step 6 [no] {ip | ipv6} address ip-prefix/masklen Configures IP addresses on the interface. The
[eui64] [secondary] [preferred] specified address can be declared as either:
Example: • preferred—The default source address
for traffic from the interface.
apic1(config-leaf-if)# ip address
10.1.1.1/24 • secondary—The secondary address of the
apic1(config-leaf-if)# ipv6 address interface.
2001::1/64 preferred
With the optional eui64 keyword, the host can
assign itself a 64-bit Extended Unique Identifier
(EUI).
Examples
This example shows how to assign a layer 3 port to a named L3Out.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface eth 1/20
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf v1 l3out out1
apic1(config-leaf-if)# ip address 10.1.1.1/24
apic1(config-leaf-if)# ipv6 address 2001::1/64 preferred
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface eth 1/5
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vlan-domain member d1
apic1(config-leaf-if)# exit
apic1(config-leaf)# interface ethernet 1/5.1000
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf v1 l3out out1
apic1(config-leaf-if)# ip address 10.1.1.1/24
apic1(config-leaf-if)# ipv6 address 2001::1/64 preferred
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# interface vlan 200
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf v1
apic1(config-leaf-if)# ip address 10.1.1.1/24
apic1(config-leaf-if)# exit
apic1(config-leaf)# interface ethernet 1/4
apic1(config-leaf-if)# vlan-domain member d1
apic1(config-leaf-if)# switchport trunk allowed vlan 200 tenant t1 external-svi l3out out1
• Import—Route-map for routes imported into the routing protocol on the L3Out. By default, all
routes are imported. You can control specific routes to be imported by using one or more match
prefix-list or match community-list statements.
• Shared—Route-map that contains the routes and the contract provider/consumer policy that will be
used for leaking the routes from this VRF to any other VRF that has the contract association.
These route-maps are created when you associate a leaf to the L3Out through the vrf context tenant
tenant-name vrf vrf-name l3out l3out-name command.
• The scope of the route-maps under the named L3Out is always global and is applicable on all nodes
where the Named L3Out is deployed.
• All commands under the route-map (such as match prefix-list,match community-list, match
bridge-domain) are the same as the route-map configuration for the Basic Mode discussed in the previous
sections.
Procedure
Step 3 [no] vrf context tenant tenant-name vrf Configures a tenant VRF on the node.
vrf-name l3out l3out-name
Example:
apic1(config-leaf)# vrf context tenant
exampleCorp vrf v1 l3out out1
Step 4 Required: [no] route-map name Creates a route-map and enters route-map
configuration. This will be the import
Example:
route-map.
apic1(config-leaf-vrf)# route-map
out1_in
Step 5 Required: [no] ip prefix-list list-name permit Creates a prefix-list under the route-map.
prefix/masklen [le {32 | 128}]
Example:
apic1(config-leaf-vrf-route-map)# ip
prefix-list p1 permit 15.1.1.0/24
Step 9 Required: [no] route-map name Creates a route-map and enters route-map
configuration. This will be the export
Example:
route-map.
apic1(config-leaf-vrf)# route-map
out1_out
Step 10 Required: [no] ip prefix-list list-name permit Creates a prefix-list under the route-map.
prefix/masklen [le {32 | 128}]
Example:
apic1(config-leaf-vrf-route-map)# ip
prefix-list p2 permit 16.1.1.0/24
Step 11 Required: [no] match prefix-list list-name Matches a prefix-list that has already been
created and enters the match mode to configure
Example:
the route-control profile for the prefix-list.
apic1(config-leaf-vrf-route-map)# match
prefix-list p2
Step 12 Required: set tag name Sets the tag value. The name parameter is an
unsigned integer.
Example:
apic1(config-leaf-vrf-route-map-match)#
set tag 100
Step 14 Required: [no] match bridge-domain Matches a bridge domain in order to export its
list-name public subnets through the protocol.
Example:
apic1(config-leaf-vrf-route-map)# match
bridge-domain bd1
Step 16 Required: [no] route-map name Creates a route-map and enters route-map
configuration. This will be the shared
Example:
route-map.
apic1(config-leaf-vrf)# route-map
out1_shared
Step 17 Required: [no] ip prefix-list list-name permit Creates a prefix-list under the route-map.
prefix/masklen [le {32 | 128}]
Example:
apic1(config-leaf-vrf-route-map)# ip
prefix-list p3 permit 16.10.1.0/24
Step 18 Required: [no] match prefix-list list-name Matches a prefix-list that has already been
created and enters the match mode to configure
Example:
the route-control profile for the prefix-list.
apic1(config-leaf-vrf-route-map)# match
prefix-list p3
Step 19 Required: contract provider name Adds contract, required to leak routes
(matching this prefix-list) from this VRF.
Example:
apic1(config-leaf-vrf-route-map-match)#
contract provider default
Examples
This example shows how to configure route maps for a named L3Out.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant exampleCorp vrf v1 l3out out1
Procedure
Step 4 vrf member tenant tenant-name vrf vrf-name Specifies the VRF instance to associate with
subsequent policy configuration mode
Example:
commands.
apic1(config-bgp)# vrf member tenant
exampleCorp vrf v100
Step 8 update-source ethernet interface-range Update the Source IP for BGP Packets to one
of loopback, physical, sub-interface or SVI
Example:
interfaces..
apic1(config-leaf-bgp-vrf-neighbor)#
update-source ethernet 1/3
Examples
This example shows how to configure BGP routing protocol for a named L3Out.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router bgp 100
apic1(config-bgp)# vrf member tenant exampleCorp vrf v1
apic1(config-leaf-bgp-vrf)# neighbor 192.0.2.229 l3out out1
apic1(config-leaf-bgp-vrf-neighbor)# remote-as 300
apic1(config-leaf-bgp-vrf-neighbor)# allow-self-as-count 5
apic1(config-leaf-bgp-vrf-neighbor)# update-source ethernet 1/3
Procedure
Step 4 vrf member tenant tenant-name vrf vrf-name Enables a VRF in the OSPF session.
Example:
apic1(config-leaf-ospf)# vrf member
tenant exampleCorp vrf v100
Step 6 area area-id loopback loopback-address When OSPF is used as a connectivity protocol
for BGP, OSPF advertises the loopback
Example:
address which is used as the source of the BGP
apic1(config-leaf-ospf-vrf)# area session. Note that the loopback IP address and
0.0.0.1 loopback 192.0.20.11
not the loopback ID is used. In this case, a
BGP session relying on OSPF will use the
same loopback IP address in its update-source
command.
Step 11 vlan-domain member domain-name Assign a VLAN domain to the interface. The
VLAN domain must have already been created
Example:
using the vlan-domain command in the global
apic1(config-leaf-if)# vlan-domain configuration mode.
member dom1
Step 13 vrf member tenant tenant-name vrf vrf-name Attaches the interface to the tenant VRF.
l3out l3out-name
Example:
apic1(config-leaf-if)# vrf member tenant
exampleCorp vrf v1 l3out out1
Step 14 [no] {ip | ipv6} address ip-prefix/masklen Configures IP addresses on the interface. The
[eui64] [secondary] [preferred] specified address can be declared as either:
Example: • preferred—The default source address
for traffic from the interface.
apic1(config-leaf-if)# ip address
10.1.1.1/24 • secondary—The secondary address of
apic1(config-leaf-if)# ipv6 address the interface.
2001::1/64 preferred
With the optional eui64 keyword, the host can
assign itself a 64-bit Extended Unique
Identifier (EUI).
In this mode, you can also configure ipv6
link-local, mac address, mtu, and other layer
3 properties on the interface.
Step 15 {ip | ipv6} router ospf default area area-id Creates an OSPF routing process and enters
OSPF policy configuration.
Example:
apic1(config-leaf-if)# ip router ospf
default area 0.0.0.1
Examples
This example shows how to configure OSPF routing protocol for a named L3Out.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router ospf default
apic1(config-leaf-ospf)# vrf member tenant exampleCorp vrf v1
apic1(config-leaf-ospf-vrf)# area 0.0.0.1 l3out out1
apic1(config-leaf-ospf-vrf)# area 0.0.0.1 loopback 192.0.20.11
apic1(config-leaf-ospf-vrf)# area 0.0.0.1 nssa
apic1(config-leaf-ospf-vrf)# exit
apic1(config-leaf-ospf)# exit
apic1(config-leaf)# interface eth 1/20
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member tenant exampleCorp vrf v1 l3out out1
apic1(config-leaf-if)# ip address 10.1.1.1/24
Procedure
Step 4 vrf member tenant tenant-name vrf vrf-name Specifies the VRF instance to associate with
subsequent configuration mode commands.
Example:
apic1(config-eigrp)# vrf member tenant
exampleCorp vrf v100
Step 5 autonomous-system asn l3out l3out-name Enters Autonomous System configuration for
EIGRP.
Example:
apic1(config-eigrp-vrf)#
autonomous-system 500 l3out out1
Step 9 vlan-domain member domain-name Assign a VLAN domain to the interface. The
VLAN domain must have already been created
Example:
using the vlan-domain command in the global
apic1(config-leaf-if)# vlan-domain configuration mode.
member dom1
Step 11 vrf member tenant tenant-name vrf vrf-name Attaches the interface to the tenant VRF.
l3out l3out-name
Example:
apic1(config-leaf-if)# vrf member tenant
exampleCorp vrf v1 l3out out1
Step 12 [no] {ip | ipv6} address ip-prefix/masklen Configures IP addresses on the interface. The
[eui64] [secondary] [preferred] specified address can be declared as either:
Example: • preferred—The default source address
for traffic from the interface.
apic1(config-leaf-if)# ip address
10.1.1.1/24 • secondary—The secondary address of
apic1(config-leaf-if)# ipv6 address the interface.
2001::1/64 preferred
With the optional eui64 keyword, the host can
assign itself a 64-bit Extended Unique
Identifier (EUI).
In this mode, you can also configure ipv6
link-local, mac address, mtu, and other layer
3 properties on the interface.
Step 13 {ip | ipv6} router eigrp default Sets EIGRP policies to default.
Example:
apic1(config-leaf-if)# ip router eigrp
default
Examples
This example shows how to configure EIGRP routing protocol for a named L3Out.
apic1# configure
apic1(config)# leaf 101
apic1(config-leaf)# router eigrp default
apic1(config-eigrp)# vrf member tenant exampleCorp vrf v1
apic1(config-eigrp-vrf)# autonomous-system 500 l3out out1
apic1(config-eigrp-vrf)# exit
apic1(config-eigrp)# exit
Procedure
Step 3 external-l3 epg epg-name l3out l3out-name Enters the external-l3 EPG configuration mode.
Example:
apic1(config-tenant)# external-l3 epg
epg1 l3out out1
apic1(config-tenant-l3ext-epg)# match ip
192.0.20.0/24
apic1(config-tenant-l3ext-epg)# match
ipv6 2001::1/64
Step 5 contract consumer contract-name Specifies the consumer contract for the EPG.
Example:
Step 6 contract provider contract-name Specifies the provider contract for the EPG.
Example:
apic1(config-tenant-l3ext-epg)# contract
provider cProvider1
Examples
This example shows how to configure an external layer 3 EPG for a named L3Out.
apic1# configure
apic1(config)# tenant exampleCorp
apic1(config-tenant)# external-l3 epg epg1 l3out out1
apic1(config-tenant-l3ext-epg)# match ip 192.0.20.0/24
apic1(config-tenant-l3ext-epg)# match ipv6 2001::1/64
apic1(config-tenant-l3ext-epg)# contract consumer cConsumer1
apic1(config-tenant-l3ext-epg)# contract provider cProvider1
• Configurable ND subnets for external networks, and arbitrary subnet configurations for pervasive bridge
domains are not supported.
• Per Interface
• Control of ND packets (NS/NA)
• Neighbor Solicitation Interval
• Neighbor Solicitation Retry count
• Control of RA packets
• Suppress RA
• Suppress RA MTU
• RA Interval, RA Interval minimum, Retransmit time
Configuring a Tenant, VRF, and Bridge Domain with IPv6 Neighbor Discovery
on the Bridge Domain Using the NX-OS Style CLI
Procedure
Step 1 Configure an IPv6 neighbor discovery interface policy and assign it to a bridge domain:
a) Create an IPv6 neighbor discovery interface policy:
Example:
Step 2 Configure an IPV6 bridge domain subnet and neighbor discovery prefix policy on the subnet:
Example:
Procedure
Step 2 tenant tenant_name Creates a tenant and enters the tenant mode.
Example:
Step 4 ipv6 nd mtu mtu value Assigns an MTU value to the IPv6 ND policy.
Example:
apic1(config-tenant-template-ipv6-nd)#
ipv6 nd mtu 1500
apic1(config-tenant-template-ipv6)# exit
apic1(config-tenant-template)# exit
apic1(config-tenant)#
Step 7 vrf member VRF_name Associates the VRF with the Layer 3 Out.
Example:
Step 8 external-l3 epg instp l3out l3extOut001 Assigns the Layer 3 Out and the VRF to a
Layer 3 interface.
Example:
Step 10 vrf context tenant ExampleCorp vrf pvn1 Associates the VRF to the leaf switch.
l3out l3extOut001
Example:
apic1(config-leaf-vrf)# exit
Step 12 vrf member tenant ExampleCorp vrf pvn1 Specifies the associated Tenant, VRF, Layer
l3out l3extOut001 3 Out in the interface.
Example:
Step 13 ipv6 address 2001:20:21:22::2/64 preferred Specifies the primary or preferred IPv6
address.
Example:
Step 14 ipv6 nd prefix 2001:20:21:22::2/64 1000 Configures the IPv6 ND prefix policy under
1000 the Layer 3 interface.
Example:
Microsoft NLB
Configuring Microsoft NLB in Unicast Mode Using the NX-OS Style CLI
This task configures Microsoft NLB to flood all of the ports in the bridge domain.
Procedure
Configuring Microsoft NLB in Multicast Mode Using the NX-OS Style CLI
This task configures Microsoft NLB to flood only on certain ports in the bridge domain.
Procedure
Step 5 [no] endpoint {ip | ipv6} ip-address epnlb Configures Microsoft NLB in static multicast
mode mode-mcast--static mac mac-address mode, where:
Example: • ip-address is the Microsoft NLB cluster
apic1 (config-tenant-app-epg)# endpoint VIP.
ip 192.0.2.2/32 epnlb mode
mode-mcast--static mac 03:BF:01:02:03:04 • mac-address is the Microsoft NLB cluster
MAC address.
Configuring Microsoft NLB in IGMP Mode Using the NX-OS Style CLI
This task configures Microsoft NLB to flood only on certain ports in the bridge domain.
Procedure
Step 5 [no] endpoint {ip | ipv6} ip-address epnlb Configures Microsoft NLB in IGMP mode,
mode mode-mcast-igmp group where:
multicast-IP-address
• ip-address is the Microsoft NLB cluster
Example: VIP.
apic1 (config-tenant-app-epg)# endpoint
ip 192.0.2.2/32 epnlb mode
• multicast-IP-address is the multicast IP
mode-mcast-igmp group 1.3.5.7 for the NLB endpoint group.
MLD Snooping
Configuring and Assigning an MLD Snooping Policy to a Bridge Domain using
the NX-OS Style CLI
Before you begin
• Create the tenant that will consume the MLD Snooping policy.
• Create the bridge domain for the tenant, where you will attach the MLD Snooping policy.
Procedure
Step 3 template ipv6 mld snooping policy Creates an MLD snooping policy. The example
policy-name NX-OS style CLI sequence creates an MLD
snooping policy named mldPolicy1.
Example:
apic1(config-tenant-template-ip-mld-snooping)#
ipv6 mld snooping
apic1(config-tenant-template-ip-mld-snooping)#
no ipv6 mld snooping
Step 5 [no] ipv6 mld snooping fast-leave Enables or disables IPv6 MLD snooping
fast-leave processing.
Example:
apic1(config-tenant-template-ip-mld-snooping)#
ipv6 mld snooping fast-leave
apic1(config-tenant-template-ip-mld-snooping)#
no ipv6 mld snooping fast-leave
Step 6 [no] ipv6 mld snooping querier Enables or disables IPv6 MLD snooping
querier processing. For the enabling querier
Example:
option to be effectively enabled on the assigned
policy, you must also enable the querier option
apic1(config-tenant-template-ip-mld-snooping)#
ipv6 mld snooping querier in the subnets assigned to the bridge domains
apic1(config-tenant-template-ip-mld-snooping)# to which the policy is applied, as described in
no ipv6 mld snooping querier Step 14, on page 129.
Step 7 ipv6 mld snooping Changes the IPv6 MLD snooping last member
last-member-query-interval parameter query interval parameter. The example NX-OS
style CLI sequence changes the IPv6 MLD
Example:
snooping last member query interval parameter
to 25 seconds. Valid options are 1-25. The
apic1(config-tenant-template-ip-mld-snooping)#
ipv6 mld snooping default is 1 second.
last-member-query-interval 25
Step 8 ipv6 mld snooping query-interval parameter Changes the IPv6 MLD snooping query
interval parameter. The example NX-OS style
Example:
CLI sequence changes the IPv6 MLD snooping
query interval parameter to 300 seconds. Valid
apic1(config-tenant-template-ip-mld-snooping)#
ipv6 mld snooping query-interval 300 options are 1-18000. The default is 125
seconds.
Step 9 ipv6 mld snooping query-max-response-time Changes the IPv6 MLD snooping max query
parameter response time. The example NX-OS style CLI
sequence changes the IPv6 MLD snooping
Example:
max query response time to 25 seconds. Valid
options are 1-25. The default is 10 seconds.
apic1(config-tenant-template-ip-mld-snooping)#
ipv6 mld snooping
query-max-response-time 25
Step 10 ipv6 mld snooping startup-query-count Changes the IPv6 MLD snooping number of
parameter initial queries to send. The example NX-OS
style CLI sequence changes the IPv6 MLD
Example:
Step 11 ipv6 mld snooping startup-query-interval Changes the IPv6 MLD snooping time for
parameter sending initial queries. The example NX-OS
style CLI sequence changes the IPv6 MLD
Example:
snooping time for sending initial queries to
300 seconds. Valid options are 1-18000. The
apic1(config-tenant-template-ip-mld-snooping)#
ipv6 mld snooping default is 31 seconds.
startup-query-interval 300
apic1(config-tenant-template-ip-mld-snooping)#
exit
apic1(config-tenant)#
Step 15 ipv6 mld snooping policy policy-name Associates the bridge domain with an MLD
snooping policy. The example NX-OS style
Example:
CLI sequence associates the bridge domain
with an MLD snooping policy named
apic1(config-tenant-interface)# ipv6
mld snooping policy mldPolicy1 mldPolicy1.
apic1(config-tenant-interface)# exit
apic1(config-tenant)#
Configuring HSRP
Configuring HSRP in Cisco APIC Using Inline Parameters in NX-OS Style CLI
HSRP is enabled when the leaf switch is configured.
Procedure
Configuring HSRP in Cisco APIC Using Template and Policy in NX-OS Style
CLI
HSRP is enabled when the leaf switch is configured.
Procedure
All tenant WAN connections use a single session on the spine switches where the WAN routers are connected.
This aggregation of tenant BGP sessions towards the Data Center Interconnect Gateway (DCIG) improves
control plane scale by reducing the number of tenant BGP sessions and the amount of configuration required
for all of them. The network is extended out using Layer 3 subinterfaces configured on spine fabric ports.
Transit routing with shared services using GOLF is not supported.
A Layer 3 external outside network (L3extOut) for GOLF physical connectivity for a spine switch is specified
under the infra tenant, and includes the following:
• LNodeP (l3extInstP is not required within the L3Out in the infra tenant. )
• A provider label for the L3extOut for GOLF in the infra tenant.
• OSPF protocol policies
• BGP protocol policies
All regular tenants use the above-defined physical connectivity. The L3extOut defined in regular tenants
requires the following:
• An l3extInstP (EPG) with subnets and contracts. The scope of the subnet is used to control import/export
route control and security policies. The bridge domain subnet must be set to advertise externally and it
must be in the same VRF as the application EPG and the GOLF L3Out EPG.
• Communication between the application EPG and the GOLF L3Out EPG is governed by explicit contracts
(not Contract Preferred Groups).
• An l3extConsLbl consumer label that must be matched with the same provider label of an L3Out for
GOLF in the infra tenant. Label matching enables application EPGs in other tenants to consume the
LNodeP external L3Out EPG.
• The BGP EVPN session in the matching provider L3extOut in the infra tenant advertises the tenant
routes defined in this L3Out.
Note Cisco ACI does not support IP fragmentation. Therefore, when you configure Layer 3 Outside (L3Out)
connections to external routers, or multipod connections through an Inter-Pod Network (IPN), it is critical
that the interface MTU is set appropriately on both ends of a link. On some platforms, such as Cisco ACI,
Cisco NX-OS, and Cisco IOS, the configurable MTU value does not take into account the ethernet headers
(matching IP MTU, and excluding the 14-18 ethernet header size), while other platforms, such as IOS-XR,
include the ethernet header in the configured MTU value. A configured value of 9000 results in a max IP
packet size of 9000 bytes in Cisco ACI, Cisco NX-OS, and Cisco IOS, but results in a max IP packet size of
8986 bytes for an IOS-XR untagged interface.
For the appropriate MTU values for each platform, see the relevant configuration guides.
We highly recommend that you test the MTU using CLI-based commands. For example, on the Cisco NX-OS
CLI, use a command such as ping 1.1.1.1 df-bit packet-size 9000 source-interface ethernet 1/1.
Configuration Tasks to Configure Cisco ACI GOLF Services Using the NX-OS
Style CLI
Perform the following tasks to configure GOLF services (using the BGP EVPN protocol), with the NX-OS
style CLI:
• Configure the infra tenant for BGP EVPN, including the VLAN domain, VRF, Interface IP addressing,
and OSPF.
• Configure BGP on the spine node to support BGP EVPN.
• Configure a tenant for BGP EVPN.
• Configure the BGP EVPN route target, route map, and prefix-epg for the tenant.
• Configure BGP address-families to enable distributing BGP EVPN type-2 (MAC-IP) host routes to the
DCIG, with the host-rt-enable command .
Configuring a Spine and the Infra Tenant for BGP EVPN, Using the NX-OS Style
CLI
This task describes how to configure the infra tenant for BGP EVPN, including the VLAN domain, VRF,
Interface IP addressing, and OSPF in the following steps:
Procedure
Step 4 vrf context tenanttenant-name vrf vrf-name Associates the VRF with the tenant.
apic1(config-spine)# vrf context tenant
infra vrf overlay-1
Step 8 vlan-domain member vlan-domain-name Associates the interface with the VLAN
domain.
apic1(config-spine-if)# vlan-domain
member evpn-dom
Step 11 vrf member tenanttenant-name vrf vrf-name Associates the interface with the overlay-1
VRF and the infra tenant.
apic1(config-spine-if)# vrf member
tenant infra vrf overlay-1
Step 14 ip router ospf default areaospf-area-id Sets the default OSPF area ID for the interface.
apic1(config-spine-if)# ip router ospf
default area 0.0.0.150
Step 17 vlan-domain member vlan-domain-name Associates the interface with the VLAN
domain.
apic1(config-spine-if)# vlan-domain
member evpn-dom
Step 20 vrf member tenanttenant-name vrf vrf-name Associates the interface with the overlay-1
VRF and the infra tenant.
apic1(config-spine-if)# vrf member
tenant infra vrf overlay-1
Step 23 ip router ospf default areaospf-area-id Sets the default OSPF area ID for the interface.
apic1(config-spine-if)# ip router ospf
default area 0.0.0.200
Step 26 vrf member tenant tenant-name vrf vrf-name Associates the Router OSPF policy with the
overlay-1 VRF and infra tenant.
apic1(config-spine-ospf)# vrf member
tenant infra vrf overlay-1
Step 27 area area-id loopback loopback-ip-address Configure an OSPF area for the OSPF policy.
apic1(config-spine-ospf-vrf)# area
0.0.0.150 loopback 10.10.5.3
Route Target Configuration between the Spine Switches and the DCI
There are two ways to configure EVPN route targets (RTs) for the GOLF VRFs: Manual RT and Auto RT.
The route target is synchronized between ACI spines and DCIs through OpFlex. Auto RT for GOLF VRFs
has the Fabric ID embedded in the format: – ASN: [FabricID] VNID
If two sites have VRFs deployed as in the following diagram, traffic between the VRFs can be mixed.
Site 1 Site 2
When routes are received from the GOLF spine at one site, the outbound peer policy towards the GOLF spine
at another site filters the routes based on the community in the inbound peer policy. A different outbound peer
policy strips off the community towards the WAN. All the route-maps are at peer level.
Procedure
Step 2 Configure the outbound peer policy to filter routes based on the community in the inbound peer policy.
Example:
ip community-list standard test-com permit 1:1
Step 3 Configure the outbound peer policy to filter the community towards the WAN.
Example:
ip community-list standard test-com permit 1:1
update-source loopback0
send-community both
route-map multi-site-in in
send-community both
Configuring BGP to Support BGP EVPN on a Spine, Using the NX-OS Style CLI
This task shows how to configure BGP on the spine to support BGP EVPN in the following steps:
Procedure
Step 3 router bgp AS-number Configures BGP for the spine node.
apic1(config-spine)# router bgp 100
Step 4 vrf context tenanttenant-name vrf vrf-name Associates the Router BGP policy with the
infra tenant and the overlay-1 VRF.
apic1(config-spine-bgp)# vrf context
tenant infra vrf overlay-1
Step 5 vrf context tenanttenant-name vrf vrf-name Associates the Router BGP policy with the
infra tenant and the overlay-1 VRF.
apic1(config-spine-bgp-vrf)# vrf context
tenant infra vrf overlay-1
Step 6 neighbor neighbor-ip-address evpn Configures the IP address for an EVPN BGP
neighbor.
apic1(config-spine-bgp-vrf)# neighbor
10.10.4.1 evpn
Step 11 neighbor neighbor-ip-address evpn Configures the IP address for an EVPN BGP
neighbor.
apic1(config-spine-bgp-vrf)# neighbor
10.10.5.1 evpn
Configuring a Tenant for BGP EVPN Using the NX-OS Style CLI
This task shows how to configure a tenant for BGP EVPN in the following steps:
Procedure
Step 6 vrf member vrf-name Associates the bridge domain with the VRF
and tenant.
apic1(config-tenant-bd)# vrf member
vrf-sky
Step 12 vrf member vrf-name Associates the bridge domain with the VRF
and tenant.
apic1(config-tenant-bd)# vrf member
vrf-sky
Procedure
Step 3 vrf context tenanttenant-name vrf vrf-name Enters creates a VRF or enters VRF
configuration mode.
apic1(config-spine)# vrf context tenant
sky vrf vrf_sky
Step 4 address-family { ipv4 | ipv6 } unicast Sets IPv4 or IPv6 unicast address family for
the VRF.
apic1(config-spine-vrf)# address-family
ipv4 unicast
Step 8 route-map route-map-name Creates a route map for EVPN (with prefix
learned from a transit network).
apic1(config-spine-vrf)# route map rmap
Step 9 ip prefix-list ip-pl-name permit A.B.C.D/LEN Adds an IP prefix list to the route map to
permit traffic from the specified subnet.
apic1(config-spine-vrf-route-map)# ip
prefix-list pl permit 11.10.10.0/24
Step 12 match prefix-list pl-name Sets the route-map to match the specified
prefix-list.
apic1(config-spine-vrf-route-map)# match
prefix-list pl
Step 15 evpn export maproute-map-name label Assigns a consumer label to the VRF.
consumer-label-name apic1(config-spine-vrf)# evpn export
map rmap label evpn-aci
Step 16 route-map route-map-name Creates a route map for EVPN (with prefix
learned from a transit network).
apic1(config-spine-vrf)# route map rmap2
Step 19 match prefix-list pl-name Sets the route-map to match the specified
prefix-list.
apic1(config-spine-vrf-route-map)# match
prefix-list pl
Step 22 evpn export maproute-map-name label Assigns a consumer label to the VRF.
consumer-label-name apic1(config-spine-vrf)# evpn export
map rmap label evpn-aci2
Enabling Distributing BGP EVPN Type-2 Host Routes to a DCIG Using the NX-OS
Style CLI
Procedure
apic1(config-bgp-af)# exit
Cisco ACI GOLF Configuration Example, Using the NX-OS Style CLI
These examples show the CLI commands to configure GOLF Services, which uses the BGP EVPN protocol
over OSPF for WAN routers that are connected to spine switches.
configure
vlan-domain evpn-dom dynamic
exit
spine 111
# Configure Tenant Infra VRF overlay-1 on the spine.
vrf context tenant infra vrf overlay-1
router-id 10.10.3.3
exit
Configure
spine 111
router bgp 100
vrf member tenant infra vrf overlay- 1
neighbor 10.10.4.1 evpn
label golf_aci
update-source loopback 10.10.4.3
remote-as 100
exit
neighbor 10.10.5.1 evpn
label golf_aci2
update-source loopback 10.10.5.3
remote-as 100
exit
exit
exit
configure
tenant sky
vrf context vrf_sky
exit
bridge-domain bd_sky
vrf member vrf_sky
exit
interface bridge-domain bd_sky
ip address 59.10.1.1/24
exit
bridge-domain bd_sky2
vrf member vrf_sky
exit
interface bridge-domain bd_sky2
ip address 59.11.1.1/24
exit
exit
Configuring the BGP EVPN Route Target, Route Map, and Prefix EPG for the Tenant
The following example shows how to configure a route map to advertise bridge-domain subnets through BGP
EVPN.
configure
spine 111
vrf context tenant sky vrf vrf_sky
address-family ipv4 unicast
route-target export 100:1
route-target import 100:1
exit
route-map rmap
ip prefix-list p1 permit 11.10.10.0/24
match bridge-domain bd_sky
exit
match prefix-list p1
exit
route-map rmap2
match bridge-domain bd_sky
exit
match prefix-list p1
exit
exit
Procedure
Step 1 Verify that HostLeak object is enabled under the VRF-AF in question, by entering a command such as the
following in the spine-switch CLI:
Example:
spine1# ls /mit/sys/bgp/inst/dom-apple/af-ipv4-ucast/
ctrl-l2vpn-evpn ctrl-vpnv4-ucast hostleak summary
Step 2 Verify that the config-MO has been successfully processed by BGP, by entering a command such as the
following in the spine-switch CLI:
Example:
spine1# show bgp process vrf apple
Look for output similar to the following:
Information for address family IPv4 Unicast in VRF apple
Table Id : 0
Table state : UP
Table refcount : 3
Peers Active-peers Routes Paths Networks Aggregates
0 0 0 0 0 0
Redistribution
None
Step 3 Verify that the public BD-subnet has been advertised to DCIG as an EVPN type-5 route:
Example:
spine1# show bgp l2vpn evpn 10.6.0.0 vrf overlay-1
Route Distinguisher: 192.41.1.5:4123 (L3VNI 2097154)
BGP routing table entry for [5]:[0]:[0]:[16]:[10.6.0.0]:[0.0.0.0]/224, version 2088
Paths: (1 available, best #1)
Flags: (0x000002 00000000) on xmit-list, is not in rib/evpn
Multipath: eBGP iBGP
Advertised path-id 1
Path type: local 0x4000008c 0x0 ref 1, path is valid, is best path
AS-Path: NONE, path locally originated
192.41.1.1 (metric 0) from 0.0.0.0 (192.41.1.5)
Origin IGP, MED not set, localpref 100, weight 32768
Received label 2097154
Community: 1234:444
Extcommunity:
RT:1234:5101
4BYTEAS-GENERIC:T:1234:444
In the Path type entry, ref 1 indicates that one route was sent.
Step 4 Verify whether the host route advertised to the EVPN peer was an EVPN type-2 MAC-IP route:
Example:
spine1# show bgp l2vpn evpn 10.6.41.1 vrf overlay-1
Route Distinguisher: 10.10.41.2:100 (L2VNI 100)
BGP routing table entry for [2]:[0]:[2097154]:[48]:[0200.0000.0002]:[32]:[10.6.41
.1]/272, version 1146
Shared RD: 192.41.1.5:4123 (L3VNI 2097154)
Paths: (1 available, best #1)
Flags: (0x00010a 00000000) on xmit-list, is not in rib/evpn
Multipath: eBGP iBGP
Advertised path-id 1
Path type: local 0x4000008c 0x0 ref 0, path is valid, is best path
AS-Path: NONE, path locally originated
EVPN network: [5]:[0]:[0]:[16]:[10.6.0.0]:[0.0.0.0] (VRF apple)
10.10.41.2 (metric 0) from 0.0.0.0 (192.41.1.5)
Origin IGP, MED not set, localpref 100, weight 32768
Received label 2097154 2097154
Extcommunity:
RT:1234:16777216
The Shared RD line indicates the RD/VNI shared by the EVPN type-2 route and the BD subnet.
The EVPN Network line shows the EVPN type-5 route of the BD-Subnet.
The Path-id advertised to peers indicates the path advertised to EVPN peers.
Step 5 Verify that the EVPN peer (a DCIG) received the correct type-2 MAC-IP route and the host route was
successfully imported into the given VRF, by entering a command such as the following on the DCIG device
(assuming that the DCIG is a Cisco ASR 9000 switch in the example below):
Example:
RP/0/RSP0/CPU0:asr9k#show bgp vrf apple-2887482362-8-1 10.6.41.1
Tue Sep 6 23:38:50.034 UTC
BGP routing table entry for 10.6.41.1/32, Route Distinguisher: 44.55.66.77:51
Versions:
Process bRIB/RIB SendTblVer
Speaker 2088 2088
Last Modified: Feb 21 08:30:36.850 for 28w2d
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
Local
192.41.1.1 (metric 42) from 10.10.41.1 (192.41.1.5)
Received Label 2097154
Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate,
imported
Received Path ID 0, Local Path ID 1, version 2088
Community: 1234:444
In this output, the received RD, next hop, and attributes are the same for the type-2 route and the BD subnet.
Multipod_Fabric
About Multipod Fabric
Multipod enables provisioning a more fault tolerant fabric comprised of multiple pods with isolated control
plane protocols. Also, multipod provides more flexibility with regard to the full mesh cabling between leaf
and spine switches. For example, if leaf switches are spread across different floors or different buildings,
multipod enables provisioning multiple pods per floor or building and providing connectivity between pods
through spine switches.
Multipod uses MP-BGP EVPN as the control-plane communication protocol between the ACI spines in
different Pods.
WAN routers can be provisioned in the IPN, directly connected to spine switches or connected to border leaf
switches. Multipod uses a single APIC cluster for all the pods; all the pods act as a single fabric. Individual
APIC controllers are placed across the pods but they are all part of a single APIC cluster.
Procedure
Step 2 [no] system switch-id serial-number switch-id For each switch in the multipod fabric, declare
switch-name [pod pod-id] [role {leaf | spine}] the associated pod and the role (leaf or spine)
of the switch. Repeat this command for each
Example:
leaf and spine switch in the multipod fabric.
apic1(config)# system switch-id
SAL1748H56D 201 ifav4-spine1 pod 1 role
spine
Example
This example shows how to assign spine and leaf switches in a two-pod fabric.
apic1# configure
apic1(config)# system switch-id SAL1748H56D 201 ifav4-spine1 pod 1 role spine
apic1(config)# system switch-id SAL1938P7A6 202 ifav4-spine3 pod 1 role spine
apic1(config)# system switch-id SAL1819RXP4 101 ifav4-leaf1 pod 1 role leaf
apic1(config)# system switch-id SAL1803L25H 102 ifav4-leaf2 pod 1 role leaf
apic1(config)# system switch-id SAL1934MNY0 103 ifav4-leaf3 pod 1 role leaf
apic1(config)# system switch-id SAL1934MNY3 104 ifav4-leaf4 pod 1 role leaf
apic1(config)# system switch-id SAL1931LA3B 203 ifav4-spine2 pod 2 role spine
apic1(config)# system switch-id FGE173400A9 204 ifav4-spine4 pod 2 role spine
apic1(config)# system switch-id SAL1938PHBB 105 ifav4-leaf5 pod 2 role leaf
apic1(config)# system switch-id SAL1942R857 106 ifav4-leaf6 pod 2 role leaf
apic1(config)# system pod 1 tep-pool 10.0.0.0/16
apic1(config)# system pod 2 tep-pool 10.1.0.0/16
What to do next
Configure fabric-external connectivity.
Procedure
Step 12 [no] route-target extended ASN4:NN Route targets are carried as extended
community attributes. Enter the community
Example:
number in the AA4:NN2 format:
apic1(config-fabric-external)# 1-4294967295: 1-65535.
route-target extended 5:16
Step 13 exit
Example
This example shows how to configure fabric-external connectivity for a multipod fabric.
apic1# configure
apic1(config)# fabric-external 1
apic1(config-fabric-external)# bgp evpn peering
apic1(config-fabric-external)# pod 1
apic1(config-fabric-external-pod)# interpod data hardware-proxy 100.11.1.1/32
apic1(config-fabric-external-pod)# bgp evpn peering
apic1(config-fabric-external-pod)# exit
apic1(config-fabric-external)# pod 2
apic1(config-fabric-external-pod)# interpod data hardware-proxy 200.11.1.1/32
apic1(config-fabric-external-pod)# bgp evpn peering
apic1(config-fabric-external-pod)# exit
apic1(config-fabric-external)# route-map interpod-import
apic1(config-fabric-external-route-map)# ip prefix-list default permit 0.0.0.0/0
apic1(config-fabric-external-route-map)# exit
apic1(config-fabric-external)# route-target extended 5:16
apic1(config-fabric-external)# exit
What to do next
Configure spine interfaces and OSPF.
Procedure
Step 7 [no] vlan-domain member domain-name The VLAN domain must already exist, having
been created using the vlan-domain
Example:
domain-name command in the global
apic1(config-spine)# vlan-domain member configuration mode.
l3Dom
Step 9 [no] interface ethernet type/slot.subinterface Encapsulation for the subinterface must be 4.
Example:
apic1(config-spine)# interface ethernet
1/1.4
Step 10 [no] vrf member tenant infra vrf vrf-name Configure the interface as a member of the
tenant VRF.
Example:
apic1(config-spine-if)# vrf member
tenant infra vrf overlay-1
Step 18 [no] area area loopback ip-address Advertise the loopback address through OSPF.
This address is used by BGP EVPN sessions
Example:
for peering.
apic1(config-spine-ospf-vrf)# area
0.0.0.0 loopback 201.201.201.201
Step 19 [no] area area interpod peering Enable inter-pod peering on the OSPF area,
which will set up BGP EVPN sessions
Example:
automatically using the loopback address
apic1(config-spine-ospf-vrf)# area advertised by OSPF.
0.0.0.0 interpod peering
Example
apic1# configure
The remote leaf switches are added to an existing pod in the fabric. All policies deployed in the main datacenter
are deployed in the remote switches, which behave like local leaf switches belonging to the pod. In this
topology, all unicast traffic is through VXLAN over Layer 3. Layer 2 Broadcast, Unknown Unicast, and
Multicast (BUM) messages are sent using Head End Replication (HER) tunnels without the use of Multicast.
All local traffic on the remote site is switched directly between endpoints, whether physical or virtual. Any
traffic that requires use of the spine switch proxy is forwarded to the main datacenter.
The APIC system discovers the remote leaf switches when they come up. From that time, they can be managed
through APIC, as part of the fabric.
Note • All inter-VRF traffic goes to the spine switch before being forwarded.
• Before decommissioning a remote leaf, you must first delete the vPC.
You can configure Remote Leaf in the APIC GUI, either with and without a wizard, or use the REST API or
the NX-OS style CLI.
Sub-Leaf Switches
• Only Cisco Nexus 9000 series switches with names that end in EX, and later (for example,
N9K-C93180LC-EX) are supported as sub-leaf switches.
• Only Cisco Nexus 9000 series switches with names that end in EX, and later (for example,
N9K-C93180LC-EX) are supported as tier 2 switches if there are sub-leaf switches attached to them.
• Sub-leaf switches attached to remote leaf switches is not supported.
Note In Cisco APIC Release 3.2(x), the following features are supported that were not previously:
• FEX devices connected to remote leaf switches
• Cisco AVS with VLAN and Cisco AVS with VXLAN
• Cisco ACI Virtual Edge with VLAN and ACI Virtual Edge with VXLAN
• The Cisco Nexus 9336C-FX2 switch is now supported for remote leaf switches
Stretching of L3out SVI between local leaf switches (ACI main data center switches) and remote leaf switches
is not supported.
The following deployments and configurations are not supported with the remote leaf switch feature:
• APIC controllers directly connected to remote leaf switches
• Orphan port-channel or physical ports on remote leaf switches, with a vPC domain (this restriction applies
for releases 3.1 and earlier)
• With and without service node integration, local traffic forwarding within a remote location is only
supported if the consumer, provider, and services nodes are all connected to Remote Leaf switches are
in vPC mode
Full fabric and tenant policies are supported on remote leaf switches, in this release, except for the following
features:
• ACI Multi-Site
• Layer 2 Outside Connections (except Static EPGs)
• 802.1Q Tunnels
• Copy services with vzAny contract
• FCoE connections on remote leaf switches
• Flood in encapsulation for bridge domains or EPGs
• Fast Link Failover policies
• Managed Service Graph-attached devices at remote locations
• Netflow
• PBR Tracking on remote leaf switches (with system-level global GIPo enabled)
• Q-in-Q Encapsulation Mapping for EPGs
• Traffic Storm Control
• Cloud Sec and MacSec Encryption
• First Hop Security
• PTP
• Layer 3 Multicast routing on remote leaf switches
• Openstack and Kubernetes VMM domains
• Maintenance mode
• Troubleshooting Wizard
• Transit L3Out across remote locations, which is when the main Cisco ACI datacenter pod is a transit
between two remote locations (the L3Out in RL location-1 and L3Out in RL location-2 are
advertising prefixes for each other)
• Traffic forwarding directly across two remote leaf vPC pairs in the same remote datacenter or across
datacenters
Bandwidth in the WAN must be a minimum of 100 Mbps and maximum supported latency is 300 msecs.
• It is recommended, but not required to connect the pair of remote leaf switches with a vPC. The switches
on both ends of the vPC must be remote leaf switches at the same remote datacenter.
• Configure the northbound interfaces as Layer 3 sub-interfaces on VLAN-4, with unique IP addresses.
If you connect more than one interface from the remote leaf switch to the router, configure each interface
with a unique IP address.
• Enable OSPF on the interfaces.
• The IP addresses in the remote leaf switch TEP Pool subnet must not overlap with the pod TEP subnet
pool. The subnet used must be /24 or lower.
• Multipod is supported, but not required, with the Remote Leaf feature.
• When connecting a pod in a single-pod fabric with remote leaf switches, configure an L3Out from a
spine switch to the WAN router and an L3Out from a remote leaf switch to the WAN router, both using
VLAN-4 on the switch interfaces.
• When connecting a pod in a multipod fabric with remote leaf switches, configure an L3Out from a spine
switch to the WAN router and an L3Out from a remote leaf switch to the WAN router, both using VLAN-4
on the switch interfaces. Also configure a multipod-internal L3Out using VLAN-5 to support traffic that
crosses pods destined to a remote leaf switch. The regular multipod and multipod-internal connections
can be configured on the same physical interfaces, as long as they use VLAN-4 and VLAN-5.
• When configuring the Multipod-internal L3Out, use the same router ID as for the regular multipod L3Out,
but deselect the Use Router ID as Loopback Address option for the router-id and configure a different
loopback IP address. This enables ECMP to function.
Procedure
Step 4 Configure two L3Outs for the infra tenant, one for the remote leaf connections and one for the multipod IPN.
Example:
Step 5 Configure the spine switch interfaces and sub-interfaces to be used by the L3Outs.
Example:
Step 6 Configure the remote leaf switch interface and sub-interface used for communicating with the main fabric
pod.
Example:
(config)# leaf 101
apic1(config-leaf)# vrf context tenant infra vrf overlay-1 l3out rl-wan-test
apic1(config-leaf-vrf)# exit
apic1(config-leaf)#
apic1(config-leaf)# interface ethernet 1/49
Example
The following example provides a downloadable configuration:
apic1# configure
apic1(config)# system remote-leaf-site 5 pod 2 tep-pool 192.0.0.0/16
apic1(config)# system switch-id FDO210805SKD 109 ifav4-leaf9 pod 2
remote-leaf-site 5 node-type remote-leaf-wan
apic1(config)# vlan-domain ospfDom
apic1(config-vlan)# vlan 4-5
apic1(config-vlan)# exit
apic1(config)# tenant infra
apic1(config-tenant)# l3out rl-wan-test
apic1(config-tenant-l3out)# vrf member overlay-1
apic1(config-tenant-l3out)# exit
apic1(config-tenant)# l3out ipn-multipodInternal
apic1(config-tenant-l3out)# vrf member overlay-1
apic1(config-tenant-l3out)# exit
apic1(config-tenant)# exit
apic1(config)#
apic1(config)# spine 201
apic1(config-spine)# vrf context tenant infra vrf overlay-1 l3out rl-wan-test
apic1(config-spine-vrf)# exit
apic1(config-spine)# vrf context tenant infra vrf overlay-1 l3out ipn-multipodInternal
apic1(config-spine-vrf)# exit
apic1(config-spine)#
apic1(config-spine)# interface ethernet 8/36
apic1(config-spine-if)# vlan-domain member ospfDom
apic1(config-spine-if)# exit
apic1(config-spine)# router ospf default
apic1(config-spine-ospf)# vrf member tenant infra vrf overlay-1
apic1(config-spine-ospf-vrf)# area 5 l3out rl-wan-test
apic1(config-spine-ospf-vrf)# exit
apic1(config-spine-ospf)# exit
apic1(config-spine)#
apic1(config-spine)# interface ethernet 8/36.4
apic1(config-spine-if)# vrf member tenant infra vrf overlay-1 l3out rl-wan-test
apic1(config-spine-if)# ip router ospf default area 5
apic1(config-spine-if)# exit
apic1(config-spine)# router ospf multipod-internal
apic1(config-spine-ospf)# vrf member tenant infra vrf overlay-1
apic1(config-spine-ospf-vrf)# area 5 l3out ipn-multipodInternal
apic1(config-spine-ospf-vrf)# exit
apic1(config-spine-ospf)# exit
apic1(config-spine)#
apic1(config-spine)# interface ethernet 8/36.5
apic1(config-spine-if)# vrf member tenant infra vrf overlay-1 l3out ipn-multipodInternal
apic1(config-spine-if)# ip router ospf multipod-internal area 5
apic1(config-spine-if)# exit
apic1(config-spine)# exit
apic1(config)#
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant infra vrf overlay-1 l3out rl-wan-test
apic1(config-leaf-vrf)# exit
apic1(config-leaf)#
apic1(config-leaf)# interface ethernet 1/49
apic1(config-leaf-if)# vlan-domain member ospfDom
apic1(config-leaf-if)# exit
apic1(config-leaf)# router ospf default
apic1(config-leaf-ospf)# vrf member tenant infra vrf overlay-1
apic1(config-leaf-ospf-vrf)# area 5 l3out rl-wan-test
apic1(config-leaf-ospf-vrf)# exit
apic1(config-leaf-ospf)# exit
apic1(config-leaf)#
apic1(config-leaf)# interface ethernet 1/49.4
apic1(config-leaf-if)# vrf member tenant infra vrf overlay-1 l3out rl-wan-test
apic1(config-leaf-if)# ip router ospf default area 5
apic1(config-leaf-if)# exit
Transit Routing
Transit Routing in the ACI Fabric
The Cisco APIC software supports external Layer 3 connectivity with OSPF (NSSA) and iBGP. The fabric
advertises the tenant bridge domain subnets out to the external routers on the External Layer 3 Outside (L3Out)
connections. The routes that are learned from the external routers are not advertised to other external routers.
The fabric behaves like a stub network that can be used to carry the traffic between the external Layer 3
domains.
In transit routing, multiple L3Out connections within a single tenant and VRF are supported and the APIC
advertises the routes that are learned from one L3Out connection to another L3Out connection. The external
Layer 3 domains peer with the fabric on the border leaf switches. The fabric is a transit Multiprotocol-Border
Gateway Protocol (MP-BGP) domain between the peers.
The configuration for external L3Out connections is done at the tenant and VRF level. The routes that are
learned from the external peers are imported into MP-BGP at the ingress leaf per VRF. The prefixes that are
learned from the L3Out connections are exported to the leaf switches only where the tenant VRF is present.
Note For cautions and guidelines for configuring transit routing, see Guidelines for Transit Routing, on page 167
Figure 12:
In the examples in this chapter, the Cisco ACI fabric has 2 leaf switches and two spine switches, that are
controlled by an APIC cluster. The border leaf switches 101 and 102 have L3Outs on them providing
connections to two routers and thus to the Internet. The goal of this example is to enable traffic to flow from
EP 1 to EP 2 on the Internet into and out of the fabric through the two L3Outs.
In this example, the tenant that is associated with both L3Outs is t1, with VRF v1.
Before configuring the L3Outs, configure the nodes, ports, functional profiles, AEPs, and a Layer 3 domain.
You must also configure the spine switches 104 and 105 as BGP route reflectors.
Configuring transit routing includes defining the following components:
1. Tenant and VRF
2. Node and interface on leaf 101 and leaf 102
3. Primary routing protocol on each L3Out (used to exchange routes between border leaf switch and external
routers; in this example, BGP)
4. Connectivity routing protocol on each L3Out (provides reachability information for the primary protocol;
in this example, OSPF)
5. Two external EPGs
6. One route map
7. At least one filter and one contract
8. Associate the contract with the external EPGs
Note For transit routing cautions and guidelines, see Guidelines for Transit Routing, on page 167.
The following table lists the names that are used in the examples in this chapter:
Property Names for L3Out1 on Node 101 Names for L3Out2 on Node 102
Tenant t1 t1
VRF v1 v1
Route map rp1 with ctx1 and route destination rp2 with ctx2 and route destination
192.168.1.0/24 192.168.2.0/24
Transit Routing with a Single L3Out Before APIC, release 2.3(1f), transit routing was not supported
Profile within a single L3Out profile. In APIC, release 2.3(1f) and later,
you can configure transit routing with a single L3Out profile, with
the following limitations:
• If the VRF is unenforced, an external subnet (l3extSubnet)
of 0.0.0.0/0 can be used to allow traffic between the routers
sharing the same L3EPG.
• If the VRF is enforced, an external default subnet (0.0.0.0/0)
cannot be used to match both source and destination prefixes
for traffic within the same Layer 3 EPG. To match all traffic
within the same Layer 3 EPG, the following prefixes are
supported:
• IPv4
• 0.0.0.0/1—with External Subnets for the External
EPG
• 128.0.0.0/1—with External Subnets for the
External EPG
• 0.0.0.0/0—with Import Route Control Subnet,
Aggregate Import
• IPv6
• 0::0/1—with External Subnets for the External
EPG
• 8000::0/1—with External Subnets for the External
EPG
• 0:0/0—with Import Route Control Subnet,
Aggregate Import
Shared Routes: Differences in Hardware Routes shared between VRFs function correctly on Generation 2
Support switches (Cisco Nexus N9K switches with "EX" or "FX" on the
end of the switch model name, or later; for example,
N9K-93108TC-EX). On Generation 1 switches, however, there
may be dropped packets with this configuration, because the
physical ternary content-addressable memory (TCAM) tables that
store routes do not have enough capacity to fully support route
parsing.
OSPF or EIGRP in Back to Back Cisco APIC supports transit routing in export route control policies
Configuration that are configured on the L3Out. These policies control which
transit routes (prefixes) are redistributed into the routing protocols
in the L3Out. When these transit routes are redistributed into
OSPF or EIGRP, they are tagged 4294967295 to prevent routing
loops. The Cisco ACI fabric does not accept routes matching this
tag when learned on an OSPF or EIGRP L3Out. However, in the
following cases, it is necessary to override this behavior:
• When connecting two Cisco ACI fabrics using OSPF or
EIGRP.
• When connecting two different VRFs in the same Cisco ACI
fabric using OSPF or EIGRP.
Advertising BD Subnets Outside the The import and export route control policies only apply to the
Fabric transit routes (the routes that are learned from other external peers)
and the static routes. The subnets internal to the fabric that are
configured on the tenant BD subnets are not advertised out using
the export policy subnets. The tenant subnets are still permitted
using the IP prefix-lists and the route-maps but they are
implemented using different configuration steps. See the following
configuration steps to advertise the tenant subnets outside the
fabric:
1. Configure the tenant subnet scope as Public Subnet in the
subnet properties window.
2. Optional. Set the Subnet Control as ND RA Prefix in the
subnet properties window.
3. Associate the tenant bridge domain (BD) with the external
Layer 3 Outside (L3Out).
4. Create contract (provider or consumer) association between
the tenant EPG and the external EPG.
Setting the BD subnet to Public scope and associating the BD
to the L3Out creates an IP prefix-list and the route-map
sequence entry on the border leaf for the BD subnet prefix.
Advertising a Default Route For external connections to the fabric that only require a default
route, there is support for originating a default route for OSPF,
EIGRP, and BGP L3Out connections. If a default route is received
from an external peer, this route can be redistributed out to another
peer following the transit export route control as described earlier
in this article.
A default route can also be advertised out using a Default Route
Leak policy. This policy supports advertising a default route if it
is present in the routing table or it always supports advertising a
default route. The Default Route Leak policy is configured in the
L3Out connection.
When creating a Default Route Leak policy, follow these
guidelines:
• For BGP, the Always property is not applicable.
• For BGP, when configuring the Scope property, choose
Outside.
• For OSPF, the scope value Context creates a type-5 LSA
while the Scope value Outside creates type-7 LSA. Your
choice depends on the area type configured in the L3Out. If
the area type is Regular, set the scope to Context. If the area
type is NSSA, set the scope to Outside.
• For EIGRP, when choosing the Scope property, you must
choose Context.
For an example of the commands for these prerequisites, see NX-OS Style CLI Example: L3Out Prerequisites,
on page 12.
Procedure
• The second L3Out is on node 102, which is named nodep2. Node 102 is configured with router ID
22.22.22.203. It has a routed interface ifp2 at eth1/3, with the IP address, 23.23.23.1/24.
Example:
apic1(config)# leaf 101
apic1(config-leaf)# vrf context tenant t1 vrf v1
apic1(config-leaf-vrf)# router-id 11.11.11.103
apic1(config-leaf-vrf)# exit
apic1(config-leaf)# interface ethernet 1/3
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member tenant t1 vrf v1
apic1(config-leaf-if)# ip address 12.12.12.3/24
apic1(config-leaf-if)# exit
apic1(config-leaf)# exit
apic1(config)# leaf 102
apic1(config-leaf)# vrf context tenant t1 vrf v1
apic1(config-leaf-vrf)# router-id 22.22.22.203
apic1(config-leaf-vrf)# exit
apic1(config-leaf)# interface ethernet 1/3
apic1(config-leaf-if)# vlan-domain member dom1
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member tenant t1 vrf v1
apic1(config-leaf-if)# ip address 23.23.23.3/24
apic1(config-leaf-if)# exit
apic1(config-leaf)# exit
apic1(config-leaf-vrf)# exit
apic1(config-leaf)# router bgp 100
apic1(config-leaf-bgp)# vrf member tenant t1 vrf v1
apic1(config-leaf-bgp-vrf)# neighbor 25.25.25.2
apic1(config-leaf-bgp-vrf-neighbor)# route-map rp2 in
apic1(config-leaf-bgp-vrf-neighbor)# route-map rp1 out
apic1(config-leaf-bgp-vrf-neighbor)# exit
apic1(config-leaf-bgp-vrf)# exit
apic1(config-leaf-bgp)# exit
apic1(config-leaf)# exit
Step 7 Create filters (access lists) and contracts to enable the EPGs to communicate.
Example:
apic1(config)# tenant t1
apic1(config-tenant)# access-list http-filter
apic1(config-tenant-acl)# match ip
apic1(config-tenant-acl)# match tcp dest 80
apic1(config-tenant-acl)# exit
apic1(config-tenant)# contract httpCtrct
apic1(config-tenant-contract)# scope vrf
apic1(config-tenant-contract)# subject subj1
apic1(config-tenant-contract-subj)# access-group http-filter both
apic1(config-tenant-contract-subj)# exit
apic1(config-tenant-contract)# exit
apic1(config-tenant)# exit
apic1(config-leaf-if)# no switchport
apic1(config-leaf-if)# vrf member tenant t1 vrf v1
apic1(config-leaf-if)# ip address 12.12.12.3/24
apic1(config-leaf-if)# exit
apic1(config-leaf)# router bgp 100
apic1(config-leaf-bgp)# vrf member tenant t1 vrf v1
apic1(config-leaf-bgp-vrf)# neighbor 15.15.15.2
apic1(config-leaf-bgp-vrf-neighbor)# exit
apic1(config-leaf-bgp-vrf)# exit
apic1(config-leaf-bgp)# exit
apic1(config-leaf)# router ospf default
apic1(config-leaf-ospf)# vrf member tenant t1 vrf v1
apic1(config-leaf-ospf-vrf)# area 0.0.0.0 loopback 40.40.40.1
apic1(config-leaf-ospf-vrf)# exit
apic1(config-leaf-ospf)# exit
apic1(config-leaf)# exit
apic1(config)# tenant t1
apic1(config-tenant)# external-l3 epg extnw1
apic1(config-tenant-l3ext-epg)# vrf member v1
apic1(config-tenant-l3ext-epg)# match ip 192.168.1.0/24
apic1(config-tenant-l3ext-epg)# exit
apic1(config-tenant)# external-l3 epg extnw2
apic1(config-tenant-l3ext-epg)# vrf member v1
apic1(config-tenant-l3ext-epg)# match ip 192.168.2.0/24
apic1(config-tenant-l3ext-epg)# exit
apic1(config-tenant)# exit
apic1(config)# tenant t1
apic1(config-tenant)# access-list http-filter
apic1(config-tenant-acl)# match ip
apic1(config-tenant-acl)# match tcp dest 80
apic1(config-tenant-acl)# exit
apic1(config-tenant)# contract httpCtrct
apic1(config-tenant-contract)# scope vrf
apic1(config-tenant-contract)# subject http-subj
apic1(config-tenant-contract-subj)# access-group http-filter both
apic1(config-tenant-contract-subj)# exit
apic1(config-tenant-contract)# exit
apic1(config-tenant)# exit
apic1(config)# tenant t1
apic1(config-tenant)# external-l3 epg extnw1
apic1(config-tenant-l3ext-epg)# vrf member v1
apic1(config-tenant-l3ext-epg)# contract provider httpCtrct
apic1(config-tenant-l3ext-epg)# exit
apic1(config-tenant)# external-l3 epg extnw2
apic1(config-tenant-l3ext-epg)# vrf member v1
apic1(config-tenant-l3ext-epg)# contract consumer httpCtrct
apic1(config-tenant-l3ext-epg)# exit
apic1(config-tenant)# exit
apic1(config)#