Broadcom Netxtreme Adapters Owner's Manual en Us
Broadcom Netxtreme Adapters Owner's Manual en Us
Broadcom Netxtreme Adapters Owner's Manual en Us
Functional Description
Broadcom NetXtreme Gigabit Ethernet adapters connect a PCI, PCI-X (BCM5701 and BCM5703), or PCI Express (BCM5719,
BCM5720, BCM5721, and BCM5722) compliant system to a Gigabit Ethernet network. Broadcom NetXtreme Gigabit Ethernet
adapters incorporate a technology that transfers data at a maximum rate of 1 gigabit per second10 times the rate of Fast
Ethernet adapters.
Broadcom NetXtreme Gigabit Ethernet adapters target the increased congestion experienced at the backbone and system in
today's networks and provide a future upgrade path for systems that require more bandwidth than Fast Ethernet can provide.
Using the Broadcom teaming software, you can split your network into virtual LANs (VLANs) as well as group multiple network
adapters together into teams to provide network load balancing and fault tolerance functionality. See Teaming and Broadcom
Gigabit Ethernet Teaming Services for detailed information about teaming. See Virtual LANs for a description of VLANs. See
Configuring Teaming for instructions on configuring teaming and creating VLANs on Windows operating systems.
Features
The following is a list of the Broadcom NetXtreme Gigabit Ethernet adapter features for all supported operating systems:
PCI Express x1 Lane support (BCM5721, and BCM5722)
Gigabit Ethernet (IEEE Std 802.3-1999)
Logical Link Control (IEEE Std 802.2)
Flow Control (IEEE Std 802.3x)
Standard Ethernet frame size (1518 bytes)
TBI (SerDes style) transceiver interfaces (except for BCM5721 and BCM5722)
Jumbo frames (up to 9 KB) (except for BCM5721 and BCM5722)
Layer-2 Priority Encoding (IEEE 802.1p)
High-speed on-chip RISC processor
Adaptive interrupt frequency
Up to 4 classes of service (CoS)
Up to 4 send rings and receive rings
Integrated 96 KB frame buffer memory
GMI/MII Management Interface
Statistics for SNMP MIB II, Ethernet-like MIB, and Ethernet MIB (IEEE Std 802.3z, Clause 30)
4 unique MAC unicast addresses
Support for multicast addresses via 128 bits hashing hardware function
Serial EEPROM or serial NVRAM flash memory
Supports PXE 2.1 specification (Linux Red Hat PXE Server, Windows Server, Intel APITEST, DOS UNDI)
JTAG support
PCI v2.3 32/64-bit, 33/66 MHz Bus Interface (BCM5701, BCM5703)
Power Management
Wake on LAN (Magic Packet, Wake Up Frame, specific pattern) is supported at 10/100 Mbps operation only.
NOTES:
Adapter speed connection when the system is down waiting for a wake-up signal is either 10 Mbps or 100 Mbps, but
can return to 1000 Mbps when the system is up and running if connected to a 1000 Mbps capable switch. Systems
intending to use Wake on LAN (WOL) should be connected to a switch capable of both 1000 and 10/100 Mbps speeds.
Dell supports Wake on LAN on one adapter in the system at a time.
LED Appearance
Network State
Off
No network activity
Blinking
Network activity
NOTE: See Broadcom Gigabit Ethernet Teaming Services for detailed information on the following topics:
Glossary of Terms and Acronyms
Teaming Concepts
Software Components
Hardware Requirements
Supported Teaming by Processor
Configuring Teaming by Operating System
Supported Features by Team Type
Selecting a Team Type
Teaming Mechanisms
Architecture
Types of Teams
Driver Support by Operating System
Supported Teaming Speeds
Teaming and Other Advanced Networking Features
General Network Considerations
Application Considerations
Troubleshooting Teaming Problems
Frequently-Asked Questions
Event Log Messages
Overview
Adapter teaming allows you to group network adapters together to function as a team. The benefits of teaming include
allowing membership to VLANs, providing load balancing between adapters, and offering fault tolerance. These benefits can
be combined such that you can couple the functionality of load balancing for the load balance members and the capability of
employing a failover with having the team participate on different VLANs.
Broadcom Advanced Server Program (BASP) is the Broadcom teaming software for Windows Server 2008 operating systems.
For Windows operating systems, BASP is configured through the Broadcom Advanced Control Suite (BACS) utility. For Linux
operating systems, teaming is done with channel bonding (see Teaming with Channel Bonding).
BASP supports four types of load balancing teams:
Smart Load Balancing and Failover
Link Aggregation (802.3ad)
Generic Trunking (FEC/GEC)/802.3ad-Draft Static
SLB (Auto-Fallback Disable)
Types of Teams
The available types of teams for the supported operating systems are shown in the following table:
Table 1. Types of Teams
Operating System
Linux
Limitations of Smart Load Balancing and Failover/SLB (Auto-Fallback Disable) Types of Teams
Smart Load Balancing (SLB) is a protocol-specific scheme. The level of support for IP, IPX, and NetBEUI protocols is listed
below.
Table 2: Smart Load Balancing
Operating System
Protocol
IP
IPX
NetBEUI
IP
IPX
NetBEUI
N/S
N/S
N/S
N/S
N/S
N/S
Operating System
Protocol
IP
IP
IPX
NetBEUI
IPX
NetBEUI
N/S
N/S
N/S
N/S
N/S
N/S
Legend:
Y = yes
N = no
The Smart Load Balancing type of team works with all Ethernet switches without having to configure the switch ports to any
special trunking mode. Only IP traffic is load-balanced in both inbound and outbound directions. IPX traffic is load-balanced in
the outbound direction only. Other protocol packets are sent and received through one primary interface only. Failover for
non-IP traffic is supported only for Broadcom network adapters. The Generic Trunking type of team requires the Ethernet
switch to support some form of port trunking mode (for example, Cisco's Gigabit EtherChannel or other switch vendor's Link
Aggregation mode). The Generic Trunking type of team is protocol-independent, and all traffic should be load-balanced and
fault-tolerant.
NOTE: If you do not enable LiveLink when configuring teams, disabling Spanning Tree Protocol (STP) at the switch is
recommended. This minimizes the downtime due to the spanning tree loop determination when failing over. LiveLink
mitigates such issues.
LiveLink Functionality
LiveLink functionality is a feature of BASP that is available only for the Smart Load Balancing and Failover type of
teaming. The purpose of LiveLink is to detect network connectivity beyond the switch and to route traffic only through team
VLAN Overview
Virtual LANs (VLANs) allow you to split your physical LAN into logical parts, to create logical segmentation of workgroups, and
to enforce security policies for each logical segment. Each defined VLAN behaves as its own separate network with its traffic
and broadcasts isolated from the others, increasing bandwidth efficiency within each logical group. Up to 64 VLANs (63
tagged and 1 untagged) can be defined for each Broadcom adapter on your server, depending on the amount of memory
available in your system.
VLANs can be added to a team to allow multiple VLANs with different VLAN IDs. A virtual adapter is created for each VLAN
added.
Although VLANs are commonly used to create individual broadcast domains and/or separate IP subnets, it is sometimes
useful for a server to have a presence on more than one VLAN simultaneously. Broadcom adapters support multiple VLANs on
a per-port or per-team basis, allowing very flexible network configurations.
An IP subnet consisting of the Main Server, PC #3, and PC #5. This subnet represents an engineering group.
VLAN #2
Includes the Main Server, PCs #1 and #2 via shared media segment, and PC #5. This VLAN is a software
development group.
VLAN #3
Includes the Main Server, the Accounting Server and PC #4. This VLAN is an accounting group.
A high-use server that needs to be accessed from all VLANs and IP subnets. The Main Server has a Broadcom
adapter installed. All three IP subnets are accessed via the single physical adapter interface. The server is
attached to one of the switch ports, which is configured for VLANs #1, #2, and #3. Both the adapter and the
Main Server
connected switch port have tagging turned on. Because of the tagging VLAN capabilities of both devices, the
server is able to communicate on all three IP subnets in this network, but continues to maintain broadcast
Available to VLAN #3 only. The Accounting Server is isolated from all traffic on VLANs #1 and #2. The switch
port connected to the server has tagging turned off.
PCs #1 and
#2
Attached to a shared media hub that is then connected to the switch. PCs #1 and #2 belong to VLAN #2 only,
and are logically in the same IP subnet as the Main Server and PC #5. The switch port connected to this
segment has tagging turned off.
PC #3
A member of VLAN #1, PC #3 can communicate only with the Main Server and PC #5. Tagging is not enabled
on PC #3 switch port.
PC #4
A member of VLAN #3, PC #4 can only communicate with the servers. Tagging is not enabled on PC #4 switch
port.
PC #5
A member of both VLANs #1 and #2, PC #5 has an Broadcom adapter installed. It is connected to switch port
#10. Both the adapter and the switch port are configured for VLANs #1 and #2 and have tagging enabled.
NOTE: VLAN tagging is only required to be enabled on switch ports that create trunk links to other switches, or on ports
connected to tag-capable end-stations, such as servers or workstations with Broadcom adapters.
CIM
The Common Information Model (CIM) is an industry standard defined by the Distributed Management Task Force (DMTF).
Microsoft implements CIM on Windows platforms such as Windows Server 2008. Broadcom will support CIM on Windows
Server 2008 platforms.
Broadcom's implementation of CIM will provide various classes to provide information to users through CIM client applications.
Note that Broadcom CIM data provider will provide data only, and users can choose their preferred CIM client software to
browse the information exposed by Broadcom CIM provider.
Broadcom CIM provider provides information through BRCM_NetworkAdapter and BRCM_ExtraCapacityGroup classes.
BRCM_NetworkAdapter class provides network adapter information pertaining to a group of adapters, including both
Broadcom and other vendors' controllers. BRCM_ExtraCapacityGroup class provides team configuration for the Broadcom
Advanced Server Program (BASP) Program. Current implementation will provide team information and information of physical
network adapters in the team.
Broadcom Advanced Server Program provides events through event logs. Users can use the "Event Viewer" provided by
Windows Server 2008, or use CIM to inspect or monitor these events. Broadcom CIM provider will also provide event
information through the CIM generic event model. These events are __InstanceCreationEvent, __InstanceDeletionEvent and
__InstanceModificationEvent, and are defined by CIM. CIM requires the client application to register the events from the client
application using queries, as examples shown below in order to receive events properly.
SELECT * FROM __InstanceModificationEvent
where TargetInstance ISA "BRCM_NetworkAdapter"
SELECT * FROM __InstanceModificationEvent
where TargetInstance ISA "BRCM_ExtraCapacityGroup"
SELECT * FROM __InstanceCreationEvent
where TargetInstance ISA "BRCM_NetworkAdapter"
SELECT * FROM __InstanceDeletionEvent
where TargetInstance ISA "BRCM_NetworkAdapter"
SELECT * FROM __InstanceCreationEvent
where TargetInstance ISA "BRCM_ActsAsSpare"
SELECT * FROM __InstanceDeletionEvent
where TargetInstance ISA "BRCM_ActsAsSpare"
For detailed information about these events, see the CIM documentation at
http://www.dmtf.org/standards/published_documents/DSP0004V2.3_final.pdf.
SNMP
BASP Subagent
The BASP subagent, baspmgnt.dll, is designed for Windows Server 2008 SNMP service. It is required to install the SNMP
service before installing the BASP subagent.
The BASP subagent allows an SNMP manager software to actively monitor the configurations and performance of the
Broadcom Advanced Server features. The subagent also provides an alarm trap to an SNMP manager to inform the manager
of any changes to the conditions of the BASP component.
The BASP subagent allows monitoring of the configurations and statistics for the BASP teams, the physical NIC adapters
participating in a team, and the virtual NIC adapters created as the result of teaming. Non-teamed NIC adapters are not
monitored at this time. The BASP configuration data includes information such as team IDs, physical/virtual/VLAN/team
adapter IDs, physical/virtual/VLAN/team/ adapter descriptions, and MAC addresses of the adapters.
The statistics include detailed information such as data packets transmitted and received for the physical/virtual/VLAN/team
adapters.
The alarm trap forwards information about the changes in configuration of the physical adapters participating in a team, such
BASP Extensible-Agent
The Broadcom NetXtreme Gigabit Ethernet Controller Extended Information SNMP extensible-agent, bcmif.dll, is designed for
Windows Server 2008 SNMP service.
The extensible-agent allows the SNMP manager software to actively monitor the configurations of the Broadcom NetXtreme
adapter. It is intended to supplement the information already provided by the standard SNMP Management Network Interface
information.
The extensible-agent provides in-depth information about a Broadcom NetXtreme adapter such as:
MAC address
Bound IP address
IP subnet mask
Physical link status
Adapter state
Line speed
Duplex mode
Memory range
Interrupt setting
Bus number
Device number
Function number
To monitor this information, a SNMP manager needs to load the Broadcom Extended information MIB file to allow monitoring
of the information described above. This file, bcmif.mib, is included on the Broadcom NetXtreme adapter installation CD.
The monitored workstation requires the installation of the Broadcom Extended Information SNMP extensible-agent, bcmif.dll,
and requires the Microsoft Windows Server 2008 SNMP service to be installed and loaded.
Please read all Restrictions and Disclaimers.
Back to Contents Page
System Requirements
Before you install the Broadcom NetXtreme Gigabit Ethernet adapter, verify that your system meets the requirements listed
for your operating system:
Hardware Requirements
Pentium-based system that meets operating system requirements
One open 32-bit or 64-bit PCI and/or PCI Express slot
128-MB RAM (minimum) for Windows and Linux.
Microsoft Windows
One of the following versions of Microsoft Windows:
Windows Server 2012
Windows Server 2008 Family
Linux
NOTE: The current version of the adapter driver has been tested on the latest Red Hat, SuSE, and other Linux distributions
for i386, ia64, and x86_64 CPU architectures using 2.6.x kernels. The driver has been tested up to kernel version 2.6.13. The
driver should work on other little endian or big endian CPU architectures, but only very limited testing has been done on some
of these machines. The Makefile may have to be modified to include architecture-specific compile switches, and some minor
changes in the source files may also be required. On these machines, patching the driver into the kernel is recommended.
Safety Precautions
Preinstallation Checklist
1. Verify that your server meets the hardware and software requirements listed under System Requirements.
2. Verify that your server is using the latest BIOS.
NOTE: If you acquired the adapter software on a disk or from the Dell support website
(http://www.support.dell.com), verify the path to the adapter driver files.
3. If your system is active, shut it down.
4. When system shutdown is complete, turn off the power and unplug the power cord.
5. Holding the adapter card by the edges, remove it from its shipping package and place it on an antistatic surface.
6. Check the adapter for visible signs of damage, particularly on the card edge connector. Never attempt to install any
damaged adapter.
Connector
Media
Maximum Distance
10BASE-T
RJ-45
Category 3, 4, or 5 UTP
100/1000BASE-T 1
RJ-45
Category 5 2 UTP
1 1000BASE-T
signaling requires four twisted pairs of Category 5 balanced cabling, as specified in ISO/IEC
11801:1995 and EIA/TIA-568-A (1995) and tested using procedures defined in TIA/EIA TSB95.
2 Category 5 is the minimum requirement. Category 5e and Category 6 are fully supported.
2. Connect one end of the cable to the adapter.
3. Connect the other end of the cable to an RJ-45 Ethernet network port.
NOTE: After the cable is properly connected at both ends, the port LEDs on the adapter should be functional. See
Table1:"10/100/1000BASE-T Cable Specifications" for a description of network link and activity indications
Please read all Restrictions and Disclaimers.
Back to Contents Page
Broadcom Boot Agent Driver Software: Broadcom NetXtreme BCM57XX User Guide
Overview
Broadcom NetXtreme Gigabit Ethernet adapters support Preboot Execution Environment (PXE), Remote Program Load (RPL), iSCSI boot, and Bootstrap
Protocol (BootP). Multi-Boot Agent (MBA) is a software module that allows your networked system to boot with the images provided by remote systems
across the network. The Broadcom MBA driver complies with the PXE 2.1 specification and is released with both monolithic and split binary images. This
provides flexibility to users in different environments where the motherboard may or may not have built-in base code.
The MBA module operates in a client/system environment. A network consists of one or more boot systems that provide boot images to multiple systems
through the network. The Broadcom implementation of the MBA module has been tested successfully in the following environments:
Linux Red Hat PXE Server. Broadcom PXE clients are able to remotely boot and use network resources (NFS mount, and so forth) and to perform
Linux installations. In the case of a remote boot, the Linux universal driver binds seamlessly with the Broadcom Universal Network Driver Interface
(UNDI) and provides a network interface in the Linux remotely-booted client environment.
Intel APITEST. The Broadcom PXE driver passes all API compliance test suites.
MS-DOS UNDI. The MS-DOS Universal Network Driver Interface (UNDI) seamlessly binds with the Broadcom UNDI to provide a network device driver
interface specification (NDIS2) interface to the upper layer protocol stack. This allows systems to connect to network resources in an MS-DOS
environment.
Remote Installation Service (RIS). The Broadcom PXE clients are able to remotely boot to a Windows Server 2008 system running RIS to initialize
and install Windows Server 2008 and prior operating systems.
Windows Deployment Service (WDS). For Windows Server 2003 SP2, RIS was replaced by WDS, which offers a Broadcom PXE client to install
Windows operating systems, including Windows Server 2008.
Broadcom Boot Agent Driver Software: Broadcom NetXtreme BCM57XX User Guide
a. Select the adapter to configure and press Enter. The Main Menu displays.
b. Select MBA Configuration to display the MBA Configuration menu.
3. Use the UP ARROW and DOWN ARROW keys to move to the Boot Protocol menu item. Then use the RIGHT ARROW or LEFT ARROW key to select the
boot protocol of choice if other boot protocols besides Preboot Execution Environment (PXE) are available. If available, other boot protocols include
Remote Program Load (RPL) and Bootstrap Protocol (BOOTP).
NOTE: For iSCSI boot-capable LOMs, the boot protocol is set via the BIOS. See your system documentation for more information.
NOTE: If you have multiple adapters in your system and you are unsure which adapter you are configuring, press CTRL+F6, which causes the port
LEDs on the adapter to start blinking.
4. Use the UP ARROW, DOWN ARROW, LEFT ARROW, and RIGHT ARROW keys to move to and change the values for other menu items, as desired.
5. Press F4 to save your settings.
6. Press ESC when you are finished.
Using uEFI
1. Restart your system.
2. Enter the System Setup or Device Setting configuration menu.
3. Select the device on which you want to change MBA settings.
4. Select MBA Configuration Menu.
5. Use the drop-down menu to select the boot protocol of choice, if boot protocols other than Preboot Execution Environment (PXE) are available. If
available, other boot protocols include iSCSI and Bootstrap Protocol (BOOTP).
NOTE: For iSCSI boot-capable LOMs, the boot protocol is set via the BIOS. See your system documentation formore information.
6. Use the UP ARROW, DOWN ARROW, LEFT ARROW, and RIGHT ARROW keys to move to and change the values for other menu items, as desired.
7. Select Back to go to Main menu
8. Select Finish to save and exit.
Broadcom Boot Agent Driver Software: Broadcom NetXtreme BCM57XX User Guide
The Red Hat Enterprise Linux has PXE Server support. It allows users to remotely perform a complete Linux installation over the network. The distribution
comes with the boot images boot kernel (vmlinuz) and initial ram disk (initrd), which are located on the Red Hat disk#1:
/images/pxeboot/vmlinuz
/images/pxeboot/initrd.img
Refer to the Red Hat documentation for instructions on how to install PXE Server on Linux.
The Initrd.img file distributed with Red Hat 8.0, however, does not have a Linux network driver for the Broadcom NetXtreme Gigabit Ethernet adapter.
This version requires drivers that are not part of the standard distribution. You can create a driver disk for the Broadcom NetXtreme Gigabit Ethernet adapter
from the image distributed with the driver source media. Refer to the Linux Readme.txt file for more information.
A remote boot does not require a standard Linux network driver for the Broadcom NetXtreme Gigabit Ethernet adapter. After the PXE client downloads the
Linux kernel and initial ram disk, the Linux universal driver that came with the Linux distribution binds with the UNDI code of the PXE to form a Linux network
driver.
iSCSI Boot
Broadcom NetXtreme Gigabit Ethernet adapters support iSCSI boot to enable network boot of operating systems to diskless
systems. The iSCSI boot allows a Windows or Linux operating system boot from an iSCSI target machine located remotely
over a standard IP network.
For both Windows and Linux operating systems, iSCSI boot can be configured to boot with two distinctive paths: non-offload
(also known as Microsoft/Open-iSCSI initiator) and offload (Broadcom's offload iSCSI driver or HBA). Configuration of the
path is set with the HBA Boot Mode option located on the General Parameters screen of the iSCSI Configuration utility.
See Table1 for more information on all General Parameters screen configuration options.
Description
TCP/IP
parameters via
DHCP
This option is specific to IPv4. Controls whether the iSCSI boot host software acquires the IP address
information using DHCP (Enabled) or use a static IP configuration (Disabled).
This option is specific to IPv6. Controls whether the iSCSI boot host software will configure a stateless
IP
link-local address and/or stateful address if DHCPv6 is present and used (Enabled). Router Solicit packets
Autoconfiguration are sent out up to three times with 4 second intervals in between each retry. Or use a static IP
configuration (Disabled).
iSCSI
parameters via
DHCP
Controls whether the iSCSI boot host software acquires its iSCSI target parameters using DHCP (Enabled)
or through a static configuration (Disabled). The static information is entered through the iSCSI Initiator
Parameters Configuration screen.
CHAP
Authentication
Controls whether the iSCSI boot host software uses CHAP authentication when connecting to the iSCSI
target. If CHAP Authentication is enabled, the CHAP ID and CHAP Secret are entered through the iSCSI
Initiator Parameters Configuration screen.
DHCP Vendor ID
Controls how the iSCSI boot host software interprets the Vendor Class ID field used during DHCP. If the
Vendor Class ID field in the DHCP Offer packet matches the value in the field, the iSCSI boot host
software looks into the DHCP Option 43 fields for the required iSCSI boot extensions. If DHCP is disabled,
this value does not need to be set.
Link Up Delay
Time
Controls how long the iSCSI boot host software waits, in seconds, after an Ethernet link is established
before sending any data over the network. The valid values are 0 to 255. As an example, a user may
need to set a value for this option if a network protocol, such as Spanning Tree, is enabled on the switch
interface to the client system.
Use TCP
Timestamp
Target as First
HDD
Allows specifying that the iSCSI target drive will appear as the first hard drive in the system.
Controls the number of connection retries the iSCSI Boot initiator will attempt if the iSCSI target LUN is
busy.
IP Version
This option specific to IPv6. Toggles between the IPv4 or IPv6 protocol. All IP settings will be lost when
switching from one protocol version to another.
Definition
"iscsi:"
A literal string
Separator
<protocol>
The IP protocol used to access the iSCSI target. Currently, only TCP is supported so the protocol is 6.
<port>
The port number associated with the protocol. The standard port number for iSCSI is 3260.
<LUN>
The Logical Unit Number to use on the iSCSI target. The value of the LUN must be represented in
hexadecimal format. A LUN with an ID OF 64 would have to be configured as 40 within the option 17
parameter on the DHCP server.
<targetname> The target name in either IQN or EUI format (refer to RFC 3720 for details on both IQN and EUI formats). An
example IQN name would be "iqn.1995-05.com.broadcom:iscsi-target".
DHCP Option 43, Vendor-Specific Information
DHCP option 43 (vendor-specific information) provides more configuration options to the iSCSI client than DHCP option 17. In
this configuration, three additional suboptions are provided that assign the initiator IQN to the iSCSI boot client along with
two iSCSI target IQNs that can be used for booting. The format for the iSCSI target IQN is the same as that of DHCP
option17, while the iSCSI initiator IQN is simply the initiator's IQN.
NOTE: DHCP Option 43 is supported on IPv4 only.
The suboptions are listed below.
Table 3: DHCP Option 43 Suboption Definition
Suboption Definition
203
Using DHCP option 43 requires more configuration than DHCP option 17, but it provides a richer environment and provides
more configuration options. Broadcom recommends that customers use DHCP option 43 when performing dynamic iSCSI boot
configuration.
Configuring the DHCP Server
Configure the DHCP server to support option 17 or option 43.
NOTE: If using Option 43, you also need to configure Option 60. The value of Option 60 should match the DHCP Vendor
ID value. The DHCP Vendor ID value is BRCM ISAN, as shown in General Parameters of the iSCSI Boot Configuration
menu.
203
NOTE: In Table4, the brackets [ ] are required for the IPv6 addresses.
The content of option 17 should be <2-byte Option Number 201|202|203> <2-byte length> <data>.
Refer to the silent.txt file for the specific driver installer application for instructions on how to extract the
individual Windows NetXtreme drivers.
Other software required:
Bindview.exe (Windows Server 2008 R2 only; see KB976042)
Procedure:
1. Remove any local hard drives on the system to be booted (the "remote system").
2. Load the latest Broadcom MBA and iSCSI boot images onto NVRAM of the adapter.
3. Configure the BIOS on the remote system to have the Broadcom MBA as the first bootable device, and the CDROM as
the second device.
4. Configure the iSCSI target to allow a connection from the remote device. Ensure that the target has sufficient disk
space to hold the new O/S installation.
5. Boot up the remote system. When the Preboot Execution Environment (PXE) banner displays, press Ctrl+S to enter
the PXE menu.
6. At the PXE menu, set Boot Protocol to iSCSI.
7. Enter the iSCSI target parameters.
8. In General Parameters, set the Boot to Target parameter to One-Time Disabled.
9. Save the settings and reboot the system.
The remote system should connect to the iSCSI target and then boot from the DVDROM device.
10. Boot to DVD and begin installation.
11. Answer all the installation questions appropriately (specify the Operating System you want to install, accept the license
terms, etc.).
When the Where do you want to install Windows? window appears, the target drive should be visible. This is a
drive connected via the iSCSI boot protocol, located in the remote iSCSI target.
12. Select Next to proceed with Windows Server 2008 R2 installation.
A few minutes after the Windows Server 2008 R2 DVD installation process starts, a system reboot will follow. After
the reboot, the Windows Server 2008 R2 installation routine should resume and complete the installation.
13. Following another system restart, check and verify that the remote system is able to boot to the desktop.
14. After Windows Server 2008 R2 is booted up, load the driver and run Bindview.exe.
a. Select All Services.
b. Under WFP Lightweight Filter you should see Binding paths for the AUT. Right-click and disable them. When
done, close out of the application.
15. Verify that the OS and system are functional and can pass traffic by pinging a remote system's IP, etc.
NOTE: The Microsoft procedure injects only the eVBD and NDIS drivers. Broadcom recommends that all drivers
(eVBD, VBD, BXND, OIS, FCoE, and NetXtreme I NDIS) be injected.
The following procedure prepares the image for installation and booting:
1. Remove any local hard drives on the system to be booted (the "remote system").
2. Load the latest Broadcom MBA and iSCSI boot images into the NVRAM of the adapter.
3. Configure the BIOS on the remote system to have the Broadcom MBA as the first bootable device and the CDROM as
the second device.
4. Configure the iSCSI target to allow a connection from the remote device. Ensure that the target has sufficient disk
space to hold the new O/S installation.
5. Boot up the remote system. When the Preboot Execution Environment (PXE) banner displays, press Ctrl+S to enter
the PXE menu.
6. At the PXE menu, set Boot Protocol to iSCSI.
7. Enter the iSCSI target parameters.
8. In General Parameters, set the Boot to Target parameter to One-Time Disabled.
9. Save the settings and reboot the system.
The remote system should connect to the iSCSI target and then boot from the DVDROM device.
10. Boot from DVD and begin installation.
11. Answer all the installation questions appropriately (specify the Operating System you want to install, accept the license
terms, etc.).
When the Where do you want to install Windows? window appears, the target drive should be visible. This is a
drive connected via the iSCSI boot protocol, located in the remote iSCSI target.
12. Select Next to proceed with Windows 2012 installation.
A few minutes after the Windows 2012 DVD installation process starts, a system reboot will occur. After the reboot,
the Windows 2012 installation routine should resume and complete the installation.
13. Following another system restart, check and verify that the remote system is able to boot to the desktop.
14. After Windows 2012 boots to the OS, Broadcom recommends running the driver installer to complete the Broadcom
driver and application installation.
Linux iSCSI Boot Setup
Linux iSCSI boot is supported on Red Hat Enterprise Linux 5.5 and later and SUSE Linux Enterprise Server 11 SP1 and later in
both the offload and non-offload paths. Note that SLES 10.x and SLES 11 have support only for the non-offload path.
1. For driver update, obtain the latest Broadcom Linux driver CD.
2. Configure the iSCSI Boot Parameters for DVD direct install to target by disabling the Boot from target option on the
network adapter.
3. Configure to install via the non-offload path by setting HBA Boot Mode to Disabled in the NVRAM Configuration.
(Note: This parameter cannot be changed when the adapter is in Multi-Function mode.). Note that, for RHEL6.2 and
SLES11SP2 and newer, installation via the offload path is supported. For this case, set the HBA Boot Mode to Enabled
in the NVRAM Configuration.
4. Change the boot order as follows:
a. Boot from the network adapter.
b. Boot from the CD/DVD driver.
5. Reboot the system.
iscsiboot
boot.multipath
NOTE: The file and folder names used in this procedure are examples only. You can specify your own file and folder
names for your slipstream project.
1. For Windows Server 2008 R2 and SP2, install the Windows Automated Installation Kit (AIK).
or
For Windows Server 2012 and 2012 R2, install the Windows Assessment and Deployment Kit (ADK).
2. Use the following commands to create a temporary directory and set it as the current directory for all later steps:
md C:\Temp\x
cd /d C:\Temp\x
3. Use the following commands to create two subdirectories:
md src
md mnt
4. Use the following command to copy the original DVD into the src subdirectory.
xcopy N:\ .\src /e /c /i /f /h /k /y /q
Note that in this example, the installation DVD is in the N: drive.
5. Open a Deployment and Imaging Tools command prompt in elevated mode. Then, set c:\Temp\x as the current
directory.
Note that you will use this command prompt window in all subsequent steps.
the following commands to add the following driver to the currently mounted image:
/image:.\mnt /add-driver /driver:C:\Temp\b57nd60a\b57nd60a.inf
the following command to unmount the boot.wim image:
/unmount-wim /mountdir:.\mnt /commit
the following command to determine the index of the desired SKU in the install.wim image:
/get-wiminfo /wimfile:.\src\sources\install.wim
For example, in Windows Server 2012, index 2 is identified as "Windows Server 2012 SERVERSTANDARD."
11. Enter the following command to mount the install.wim image:
dism /mount-wim /wimfile:.\src\sources\install.wim /index:X /mountdir:.\mnt
Note that X is a placeholder for the index value that you obtained in step 10.
12. Enter the following commands to add the driver to the currently mounted image:
dism /image:.\mnt /add-driver /driver:C:\Temp\b57nd60a\b57nd60a.inf
13. Enter the following command to unmount the install.wim image:
dism /unmount-wim /mountdir:.\mnt /commit
14. Enter the following command to create an .iso file:
oscdimg -e -h -m -n -lslipstream -bootdata:2#p0,e,b"c:\Program Files\Windows
AIK\Tools\PETools\amd64\boot\etfsboot.com"#pEF,e,b"c:\Program Files\Windows
AIK\Tools\PETools\amd64\boot\efisys.bin" c:\temp\x\src c:\temp\Win20xxMOD.iso
Note that Platform is a placeholder for the architecture of the operating system that you want to install, such as
amd64 or x86. Also, xx in the file names is a placeholder for the Windows Server OS version (2012, 2008R2,
2008SP2.)
15. Using a DVD-burning application, burn the .iso file you created to a DVD.
16. Use the DVD that you created in step 15 to install the applicable Windows Server version.
Booting
After that the system has been prepared for an iSCSI boot and the operating system is present on the iSCSI target, the last
step is to perform the actual boot. The system will boot to Windows or Linux over the network and operate as if it were a
local disk drive.
1. Reboot the server.
2. Select CTRL+S.
3. From the Main menu, select General Parameters and configure the Boot to iSCSI target option to Enabled.
If CHAP authentication is needed, enable CHAP authentication after determining that booting is successful (see Enabling CHAP
Authentication).
Virtual LANs
Virtual LAN (VLAN) tagging is not supported for iSCSI boot with the Microsoft iSCSI Software Initiator.
NOTE: Microsoft recommends against this method. Toggling the boot path from NDIS to HBA or vice versa
after installation is completed is not recommended.
Problem: The Xen hypervisor will not start when booting from an iSCSI image created with the RHEL 5.4 Xen kernel.
Solution: This is a known third-party issue. To work around this issue, disable the Xen hypervisor's EDD feature by editing
the grub.conf file in the boot/grub folder to add the edd=off switch to the end of the kernel line. For example,
kernel /xen.gz edd=off.
Problem: Unable to connect to an EqualLogic target using Windows Server 2008 and higher.
Solution: Add an exception to your firewall to allow ICMP echo requests.
Problem: Installing Windows onto an iSCSI target via iSCSI boot fails when connecting to a 1Gbps switch port.
Solution: This is a limitation relating to adapters that use SFP+ as the physical connection. SFP+ defaults to 10Gbps
operation and does not support autonegotiation.
Preinstallation Requirementatas
Before you can successfully install the NDIS2 driver software, the Broadcom network adapter must be physically installed in
the server. Networking software that is appropriate to the operating system (such as Microsoft LAN Manager 2.2 for MS-DOS)
must already be running on your server.
B57$
B572$
B573$
B574$
The LineSpeed keyword is used to force the speed of the network connection. The LineSpeed keyword requires a decimal
number and of either 10, 100, or 1000. Technically, a line speed of 1000 Mbit/s cannot be forced and can be achieved only
through auto-negotiation. For the sake of simplicity, the driver performs auto-negotiation when the line speed is set to a
value of 1000. Forced 1000 Mbit/s speed is not needed for copper links; auto-negotiation is the proper supported
configuration under the IEEE Ethernet specification.
The Duplex keyword is used to force the duplex mode of the adapter. The Duplex keyword requires a text string of either
HALF or FULL. When the Duplex keyword is used, the LineSpeed keyword must also be used. If neither keyword is used, the
network adapter defaults to auto-negotiation mode.
The NodeAddress keyword specifies the network address used by the adapter. If a multicast address or a broadcast address is
specified, the adapter uses the default MAC address.
The FixCheckSumOff keyword turns off the driver workaround for the TCP/IP stack to recognize the ones complement version
of the checksum.
Example entries for the LineSpeed, Duplex, and NodeAddress keywords are shown below:
[B57]
DRIVERNAME = B57$
BUSNUM = 3
DEVNUM = 10
PORTNUM = 0
LINESPEED = 100
Limitations
The current version of the adapter driver has been tested on the latest Red Hat, SuSE, and other Linux distributions for i386,
ia64, and x86_64 CPU architectures using 2.6.x kernels. The driver has been tested up to kernel version 2.6.13. The driver
should work on other little endian or big endian CPU architectures, but only very limited testing has been done on some of
these machines. The Makefile may have to be modified to include architecture-specific compile switches, and some minor
changes in the source files may also be required. On these machines, patching the driver into the kernel is recommended.
Packaging
The Linux TG3 driver is released in the following packaging formats (file names):
Source RPM (tg3-version.3dkms.src.rpm)
Source RPM (tg3-version.3dkms.noarch.rpm)
Supplemental (tg3_sup-version.tar.gz)
Compressed tar (tg3-version.tar.gz)
Identical source files to build the driver are included in both RPM and TAR source packages. The tar file contains additional
utilities such as patches and driver disk images for network installation.
Network Installations
For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver disk that contains the tg3
driver may be needed. The driver disk images for the most recent Red Hat versions are included. Boot drivers for other Linux
/usr/share/hwdata/pci.ids /usr/share/hwdata/old.pci.ids
/usr/share/hwdata/pci.ids.new /usr/share/hwdata/pci.ids
/usr/share/hwdata/pcitable /usr/share/hwdata/old.pcitable
/usr/share/hwdata/pcitable.new /usr/share/hwdata/pcitable
NOTE: The paths above are for Red Hat distributions. These paths may be different on other distributions.
Driver Messages
The following are the most common sample messages that may be logged in the /var/log/messages file. Use dmesg -nlevel
to control the level at which messages appear on the console. Most systems are set to level 6 by default.
Driver Sign on
tg3.c:version (date)
NIC Detected
eth#: Tigon3 [partno (BCM95xxx) rev 4202 PHY (57xx) (PCI Express) 10/100/1000BaseT Ethernet
:00:xx:xx:xx:xx:xx
eth#: RXcsums [1] LinkChg REG [0] MIirq [0] ASF [0] Split [0] Wirespeed [1]TSOcap [1]
eth#: dma_rwctrl [76180000]
ACPI : PCI interrupt 0000:02:02.0 [A] -> GSI 26 (level,low) -> IRQ 233
Flow Control
Packaging
The VMware driver is released in the following packaging format.
Table 1: VMware Driver Packaging
Format
Drivers
Drivers
Download, Install, and Update Drivers
To download, install, or update the VMware ESX/ESXi driver for NetXtreme I GbE network adapters, see
http://www.vmware.com/support.
Driver Parameters
NetQueue
The optional parameter force_netq can be used to set the number of Rx and Tx net queues. BCM57XX devices that support
NetQueue are the BCM5718, BCM5719, BCM5720, BCM5721, and BCM5722.
By default, the driver tries to use the optimal number of NetQueues. To explicitly force the number of queues, set the number
of NetQueues per port via the following command:
esxcfg-module -s force_netq=x,x,x.... tg3
Allowed values for x are 1 to 15:
115 will force the number of NetQueues for the given NIC.
0 disables NetQueue.
1 specifies to use the default driver NetQueue value.
the number of "x" entries can go up to 32, which means the maximum supported NICs = 32.
Example usage:
esxcfg-module -s force_netq=-1,0,1,2 tg3]
tg3 NIC 0: Use the default number of NetQueues.
tg3 NIC 1: Disable the NetQueue feature.
tg3 NIC 2: Use 1 NetQueue.
tg3 NIC 3: Use 2 NetQueues.
Note that the NIC # above does not correspond to the vmnic<#>. The NIC number is the system vmnic probe order number.
Optimally, the number of NetQueues matches the number of CPUs on the machine.
Driver Parameters
Several optional parameters can be supplied as a command line argument to the vmkload_mod command. These parameters
Driver Defaults
Table 2: VMware Driver Defaults
Parameter
Default Value
Speed
Flow Control
MTU
Rx Ring Size
Rx Jumbo Ring
Size
100 (range 0255). Not all chips support the jumbo ring and some chips that suppoort jumbo frames
do not use the jumbo ring.
Tx Ring Size
511 (range (MAX_SKB_FRAGS+1) 511). MAX_SKB_FRAGS varies on different kernels and different
architectures. On a 2.6 kernel for x86, MAX_SKB_FRAGS is 18.
Coalesce RX
Microseconds
20 (range 01023)
Coalesce RX
Microseconds irq
20 (range 0255)
Coalesce rx
frames
5 (range 01023)
Coalesce rx
frames irq
5 (range 0255)
Coalesce TX
Microseconds
72 (range 01023)
Coalesce tx usecs
irq
20 (range 0255)
Coalesce tx
frames
53 (range 01023)
Coalesce tx
frames irq
5 (range 0255)
Coalesce stats
usecs
1000000 (aprox. 1 sec.). Some coalescing parameters are not used or have different defaults on some
chips.
MSI
Enabled (if supported by the chip and passed the interrupt test).
WoL
Disabled
Driver Messages
The following are the most common sample messages that may be logged in the file /var/log/messages. Use dmesg -n
<level> to control the level at which messages will appear on the console. Most systems are set to level 6 by default. To see
all messages, set the level higher.
Driver Sign On
tg3.c:v3.118g (Jan 4, 2012)
NIC Detected
vmnic0:
vmnic0:
vmnic0:
vmnic0:
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
Back to Contents Page
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
NOTE: BASP is not available on Windows Small Business Server (SBS) 2008.
To install the Broadcom NetXtreme I drivers and management applications
1. When the Found New Hardware Wizard appears, click Cancel.
2. Insert the Dell-provided CD into the CD or DVD drive or download the software driver package from the Dell website at
http://support.dell.com/.
3. On the driver source media, or from the location to which you downloaded the software driver package, open the folder
for your operating system, open the Driver_Management_Apps_Installer folder, and then double-click Setup.exe to
open the InstallShield Wizard.
4. Click Next to continue.
5. After you review the license agreement, click I accept the terms in the license agreement and then click Next to
continue.
6. Select the features you want installed.
7. Select how you want to install the NetXtreme I drivers and then click Next.
8. Click Install.
9. Click Finish to close the wizard.
10. The installer will determine if a system restart is necessary. Follow the on-screen instructions.
To install the Microsoft iSCSI Software Initiator for iSCSI Crash Dump
If supported and if you will use the Broadcom iSCSI Crash Dump utility, it is important to follow the installation sequence:
Run the installer
Install the Microsoft iSCSI Software Initiator along with the patch (MS KB939875)
NOTE: If performing an upgrade of the device drivers from the installer, re-enable iSCSI Crash Dump from the
Advanced section of the BACS Configuration tab.
Perform this procedure after running the installer to install the device drivers and the management applications.
1. Install Microsoft iSCSI Software Initiator (version 2.06 or later) if not included in your OS. To determine when you need
to install the Microsoft iSCSI Software Initiator, see Table1. To download the iSCSI Software Initiator from Microsoft,
go to http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=18986.
2. Install Microsoft patch for iSCSI crash dump file generation (Microsoft KB939875) from
http://support.microsoft.com/kb/939875. To determine if you need to install the Microsoft patch, see Table1.
Table 1: Windows Operating Systems and iSCSI Crash Dump
Operating System
NDIS
Windows Server 2008
No
No
No
No
No
No
No
OIS
Windows Server 2008
No
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
To perform a silent install from within the installer source folder
Type the following:
setup /s /v/qn
To perform a silent upgrade from within the installer source folder
Type the following:
setup /s /v/qn
To perform a silent reinstall of the same installer
Type the following:
setup /s /v"/qn REINSTALL=ALL"
NOTE: The REINSTALL switch should only be used if the same installer is already installed on the system. If upgrading an
earlier version of the installer, use setup /s /v/qn as listed above.
To perform a silent install by feature
Use the ADDSOURCE to include any of the features listed below.
Type the following according to platform:
IA32 platforms: setup /s /v"/qn ADDSOURCE=Driversi32,BACSi32,BASPi32,SNMPi32,CIMi32"
AMD/EM64T platforms: setup /s /v"/qn ADDSOURCE=Driversa64,BACSa64,BASPa64,SNMPa64,CIMa64"
The following command-line statement installs only the Broadcom drivers according to platform:
IA32 platforms: setup /s /v"/qn ADDSOURCE=Driversi32"
AMD64 platforms: setup /s /v"/qn ADDSOURCE=Driversa64"
NOTE: The Broadcom device drivers are a required feature and are always installed, even if you do not specify
ADDSOURCE.
To perform a silent install from within a batch file
To perform a silent install from within a batch file and to wait for the install to complete before continuing with the next
command line, type the following:
start /wait setup /s /w /v/qn
To perform a silent install to force a downgrade (default is NO)
setup /s /v" /qn DOWNGRADE=Y"
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
10. The installer will determine if a system restart is necessary. Follow the on-screen instructions.
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
the following modification to the b06nd.inf file located in the I386 directory as follows:
1. Locate [Manufacturer] header within the file.
2. Review the line below it which reads: %brcm% = broadcom, ntx86, ntamd64, ntia64 or equivalent.
3. Modify that line to read: %brcm% = broadcom.ntx86, ntamd64, ntia64. The change made replaces the comma and
space after "broadcom" with a period.
4. Save the file.
5. Restart the RIS service (binlsvc) or WDS services (wdsserver).
WinPE 2.0
The Microsoft Windows Server 2008 method of inserting the NetXtreme I monolithic driver in a WinPe 2.0 image is different
from the Windows Server 2008 R2 method, as discussed below.
By default, the monolithic driver is not included in the boot.wim and install.wim files that come with either the Microsoft
Windows Server 2008 CD or the Windows Server 2008 R2 CD. Microsoft's Windows Automated Installation Kit (AIK) allows
you to modify the default boot.wim and install.wim files, and create WinPE 2.0 images to include the NetXtreme I monolithic
driver in the Windows Server installation.
To insert the monolithic driver into a WinPE 2.0 boot image (Windows Server 2008)
To insert Broadcom's NetXtreme I monolithic driver in a WinPE 2.0 image, download AIK from http://www.microsoft.com/enus/download/default.aspx and install.
After installing AIK, copy the latest monolithic driver to a directory on the local hard drive of the system you installed the AIK.
Follow the procedure below to insert the monolithic driver into a WinPE 2.0 boot image.
1. From All Programs, open Windows AIK and select Windows PE Tools Command prompt.
2. At the command prompt, run the copype.cmd script. The script requires two arguments: hardware architecture and
destination location.
copype.cmd <arch> <destination>
For example: copype x86 c:\VistaPEx86
NOTE: The directory structure c:\VistaPEx86 is used throughout this procedure.
3. Mount the base image to a local directory so that you can add or remove packages by typing:
imagex /mountrw c:\VistaPEx86\winpe.wim 1 c:\VistaPEx86\mount
4. Place the monolithic driver and inf file in c:\drivers\x32\ by typing:
peimg /inf=c:\Drivers\x32\b06nd.inf c:\VistaPEx86\mount\windows
AIK inserts the driver into the WinPE 2.0 image.
5. To complete the customization of the image, prepare the image for deployment, type:
peimg /prep c:\VistaPEx86\mount\windows
6. When asked to continue and have the program prepare the image for deployment, type:
yes
7. To commit the changes to the original image file (Winpe.wim), type:
imagex /unmount c:\VistaPEx86\mount /commit
8. To replace the default Boot.wim file in the \ISO directory with your new custom image, type:
copy c:\VistaPex86\winpe.wim c:\VistaPEx86\ISO\sources\boot.wim
To add a device driver to an offline Windows PE image (Windows Server 2008 R2)
This procedure demonstrates how to use the Deployment Image Servicing and Management (DISM) tool to add a device
driver (.inf) to an offline Windows PE image. Before running a DISM command, first mount the Windows PE image.
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
1. Mount the base image by using the DISM tool to a local Windows PE directory. For example:
Dism /Mount-WIM /WimFile:c:\winpe_x86\winpe.wim /index:1 /MountDir:c:\winpe_x86\mount
NOTE: The directory structure c:\winpe_x86 is used throughout this procedure.
2. Add the .inf file to the base image by using the dism command with the /Add-Driver option. For example Driver.inf is
the Broadcom driver, evnd.inf is the driver for the 10 Gbps devices, and b06nd.inf is the driver for the 1 Gbps devices.
Dism /image:<path_to_image> /Add-Driver /Driver:c:\winpe_x86\mount\Windows\driver.inf
3. Repeat steps 1 and 2 for each additional device driver.
4. Unmount the image after modifying it.
dism /unmount-wim /Mountdir:c:\winpe_x86\mount /commit
5. After unmounting the image, burn it to the designated media.
To create a bootable CD-ROM
1. On your technician computer, at the command prompt, create an .iso file by typing:
oscdimg -n -bc:\VistaPEx86\etfsboot.com c:\VistaPEx86\ISO C:\VistaPEx86\VistaPEx86.iso
2. Burn the iso image to a CD.
WinPE 3.1
To insert the NetXtreme I monolithic driver into a WinPE 3.1 boot image (Windows server 2008 R2 SP1)
1. Open the WinPE image and mount it:
Dism /Mount-WIM /WimFile:c:\WinPEx64\winpe.wim /index:1 /MountDir:c:\WinPEx64\mount
2. Use the following commands to insert the Broadcom NetXtreme I drivers into the WinPE 3.1 image:
eVBD driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\evbd.inf
NetXtreme I NDIS 5.1 driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\b57amd64.inf
NetXtreme I NDIS driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\bxnd.inf
NetXtreme I NDIS 6.0 driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\b57nd60a.inf
NetXtreme I VBD driver:
Dism /image:c:\WinPEx64\mount /Add-Driver /Driver:c:\Drivers\x64\bxvbd.inf
3. Close the WinPE image and unmount it:
Dism /unmount-wim /Mountdir:c:\WinPEx64\mount /commit
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
These make up two separate sections that can be configured: one for standard RJ-45 copper interfaces (params_utp) and one
for fiber devices (params_fiber).
1. As described in the file, replace the value above in quotation marks under the correct section, depending upon the
network adapter in your system. The available values are shown below.
Options for copper interfaces:
Auto (1 Gbps is enabled when that speed is supported) = "0"
10 Mbps Half Duplex = "65794"
10 Mbps Full Duplex = "258"
100 Mbps Half Duplex = "66050"
100 Mbps Full Duplex = "514"
Options for fiber interfaces:
Auto (1 Gbps is enabled when that speed is supported) = "0"
1 Gbps Full Duplex = "771"
Auto with 1 Gbps Fallback = "33539"
Hardware default = "65283"
An example is provided in the file showing how to configure a copper interface for a 10 Mbps Full Duplex connection. The
example is shown below.
hkr, , req_medium,
2, "258"
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
NOTES:
The Power Management tab is available only for servers that support power management.
To enable Wake on LAN (WOL) when the computer is on standby, click Allow the device to bring the computer out
of standby box.
If you select Only allow management stations to bring the computer out of standby, the computer can be
brought out of standby only by Magic Packet.
CAUTION! Do not select Allow the computer to turn off the device to save power for any adapter that is
a member of a team.
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
Using WS-MAN
To use the WS-MAN communication protocol, follow the instructions in the following sections:
WS-MAN Windows Server Configuration
WS-MAN Windows Client Installation
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
3. Under Computer Configuration, open the Administrative Templates folder and then open the Windows
Components folder.
4. Select Windows Remote Management (WinRM).
5. Under Windows Remote Management (WinRM), select WinRm Client.
6. Under WinRM Client, double-click Trusted Hosts.
7. In the TrustedHostsList, enter the host names of the clients. If all clients are trusted then enter an asterisk (*) only.
8. Select WinRM Service.
9. Enable Allow Basic Authentication.
10. Enable Allow unencrypted traffic.
11. Close the Group Policy wIndow.
12. From the command prompt, run the following command to configure WinRM with default settings:
winrm qc or winrm quickconfig
13. When the tool displays "Make these changes[y/n]?", enter "y".
14. Enter one of the following commands to check whether an HTTP listener is created:
winrm enumerate winrm/confg/listener
or
winrm e winrm/config/Listener
15. Enter the following command from the command prompt to test locally.
winrm id
Step 5: Perform HTTPS Configuration on the Server (to use HTTPS rather than HTTP)
This step consists of two distinct processes: generating a self-signed certificate, if certificate does not exist, and importing it
to a Windows server. If one does not already exist, you must configure a self-signed certificate on the Windows server to
enable HTTPS/SSL communication with the BACS GUI on the Windows client. The Windows client also must be configured with
the self-signed certificate. See Perform HTTPS Configuration (if you plan to use HTTPS).
NOTE: The self-signed certificate can be created on any Windows server. The server does not require BACS to be installed.
The self-signed certificate generated on any Windows server should be copied on the local drive of client.
1. Click Start (or press the Windows logo key) and select Run.
2. Enter gpedit.msc to open the local Group Policy editor.
3. Under Computer Configuration, open the Administrative Templates folder and then open the Windows
Components folder.
4. Select Windows Remote Management (WinRM).
5. Under Windows Remote Management (WinRM), select WinRm Client.
6. Under WinRM Client, double-click Trusted Hosts.
7. In the TrustedHostsList, enter the host names of the clients. If all clients are trusted then enter an asterisk (*) only.
8. Select WinRM Service.
9. Enable Allow Basic Authentication.
To generate a self-signed certificate for the Windows Server:
Openssl on Windows can be used to generate the self-signed certificate, as follows:
1. Enter the following command to generate a private key:
openssl genrsa -des3 -out server.key 1024
2. You are prompted to enter a passphrase. Be sure to remember the passphrase.
3. Use the following steps to generate a Certificate Signing Request (CSR).
During the generation of the CSR, you are prompted for several pieces of information. When prompted for the
"Common Name", enter the Windows Server host name or IP address.
Enter the following command (sample responses are shown):
openssl req -new -key server.key -out server.csr
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
If this command does not work, try the following:
openssl req new key server.key out server.csr config openssl.cnf
The openssl.cnf file should be placed in the same directory where openssl is placed. Openssl.cnf is located in the
folder C:\Program Files (x86)\GnuWin32\share.
The following information is requested:
Country Name (2 letter code) []:US
State or Province Name (full name) []: California
Locality Name (e.g., city) []: Irvine
Organization Name (e.g., company) []: Broadcom Corporation
Organizational Unit Name (e.g., section) []: Engineering
Common Name (e.g., YOUR name) []: Enter the host name or IP address of the Windows server. For iPv6, enter
the Common Name in the format [xyxy:xxx:....::xxx], including the brackets [ ].
(Optional) Email Address []:
Enter the following additional attributes to be sent with your certificate request:
A challenge password []:password1
An optional company name []:
4. Remove the passphrase from the key.
Enter the following commands:
cp server.key server.key.org
openssl rsa -in server.key.org -out server.key
5. Generate a self-signed certificate:
To generate a self-signed certificate which is active for 365 days, enter the following command:
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
The following output displays:
Signature ok
subject=/C=US/ST=California/L=Irvine/O=Broadcom Corporation/OU=Engineering/CN=MGMTAPPLAB3/emailAddress=
Getting Private key
6. Enter the following command to verify the generated self-signed certificate.
openssl verify server.crt
The following output displays:
server.crt:/C=US/ST=California/L=Irvine/O=Broadcom Corporation/OU=Engineering/CN=MGMTAPPLAB3/emailAddress=
error 18 at 0 depth lookup:self signed certificate
OK
Ignore the error message "error 18 at 0 depth lookup:self signed certificate". This error indicates that this is a selfsigned certificate.
7. Convert the certificate from "crt" to "pkcs12" format, as follows:
For a Windows server, the certificate should be in pkcs12 format. Enter the following command:
openssl pkcs12 -export -in server.crt -inkey server.key -out hostname.pfx
You will be prompted for the following:
Enter Export Password:
Verifying - Enter Export Password:
Enter the password and be sure to remember it. The password is required when importing the certificate on the
Windows server and client.
8. Make a copy of the certificate file server.crt and place it on the server where BACS will be installed, so that it can be
imported. If you plan to use a Windows client to connect to the server running BACS, then the certificate also needs to
be transferred (copied and pasted) to the client system.
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
NOTE: A separate certificate must be generated for an IPv4 address, IPv6 address, and Hostname.
To instal the self-signed certificate on Windows server:
Transfer the file hostname.pfx you generated on the Windows server before you install the certificate:
1. Click Start (or press the Windows logo key) and select Run.
2. Enter MMC and click OK.
3. Click File > Add/Remove Snap-in.
4. Click Add.
5. Select Certificates and click Add.
6. Select Computer account.
7. Click Next and then click Finish.
8. Click Close, then click OK.
9. Open the Certificates (Local Computer) folder and then open the Personal folder.
10. Right-click Certificates, select All Tasks and then click Import.
11. Click Next to begin the Certificate Import Wizard.
12. Browse to select hostname.pfx.
13. When you are prompted for the password for the private key, enter the same password you created in To generate a
self-signed certificate for the Windows Server:.
14. Follow the instructions, select the defaults, and continue.
The certificate is shown as installed on the right side of the window. The name will be the name you specified while
creating a self-signed certificate.
15. Right-click on the certificate and select Properties.
A dialog box displays, as follows:
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
16. Ensure that only Server Authentication is enabled, as shown in the figure.
17. Open Trusted Root Certification Authorities and then open Certificates.
18. Follow the instructions from Step 11. to Step 17.
NOTE: See Perform HTTPS Configuration (if you plan to use HTTPS) for instructions on importing the self-signed
certificate on a client.
Step 6: Configure WinRM HTTPS/SSL on the Server
1. Create WinRM Listener, as follows:
a. Click Start (or press the Windows logo key) and select Run.
b. Enter MMC and click OK.
c. Select the self-signed certificate from the Personal store.
For example, if the certificate is created with a host name, the host name will appear.
d. Double-click the certificate to open it.
e. Click the Details tab.
f. Scroll down and select the Thumbprint field.
g. Select and copy the thumbprint in the Details window so you can insert it in the next step.
h. Return to the command prompt.
i. Enter the following command:
winrm create winrm/config/Listener?Address=*+Transport=
HTTPS @{Hostname="<HostName or IPAddress>";
CertificateThumbprint="<paste from the previous step and remove the spaces>"}
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
NOTES:
If the certificate was generated using the host name, enter the host name. If it was generated
using the IP address, enter the IP address. For an IPv6 address, use brackets [ ] around the
address.
If HTTPS is configured in your system, the listener must be deleted before creating a new HTTPS
listener. Use the following command:
winrm delete winrm/config/Listener?Address=*+Transport=HTTPS
j. The above command creates a listener on the HTTPS port (5986) using any/all network address of the server,
and my SelfSSL generated certificate.
k. You can use the winrm command to modify or set the HTTPS listener, as WinRM listeners can be configured on
any user defined port.
l. From command prompt, run the following command to verify that the listener(s) that have been configured:
winrm e winrm/config/listener
2. Test HTTPS/SSL connection on the server.
a. At the command prompt on the server, enter the following command:
winrs -r:https://yourserver:5986 -u:username -p:password hostname
b. If setup correctly, the output of the command shows the server host name.
c. To check WinRM Service Configuration, run the following command:
winrm get winrm/config/service
Step 7: Additional Server Configuration
If necessary, modify the firewall rules as follows:
Windows Server 2008 R2
1. From the Administrative Tools menu, open Windows Firewall with Advanced Security.
2. Right-click Inbound Rules and select New Rule.
The new rule wizard opens.
3. Select Port and click Next.
4. On the Protocol and Ports screen, select TCP and enter the specific port, for example, 5985 for HTTP or 5986 for
HTTPS.
5. Click Next.
6. On the Action screen, select Allow the connection and click Next.
7. For Profile, you can select all three profiles if your server is in a workgroup.
8. Specify a name for the rule and click Finish.
9. Ensure that the new rule and is enabled (the green check box is selected).
Windows XP
1. Click Start > Control Panel, and then double-click Windows Firewall.
2. Click the Exceptions tab
3. Click Add Port.
4. Enter a meaningful Name, for example "WinRM rule" and port number, for example, 5985 for HTTP or 5986 for HTTPS.
5. Click OK.
Useful WinRM Commands
Command
Description
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
winrm enumerate winrm/config/Listener or winrm e
winrm/config/Listener
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
k. Browse to select the .pfx file you generated in To generate a self-signed certificate for the Windows Server:.
Change the selection in the Files of type list to Personal Information Exchange (*.pfxas, *.p12), select the
hostname.pfx file and click Open.
l. Enter the password you assigned to the private key and click Next.
3. Configure WinRM HTTPS/SSL
You can run winrm from a client to retrieve information from the server using WinRM HTTPS connection. Use the
following steps to test the WinRM HTTPS/SSL connection from client:
a. To retrieve the server operating system information, enter the following command.
winrm e wmi/root/cimv2/Win32_OperatingSystem -r:https://yourservername u:username
-p:password -skipCAcheck
b. To retrieve the server WinRM identity information, enter the following command.
winrm id -r:https://yourservername -u:username -p:password -skipCAcheck
c. To enumerate Windows services on the server, enter the following command.
winrm e wmicimv2/Win32_service -r:https://yourservername -u:username p:password skipCAcheck
NOTE: It is important to use -skipCAcheck switch in the winrm command line testing, as the certificate is
self-generated and not imported on the client. Otherwise, the following error message displays: WSManFault.
Using WMI
No special configuration is required to use WMI on the Windows client. Perform the steps in the following sections to configure
WMI on the Windows server.
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
For more information, see Securing a Remote WMI Connection on the Microsoft Developer Network site.
Windows Driver and Management Application Installation: Broadcom NetXtreme BCM57XX User Guide
On a Windows XP Pro or Windows 2003 Server computer, ensure that remote logons are not being coerced to the
GUEST account (referred to as "ForceGuest", which is enabled by default on computers that are not attached to a
domain). Open the Local Security Policy editor by clicking Start > Run and entering secpol.msc. Open the Local
Policies node and select Security Options. Then, scroll down to the setting titled Network access: Sharing and
security model for local accounts. If this is set to Guest only, change it to Classic and restart the computer.
In Windows Vista and Windows 7, in order to let all users in the administrator group connect using the WMI
namespace, the user might need to change the LocalAccountTokenFilterPolicy as needed.
Please read all Restrictions and Disclaimers.
Back to Contents Page
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Back to Contents Page
Introduction
Glossary
Teaming Concepts
Software Components
Hardware Requirements
Supported Teaming by Processor
Configuring Teaming by Operating System
Supported Features by Team Type
Selecting a Team Type
This section describes the technology and implementation considerations when working with the network teaming services
offered by the Broadcom software shipped with systems. The goal of Broadcom teaming services is to provide fault tolerance
and link aggregation across a team of two or more adapters. The information in this document is provided to assist IT
professionals during the deployment and troubleshooting of system applications that require network fault tolerance and load
balancing.
Glossary
Table 1. Glossary
Item
Definition
ARP
BACS
BASP
DNS
G-ARP
Generic Trunking
Switch-dependent load balancing and failover type of team in which the intermediate driver manages
(FEC/GEC)/802.3adoutgoing traffic and the switch manages incoming traffic.
Draft Static
HSRP
ICMP
IGMP
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
IP
Internet Protocol
LACP
Link Aggregation
(802.3ad)
Switch-dependent load balancing and failover type of team with LACP in which the intermediate driver
manages outgoing traffic and the switch manages incoming traffic.
LOM
LAN on Motherboard
MAC
NDIS
NLB
PXE
RAID
Switch-independent failover type of team in which the primary team member handles all incoming and
Smart Load Balance
outgoing traffic while the standby team member is idle until a failover event (for example, loss of link
and Failover
occurs). The intermediate driver (BASP) manages incoming/outgoing traffic.
Smart Load
Balancing (SLB)
Switch-independent load balancing and failover type of team, in which the intermediate driver
manages outgoing/incoming traffic.
TCP
UDP
WINS
WLBS
Teaming Concepts
Network Addressing
Teaming and Network Addresses
Description of Teaming Types
The concept of grouping multiple physical devices to provide fault tolerance and load balancing is not new. It has been around
for years. Storage devices use RAID technology to group individual hard drives. Switch ports can be grouped together using
technologies such as Cisco Gigabit EtherChannel, IEEE 802.3ad Link Aggregation, Bay Network Multilink Trunking, and
Extreme Network Load Sharing. Network interfaces on Dell servers can be grouped together into a team of physical ports
called a virtual adapter.
Network Addressing
To understand how teaming works, it is important to understand how node communications work in an Ethernet network. This
document is based on the assumption that the reader is familiar with the basics of IP and Ethernet network communications.
The following information provides a high-level overview of the concepts of network addressing used in an Ethernet network.
Every Ethernet network interface in a host platform, such as a computer system, requires a globally unique Layer 2 address
and at least one globally unique Layer 3 address. Layer 2 is the Data Link Layer, and Layer 3 is the Network layer as defined
in the OSI model. The Layer 2 address is assigned to the hardware and is often referred to as the MAC address or physical
address. This address is pre-programmed at the factory and stored in NVRAM on a network interface card or on the system
motherboard for an embedded LAN interface. The Layer 3 addresses are referred to as the protocol or logical address
assigned to the software stack. IP and IPX are examples of Layer 3 protocols. In addition, Layer 4 (Transport Layer) uses port
numbers for each network upper level protocol such as Telnet or FTP. These port numbers are used to differentiate traffic
flows across applications. Layer 4 protocols such as TCP or UDP are most commonly used in today's networks. The
combination of the IP address and the TCP port number is called a socket.
Ethernet devices communicate with other Ethernet devices using the MAC address, not the IP address. However, most
applications work with a host name that is translated to an IP address by a Naming Service such as WINS and DNS.
Therefore, a method of identifying the MAC address assigned to the IP address is required. The Address Resolution Protocol
for an IP network provides this mechanism. For IPX, the MAC address is part of the network address and ARP is not required.
ARP is implemented using an ARP Request and ARP Reply frame. ARP Requests are typically sent to a broadcast address while
the ARP Reply is typically sent as unicast traffic. A unicast address corresponds to a single MAC address or a single IP
address. A broadcast address is sent to all devices on a network.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
A team of adapters function as a single virtual network interface and does not appear any different to other network devices
than a non-teamed adapter. A virtual network adapter advertises a single Layer 2 and one or more Layer 3 addresses. When
the teaming driver initializes, it selects one MAC address from one of the physical adapters that make up the team to be the
Team MAC address. This address is typically taken from the first adapter that gets initialized by the driver. When the system
hosting the team receives an ARP request, it selects one MAC address from among the physical adapters in the team to use
as the source MAC address in the ARP Reply. In Windows operating systems, the IPCONFIG /all command shows the IP and
MAC address of the virtual adapter and not the individual physical adapters. The protocol IP address is assigned to the virtual
network interface and not to the individual physical adapters.
For switch-independent teaming modes, all physical adapters that make up a virtual adapter must use the unique MAC
address assigned to them when transmitting data. That is, the frames that are sent by each of the physical adapters in the
team must use a unique MAC address to be IEEE compliant. It is important to note that ARP cache entries are not learned
from received frames, but only from ARP requests and ARP replies.
Teaming Type
Switch-Dependent
Link Aggregation Control
(Switch must support Protocol support is required on
specific type of
the switch
team)
members)
Load
Failover
Balancing
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
(LOM) devices. Transmit load balancing is achieved by creating a hashing table using the source and destination IP addresses
and TCP/UDP port numbers.The same combination of source and destination IP addresses and TCP/UDP port numbers will
generally yield the same hash index and therefore point to the same port in the team. When a port is selected to carry all the
frames of a given socket, the unique MAC address of the physical adapter is included in the frame, and not the team MAC
address. This is required to comply with the IEEE 802.3 standard. If two adapters transmit using the same MAC address, then
a duplicate MAC address situation would occur that the switch could not handle.
Receive load balancing is achieved through an intermediate driver by sending gratuitous ARPs on a client-by-client basis
using the unicast address of each client as the destination address of the ARP request (also known as a directed ARP). This is
considered client load balancing and not traffic load balancing. When the intermediate driver detects a significant load
imbalance between the physical adapters in an SLB team, it will generate G-ARPs in an effort to redistribute incoming frames.
The intermediate driver (BASP) does not answer ARP requests; only the software protocol stack provides the required ARP
Reply. It is important to understand that receive load balancing is a function of the number of clients that are connecting to
the system through the team interface.
SLB receive load balancing attempts to load balance incoming traffic for client machines across physical ports in the team. It
uses a modified gratuitous ARP to advertise a different MAC address for the team IP Address in the sender physical and
protocol address. This G-ARP is unicast with the MAC and IP Address of a client machine in the target physical and protocol
address respectively. This causes the target client to update its ARP cache with a new MAC address map to the team IP
address. G-ARPs are not broadcast because this would cause all clients to send their traffic to the same port. As a result, the
benefits achieved through client load balancing would be eliminated, and could cause out-of-order frame delivery. This
receive load balancing scheme works as long as all clients and the teamed system are on the same subnet or broadcast
domain.
When the clients and the system are on different subnets, and incoming traffic has to traverse a router, the received traffic
destined for the system is not load balanced. The physical adapter that the intermediate driver has selected to carry the IP
flow carries all of the traffic. When the router sends a frame to the team IP address, it broadcasts an ARP request (if not in
the ARP cache). The server software stack generates an ARP reply with the team MAC address, but the intermediate driver
modifies the ARP reply and sends it over a particular physical adapter, establishing the flow for that session.
The reason is that ARP is not a routable protocol. It does not have an IP header and therefore, is not sent to the router or
default gateway. ARP is only a local subnet protocol. In addition, since the G-ARP is not a broadcast packet, the router will
not process it and will not update its own ARP cache.
The only way that the router would process an ARP that is intended for another network device is if it has Proxy ARP enabled
and the host has no default gateway. This is very rare and not recommended for most applications.
Transmit traffic through a router will be load balanced as transmit load balancing is based on the source and destination IP
address and TCP/UDP port number. Since routers do not alter the source and destination IP address, the load balancing
algorithm works as intended.
Configuring routers for Hot Standby Routing Protocol (HSRP) does not allow for receive load balancing to occur in the adapter
team. In general, HSRP allows for two routers to act as one router, advertising a virtual IP and virtual MAC address. One
physical router is the active interface while the other is standby. Although HSRP can also load share nodes (using different
default gateways on the host nodes) across multiple routers in HSRP groups, it always points to the primary MAC address of
the team.
Generic Trunking
Generic Trunking is a switch-assisted teaming mode and requires configuring ports at both ends of the link: server interfaces
and switch ports. This is often referred to as Cisco Fast EtherChannel or Gigabit EtherChannel. In addition, generic trunking
supports similar implementations by other switch OEMs such as Extreme Networks Load Sharing and Bay Networks or IEEE
802.3ad Link Aggregation static mode. In this mode, the team advertises one MAC Address and one IP Address when the
protocol stack responds to ARP Requests. In addition, each physical adapter in the team uses the same team MAC address
when transmitting frames. This is possible since the switch at the other end of the link is aware of the teaming mode and will
handle the use of a single MAC address by every port in the team. The forwarding table in the switch will reflect the trunk as a
single virtual port.
In this teaming mode, the intermediate driver controls load balancing and failover for outgoing traffic only, while incoming
traffic is controlled by the switch firmware and hardware. As is the case for Smart Load Balancing, the BASP intermediate
driver uses the IP/TCP/UDP source and destination addresses to load balance the transmit traffic from the server. Most
switches implement an XOR hashing of the source and destination MAC address.
Link Aggregation (IEEE 802.3ad LACP)
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Link Aggregation is similar to Generic Trunking except that it uses the Link Aggregation Control Protocol to negotiate the
ports that will make up the team. LACP must be enabled at both ends of the link for the team to be operational. If LACP is not
available at both ends of the link, 802.3ad provides a manual aggregation that only requires both ends of the link to be in a
link up state. Because manual aggregation provides for the activation of a member link without performing the LACP message
exchanges, it should not be considered as reliable and robust as an LACP negotiated link. LACP automatically determines
which member links can be aggregated and then aggregates them. It provides for the controlled addition and removal of
physical links for the link aggregation so that no frames are lost or duplicated. The removal of aggregate link members is
provided by the marker protocol that can be optionally enabled for Link Aggregation Control Protocol (LACP) enabled
aggregate links.
The Link Aggregation group advertises a single MAC address for all the ports in the trunk. The MAC address of the Aggregator
can be the MAC addresses of one of the MACs that make up the group. LACP and marker protocols use a multicast destination
address.
The Link Aggregation control function determines which links may be aggregated and then binds the ports to an Aggregator
function in the system and monitors conditions to determine if a change in the aggregation group is required. Link
aggregation combines the individual capacity of multiple links to form a high performance virtual link. The failure or
replacement of a link in an LACP trunk will not cause loss of connectivity. The traffic will simply be failed over to the
remaining links in the trunk.
SLB (Auto-Fallback Disable)
This type of team is identical to the Smart Load Balance and Failover type of team, with the following exceptionwhen the
standby member is active, if a primary member comes back on line, the team continues using the standby member rather
than switching back to the primary member. This type of team is supported only for situations in which the network cable is
disconnected and reconnected to the network adapter. It is not supported for situations in which the adapter is
removed/installed through Device Manager or Hot-Plug PCI.
If any primary adapter assigned to a team is disabled, the team functions as a Smart Load Balancing and Failover type of
team in which auto-fallback occurs.
Software Components
Teaming is implemented via an NDIS intermediate driver in the Windows Operating System environment. This software
component works with the miniport driver, the NDIS layer, and the protocol stack to enable the teaming architecture (see
Figure1). The miniport driver controls the host LAN controller directly to enable functions such as sends, receives, and
interrupt processing. The intermediate driver fits between the miniport driver and the protocol layer multiplexing several
miniport driver instances, and creating a virtual adapter that looks like a single adapter to the NDIS layer. NDIS provides a
set of library functions to enable the communications between either miniport drivers or intermediate drivers and the protocol
stack. The protocol stack implements IP, IPX and ARP. A protocol address such as an IP address is assigned to each miniport
device instance, but when an Intermediate driver is installed, the protocol address is assigned to the virtual team adapter and
not to the individual miniport devices that make up the team.
The Broadcom supplied teaming support is provided by three individual software components that work together and are
supported as a package. When one component is upgraded, all the other components must be upgraded to the supported
versions. Table3 describes the three software components and their associated files for supported operating systems.
Table 3. Broadcom Teaming Software Component
Software Component
Broadcom Name
Windows
Miniport Driver
B57xp32.sys
B57w2k.sys
tg3
B57amd64.sys
B57xp64.sys
Intermediate Driver
Baspxp32.sys
Baspw2k.sys
Broadcom Advanced Server Program (BASP)
bonding
Basamd64.sys
Baspxp64.sys
Hardware Requirements
BACS
Linux
N/A
b57nd60x.sys
N/A
b57nd60a.sys
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Repeater Hub
Switching Hub
Router
The various teaming modes described in this document place certain restrictions on the networking equipment used to
connect clients to teamed systems. Each type of network interconnect technology has an effect on teaming as described in the
following sections.
Repeater Hub
A Repeater Hub allows a network administrator to extend an Ethernet network beyond the limits of an individual segment.
The repeater regenerates the input signal received on one port onto all other connected ports, forming a single collision
domain. This means that when a station attached to a repeater sends an Ethernet frame to another station, every station
within the same collision domain will also receive that message. If two stations begin transmitting at the same time, a
collision occurs, and each transmitting station must retransmit its data after waiting a random amount of time.
The use of a repeater requires that each station participating within the collision domain operate in half-duplex mode.
Although half-duplex mode is supported for Gigabit Ethernet adapters in the IEEE 802.3 specification, half-duplex mode is not
supported by the majority of Gigabit Ethernet adapter manufacturers. Therefore, half-duplex mode is not considered here.
Teaming across hubs is supported for troubleshooting purposes (such as connecting a network analyzer) for SLB teams only.
Switching Hub
Unlike a repeater hub, a switching hub (or more simply a switch) allows an Ethernet network to be broken into multiple
collision domains. The switch is responsible for forwarding Ethernet packets between hosts based solely on Ethernet MAC
addresses. A physical network adapter that is attached to a switch may operate in half-duplex or full-duplex mode.
To support Generic Trunking and 802.3ad Link Aggregation, a switch must specifically support such functionality. If the switch
does not support these protocols, it may still be used for Smart Load Balancing.
Router
A router is designed to route network traffic based on Layer 3 or higher protocols, although it often also works as a Layer 2
device with switching capabilities. The teaming of ports connected directly to a router is not supported.
Configuration Tool
Bonding
The Broadcom Advanced Control Suite (BACS) utility is designed to run in 32-bit and 64-bit Windows Server 2008. BACS is
used to configure load balancing and fault tolerance teaming, and VLANs. In addition, it displays the MAC address, driver
version, and status information about each network adapter. BACS also includes a number of diagnostics tools such as
hardware diagnostics, cable testing, and a network topology test.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Table 5. Comparison of Team Types
Type of Team
Fault
Load
Tolerance Balancing
SwitchIndependent
Switch-Dependent Dynamic Link
Static Trunking
Aggregation
(IEEE
802.3ad)
Function
SLB with
Standbya
SLB
Generic Trunking
Link
Aggregation
28
28
28
28
Number of teams
16
16
16
16
Yes
Yes
Yes
Yes
Yes
Yes
Switch-dependent
Switchdependent
TX load balancing
No
Yes
Yes
Yes
Yes
Yes (performed by
the switch)
Yes (performed
by the switch)
RX load balancing
No
No
No
Yes
Yes
No
No
No
No
Yes
Yes
Yes (switchdependent)
Yes
Yes
Yes
No
No
Yes
Yes
No (must be the
same speed)
Yes
No
Yes
Yes
Yes
Yesb
Yesb
Yesb
Yesb
No
Yes (IPX
outbound traffic Yes
only)
Yes
No
No
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
No
Yes
Yes
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Teaming Mechanisms
Architecture
Types of Teams
Driver Support by Operating System
Supported Teaming Speeds
Architecture
The Broadcom Advanced Server Program is implemented as an NDIS intermediate driver (see Figure2). It operates below
protocol stacks such as TCP/IP and IPX and appears as a virtual adapter. This virtual adapter inherits the MAC Address of the
first port initialized in the team. A Layer 3 address must also be configured for the virtual adapter. The primary function of
BASP is to balance inbound (for SLB) and outbound traffic (for all teaming modes) among the physical adapters installed on
the system selected for teaming. The inbound and outbound algorithms are independent and orthogonal to each other. The
outbound traffic for a particular session can be assigned to a given port while its corresponding inbound traffic can be
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
assigned to a different port.
Figure 2. Intermediate Driver
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
performs load balancing on a TCP session basis and not on a packet-by-packet basis.
In the Outbound Flow Hash Entries, statistics counters are also updated after classification. The load-balancing engine uses
these counters to periodically distribute the flows across teamed ports. The outbound code path has been designed to achieve
best possible concurrency where multiple concurrent accesses to the Outbound Flow Hash Table are allowed.
For protocols other than TCP/IP, the first physical adapter will always be selected for outbound packets. The exception is
Address Resolution Protocol (ARP), which is handled differently to achieve inbound load balancing.
Protocol Support
ARP and IP/TCP/UDP flows are load balanced. If the packet is an IP protocol only, such as ICMP or IGMP, then all data flowing
to a particular IP address will go out through the same physical adapter. If the packet uses TCP or UDP for the L4 protocol,
then the port number is added to the hashing algorithm, so two separate L4 flows can go out through two separate physical
adapters to the same IP address.
For example, assume the client has an IP address of 10.0.0.1. All IGMP and ICMP traffic will go out the same physical adapter
because only the IP address is used for the hash. The flow would look something like this:
IGMP ------> PhysAdapter1 ------> 10.0.0.1
ICMP ------> PhysAdapter1 ------> 10.0.0.1
If the server also sends an TCP and UDP flow to the same 10.0.0.1 address, they can be on the same physical adapter as
IGMP and ICMP, or on completely different physical adapters from ICMP and IGMP. The stream may look like this:
IGMP ------> PhysAdapter1 ------> 10.0.0.1
ICMP ------> PhysAdapter1 ------> 10.0.0.1
TCP------> PhysAdapter1 ------> 10.0.0.1
UDP------> PhysAdatper1 ------> 10.0.0.1
Or the streams may look like this:
IGMP ------> PhysAdapter1 ------> 10.0.0.1
ICMP ------> PhysAdapter1 ------> 10.0.0.1
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
TCP------> PhysAdapter2 ------> 10.0.0.1
UDP------> PhysAdatper3 ------> 10.0.0.1
The actual assignment between adapters may change over time, but any protocol that is not TCP/UDP based goes over the
same physical adapter because only the IP address is used in the hash.
Performance
Modern network interface cards provide many hardware features that reduce CPU utilization by offloading certain CPU
intensive operations (see Teaming and Other Advanced Networking Properties). In contrast, the BASP intermediate driver is a
purely software function that must examine every packet received from the protocol stacks and react to its contents before
sending it out through a particular physical interface. Though the BASP driver can process each outgoing packet in near
constant time, some applications that may already be CPU bound may suffer if operated over a teamed interface. Such an
application may be better suited to take advantage of the failover capabilities of the intermediate driver rather than the load
balancing features, or it may operate more efficiently over a single physical adapter that provides a particular hardware
feature such as Large Send Offload.
Types of Teams
Switch-Independent
The Broadcom Smart Load Balancing type of team allows two to eight physical adapters to operate as a single virtual adapter.
The greatest benefit of the SLB type of team is that it operates on any IEEE compliant switch and requires no special
configuration.
Smart Load Balancing and Failover
SLB provides for switch-independent, bidirectional, fault-tolerant teaming and load balancing. Switch independence implies
that there is no specific support for this function required in the switch, allowing SLB to be compatible with all switches. Under
SLB, all adapters in the team have separate MAC addresses. The load-balancing algorithm operates on Layer 3 addresses of
the source and destination nodes, which enables SLB to load balance both incoming and outgoing traffic.
The BASP intermediate driver continually monitors the physical ports in a team for link loss. In the event of link loss on any
port, traffic is automatically diverted to other ports in the team. The SLB teaming mode supports switch fault tolerance by
allowing teaming across different switches- provided the switches are on the same physical network or broadcast domain.
Network Communications
The following are the key attributes of SLB:
Failover mechanism (link loss detection)
Load balancing algorithm. Inbound and outbound traffic are balanced through a Broadcom proprietary mechanism
based on L4 flows.
Outbound load balancing using MAC address is not supported.
Outbound load balancing using IP address is supported.
Multivendor teaming is supported (must include at least 1 Broadcom Ethernet adapter as a team member).
Applications
The Smart Load Balance and Failover algorithm is most appropriate in home and small business environments where cost is a
concern or commodity switching equipment is used. Smart Load Balance and Failover teaming works with unmanaged Layer 2
switches and is a cost-effective way of getting redundancy and link aggregation at the system. Smart Load Balance and
Failover also supports the teaming physical adapters having different link capabilities. In addition, Smart Load Balance and
Failover is recommended when switch fault tolerance is required.
Configuration Recommendations
The Smart Load Balance and Failover type of team supports connecting the teamed ports to hubs and switches if they are on
the same broadcast domain. It does not support connecting to a router or Layer 3 switches because the ports must be on the
same subnet.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Switch-Dependent
Generic Static Trunking
This mode supports a variety of environments where the adapter link partners are statically configured to support a
proprietary trunking mechanism. This mode could be used to support Lucent's Open Trunk, Cisco's Fast EtherChannel (FEC),
and Cisco's Gigabit EtherChannel (GEC). In the static mode, as in generic link aggregation, the switch administrator needs to
assign the ports to the team, and this assignment cannot be altered by the BASP, as there is no exchange of the Link
Aggregation Control Protocol (LACP) frame.
With this mode, all adapters in the team are configured to receive packets for the same MAC address. Trunking operates on
Layer 2 addresses and supports load balancing and failover for both inbound and outbound traffic. The BASP driver
determines the load-balancing scheme for outbound packets, using Layer 4 protocols previously discussed, whereas the team
link partner determines the load-balancing scheme for inbound packets.
The attached switch must support the appropriate trunking scheme for this mode of operation. Both the BASP and the switch
continually monitor their ports for link loss. In the event of link loss on any port, traffic is automatically diverted to other ports
in the team.
Network Communications
The following are the key attributes of Generic Static Trunking:
Failover mechanism (link loss detection)
Load balancing algorithm. Outbound traffic is balanced through Broadcom proprietary mechanism based L4 flows.
Inbound traffic is balanced according to a switch specific mechanism.
Outbound Load Balancing using MAC Address is not supported.
Outbound Load Balancing using IP address is supported.
Multivendor teaming is supported (must include at least one Broadcom Ethernet adapter as a team member)
Applications
Generic trunking works with switches that support Cisco Fast EtherChannel, Cisco Gigabit EtherChannel, Extreme Networks
Load Sharing and Bay Networks or IEEE 802.3ad Link Aggregation static mode. Since load balancing is implemented on Layer
2 addresses, all higher protocols such as IP, IPX, and NetBEUI are supported. Therefore, this is the recommended teaming
mode when the switch supports generic trunking modes over SLB.
Configuration Recommendations
Static trunking supports connecting the teamed ports to switches if they are on the same broadcast domain and support
generic trunking. It does not support connecting to a router or Layer 3 switches since the ports must be on the same subnet.
Dynamic Trunking (IEEE 802.3ad Link Aggregation)
This mode supports link aggregation through static and dynamic configuration via the Link Aggregation Control Protocol
(LACP). With this mode, all adapters in the team are configured to receive packets for the same MAC address. The MAC
address of the first adapter in the team is used and cannot be substituted for a different MAC address. The BASP driver
determines the load-balancing scheme for outbound packets, using Layer 4 protocols previously discussed, whereas the
team's link partner determines the load-balancing scheme for inbound packets. Because the load balancing is implemented on
Layer 2, all higher protocols such as IP, IPX, and NetBEUI are supported. The attached switch must support the 802.3ad Link
Aggregation standard for this mode of operation. The switch manages the inbound traffic to the adapter while the BASP
manages the outbound traffic. Both the BASP and the switch continually monitor their ports for link loss. In the event of link
loss on any port, traffic is automatically diverted to other ports in the team.
Network Communications
The following are the key attributes of Dynamic Trunking:
Failover mechanism Link loss detection
Load Balancing Algorithm Outbound traffic is balanced through a Broadcom proprietary mechanism based on L4
flows. Inbound traffic is balanced according to a switch specific mechanism.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Outbound Load Balancing using MAC Address - No
Outbound Load Balancing using IP Address - Yes
Multivendor teaming Supported (must include at least one Broadcom Ethernet adapter as a team member)
Applications
Dynamic trunking works with switches that support IEEE 802.3ad Link Aggregation dynamic mode using LACP. Inbound load
balancing is switch dependent. In general, the switch traffic is load balanced based on L2 addresses. In this case, all network
protocols such as IP, IPX, and NetBEUI are load balanced. Therefore, this is the recommended teaming mode when the switch
supports LACP, except when switch fault tolerance is required. SLB is the only teaming mode that supports switch fault
tolerance.
Configuration Recommendations
Dynamic trunking supports connecting the teamed ports to switches as long as they are on the same broadcast domain and
supports IEEE 802.3ad LACP trunking. It does not support connecting to a router or Layer 3 switches since the ports must be
on the same subnet.
Windows Support
BACS a
Number of teams
16
Yes
Hot add
Yes
Hot remove
Yes
Different speeds
Frame protocol
IP
Failover time
<500 ms
Fallback time
1.5 s b (approximate)
LiveLink support
Yes
MAC address
Different
Multivendor teaming
Yes
Generic Trunking
User interface
BACS
Number of teams
16
Yes
Hot add
Yes
Hot remove
Yes
Different speeds
Frame protocol
All
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Failover event
Failover time
500 ms
Fallback time
1.5 s b (approximate)
MAC address
Multivendor teaming
Yes
Dynamic Trunking
User interface
BACS
Number of teams
16
Yes
Hot add
Yes
Hot remove
Yes
Different speeds
Frame protocol
All
Failover time
<500 ms
Fallback time
1.5 s b (approximate)
MAC address
Multivendor teaming
Yes
FEC
100
GEC
1000
IEEE 802.3ad
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
each team member has been configured similarly. Settings to check include VLANs and QoS Packet Tagging, Jumbo Frames,
and the various offloads. The advanced adapter properties and teaming support are listed in Table8.
Table 8. Advanced Adapter Properties and Teaming Support
Adapter Property
Checksum Offload
Yes
No
Yesa
Jumbo Frames
Yesb
Yes
Wake on LAN
No
Checksum Offload
Checksum Offload is a property of the Broadcom network adapters that allows the TCP/IP/UDP checksums for send and
receive traffic to be calculated by the adapter hardware rather than by the host CPU. In high-traffic situations, this can allow
a system to handle more connections more efficiently than if the host CPU were forced to calculate the checksums. This
property is inherently a hardware property and would not benefit from a software-only implementation. An adapter that
supports Checksum Offload advertises this capability to the operating system so that the checksum does not need to be
calculated in the protocol stack; because the intermediate driver is located directly between the protocol layer and the
miniport driver, the protocol layer is not able to offload any checksums. Checksum Offload is only supported for IPv4 at this
time.
Jumbo Frames
The use of jumbo frames was originally proposed by Alteon Networks, Inc. in 1998 and increased the maximum size of an
Ethernet frame to a maximum size of 9000 bytes. Though never formally adopted by the IEEE 802.3 Working Group, support
for jumbo frames has been implemented in Broadcom adapters. The BASP intermediate driver supports jumbo frames,
provided that all of the physical adapters in the team also support jumbo frames and the same size is set on all adapters in
the team.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
group. VLANs also enable the administrator to enforce appropriate security and quality of service (QoS) policies. The BASP
supports the creation of 64 VLANs per team or adapter: 63 tagged and 1 untagged. The operating system and system
resources, however, limit the actual number of VLANs. VLAN support is provided according to IEEE 802.1q and is supported in
a teaming environment as well as on a single adapter. Note that VLANs are supported only with homogeneous teaming and
not in a multivendor teaming environment. The BASP intermediate driver supports VLAN tagging. One or more VLANs may be
bound to a single instance of the intermediate driver.
Wake on LAN
Wake on LAN (WOL) is a feature that allows a system to be awakened from a sleep state by the arrival of a specific packet
over the Ethernet interface. Because a virtual adapter is implemented as a software only device, it lacks the hardware
features to implement Wake on LAN and cannot be enabled to wake the system from a sleeping state via the virtual adapter.
The physical adapters, however, support this property, even when the adapter is part of a team.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Pings, however, flow between Red and Blue and between Red and Gray.
Furthermore, a failover event would cause additional loss of connectivity. Consider a cable disconnect on the Top Switch port
4. In this case, Gray would send the ICMP Request to Red 49:C9, but because the Bottom switch has no entry for 49:C9 in its
CAM Table, the frame is flooded to all its ports but cannot find a way to get to 49:C9.
Figure 3. Teaming Across Switches Without an Interswitch Link
The addition of a link between the switches allows traffic from/to Blue and Gray to reach each other without any problems.
Note the additional entries in the CAM table for both switches. The link interconnect is critical for the proper operation of the
team. As a result, it is highly advisable to have a link aggregation trunk to interconnect the two switches to ensure high
availability for the connection.
Figure 4. Teaming Across Switches With Interconnect
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Figure5 represents a failover event in which the cable is unplugged on the Top Switch port 4. This is a successful failover with
all stations pinging each other without loss of connectivity.
Figure 5. Failover Event
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Topology Change Notice (TCN)
Port Fast/Edge Port
In Ethernet networks, only one active path may exist between any two bridges or switches. Multiple active paths between
switches can cause loops in the network. When loops occur, some switches recognize stations on both sides of the switch.
This situation causes the forwarding algorithm to malfunction allowing duplicate frames to be forwarded. Spanning tree
algorithms provide path redundancy by defining a tree that spans all of the switches in an extended network and then forces
certain redundant data paths into a standby (blocked) state. At regular intervals, the switches in the network send and receive
spanning tree packets that they use to identify the path. If one network segment becomes unreachable, or if spanning tree
costs change, the spanning tree algorithm reconfigures the spanning tree topology and re-establishes the link by activating
the standby path. Spanning tree operation is transparent to end stations, which do not detect whether they are connected to
a single LAN segment or a switched LAN of multiple segments.
Spanning Tree Protocol (STP) is a Layer 2 protocol designed to run on bridges and switches. The specification for STP is
defined in IEEE 802.1d. The main purpose of STP is to ensure that you do not run into a loop situation when you have
redundant paths in your network. STP detects/disables network loops and provides backup links between switches or bridges.
It allows the device to interact with other STP compliant devices in your network to ensure that only one path exists between
any two stations on the network.
After a stable network topology has been established, all bridges listen for hello BPDUs (Bridge Protocol Data Units)
transmitted from the root bridge. If a bridge does not get a hello BPDU after a predefined interval (Max Age), the bridge
assumes that the link to the root bridge is down. This bridge then initiates negotiations with other bridges to reconfigure the
network to re-establish a valid network topology. The process to create a new topology can take up to 50 seconds. During
this time, end-to-end communications are interrupted.
The use of Spanning Tree is not recommended for ports that are connected to end stations, because by definition, an end
station does not create a loop within an Ethernet segment. Additionally, when a teamed adapter is connected to a port with
Spanning Tree enabled, users may experience unexpected connectivity problems. For example, consider a teamed adapter
that has a lost link on one of its physical adapters. If the physical adapter were to be reconnected (also known as fallback),
the intermediate driver would detect that the link has been reestablished and would begin to pass traffic through the port.
Traffic would be lost if the port was temporarily blocked by the Spanning Tree Protocol.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Layer 3 Routing/Switching
The switch that the teamed ports are connected to must not be a Layer 3 switch or router. The ports in the team must be in
the same network.
SLB Teams
SLB teams are the only teaming type not dependant on switch configuration. The server intermediate driver handles the load
balancing and fault tolerance mechanisms with no assistance from the switch. These elements of SLB make it the only team
type that maintains failover and fallback characteristics when team ports are connected directly to a hub.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Application Considerations
Teaming and Clustering
Teaming and Network Backup
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
acceptable). These interfaces serve two purposes. One adapter is used exclusively for intra-cluster heartbeat communications.
This is referred to as the private adapter and usually resides on a separate private subnetwork. The other adapter is used for
client communications and is referred to as the public adapter.
Multiple adapters may be used for each of these purposes: private, intracluster communications and public, external client
communications. All Broadcom teaming modes are supported with Microsoft Cluster Software for the public adapter only.
Private network adapter teaming is not supported. Microsoft indicates that the use of teaming on the private interconnect of a
server cluster is not supported because of delays that could possibly occur in the transmission and receipt of heartbeat
packets between the nodes. For best results, when you want redundancy for the private interconnect, disable teaming and
use the available ports to form a second private interconnect. This achieves the same end result and provides dual, robust
communication paths for the nodes to communicate over.
For teaming in a clustered environment, customers are recommended to use the same brand of adapters.
Figure7 shows a 2-node Fibre-Channel cluster with three network interfaces per cluster node: one private and two public. On
each node, the two public adapters are teamed, and the private adapter is not. Teaming is supported across the same switch
or across two switches. Figure8 shows the same 2-node Fibre-Channel cluster in this configuration.
Figure 7. Clustering With Teaming Across One Switch
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
NOTE: Microsoft Network Load Balancing is not supported with Microsoft Cluster Software.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Advanced Features
PXE is used extensively for the deployment of the cluster (installation and recovery of compute nodes). Teaming is typically
not used on the host side and it is not a part of our standard offering. Link aggregation is commonly used between switches,
especially for large configurations. Jumbo frames, although not a part of our standard offering, may provide performance
improvement for some applications due to reduced CPU overhead.
Oracle
In our Oracle Solution Stacks, we support adapter teaming in both the private network (interconnect between RAC nodes) and
public network with clients or the application layer above the database layer.
Figure 8. Clustering With Teaming Across Two Switches
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Load Balancing and Failover
Fault Tolerance
When you perform network backups in a nonteamed environment, overall throughput on a backup server adapter can be
easily impacted due to excessive traffic and adapter overloading. Depending on the number of backup servers, data streams,
and tape drive speed, backup traffic can easily consume a high percentage of the network link bandwidth, thus impacting
production data and tape backup performance. Network backups usually consist of a dedicated backup server running with
tape backup software such as NetBackup, Galaxy or Backup Exec. Attached to the backup server is either a direct SCSI tape
backup unit or a tape library connected through a fiber channel storage area network (SAN). Systems that are backed up
over the network are typically called clients or remote servers and usually have a tape backup software agent installed.
Figure9 shows a typical 1 Gbps nonteamed network environment with tape backup implementation.
Figure 9. Network Backup without Teaming
Because there are four client servers, the backup server can simultaneously stream four backup jobs (one per client) to a
multidrive autoloader. Because of the single link between the switch and the backup server, however, a 4-stream backup can
easily saturate the adapter and link. If the adapter on the backup server operates at 1 Gbps (125 MB/s), and each client is
able to stream data at 20 MB/s during tape backup, the throughput between the backup server and switch will be at 80MB/s
(20 MB/s x 4), which is equivalent to 64% of the network bandwidth. Although this is well within the network bandwidth
range, the 64% constitutes a high percentage, especially if other applications share the same link.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
For a tape backup server to reliably use adapter performance and network bandwidth when backing up clients, a network
infrastructure must implement teaming such as load balancing and fault tolerance. Data centers will incorporate redundant
switches, link aggregation, and trunking as part of their fault tolerant solution. Although teaming device drivers will
manipulate the way data flows through teamed interfaces and failover paths, this is transparent to tape backup applications
and does not interrupt any tape backup process when backing up remote systems over the network. Figure10 shows a
network topology that demonstrates tape backup in a Broadcom teamed environment and how smart load balancing can load
balance tape backup data across teamed adapters.
There are four paths that the client-server can use to send data to the backup server, but only one of these paths will be
designated during data transfer. One possible path that Client-Server Red can use to send data to the backup server is:
Example Path: Client-Server Red sends data through Adapter A, Switch 1, Backup Server Adapter A.
The designated path is determined by two factors:
1. Client-Server ARP cache; which points to the backup server MAC address. This is determined by the Broadcom
intermediate driver inbound load balancing algorithm.
2. The physical adapter interface on Client-Server Red will be used to transmit the data. The Broadcom intermediate
driver outbound load balancing algorithm determines this (see Outbound Traffic Flow and Inbound Traffic Flow (SLB
Only)).
The teamed interface on the backup server transmits a gratuitous address resolution protocol (G-ARP) to Client-Server Red,
which in turn, causes the client server ARP cache to get updated with the Backup Server MAC address. The load balancing
mechanism within the teamed interface determines the MAC address embedded in the G-ARP. The selected MAC address is
essentially the destination for data transfer from the client server. On Client-Server Red, the SLB teaming algorithm will
determine which of the two adapter interfaces will be used to transmit data. In this example, data from Client-Server Red is
received on the backup server Adapter A interface. To demonstrate the SLB mechanisms when additional load is placed on the
teamed interface, consider the scenario when the backup server initiates a second backup operation: one to Client-Server
Red, and one to Client-Server Blue. The route that Client-Server Blue uses to send data to the backup server is dependant on
its ARP cache, which points to the backup server MAC address. Because Adapter A of the backup server is already under load
from its backup operation with Client-Server Red, the backup server invokes its SLB algorithm to inform Client-Server Blue
(thru an G-ARP) to update its ARP cache to reflect the backup server Adapter B MAC address. When Client-Server Blue needs
to transmit data, it uses either one of its adapter interfaces, which is determined by its own SLB algorithm. What is important
is that data from Client-Server Blue is received by the Backup Server Adapter B interface, and not by its Adapter A interface.
This is important because with both backup streams running simultaneously, the backup server must load balance data
streams from different clients. With both backup streams running, each adapter interface on the backup server is processing
an equal load, thus load-balancing data across both adapter interfaces.
The same algorithm applies if a third and fourth backup operation is initiated from the backup server. The teamed interface
on the backup server transmits a unicast G-ARP to backup clients to inform them to update their ARP cache. Each client then
transmits backup data along a route to the target MAC address on the backup server.
Fault Tolerance
If a network link fails during tape backup operations, all traffic between the backup server and client stops and backup jobs
fail. If, however, the network topology was configured for both Broadcom SLB and switch fault tolerance, then this would
allow tape backup operations to continue without interruption during the link failure. All failover processes within the network
are transparent to tape backup software applications. To understand how backup data streams are directed during network
failover process, consider the topology in Figure8. Client-Server Red is transmitting data to the backup server through Path
1, but a link failure occurs between the backup server and the switch. Because the data can no longer be sent from Switch #1
to the Adapter A interface on the backup server, the data is redirected from Switch #1 through Switch #2, to the Adapter B
interface on the backup server. This occurs without the knowledge of the backup application because all fault tolerant
operations are handled by the adapter team interface and trunk settings on the switches. From the client server perspective,
it still operates as if it is transmitting data through the original path.
Figure 10. Network Backup With SLB Teaming Across Two Switches
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
3. All switches that the team is directly connected to must have the same hardware revision, firmware revision, and
software revision to be supported.
4. To be teamed, adapters should be members of the same VLAN. In the event that multiple teams are configured, each
team should be on a separate network.
5. Do not enter a multicast or broadcast address in the Locally Administered Address field.
6. Do not assign a Locally Administered Address on any physical adapter that is a member of a team.
7. Verify that power management is disabled on all physical members of any team (the Allow the computer to turn off
this device to save power box on the Power Management tab in adapter Properties should be clearedsee
Setting Power Management Options in "Windows Driver and Application Installation").
8. Remove any static IP address from the individual physical team members before the team is built.
9. A team that requires maximum throughput should use LACP or GEC\FEC. In these cases, the intermediate driver is only
responsible for the outbound load balancing while the switch performs the inbound load balancing.
10. Aggregated teams (802.3ad \ LACP and GEC\FEC) must be connected to only a single switch that supports IEEE
802.3a, LACP or GEC/FEC.
11. It is not recommended to connect any team to a hub, as a hub only support half duplex. Hubs should be connected to
a team for troubleshooting purposes only. Disabling the device driver of a network adapter participating in an LACP or
GEC/FEC team may have adverse affects with network connectivity. Broadcom recommends that the adapter first be
physically disconnected from the switch before disabling the device driver in order to avoid a network connection loss.
12. Verify the base (Miniport) and team (intermediate) drivers are from the same release package.Dell does not test or
support the mixing of base and teaming drivers from different CD releases.
13. Test the connectivity to each physical adapter prior to teaming.
14. Test the failover and fallback behavior of the team before placing into a production environment.
15. When moving from a nonproduction network to a production network, it is strongly recommended to test again for
failover and fallback.
16. Test the performance behavior of the team before placing it into a production environment.
Troubleshooting Guidelines
Before you call for support, make sure you have completed the following steps for troubleshooting network connectivity
problems when the server is using adapter teaming.
1. Make sure the link light is ON for every adapter and all the cables are attached.
2. Check that the matching base and intermediate drivers belong to the same Dell release and are loaded correctly.
3. Check for a valid IP address using the ipconfig command for Windows.
4. Check that STP is disabled or Edge Port/Port Fast is enabled on the switch ports connected to the team or that LiveLink
is being used.
5. Check that the adapters and the switch are configured identically for link speed and duplex.
6. If possible, break the team and check for connectivity to each adapter independently to confirm that the problem is
directly associated with teaming.
7. Check that all switch ports connected to the team are on the same VLAN.
8. Check that the switch ports are configured properly for Generic Trunking (FEC/GEC)/802.3ad-Draft Static type of
teaming and that it matches the adapter teaming type. If the system is configured for an SLB type of team, make sure
the corresponding switch ports are not configured for Generic Trunking (FEC/GEC)/802.3ad-Draft Static types of teams.
Under what circumstances is traffic not load balanced? Why is all traffic not load balanced evenly across the
team members?
Answer:
The bulk of traffic does not use IP/TCP/UDP or the bulk of the clients are in a different network. The receive
load balancing is not a function of traffic load, but a function of the number of clients that are connected to
the system.
Question:
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Answer:
Broadcom's teaming software only supports IP/TCP/UDP traffic. All other traffic is forwarded to the primary
adapter.
Question:
Which protocols are load balanced with SLB and which ones are not?
Answer:
Only IP/TCP/UDP protocols are load balanced in both directions: send and receive. IPX is load balanced on the
transmit traffic only.
Question:
Can I team a port running at 100 Mbps with a port running at 1000 Mbps?
Answer:
Mixing link speeds within a team is only supported for Smart Load Balancing teams and 802.3ad teams, as
stated earlier.
Question:
Answer:
Yes with SLB, and yes if the switch allows for it in FEC/GEC and 802.3ad.
Question:
What is the difference between adapter load balancing and Microsoft's Network Load Balancing (NLB)?
Answer:
Adapter load balancing is done at a network session level, whereas NLB is done at the system application
level.
Question:
Answer:
Yes. Teamed ports can be connected to a hub for troubleshooting purposes. However, this practice is not
recommended for normal operation because the performance would be degraded due to hub limitations.
Connect the teamed ports to a switch instead.
Question:
Answer:
No. All ports in a team must be on the same network; in a router, however, each port is a separate network
by definition. All teaming modes require that the link partner be a Layer 2 switch.
Question:
Answer:
Yes. Teaming is supported on the public network only, not on the private network used for the heartbeat link.
Question:
Answer:
A PXE client operates in an environment before the operating system is loaded; as a result, virtual adapters
have not been enabled yet. If the physical adapter supports PXE, then it can be used as a PXE client, whether
or not it is part of a virtual adapter when the operating system loads. PXE servers may operate over a virtual
adapter.
Question:
Answer:
Wake-on-LAN functionality operates in an environment before the operating system is loaded. WOL occurs
when the system is off or in standby, so no team is configured.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Question:
Answer:
Question:
What is the maximum number of teams that can be configured on the same system?
Answer:
Question:
Why does my team lose connectivity for the first 30 to 50 seconds after the primary adapter is restored
(fallback)?
Answer:
Because Spanning Tree Protocol is bringing the port from blocking to forwarding. You must enable Port Fast or
Edge Port on the switch ports connected to the team or use LiveLink to account for the STP delay.
Question:
Answer:
Smart Load Balancing can be used with multiple switches because each physical adapter in the system uses a
unique Ethernet MAC address. Link Aggregation and Generic Trunking cannot operate across switches because
they require all physical adapters to share the same Ethernet MAC address.
Question:
Answer:
The intermediate driver cannot be upgraded through the Local Area Connection Properties. It must be
upgraded using the Setup installer.
Question:
Answer:
In Broadcom Advanced Control Suite, click the BASP Statistics tab for the virtual adapter.
Question:
Answer:
Yes, but only when running NLB in a multicast mode (NLB is not supported with MS Cluster Services).
Question:
Should both the backup system and client systems that are backed up be teamed?
Answer:
Because the backup system is under the most data load, it should always be teamed for link aggregation and
failover. A fully redundant network, however, requires that both the switches and the backup clients be
teamed for fault tolerance and link aggregation.
Question:
During backup operations, does the adapter teaming algorithm load balance data at a byte-level or a sessionlevel?
Answer:
When using adapter teaming, data is only load balanced at a session level and not a byte level to prevent outof-order frames. Adapter teaming load balancing does not work the same way as other storage load balancing
mechanisms such as EMC PowerPath.
Question:
Is there any special configuration required in the tape backup software or hardware to work with adapter
teaming?
Answer:
No special configuration is required in the tape software to work with teaming. Teaming is transparent to tape
backup applications.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
Question:
Answer:
In all operating systems, the most accurate method for checking the driver revision is to physically locate the
driver file and check the properties.
Question:
Answer:
No. SLB can detect only the loss of link between the teamed port and its immediate link partner. SLB cannot
detect link failures on other ports. For more information, see LiveLink Functionality.
Question:
Answer:
Question:
Where do I monitor real time statistics for an adapter team in a Windows system?
Answer:
Use the Broadcom Advanced Control Suite (BACS) to monitor general, IEEE 802.3, and custom counters.
Cause
Corrective Action
Failed to access
The driver cannot access PCI
configuration information.
configuration space registers on
Reinstall the network
the adapter.
driver.
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
network cable is properly connection with its link partner.
connected.
The adapter has established a
link.
Network controller
The adapter has been manually
configured for 10Mb half- configured for the selected line
duplex link.
speed and duplex settings.
Network controller
configured for 10Mb fullduplex link.
Network controller
configured for 100Mb
half-duplex link.
Network controller
The adapter has been manually
configured for 100Mb full- configured for the selected line
duplex link.
speed and duplex settings.
10
Network controller
configured for 1Gb halfduplex link.
11
Network controller
configured for 1Gb fullduplex link.
12
13
14
15
Driver initialized
successfully.
16
17
18
19
Cause
Corrective Action
Broadcom Gigabit Ethernet Teaming Services: Broadcom NetXtreme BCM57XX User Guide
management interface.
device instance.
Another driver has created a Unload the conflicting device driver that uses the
conflicting device name.
name Blf.
10
11
12
13
14
15
16
17
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Back to Contents Page
BACS Interface
Connecting to a Host
Configuring Teaming
Troubleshooting BACS
BACS Interface
The BACS interface is comprised of the following regions:
Explorer View pane
Context View selector
Context View pane
Menu bar
Description pane
By default, the Explorer View pane is docked and pinned on the left side of the main window, the Context View pane on the right, the
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Context View selector below the menu bar, and the Description pane below the Context View pane. Drag the splitter between any
two panes to vary the size of the panes.
Filter View
In a multiple-host environment using several C-NICs, there can be a large number of manageable elements per adapter that can be
difficult and cumbersome to view, configure, and manage all elements. Use the filter to select a particular device function. Possible
filter views include:
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
All
Team view
NDIS view
iSCSI view
FCoE view
iSCSI Target view
FCoE Target view
Menu Bar
The following appear on the menu bar, but because the menu items are context-sensitive, not all items will be available at all times:
File menu
Team Save As: Saves the current team configurations to a file.
Team Restore: Restores any saved team configuration from a file.
Action menu
Remove Host: Removes the selected host.
Refresh Host: Refreshes the selected host.
View menu
Explorer View: Displays/hides the Explorer View pane.
Tool Bar: Displays/hides the tool bar.
Status Bar: Displays/hides the status bar.
Broadcom Logo: Displays/hides the Broadcom Logo on BACS to optimize the maximum viewable space.
Tools menu
Options: Used for configuring BACS preferences.
Teams (Windows only)
Create Teams: Creates new teams with either the Teaming Wizard or in Advanced mode.
Manage Teams: Manages existing teams with either the Teaming Wizard or in Advanced mode.
iSCSI menu
Discovery Wizard: Locates targets and helps to configure the HBA.
Manage Targets Wizard: Manages targets.
Manage iSNS Servers: Manages Internet Storage Name Service (iSNS) servers to allow discovery, management, and
configuration of iSCSI devices.
Manage Discovery Portals: Manages iSCSI discovery portals.
Discovery Wizard
The Discovery Wizard is available from the iSCSI menu. Follow the prompts in the wizard to discover iSCSI targets via the
SendTargets method or the Internet Storage Name Service (iSNS) server.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Description Pane
The Description pane provides information, configuration instructions, and options for the selected parameter in the Context View
pane.
Connecting to a Host
You can add one or more Windows or Linux hosts to manage from BACS.
To add a local host
1. From the Action menu, click Add Host.
2. For both Windows and Linux hosts, do not change the default settings. The User name and Password are not required while
connecting to the local host.
3. Select Persist if you want BACS to save the information for this host.
4. Click Ok. BACS can now be used to view information and manage the host.
To add a remote host
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
1. From the Action menu, click Add Host.
2. Type the remote host's name or IP address in the Host box.
3. Select the protocol from the Protocol list. The protocol options for Windows are WMI, WSMan, or Try All. The protocol
options for Linux are CimXML, WSMan, or Try All. The Try All option forces the GUI client to try all options.
4. Select the HTTP scheme, or the HTTPS scheme for added security.
5. Type the Port Number value you used to configure the host, if it is different than the default value of 5985.
6. Type the User name and Password.
7. Select Persist if you want BACS to save the information for this host. The host will appear in the Explorer Pane whenever you
reopen BACS, and you will not need to enter the host IP address or host name when connecting to the host. For security
reasons, you must enter the User name and Password every time.
8. Click OK.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Displays the operating system, including the version.
Platform
Displays the hardware architecture platform (for example, 32-bit or 64-bit)
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
The current IQN name is displayed. Click the IQN name to modify the host's iSCSI initiator name, and then click Apply.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
The number of PCI-E lanes connected to the bridge. This information is only available for Broadcom NetXtremeII adapters.
Bridge Speed
The clock speed on PCI-E bus. This information is only available for Broadcom NetXtremeII adapters.
Slot Number
The slot number on the system board occupied by the adapter. This item is not available for PCI Express type adapters.
Bus Speed
The bus clock signal frequency used by the adapter. This item is not available for PCI Express type adapters.
Bus Width
The number of bits that the bus can transfer at a single time to and from the adapter. This item is not available for PCI Express type
adapters.
Bus Number
Indicates the number of the bus where the adapter is installed.
Device Number
The number assigned to the adapter by the operating system.
Function Number
The port number of the adapter. For a single-port adapter, the function number is 0. For a two-port adapter, the function number for
the first port is 0, and the function number for the second port is 1.
Interrupt Request
The interrupt line number that is associated with the adapter. Valid numbers range from 2 to 25.
Memory Address
The memory mapped address that is assigned to the adapter. This value can never be 0.
MSI Version
This is the Message Signaled Interrupts (MSI) version being used. The option MSI corresponds to the PCI 2.2 specification that
supports 32 messages and a single MSI address value. The option MSI-X corresponds to the PCI 3.0 specification that supports
2,048 messages and an independent message address for each message.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Family Firmware Version
The global firmware version that represents all firmware on the device.
Management Firmware
The firmware version installed on the system.
Vendor ID
The vendor ID.
Device ID
The adapter ID.
Subsystem Vendor ID
The subsystem vendor ID.
Subsystem ID
The subsystem ID.
iSCSI Boot Version
The version of the iSCSI Boot software initiator.
External PHY Firmware Version
The external PHY firmware version.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
1. Select PortX (where X is either 0 or 1) below Adapter in the object explorer.
2. Various components of the port will be displayed below port in the object explorer. You can click on the "+" icon near Port to
expand or collapse the tree below.
3. Select Information tab in the context view panel on the right side.
MAC Address
A physical MAC (media access control) address that is assigned to the adapter by the manufacturer. The physical address is never all
0s.
Permanent MAC Address
The unique hardware address assigned to the network adapter.
iSCSI MAC Address
If an iSCSI network adapter is loaded onto the system, this parameter will display the iSCSI MAC address.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
IPv4 DHCP
The IP address is from a DHCP server if the value is Enable.
IP Address
The network address associated with the adapter. If the IP address is all 0s, the associated driver has not been bound with Internet
Protocol (IP).
IPv6 DHCP
The IP address is from a DHCP server if the value is Enable.
IPv6 IP Address
The IPv6 network address associated with the adapter.
IPv6 Scope Id
Since local-use addresses can be reused, the Scope ID for link-local addresses specifies the link where the destination is located. The
Scope ID for site-local addresses specifies the site where the destination is located. The Scope ID is relative to the sending host.
IPv6 Flow Info
The non-zero Flow Info is used to classify traffic flows. If Flow Info equals zero, then the packets are not a part of any flow.
Default Gateway
The default gateway value is the network address of the gateway that will be used by the management firmware for packets destined
for hosts external to the local network segment.
Link Status
The status of the network link.
Up. A link is established.
Down. A link is not established.
Duplex
The adapter is operating in the indicated duplex mode.
Speed
The link speed of the adapter, in megabits per second.
Offload Capabilities
The offload capabilities supported by the adapter. This information is only available for Broadcom NetXtremeII adapters.
TOE. TCP Offload Engine (TOE) allows simultaneous operation of up to 1024 fully offloaded TCP connections for 1-Gbps
network adapters and 1880 fully offloaded TCP connections for 10-Gbps network adapters to the hardware.
iSCSI. iSCSI offload for block-level transfer of data.
LSO. Large Send Offload (LSO) prevents an upper level protocol such as TCP from breaking a large data packet into a series
of smaller packets with headers appended to them.
CO. Checksum Offload (CO) allows the TCP/IP/UDP checksums for send and receive traffic to be calculated by the adapter
hardware rather than by the host CPU.
LiveLink IP Address
The network address of the LiveLink enabled adapter.
Local Connection
Identifies the module to which the blade server is attached.
Chassis SW. Chassis switch module
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Chassis PHY. Pass-through module
None. No modules attached
BASP State
Information about the status of the BASP application. This information is displayed only when there is a team (see Configuring
Teaming).
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
The Network Test option on the Diagnostics tab lets you verify IP network connectivity. This test verifies if the driver is installed
correctly and tests connectivity to a gateway or other specified IP address on the same subnet.
The network test uses TCP/IP to send ICMP packets to remote systems, then waits for a response. If a gateway is configured, the
test automatically sends packets to that system. If a gateway is not configured or if the gateway is unreachable, the test prompts for
a destination IP address.
NOTES:
The network test option is not available on adapters that are grouped into a team (see Configuring Teaming).
This feature can be used with Windows Server managed hosts only. It is not available for hosts operating on Linux or other
OSes. You can, however use BACS on a Linux client to connect to a Windows Server host and run the network test utility.
To run the network test using the BACS GUI
1. Click the name of the adapter to test in the Explorer View pane.
2. From the Select a test to run list, select Network Test.
3. To change the destination IP address, select IP address to ping, then click the browse button (...). In the Network Test
window, enter a Destination IP address, then click OK.
4. Click Run.
The results of the network test are displayed in the Status field.
To run the network test using the BACS CLI
You can use the following CLI command to perform a network diagnostic test for the specified target. This command is available for
NDIS and virtual adapters.
BACScli -t <target type> -f <target format> -i <target ID> networkdiag [-p <IP address>]
Examples:
1. The following command runs the network test for the current selected NDIS adapter.
BACScli -t NDIS -f mac -i 0010181a1b1c "networkdiag -p 192.168.1.5"
2. The following command runs the network test for the current selected virtual adapter. Since there is no IP address specified,
BACScli will use gateway address for the test.
BACScli -t VNIC -f mac -i 0010181a1b1c "networkdiag"
In Interactive mode, use the list <view> and select <idx> commands to select the desired target device. Use networkdiag [-p
<IP address>] to run the network diagnostics test for the selected target.
Examples:
1. The following command runs the network test for the currently selected NDIS adapter.
networkdiag -p 192.168.1.5
2. The following command runs the network test for the current selected virtual adapter.
networkdiag
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
You must have administrator privileges to run diagnostic tests.
The network connection is temporarily lost while these tests are running.
Some tests are not supported on all Broadcom adapters.
To run the diagnostic tests once using the BACS GUI
1. Click the name of the adapter to test in the Explorer View pane and select the Diagnostics tab.
2. From the Select a test to run list, select Diagnostic Tests.
3. Select the diagnostic tests you want to run. Click Select All to select all tests or Clear All to clear all test selections.
4. Select the number of times to run the tests from Number of loops.
5. Click Run test(s).
6. In the error message window that warns of the network connection being temporarily interrupted, click Yes. The results are
displayed in the Result field for each test.
Control Registers
This test verifies the read and write capabilities of the network adapter registers by writing various values to the registers and
verifying the results. The adapter driver uses these registers to perform network functions such as sending and receiving information.
A test failure indicates that the adapter may not be working properly.
MII Registers
This test verifies the read and write capabilities of the registers of the physical layer (PHY). The physical layer is used to control the
electrical signals on the wire and to configure network speeds such as 1000 Mbit/s.
EEPROM
This test verifies the content of the electrically erasable programmable read-only memory (EEPROM) by reading a portion of the
EEPROM and computing the checksum. The test fails if the computed checksum is different from the checksum stored in the EEPROM.
An EEPROM image upgrade does not require a code change for this test.
Internal Memory
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
This test verifies that the internal memory of the adapter is functioning properly. The test writes patterned values to the memory and
reads back the results. The test fails if an erroneous value is read back. The adapter cannot function if its internal memory is not
functioning properly.
On-Chip CPU
This test verifies the operation of the internal CPUs in the adapter.
Interrupt
This test verifies that the Network Device Driver Interface Specification (NDIS) driver is able to receive interrupts from the adapter.
Loopback MAC and Loopback PHY
These tests verify that the NDIS driver is able to send packets to and receive packets from the adapter.
Test LED
This test causes all of the port LEDs to blink 5 times for the purpose of identifying the adapter.
To run the diagnostic tests using the BACS CLI
You can use the following CLI command to run diagnostics tests on a specified target. This command is available for physical device
ports only:
BACScli -t <target type> -f <target format> -i <target ID> "diag {[-c REG ] [-c MII ] [-c EEP] [-c MEM] [-c
CPU] [-c INT] [-c MACLB ] [-c PHYLB] [-c LED] | [-c ALL]} [-l <cnt> ] [ -v <LEDIntv> ]"
Examples:
1. The following command displays all the diagnostics tests available for the current selected target.
BACScli -t PHYPORTS -f bdf -i 01:00.00 "diag"
2. The following command runs the MII and LED tests for the selected target:
BACScli -t PHYPORTS -f bdf -i 01:00.00 "diag -c MII -c LED"
3. The following command runs all the tests five times with an LED test interval of 8 ms for the selected target:
BACScli -t PHYPORTS -f bdf -i 01:00.00 "diag -c all -l 5 -v 8"
In Interactive mode, use the list <view> and select <idx> commands to select the desired target device. Use the following
command to run diagnostic tests for the selected target:
diag {[-c REG ] [-c MII ] [-c EEP] [-c MEM] [-c CPU] [-c INT] [-c MACLB ] [-c PHYLB] [-c LED] | [-c ALL]} [-l
<cnt> ] [ -v <LEDIntv> ]
Examples:
1. The following command displays all the diagnostics tests available for the current selected target.
diag
2. The following command runs the MII and LED test for the selected target.
diag -c MII -c LED
3. The following command runs all the tests five times, with an LED test interval of 8 ms for the selected target.
diag -c all -l 5 -v 8
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
You must have administrator privileges to run the cable analysis test.
The network connection is temporarily lost during an analysis.
This option is not available for NetXtreme II 10 GbE network adapters.
This option is not available for all Broadcom network adapters.
This option is available for Broadcom NetXtreme II VBD drivers.
To run a cable analysis using BACS GUI
1. Connect the cable to a port on a switch where the port is set to Auto and the Speed & Duplex driver settings are also set to
Auto.
2. Click the name of the adapter to test in the Explorer View pane.
NOTE: For Broadcom NetXtremeII adapters, select a VBD driver; for other adapters, select an NDIS driver.
3. From the Select a test to run list, select Cable Analysis.
4. Click Run.
5. In the error message window that warns of the network connection being temporarily interrupted, click Yes.
Distance
The valid cable length in meters (except when the Noise result is returned).
Status
The result of the analysis for the indicated pair.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Good. Good cable/PCB signal paths, but no gigabit link.
Crossed. Pin short or crosstalk along two or more cable/PCB signal paths.
Open. One or both pins are open for a twisted pair.
Short. Two pins from the same twisted pair are shorted together.
Noise. Persistent noise present (most likely caused by Forced 10/100).
GB Link. Gigabit link is up and running.
N/A. Algorithm failed to reach a conclusion.
Link
The link connection speed and mode.
Status
The status after the test is run, either completed or failed.
There are several factors that could have an effect on the test results:
Link partner. Various switch and hub manufacturers implement different PHYs. Some PHYs are not IEEE compliant.
Cable quality. Category 3, 4, 5, and 6 may affect the test results.
Electrical interference. The testing environment may affect the test results.
To run a cable analysis using BACS CLI
You can use the following CLI commands to run cable analysis for the specified target. This command is available for physical device
ports only.
BACScli -t <target type> -f <target format> -i <target ID> cablediag
Example:
1. The following command runs the cable diagnostics test for the current selected target.
BACScli -t PHYPORTS -f bdf -i 01:00.00 "cablediag"
In Interactive mode, use the list <view> and select <idx> commands to select the desired target device. Use the cablediag
command to run the cable analysis test for the selected target.
Example:
1. The following command runs the cable diagnostics test for the currently selected NDIS adapter.
cablediag
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
NOTES:
Information about Broadcom network adapters may be more comprehensive than information about network adapters made
by others.
Some information may not be available for all Broadcom network adapters.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
The network address associated with the adapter. If the IP address is all 0s, the associated driver has not been bound with Internet
Protocol (IP).
IPv6 IP Address
The IPv6 network address associated with the adapter.
MAC Address
A physical MAC (media access control) address that is assigned to the adapter by the manufacturer. The physical address is never all
0s.
Permanent MAC Address
The unique hardware address assigned to the network adapter.
Offload Capabilities
The offload capabilities supported by the adapter. This information is only available for Broadcom NetXtremeII adapters.
TOE. TCP Offload Engine (TOE) allows simultaneous operation of up to 1024 fully offloaded TCP connections for 1-Gbps
network adapters and 1880 fully offloaded TCP connections for 10-Gbps network adapters to the hardware.
iSCSI. iSCSI offload for block-level transfer of data.
LSO. Large Send Offload (LSO) prevents an upper level protocol such as TCP from breaking a large data packet into a series
of smaller packets with headers appended to them.
CO. Checksum Offload (CO) allows the TCP/IP/UDP checksums for send and receive traffic to be calculated by the adapter
hardware rather than by the host CPU.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Some properties may not be available for all Broadcom network adapters.
If an adapter is included as a member of a team and you change any advanced property, then you must rebuild the team to
ensure that the team's advanced properties are properly set.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Normally, the checksum function is computed by the protocol stack. When you select one of the Checksum Offload property values
(other than None), the checksum can be computed by the network adapter.
Rx Enabled. Enables receive TCP/IP/UDP checksum offload.
Tx Enabled. Enables transmit TCP/IP/UDP checksum offload.
Tx/Rx Enabled (default). Enables transmit and receive TCP/IP/UDP checksum offload.
None. Disables checksum offload.
IPv4 Large Send Offload
Normally, the TCP segmentation is done by the protocol stack. When you enable the Large Send Offload property, the TCP
segmentation can be done by the network adapter. The default setting for this property is Enabled. This property is only available for
Broadcom NetXtremeII adapters.
IPv6 Checksum Offload
Normally, the checksum function is computed by the protocol stack. When you select one of the Checksum Offload property values
(other than None), the checksum can be computed by the network adapter.
Rx Enabled. Enables receive TCP/IP/UDP checksum offload.
Tx Enabled. Enables transmit TCP/IP/UDP checksum offload.
Tx/Rx Enabled (default). Enables transmit and receive TCP/IP/UDP checksum offload.
None. Disables checksum offload.
IPv6 Large Send Offload
Normally, the TCP segmentation is done by the protocol stack. When you enable the Large Send Offload property, the TCP
segmentation can be done by the network adapter. The default setting for this property is Enabled. This property is only available for
Broadcom NetXtremeII adapters.
Jumbo Packet
Enables the network adapter to transmit and receive oversized Ethernet frames that are greater than 1514 bytes, but less than or
equal to 9000 bytes in length (9600 bytes for network adapters that operate at 10 Gbps). This property requires the presence of a
switch that is able to process jumbo frames. This property is only available for Broadcom NetXtremeII adapters.
Frame size is set at 1500 bytes by default. To increase the size of the received frames, raise the byte quantity in 500-byte
increments.
NOTE: If Jumbo Packet is set to 5000 bytes or greater on network adapters that support 10 Gbps link speed, ensure that Flow
Control is set to Auto to prevent the system performance from performing at less than optimal levels. This limitation exists on a
per-port basis.
NOTE: If SR-IOV is enabled on a virtual function (VF) on the adapter, ensure that the same jumbo packet settings is configured
on both the VF and the Microsoft synthetic adapter. You can configure these values using Windows Device Manager > Advanced
properties.
If there is a mismatch in the values, the SRIOV function will be shown the the Degraded state in Hyper-V > Networking Status.
LSO & Jumbo Frames
Enables the network adapter to transmit and receive oversized Ethernet frames that are greater than 1514 bytes, but less than 9000
bytes in length. This property requires the presence of a switch that is able to process Jumbo frames. This information is available for
some Broadcom NetXtreme adapters and is not available for all Broadcom NetXtreme II adapters.
Frame size is set at 1500 bytes by default by selecting the LSO Enabled, Jumbo Off option.
NOTE: Jumbo Frames and Large Send Offload (LSO) are mutually-exclusive properties.
Normally, the TCP segmentation is done by the protocol stack. When you enable the Large Send Offload (LSO) property, the TCP
segmentation can be done by the network adapter.
Both Disabled. LSO & Jumbo frames are both disabled.
LSO Enabled, Jumbo Off (default). LSO enabled and jumbo frames disabled.
LSO Off, Jumbo 3000. Frame size of 3000 bytes and LSO off.
LSO Off, Jumbo 4500. Frame size of 4500 bytes and LSO off.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
LSO Off, Jumbo 6000. Frame size of 6000 bytes and LSO off.
LSO Off, Jumbo 7500. Frame size of 7500 bytes and LSO off.
LSO Off, Jumbo 9000. Frame size of 9000 bytes and LSO off.
Locally Administered Address
The Locally Administered Address is a user-defined MAC address that is used in place of the MAC address originally assigned to the
network adapter. Every adapter in the network must have its own unique MAC address. This locally administered address consists of
a 12-digit hexadecimal number.
Value. Assigns a unique node address for the adapter.
Not Present (default). Uses the factory-assigned node address on the adapter.
The appropriate assigned ranges and exceptions for the locally administered address include the following:
The range is 00:00:00:00:00:01 to FF:FF:FF:FF:FF:FD.
Do not use a multicast address (least significant bit of the high byte = 1).
Do not use all 0s or all Fs.
Do not assign a Locally Administered Address on any physical adapter that is a member of a team.
Do not assign a Locally Administered Address on any iSCSI boot-enabled device.
NOTE: In some cases, network devices will allow a different combination of hexadecimal and alphanumeric characters than what
is entered, which could result in an invalid MAC address. If you assign an address, confirm the assigned address using the ipconfig
utility at the DOS prompt.
Receive Side Scaling
Allows configuring network load balancing across multiple CPUs. The default setting for this property is Enabled.
Switch Configuration.
Allows configuring of the connected switch for the network adapters.
NOTE: Switch Configuration only applies to blade configurations.
SW_Config_10G (default). Sets the switch speed to 10 Gbit/s.
SW_Config_1G. Sets the switch speed to 1 Gbit/s.
Speed & Duplex
The Speed & Duplex property sets the connection speed and mode to that of the network. Note that Full-Duplex mode allows the
adapter to transmit and receive network data simultaneously.
10 Mb Full. Sets the speed at 10 Mbit/s and the mode to Full-Duplex.
10 Mb Half. Sets the speed at 10 Mbit/s and the mode to Half-Duplex.
100 Mb Full. Sets the speed at 100 Mbit/s and the mode to Full-Duplex.
100 Mb Half. Sets the speed at 100 Mbit/s and the mode to Half-Duplex.
1 Gb Full. Sets the speed at 1000 Mb Full-Duplex mode only. Not available for 1Gb ports.
10 GB Full. Sets the speed to 10 Gbit/s and the mode to Full-Duplex. Not available for 1Gb ports.
Auto (default). Sets the speed and mode for optimum network connection (recommended).
NOTES:
Auto is the recommended setting. This setting allows the network adapter to dynamically detect the line speed of the network.
Whenever the network capability changes, the network adapter automatically detects and adjusts to the new line speed and
duplex mode. A speed of 1 Gbit/s is enabled by selecting Auto, when that speed is supported.
1 Gb Full Auto must be attached to a link partner that is also capable of a 1 Gb connection. Since the connection is limited to
a 1 Gb connection only, the Ethernet@Wirespeed feature will be disabled. If the link partner supports a 1Gb connection only,
the Wake on LAN feature may not work. Additionally, management traffic (IPMI or UMP) in the absence of an operating system
may also be affected.
10 Mb Half and 100 Mb Half settings force the network adapter to connect to the network in Half-Duplex mode. Note that the
network adapter may not function if the network is not configured to operate at the same mode.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
10 Mb Full and 100 Mb Full settings force the network adapter to connect to the network in Full-Duplex mode. The network
adapter may not function if the network is not configured to operate at the same mode.
1 Gb copper ports cannot be forced to 1 Gb.
Speed & Duplex (SerDes)
1 Gb Full. Forces the speed to 1 Gb Full based on a matching setting for its link partner.
Auto (default). Sets the speed to auto-negotiate with its link partner at the highest matching speed.
Auto with 1Gb Fallback Full. Sets the speed to auto-negotiate with its link partner, but if the attached link partner is forced
at 1 Gbit/s, it will fall back to this mode.
Hardware Default. Sets the speed to negotiate according to the setting specified by the manufacturer (see manufacturer
documentation for more information).
Wake Up Capabilities
Enables the network adapter to wake up from a low-power mode when it receives a network wake-up frame. Two types of wake-up
frames are possible: Magic Packet and Wake Up Frame. This property is only available for Broadcom NetXtreme adapters.
Both (default). Selects both Magic Packet and Wake Up Frame as wake-up frames.
Magic Packet. Selects Magic Packet as the wake-up frame.
None. Selects no wake-up frame.
Wake Up Frame. Selects Wake Up Frame as the wake-up frame and allows the network adapter to wake the system when
an event such as a ping or an Address Resolution Protocol (ARP) request is received. This option works in conjunction with the
operating system power mode saving and does not work if the Power Save setting does not enable WOL.
WOL Speed
Sets the speed at which the network adapter connects to the network while the network adapter is in Wake on LAN mode. By default,
the WOL Speed property is set to Auto. This property is only available for Broadcom NetXtreme adapters.
10 Mb. Sets the speed to 10 Mbit/s. This is the network speed when the system is in a standby mode awaiting a wake-up
frame.
100 Mb. Sets the speed to 100 Mbit/s.
Auto (default). Sets the speed for optimum network connection.
NOTES:
Wake on LAN is supported at 10 Mbit/s or 100 Mbit/s.
Wake on LAN is supported on only one adapter in the system at a time.
Priority & VLAN
Allows enabling both the prioritization of network traffic and VLAN tagging. VLAN tagging only occurs when the VLAN ID setting is
configured with a value other than 0 (zero).
Priority & VLAN Enabled (default). Allows for packet prioritization and VLAN tagging.
Priority & VLAN Disabled. Prevents packet prioritization and VLAN tagging.
Priority Enabled. Allows packet prioritization only.
VLAN Enabled. Allows VLAN tagging only.
NOTE: If an intermediate driver is managing the network adapter for VLAN tagging, the Priority & VLAN Disabled and
Priority Enabled settings should not be used. Use the Priority & VLAN Enabled setting and change the VLAN ID to 0
(zero).
VLAN ID
Enables VLAN tagging and configures the VLAN ID when Priority & VLAN Enabled is selected as the Priority & VLAN setting. The
range for the VLAN ID is 1 to 4094 and must match the VLAN tag value on the connected switch. A value of 0 (default) in this field
disables VLAN tagging.
Risk Assessment of VLAN Tagging through the NDIS Miniport Driver
Broadcom's NDIS 6.0 miniport driver provides the means to allow a system containing a Broadcom adapter to connect to a tagged
VLAN. On Windows XP systems, this support was only provided through the use of an intermediate driver (e.g., Broadcom
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Advanced Server Program - BASP). Unlike BASP, however, the NDIS 6 driver's support for VLAN participation is only for a single
VLAN ID.
Also unlike BASP, the NDIS 6.0 driver only provides VLAN tagging of the outbound packet, but does not provide filtering of
incoming packets based on VLAN ID membership. This is the default behavior of all miniport drivers. While the lack of filtering
packets based on VLAN membership may present a security issue, the following provides a risk assessment based on this driver
limitation for an IPv4 network:
A properly configured network that has multiple VLANs should maintain separate IP segments for each VLAN. This is necessary
since outbound traffic relies on the routing table to identify which adapter (virtual or physical) to pass traffic through and does
not determine which adapter based on VLAN membership.
Since support for VLAN tagging on Broadcom's NDIS 6.0 driver is limited to transmit (Tx) traffic only, there is a risk of inbound
traffic (Rx) from a different VLAN being passed up to the operating system. However, based on the premise of a properly
configured network above, the IP segmentation and/or the switch VLAN configuration may provide additional filtration to limit
the risk.
In a back-to-back connection scenario, two computers on the same IP segment may be able to communicate regardless of
their VLAN configuration since no filtration of VLAN membership is occurring. However, this scenario assumes that the security
may already be breached since this connection type is not typical in a VLAN environment.
If the risk above is not desirable and filtering of VLAN ID membership is required, then support through an intermediate driver
would be necessary.
iSCSI Crash Dump
Crash dump is used to collect information on adapters that were booted remotely using iSCSI. To enable crash dump, set to Enable
and reboot the system. If you perform an upgrade of the device drivers, re-enable iSCSI Crash Dump. If iSCSI Boot is configured
to boot in the HBA path, then this parameter cannot be changed.
Number of Receive Descriptors
The number of receive descriptors with options of Default, Minimum, and Maximum. Receive descriptors are data segments that
allow the network adapter to allocate receive packets to memory.
Number of Transmit Descriptors
The number of transmit descriptors with value options from 100 to 600 with 200 being the default value. Transmit descriptors are
data segments that allow the network adapter to monitor transmit packets in the system memory.
RSS Queues
Allows configuring RSS queues from 1 to 4. Options are RSS 1 Queue, RSS 2 Queue, and RSS 4 Queue.
Interrupt Moderation
Enables interrupt moderation, which limits the rate of interrupt to the CPU during packet transmission and packet reception. The
disabled option allows one interrupt for every packet transmission and packet reception. Enable is the default option.
Number of RSS Queues
Allows configuring RSS queues. For 1 Gbps network adapters, the RSS queue options are Auto (default), 2, 4, and 8. For 10 Gbps
network adapters, the RSS queue options are Auto (default), 2, 4, 8, and 16.
Receive Buffers
The number of receive buffers. Receive buffers are data segments that allow the network adapter to allocate receive packets to
memory. For 1 Gbps adapters, the range of valid receive buffers is 50 to 5000 in increments of 1 with 750 receive buffers as the
default value.
Receive Buffers (0=Auto)
The number of receive buffers. Receive buffers are data segments that allow the network adapter to allocate receive packets to
memory. For 10 Gbps adapters, the range of valid receive buffers is 0 to 3000 in increments of 50 with 0 receive buffers as the
default value.
Transmit Buffers (0=Auto)
The number of transmit buffers. Transmit buffers are data segments that allow the network adapter to monitor transmit packets in
the system memory. The range of valid transmit buffers is 0 to 5000 in increments of 1 with 250 transmit buffers as the default
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
value.
TCP Connection Offload (IPv4)
Enables and disables TOE offload when using the IPv4 protocol. The default is Enabled.
TCP Connection Offload (IPv6)
Enables and disables TOE offload when using the IPv6 protocol. The default is Enabled.
Pause on Exhausted Host Ring
For BCM57711 and BCM57712 network adapters, there are two possible scenarios that can trigger pause frames to be generated: a
host ring buffer is exhausted or the on-chip buffers are depleted. With RSS enabled inside the system, it is possible to achieve better
Ethernet throughput if no pause frames are being generated in a case where a host ring buffer (of multiple RSS rings) is exhausted.
The default is Disabled.
Quality of Service
Enables Quality of Service (QoS) to provide different priorities to different applications.
Recv Segment Coalescing (IPv4)
Enable Receive Segment Coalescing (IPv4). Receive Segment Coalescing is an offload technology that reduces CPU utilization for
network processing on the receive side by offloading tasks from the CPU to a network adapter.
Recv Segment Coalescing (IPv6)
Enable Receive Segment Coalescing (IPv6). Receive Segment Coalescing is an offload technology that reduces CPU utilization for
network processing on the receive side by offloading tasks from the CPU to a network adapter.
SR-IOV
Enables Single Root I/O Virtualization (SR-IOV).
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
This property is the same as Flow Control, and is only available when the blade server is attached to a switch module.
Chassis SW Speed & Duplex
This property is the same as Speed & Duplex (SerDes), and is only available when the blade server is attached to a switch module.
Viewing Statistics
The information provided on the Statistics tab allows you to view traffic statistics for both Broadcom network adapters and network
adapters made by others. Statistical information and coverage are more comprehensive for Broadcom adapters.
To view Statistics information for any installed network adapter, click the name of the adapter listed in the Explorer View pane, then
click the Statistics tab.
If any of the sections described below is not visible, then from the Context View tab on the right side of the window, select
Statistics and then select the name of the missing section.
Click Refresh to get the most recent values for each statistic. Click Reset to change all values to zero for the current BACS session.
NOTES:
Team statistics are not compiled for a Broadcom network adapter if it is disabled.
Some statistics may not be available for all Broadcom network adapters.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
General Statistics
General Statistics show the transmitted and received statistics to and from the adapter.
Frames Tx OK
A count of the frames that were successfully transmitted. This counter is incremented when the transmit status is reported as
Transmit OK.
Frames Rx OK
A count of the frames that were successfully received. This does not include frames received with frame-too-long, frame check
sequence (FCS), length, or alignment errors, nor frames lost due to internal MAC sublayer errors. This counter is incremented when
the receive status is reported as Receive OK.
Directed Frames Tx
A count of directed data frames that were successfully transmitted.
Multicast Frames Tx
A count of frames that were successfully transmitted (as indicated by the status value Transmit OK) to a group destination address
other than a broadcast address.
Broadcast Frames Tx
A count of frames that were successfully transmitted (as indicated by the transmit status Transmit OK) to the broadcast address.
Frames transmitted to multicast addresses are not broadcast frames and are excluded.
Directed Frames Rx
A count of directed data frames that were successfully received.
Multicast Frames Rx
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
A count of frames that were successfully received and are directed to an active nonbroadcast group address. This does not include
frames received with frame-too-long, FCS, length, or alignment errors, nor frames lost because of internal MAC sublayer errors. This
counter is incremented as indicated by the Receive OK status.
Broadcast Frames Rx
A count of frames that were successfully received and are directed to a broadcast group address. This count does not include frames
received with frame-too-long, FCS, length, or alignment errors, nor frames lost because of internal MAC sublayer errors. This counter
is incremented as indicated by the Receive OK status.
Frames Rx with CRC Error
The number of frames received with CRC errors.
Initiator Login Statistics
iSCSI login enables a connection for iSCSI use between the initiator and the target and is used to authenticate parties, negotiate the
session's parameters, open security association protocol, and mark the connection as belonging to an iSCSI session.
Login Accept Responses
The number of login requests accepted by the target.
Login other failed Responses
The number of login requests that were not accepted by the target.
Login Redirect Responses
The number of responses that required further action by the initiator.
Login Authentication Failed Responses
The number of login requests that failed due to party authentication failure.
Login target authentication failure
The number of instances where the login could not authenticate the target.
Login target negotiation failure
The number of instances where the login could not negotiate the sessions parameters.
Normal logout command PDU
The number of normal logout commands issued by the initiator to remove a connection from a session or to close a session.
Other logout command PDU
The number of logout commands issued by the initiator for reasons other than to remove a connection from a session or to close a
session.
Local Initiator login failures
The number of login failures likely caused by the initiator.
Initiator Instance Statistics
The statistics in this area pertain to all sessions.
Session digest errors
The number of sessions with errors due to an invalid payload or header.
Session connection timeout error
The number of sessions that were terminated due to any of the many timeout errors.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Session format error
The number of sessions with errors due to inconsistent fields, reserved fields not 0, non-existent LUN, etc.
Sessions failed
The number of failed sessions.
Custom
Custom statistics.
Total Offload iSCSI Connections
The total number of offloaded iSCSI connections.
Session Statistics
The statistics in this area only pertain to the named session.
Session Name
The name used for the session between the initiator and the target.
Session Id
The identifier used for the session between the initiator and the target.
Bytes sent
The number of bytes sent for the named session.
Bytes received
The number of bytes received for the named session.
PDU sent
The number of iSCSI PDUs sent for the named session.
PDU received
The number of iSCSI PDUs received for the named session.
Digest errors
The number of errors due to an invalid payload or header for the named session.
Connection Timeout errors
The number of connection timeout errors for the named session.
Format errors
The number of errors due to inconsistent fields, reserved fields not 0, non-existent LUN, etc. for the named session.
IEEE 802.3 Statistics
Frames Rx with Alignment Error
A count of the frames that were not an integral number of octets in length and do not pass the FCS check. This counter is
incremented when the receive status is reported as Alignment Error.
Frames Tx with one Collision
A count of the frames that were involved in a single collision and were subsequently transmitted successfully. This counter is
incremented when the result of a transmission is reported as Transmit OK, and the attempt value is 2.
Frames Tx with more than one Collision
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
A count of the frames that were involved in more than one collision and were subsequently transmitted successfully. This counter is
incremented when the transmit status is reported as Transmit OK, and the value of the attempts variable is greater than 2 and less
than or equal to the attempt limit.
Frames Tx after Deferral
A count of the frames that were delayed being transmitted on the first attempt because the medium was busy. The frames involved
in any collision are not counted.
Custom Statistics
NOTE: Custom statistics are available only for an enabled Broadcom network adapter.
Out of Recv. Buffer
The number of times the adapter ran out of Receive Buffer Descriptors. This information is only available for Broadcom NetXtremeII
adapters.
Frames size less than 64-byte with bad FCS
The number of frames with a size less than 64 bytes with bad FCS.
MAC Rx w/ Pause Command and Length = 0
MAC control frames with the pause command and a length equal to 0.
MAC Rx w/ Pause Command and Length greater than 0
MAC control frames with the pause command and a length greater than 0.
MAC Rx w/ no Pause Command
MAC control frames with no pause command.
MAC Sent X-on
MAC Transmit with X-on was on.
MAC Sent X-off
MAC Transmit with X-on was off.
Large Send Offload Transmit Requests
The number of times the adapter was requested to transmit a packet performing TCP segmentation.
Total Offload TCP Connections
The total number of offloaded TCP connections.
SR-IOV Switch Statistics
This area shows the statistics for SR-IOV switches.
Num of Active VFs
This shows the number of active Virtual Functions (VF).
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
TCP Offload Engine (TOE) for accelerating TCP over 1 GbE and 10 GbE.
Internet Small Computer Systems Interface (iSCSI) offload for accelerating network storage access featuring centralized boot
functionality (iSCSI boot).
You can also view the number of unlicensed resources and unallocated resources.
TOE and iSCSI can only be configured on certain adapters and require a license key. License keys are installed on the motherboard
for use with LOMs and are preprogrammed in add-in NICs. TOE is a technology best suited for long-term TCP connections. TOE used
in conjunction with applications that primarily deal with short-term connections may see suboptimal performance when compared to
not using TOE. A short-term connection is a TCP connection that is established for only a few seconds at most, often less than one
second. The overhead of establishing and tearing down the TCP connection with short-term connections may diminish the benefit
that TOE provides. An example of an application that uses short-term connections is a web server that is not behind a load balancer.
TCP connections for HTTP sessions are very short-lived and last only long enough to serve up a web page, after which the TCP
connection is terminated. On the other hand, the same web server behind a load balancer benefits greatly from TOE since the TCP
connections are long-lived between the web server and the load balancer. In contrast, a backup application is very likely to benefit
from TOE since backup applications establish long-term connections between the server and the client.
To view resource reservations
1. Click the name of the Broadcom NetXtreme II system device in the Explorer View pane.
2. From the Resource Reservations section, select the property you want to set.
3. Click Apply to confirm the changes to all properties. Click Reset to return the properties to their original values.
To configure resource reservations
1. Select Reserved Resources for the Offload Configuration setting. Reservations allocate the minimum number of connections
per technology. See Table1 for the maximum number of connections available for various configurations.
2. Click First Come First Served to allow connections on a first-come-first-served basis to any enabled offload technology (TOE
or iSCSI) as long as resources are available. Otherwise, click Reserved Resources to set the maximum number of
connections for each offload technology. This option is not available for 10 Gbps network adapters.
3. If allocating TOE connections, select the TCP Offload Engine (TOE) check box and slide the bar or enter a number in the
value box. This is the maximum number of connections available for TOE offload.
4. If allocating iSCSI connections, select the iSCSI check box and slide the bar or enter a number in the value box. This is the
maximum number of connections available for iSCSI offload.
5. After the configurations are complete, click Apply to save the settings or click Reset to revert back to the previous settings.
A chart of the resource allocations appears below the resource configurations.
6. If prompted, reboot the system.
Table 1: Maximum Number of Offload Connections
Offload
TOE only
Selected
1024
TOE only
Selected
864
iSCSI only
Selected
28
Viewing Licenses
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
NOTES:
The Licenses section of the Configurations tab is only available for Broadcom NetXtreme II adapters and VBD drivers.
Not all offload technologies are available with all adapters.
The Licenses section shows the number of connections available for TOE and iSCSI offload technologies.
To view licenses
1. Click the name of the Broadcom NetXtreme II system device in the Explorer View pane.
Configuring Teaming
NOTE: BACS does not support teaming on Linux systems. Linux provides a similar built-in functionality called Channel Bonding.
Refer to the Linux OS documentation for more information.
The teaming function allows you to group any available network adapters together to function as a team. Teaming is a method of
creating a virtual NIC (a group of multiple adapters that functions as a single adapter). The benefit of this approach is that it enables
load balancing and failover. Teaming is done through the Broadcom Advanced Server Program (BASP) software. For a comprehensive
description of the technology and implementation considerations of the teaming software, refer to the "Broadcom Gigabit Ethernet
Teaming Services" section of your Broadcom network adapter user guide.
Teaming can be accomplished by either of the following methods:
Using the Broadcom Teaming Wizard
Using Expert Mode
NOTES:
For further information regarding teaming protocols, see "Teaming" in your Broadcom network adapter user guide.
If you do not enable LiveLink when configuring teams, disabling Spanning Tree Protocol (STP) at the switch is recommended.
This minimizes the downtime due to spanning tree loop determination when failing over. LiveLink mitigates such issues.
BASP is available only if a system has one or more Broadcom network adapters installed.
The TCP Offload Engine (TOE), Large Send Offload (LSO), and Checksum Offload properties are enabled for a team only when
all of the members support and are configured for the feature.
To physically remove a teamed NIC from a system, you must first delete the NIC from the team. Not doing this before
shutting down the system could result in breaking the team on a subsequent reboot, which may result in unexpected team
behavior.
If an adapter is included as a member of a team and you change any advanced property, then you must rebuild the team to
ensure that the team's advanced properties are properly set.
If an adapter with a TOE key is included as a member of a team and you remove the TOE key, then you must rebuild the
team to ensure that the team does not appear as a TOE-enabled team.
You must have administrator privileges to create or modify a team.
The load balance algorithm in a team environment in which members are connected at different speeds favors members
connected with a Gigabit Ethernet link over members connected at lower speed links (100 Mbps or 10 Mbps) until a threshold
is met. This is normal behavior.
In a GEC or 802.3ad type team, disabling the device driver(s) of a team member that has IPMI enabled may prevent a
successful failover since IPMI maintains link with the attached switch.
Do not assign a Locally Administered Address on any physical adapter that is a member of a team.
Team Types
You can create four types of load balance teams:
Smart Load Balance and Failover
Link Aggregation (802.3ad) (TOE is not applicable)
Generic Trunking (FEC/GEC)/802.3ad-Draft Static (TOE is not applicable)
SLB (Auto-Fallback Disable) The Auto-Fallback Disable feature is configured for Smart Load Balance and Failover type teams
in the Teaming Wizard.
NOTE: NetXtreme II network adapters with iSCSI enabled is supported only in an SLB team type.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Smart Load Balance and Failover
In this type of team, a standby member handles the traffic if all of the load balance members fail (a failover event). All load balance
members have to fail before the standby member takes over. When one or more of the load balance members is restored (fallback),
the restored team member(s) resumes the handling of the traffic. The LiveLink feature is supported for this type of team.
Link Aggregation (802.3ad)
In this type of team, you can dynamically configure the network adapters that have been selected to participate in a given team. If
the link partner is not correctly configured for IEEE 802.3ad link configuration, errors are detected and noted. All adapters in the
team are configured to receive packets for the same MAC address. The outbound load balancing scheme is determined by the BASP
driver. The link partner of the team determines the load balancing scheme for inbound packets. In this mode, at least one of the link
partners must be in active mode.
NOTE: TOE is not applicable for Link Aggregation team type. NetXtreme II network adapters with iSCSI enabled is not supported
for Link Aggregation team type.
Generic Trunking (FEC/GEC)/802.3ad-Draft Static
This type of team is very similar to the link aggregation type, in that all adapters in the team must be configured to receive packets
for the same MAC address. This mode does not provide link aggregation control protocol (LACP) or marker protocol support. This
mode supports a variety of environments where the link partners are statically configured to support a proprietary trunking
mechanism. Trunking supports load balancing and failover for both outbound and inbound traffic.
NOTE: TOE is not applicable for Generic Trunking (FEC/GEC)/802.3ad-Draft Static team type. NetXtreme II network adapters with
iSCSI enabled is not supported for Generic Trunking (FEC/GEC)/802.3ad-Draft Static team type.
SLB (Auto-Fallback Disable)
This team is identical to Smart Load Balance and Failover, with the following exception: when the standby member is active, if a
primary member comes back online, the team continues using the standby member rather than switching back to the primary
member. This type of team is supported only for situations in which the network cable is disconnected and reconnected to the
network adapter. It is not supported for situations in which the adapter is removed/installed through Device Manager or Hot-Plug
PCI. If any primary adapter assigned to a team is disabled, the team functions as a Smart Load Balancing and Failover type of team
in which auto-fallback occurs. The LiveLink feature is supported for this type of team.
LiveLink
LiveLink is a feature of BASP that is available for the Smart Load Balancing (SLB) and SLB (Auto-Fallback Disable) type of teaming.
The purpose of LiveLink is to detect link loss beyond the switch and to route traffic only through team members that have a live link.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
3. Type the team name and then click Next. If you want to review or change any of your settings, click Back. Click Cancel to
discard your settings and exit the wizard.
NOTE: The team name cannot exceed 39 characters, cannot begin with spaces, and cannot contain any of the following
characters: & \ / : * ? < > |
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
7. From the Available Adapters list, click the adapter you want to add to the team and then click Add. Remove team members
from the Team Members list by clicking the adapter and then clicking Remove. Click Next.
NOTE: There must be at least one Broadcom network adapter assigned to the team.
NOTE: A team created with a single adapter will not have load balancing or failover capabilities.
The TCP Offload Engine (TOE), Large Send Offload (LSO) and Checksum Offload (CO) columns indicate if the TOE, LSO,
Jumbo MTU, and/or the CO properties are supported for the adapter. The TOE, LSO, Jumbo MTU, and CO properties are
enabled for a team only when all of the members support and are configured for the feature. If this is the case, then the
team offload capabilities appear on the bottom of the screen.
NOTES:
8. If an adapter is included as a member of a team and you change any advanced property, then you must rebuild the team to
ensure that the team's advanced properties are properly set.
9. If an adapter with a TOE key is included as a member of a team and you remove the TOE key, then you must rebuild the
team to ensure that the team does not appear as a TOE-enabled team.
10. Adding a network adapter to a team where its driver is disabled may negatively affect the offloading capabilities of the team.
This may have an impact on the team's performance. Therefore, it is recommended that only driver-enabled network adapters
be added as members to a team.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
11. If you want to designate one of the adapters as a standby member (optional), select Use the following member as a
standby member, then choose the standby member from the list of adapters.
12. The Auto-Fallback Disable mode feature allows the team to continue using the standby member rather than switching back to
the primary member if the primary member comes back online. To enable this feature, select Enable Auto-Fallback Disable
mode. Click Next.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
13. If you want to configure LiveLink, select Yes, otherwise select No, then click Next.
14. Select the probe interval (the number of seconds between each retransmission of a link packet to the probe target) and the
maximum number of probe retries (the number of consecutively missed responses from a probe target before a failover is
triggered).
15. Set the Probe VLAN ID to allow for connectivity with probe targets residing on a tagged VLAN. The number set must match
the VLAN ID of the probe targets as well as the port(s) on the switch to which the team is connected.
NOTE: Each LiveLink enabled team can only communicate with Probe Targets on a single VLAN. Also, VLAN ID0 is
equivalent to an untagged network. If the Probe VLAN ID is set to a value other than 0, then a VLAN must be created with an
identical VLAN tag value (see Step 18.).
16. Click the probe target at the top of the list, click Edit Target IP Address, type the target IP address in the IP Address box
for one or all probe targets, and then click OK. Click Next.
NOTE: Only the first probe target is required. You can specify up to three additional probe targets to serve as backups by
assigning IP addresses to the other probe targets.
17. Select a listed team member, click Edit Member IP Address, and then type the member IP address in the IP Address box.
Repeat for all listed team members and then click OK. Click Next.
NOTE: All of the member IP addresses must be in the same subnet as the subnet of the probe targets.
18. If you want to create a VLAN on the team, select Add VLAN, or if you want to change the settings of an existing VLAN, select
Edit VLAN, then click Next. If you do not want to create or edit a VLAN, select Skip Manage VLAN, then click Next, and
continue with the wizard from the Finish screen (see Step 20. of this procedure).
VLANs enable you to add multiple virtual adapters that are on different subnets. The benefit of this is that your system can
have one network adapter that can belong to multiple subnets.
NOTE: VLANs can only be created when all team members are Broadcom adapters.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
20. To tag the VLAN, select Tagged and then click Next. Otherwise, click Untagged, click Next, and continue with the wizard to
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
add additional VLANs (see Step 19. of this procedure).
21. Type the VLAN tag value and then click Next. The value must be between 1 and 4094.
22. Select Yes to add or manage another VLAN and then click Next. Repeat until you do not want to add or manage any
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
additional VLANs.
NOTE: You can define up to 64 VLANs per team (63 VLANs that are tagged and 1 VLAN that is not tagged). Adding several
VLANS may slow down the reaction time of the Windows interface due to memory and processor time usage for each VLAN.
The degree to which Windows performance may suffer depends on system configuration.
23. To apply and commit the changes to the team, select Commit changes to system and Exit the wizard. To apply your
changes but continue using the wizard, select Save changes and continue to manage more teams. Click Finish.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
NOTE: At any point in the Broadcom Teaming Wizard procedure, click Preview to get a visual representation of what the
team will look like before committing any changes.
24. Click the team name in the Team Management pane to view the team's properties in the Information tab, transfer and
receive data in the Statistics tab, and team customization options in the Configurations tab.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
Creating a Team
NOTE: Enabling Dynamic Host Configuration Protocol (DHCP) is not recommended for members of an SLB type of team.
1. From the Teams menu, select Create a Team, or right-click one of the devices in the "Unassigned Adapters" section and
select Create a Team. This option is not available if there are no devices listed in the "Unassigned Adapters" sections, which
means all adapters are already assigned to teams.
2. Click Expert Mode.
NOTE: If you want to always use Expert Mode to create a team, click Default to Expert Mode on next start.
3. Click the Create Team tab.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
NOTE: The Create Team tab appears only if there are teamable adapters available.
4. Click the Team Name field to enter a team name.
5. Click the Team Type field to select a team type.
6. Click Hyper-V Mode if you want to enable Windows virtualization services. See "Microsoft Virtualization with Hyper-V" in the
"Troubleshooting" topic in the NetXtreme II Network Adapter User Guide for more information about this feature.
7. Assign any available adapter or adapters to the team by moving the adapter from the Available Adapters list to the Load
Balance Members list. There must be at least one adapter in the Load Balance Members list.
8. You can assign any other available adapter to be a standby member by selecting it from the Standby Member list.
NOTE: There must be at least one Broadcom network adapter assigned to the team.
NOTE: A team created with a single adapter will not have load balancing or failover capabilities.
The TCP Offload Engine (TOE), Large Send Offload (LSO), and Checksum Offload (CO) columns indicate if the TOE, LSO,
and/or the CO properties are supported for the adapter. The TOE, LSO, and CO properties are enabled for a team only
when all of the members support and are configured for the feature. If this is the case, then the team offload capabilities
appear on the bottom of the screen.
NOTES:
9. If an adapter is included as a member of a team and you change any advanced property, then you must rebuild the team to
ensure that the team's advanced properties are properly set.
10. If an adapter with a TOE key is included as a member of a team and you remove the TOE key, then you must rebuild the
team to ensure that the team does not appear as a TOE-enabled team.
11. Adding a network adapter to a team where its driver is disabled may negatively affect the offloading capabilities of the team.
This may have an impact on the team's performance. Therefore, it is recommended that only driver-enabled network adapters
be added as members to a team.
12. Type the value for Team MTU.
13. Click Create to save the team information.
14. Repeat steps 4. through 10. to define additional teams. As teams are defined, they can be selected from the team list, but
they have not yet been created. Click the Preview tab to view the team structure before applying the changes.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
15. Click Apply/Exit to create all the teams you have defined and exit the Manage Teams window.
16. Click Yes when the message is displayed indicating that the network connection will be temporarily interrupted.
NOTES:
17. The team name cannot exceed 39 characters, cannot begin with spaces, and cannot contain any of the following characters: &
\/:*?<>|
18. Team names must be unique. If you attempt to use a team name more than once, an error message is displayed indicating
that the name already exists.
19. The maximum number of team members is 8.
20. When team configuration has been correctly performed, a virtual team adapter driver is created for each configured team.
21. If you disable a virtual team and later want to reenable it, you must first disable and reenable all team members before you
reenable the virtual team.
22. When you create Generic Trunking and Link Aggregation teams, you cannot designate a standby member. Standby members
work only with Smart Load Balancing and Failover and SLB (Auto-Fallback Disable) types of teams.
23. For an SLB (Auto-Fallback Disable) team, to restore traffic to the load balance members from the standby member, click the
Fallback button on the Team Properties tab.
24. When configuring an SLB team, although connecting team members to a hub is supported for testing, it is recommended to
connect team members to a switch.
25. Not all network adapters made by others are supported or fully certified for teaming.
26. Configure the team IP address.
a. From Control Panel, double-click Network Connections.
b. Right-click the name of the team to be configured, and then click Properties.
c. On the General tab, click Internet Protocol (TCP/IP), and then click Properties.
d. Configure the IP address and any other necessary TCP/IP configuration for the team, and then click OK when finished.
Modifying a Team
After you have created a team, you can modify the team in the following ways:
Change the type of team
Change the members assigned to the team
Add a VLAN
Modify a VLAN (using Expert Mode)
Remove a team or a VLAN (using Expert Mode)
To modify a team
1. From the Team menu, click Edit Team, or right-click one of the teams in the list and select Edit Team. This option is only
available if a team has already been created and is listed in the Team Management pane.
2. The wizard Welcome screen appears. Click Next to continue modifying a team using the wizard or click Expert Mode to work
in Expert Mode.
NOTE: The Edit Team tab in Expert Mode appears only if there are teams configured on the system.
3. Click the Edit Team tab.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
4. Make the desired changes, and then click Update. The changes have not yet been applied; click the Preview tab to view the
updated team structure before applying the changes.
5. Click Apply/Exit to apply the updates and exit the Manage Teams window.
6. Click Yes when the message is displayed indicating that the network connection will be temporarily interrupted.
Adding a VLAN
You can add virtual LANs (VLANs) to a team. This enables you to add multiple virtual adapters that are on different subnets. The
benefit of this is that your system can have one network adapter that can belong to multiple subnets. With a VLAN, you can couple
the functionality of load balancing for the load balance members, and you can employ a failover adapter.
You can define up to 64 VLANs per team (63 VLANs that are tagged and 1 VLAN that is not tagged). VLANs can only be created
when all teams members are Broadcom adapters. If you try to create a VLAN with a non-Broadcom adapter, an error message is
displayed.
To configure a team with a VLAN
1. From the Teams menu, select Add VLAN.
2. The Welcome screen appears.
3. Click Expert Mode.
4. On the Create Team tab of the Manage Teams window, click Manage VLAN(s).
5. Type the VLAN name, then select the type and ID.
6. Click Create to save the VLAN information. As VLANs are defined, they can be selected from the Team Name list, but they
have not yet been created.
7. Continue this process until all VLANs are defined, then click OK to create them.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
8. Click Yes when the message is displayed indicating that the network connection will be temporarily interrupted.
NOTE: To maintain optimum adapter performance, your system should have 64 MB of system memory for each of the
eight VLANs created per adapter.
Deleting a VLAN
The procedure below applies when you are in Expert Mode.
To delete a VLAN
1. Select the VLAN to delete.
2. From the Teams menu, select Remove VLAN.
3. Click Apply.
4. Click Yes when the message is displayed indicating that the network connection will be temporarily interrupted.
NOTE: If you delete a team, any VLANs configured for that team are also deleted.
Configuring LiveLink for a Smart Load Balancing and Failover and SLB (Auto-Fallback Disable) Team
LiveLink is a feature of BASP that is available for the Smart Load Balancing (SLB) and SLB (Auto-Fallback Disable) type of teaming.
The purpose of LiveLink is to detect link loss beyond the switch and to route traffic only through team members that have a live link.
Read the following notes before you attempt to configure LiveLink.
NOTES:
Before you begin configuring LiveLink, review the description of LiveLink. Also verify that each probe target you plan to
specify is available and working. If the IP address of the probe target changes for any reason, LiveLink must be reconfigured.
If the MAC address of the probe target changes for any reason, you must restart the team (see the "Troubleshooting" topic in
the NetXtreme II Network Adapter User Guide).
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
A probe target must be on the same subnet as the team, have a valid (not a broadcast, multicast, or unchaste), staticallyassigned IP address, and be highly available (always on).
To ensure network connectivity to the probe target, ping the probe target from the team.
You can specify up to four probe targets.
The IP address assigned to either a probe target or team member cannot have a zero as the first or last octet.
To configure LiveLink
1. From the Teams menu, select Edit Team.
2. Click Expert Mode (to configure LiveLink using the Teaming Wizard, see Using the Broadcom Teaming Wizard).
3. In the Manage Teams window, click the Edit Team tab.
4. Select Enable LiveLink. The LiveLink Configuration options appear below.
5. It is recommended to accept the default values for Probe interval (the number of seconds between each retransmission of a
link packet to the probe target) and Probe maximum retries (the number of consecutively missed responses from a probe
target before a failover is triggered). To specify different values, click the desired probe interval in the Probe interval
(seconds) list and click the desired maximum number of probe retries in the Probe maximum retries list.
6. Set the Probe VLAN ID to correspond with the VLAN where the probe target(s) resides. This will apply the appropriate VLAN
tag to the link packet based on the shared configuration of the attached switch port(s).
NOTE: Each LiveLink enabled team can only communicate with Probe Targets on a single VLAN. Also, VLAN ID0 is
equivalent to an untagged network.
7. Select Probe Target 1 and type the target IP address for one or all probe targets.
NOTE: Only the first probe target is required. You can specify up to 3 additional probe targets to serve as backups by
assigning IP addresses to the other probe targets.
8. Select one of the listed team members and type the member IP address.
NOTE: All of the member IP addresses must be in the same subnet as the probe targets.
9. Click Update. Repeat these steps for each of the other listed team members.
10. Click Apply/Exit.
Using Broadcom Advanced Control Suite 4: Broadcom NetXtreme BCM57XX User Guide
listed in the Team Management pane, then click the Statistics tab.
Click Refresh to get the most recent values for each statistic. Click Reset to change all values to zero.
Troubleshooting BACS
Problem: When attempting to open BACS on a Linux System, the following error message displays:
"Another instance of the BACS client appears to be running on this system. Only one instance of the BACS client can be running
at a time. If you are sure that no other BACS client is running, then a previous instance may have quit unexpectedly."
Solution: This message displays if you try to run a second instance of BACS. If you receive this message but are certain that no
instance of BACS is currently running, a previous instance of BACS may have quit unexpectedly. To clear that instance, remove the
file "/dev/shm/sem.Global-BACS-{C50398EE-84A7-4bc3-9F6E-25A69603B9C0}."
Back to Contents Page
Connector
Media
Maximum Distance
10BASE-T
RJ-45
100/1000BASE-T 1
RJ-45
Category 5 2 UTP
1 1000BASE-T signaling requires 4 twisted pairs of Category 5 balanced cabling, as specified in ISO/IEC 11801:1995 and
ANSI/EIA/TIA-568-A (1995) and tested for additional performance using testing procedures defined in TIA/EIA TSB95.
2 Category 5 is the minimum requirement. Category 5e and Category 6 are fully supported.
Performance Specifications
Table 2. Performance Specifications
Feature
Specification
66 MHz maximum
PCI-X Clock
133 MHz
PCI/PCI-X Data/Address
PCI Modes
Master/slave
x1 link width
Physical Characteristics
Table 3. Physical Characteristics
NIC Type
Length
Width
PCI
Power Requirements
Table 4. Power Requirements
Item
Value
+5V 5%
Power Consumption
10W
2A @ +5VDC
BCM5703
Operating Voltage
Power Consumption
4W
1.2A@ +3.3V for BCM95703A30
BCM5721
Operating voltage
+3.3V 10%
Power Consumption
2.84W
860 mA @ +3.3VDC
BCM5722
Operating voltage
+3.3V 10%
Power Consumption
1.41W
427 mA @ +3.3VDC
Environmental Specifications
Table 5. Environmental Specifications
Condition
Operating Specification
Storage Specification
Temperature
Relative Humidity
5% to 85% (non-condensing)
5% to 95% (non-condensing)
40C, 16-hour dwells at extremes 10C/hour
Altitude
Up to 10,000 ft.
Up to 35,000 ft.
Shock
CE Notice
Note that this device has been approved for non-business purposes and may be used in any environment, including residential
areas.
BSMI
Introduction
Broadcom NetXtreme User Diagnostics is an MS-DOS based application that runs a series of diagnostic tests (see
Table1:"MS-DOS Command Prompt Mode Command Options") on the Broadcom NetXtreme Gigabit Ethernet adapters in
your system. Broadcom NetXtreme User Diagnostics also allows you to update device firmware and to view and change
settings for available adapter properties. Broadcom NetXtreme User Diagnostics can be run in either of the following modes:
MS-DOS Command Prompt mode
Broadcom Command Line Interface (CLI) mode
In either mode, you can view the version of the adapter software and specify which adapter to test and which tests to
perform. The MS-DOS Command Prompt mode is useful for viewing and changing the settings for available properties,
updating and loading device firmware, viewing the version of and printing the error log (if any) to a file. The Broadcom CLI
mode is useful for enabling/disabling available properties and enabling/disabling/selecting and setting the speed and duplex
mode of available protocols.
To run Broadcom NetXtreme User Diagnostics, create an MS-DOS 6.22 bootable disk containing the B57udiag.exe file. Next,
start the system with the boot disk in drive A. See either Running in MS-DOS Command Prompt Mode or Running in
Broadcom Command Line Interface Mode for further instructions.
System Requirements
Operating System: MS-DOS 6.22
Software: B57udiag.exe
Description
b57udiag
Performs all of the tests on all of the Broadcom NetXtreme Gigabit Ethernet adapters in your system.
b57udiag -c
<num>
Specifies the adapter to test, or the adapter on which to update the firmware or to view or change the
settings for available properties.
b57udiag -cmd
b57udiag -w
<value>
b57udiag -mbap
<value>
b57udiag -mbas
<value>
b57udiag -firm
<file>
Updates the EEPROM of the selected adapter based on the match between the existing image file name
and the new image <file name>.
Examples:
b57udiag -firm ee5751c3.40a
b57udiag -firm ee5721c3.40a
b57udiag -firmall
<file>
Updates the EEPROM of all of the adapters based on the <file name> image match.
b57udiag -ver
b57udiag -pxe
<file>
b57udiag -elog
<file>
b57udiag -help
dir
setwol
setpxe
setman
setmba
setmba
setmba
setmba
setmba
setmba
setmba
-e
-s
-s
-s
-s
-s
-s
2
0
1
2
3
4
6
=
=
=
=
=
=
=
nictest
exit
Changes from the Broadcom CLI mode to the MS-DOS command prompt mode
device
Selects
device
device
device
version
help
reset
Resets
reset
reset
reset
cls
asfprg
Description
This test uses an indirect addressing method to write an increment of data to the MAC hash register
table and read back data for verification. The memory read/write is done 100 times while
incrementing test data.
A2
Control
Register
Each register specified in the configuration content defines the read-only bit and the read/write bits.
The test writes 0s and 1s to the test bits to ensure the read-only bits are not changed, and that
read/write bits are changed.
This test attempts to read the register configuration file (Ctrlreg.txt) for the register definitions. If
the file does not exist, a default register offset and mask bits are used.
OffsetRead-Only MaskRead/Write Mask
0x000004000x000000000x007FFF8C
0x000004040x038001070x00000000
A3
Interrupt
This test verifies the interrupt functionality. It enables an interrupt and waits 500 ms for the
interrupt to occur and reports an error if it cannot generate the interrupt.
A4
A5
PCI Cfg
Register
A1
This test verifies the access integrity of the PCI configuration registers.
B1
Scratch Pad
B2
BD SRAM
This test tests the Buffer Descriptor (BD) SRAM. This test performs in the same way as the Scratch
Pad Test described in B1.
B3
DMA SRAM
This test tests the direct memory access (DMA) SRAM by performing the Scratch Pad Test described
in test B1.
B4
MBUF SRAM
This test tests the memory access buffer (MBUF) SRAM by performing the Scratch Pad Test
described in test B1.
B5
MBUF SRAM
via DMA
This test uses 8 data test patterns. A 0x1000-sized data buffer is used for this test. Before each
pattern test, the buffer is initialized and filled with the test pattern. It then performs a 0x1000-sized
transmit DMA from the host buffer to the adapter MBUF memory.
The test verifies the data integrity in the adapter MBUF memory against the host memory and
repeats the DMA for the entire MBUF buffer. Then, the test performs a receive DMA from the
adapter to the host. The 0x1000-byte test buffer is cleared to 0 before each receive DMA. After the
test verifies the integrity of the data, the test is repeated for the entire MBUF SRAM range. The 8
test patterns are described below.
Test Pattern Description
16 00s and 16 FF'sFills the entire host DMA buffer with 16 bytes of 00s and then
16 bytes of FF's.
16 FF's and 16 00sFills the entire host DMA buffer with 16 bytes of FF's and then
16 bytes of 00s.
32 00s and 32 FF'sFills the entire host DMA buffer with 32 bytes of 00s and then
32 bytes of FF's.
32 FF's and 32 00sFills the entire host DMA buffer with 32 bytes of FF's and then
32 bytes of 00s.
00000000Fills the entire host DMA buffer with all 00s.
FFFFFFFFFills the entire host DMA buffer with all FF's.
AA55AA55Fills the entire host DMA buffer with data 0xAA55AA55.
55AA55AAFills the entire host DMA buffer with data 0x55AA55AA.
B7
CPU GPR
This test tests the CPU General Purpose registers and performs in the same way as the Scratch Pad
Test (B1) over 3 different voltages (1.1V, 1.2V, and 1.3V).
NVRAM
Incremental test data is used in the electrically erasable programmable read-only memory
(EEPROM) test. The test fills the test range with test data and reads the data back to verify the
content. Afterwards, the test fills the test range with 0s to clear the memory.
C2
CPU
This test opens the Cpu.bin file. If the file exists and content is good, the test loads code to the RX
CPU and TX CPU and verifies the CPU execution.
C3
DMA
This test tests both high-priority direct memory access (DMA) and low-priority DMA. The test moves
data from the host memory to the adapter SRAM and verifies the data. The test then moves data
back to the host memory to again verify the data.
MII
The medium independent interface (MII) test function is identical to that of the Control Register
Test (A2). Each register specified in the configuration contents defines the read-only and read/write
bits. The test writes 0s and 1s to the test bits to ensure that the read-only bit values are not
changed and that the read/write bits are changed.
The test attempts to read the register configuration file (Miireg.txt) for the register definitions. If the
file does not exist, the following table is used:
OffsetRead-Only MaskRead/Write Mask
0x000x00000x7180
0x020xFFFF0x0000
0x030xFFFF0x0000
0x040x00000xFFFF
0x050xEFFF0x0000
0x060x00010x0000
0x070x08000xB7FF
0x080xFFFF0x0000
0x090x00000xFF00
0x0A0x7C000x0000
0x100x00000xFFBF
0x110x7C000x0000
0x190x7C000x0000
0x1E 0x00000xFFFF
0x1F0x00000xFFFF
C4
C5
VPD
The VPD test first saves the contents of the vital product data (VPD) memory before performing the
test. The test then writes 1 of the 5 test data patterns (0xFF, 0xAA, 0x55, increment data, or
decrement data) into VPD memory. By default, an incremental data pattern is used. The test writes
and reads back the data for the entire test range, and then restores the original contents of the
VPD memory.
Reset Test. This test sets the reset bit and polls for self-clearing bits. This test verifies the reset
value of the registers.
Event Mapping Test. This test sets the SMB_ATTN bit. By changing ASF_ATTN_ LOC bits, the test
verifies the mapping bits in TX_CPU or RX_CPU event bits.
Counter Test
C6
ASF
Hardware
Clears WG_TO, HB_TO, PA_TO, PL_TO, RT_TO bits (by setting the bits) and ensures that the
bits clear.
Clears the timestamp counter. Writes a 1 to each of the PL, PA, HB, WG, RT counters. Sets
the TSC_EN bit.
Polls each PA_TO bit and counts up to 50. Checks if the PL_TO bit is set at the end of the
count to 50. Continues to count up to 200. Checks if all other TO bits are set and verifies if
the timestamp counter is incremented.
C7
Expansion
ROM
This test tests the ability to enable, disable, and access the expansion read-only memory (ROM) on
the adapter.
C8
CPU Fetch
This test tests the PCU instruction fetch logic 100 times.
MAC
Loopback
This test is an internal loopback data transmit/receive test. It initializes the medium access control
(MAC) into an internal loopback mode and transmits 100 packets. The data should be routed back
to the receive channel and received by the receive routine, which verifies the integrity of data. A
100-Mbit/s data rate is used for this test unless Gigabit Ethernet is enabled.
D2
PHY
Loopback
This test is same as the MAC loopback test (D1), except that the data is routed back via a physical
layer device (PHY). A 100-Mbit/s data rate is used for this test unless Gigabit Ethernet is enabled.
D5
MII
This test tests the autopolling and PHY interrupt capabilities. These are functions of the PHY.
Miscellaneous
D6
MSI
This test tests the message signal interrupt (MSI) capability of the adapter. Refer to PCI
Specification, version 2.3, for the MSI definition.
0 */
1 */
2 */
3 */
4 */
offset
5 */
offset
6 */
offset
7 */
offset
8 */
9 */
10 */
11 */
12 */
13 */
14 */
15 */
16 */
17 */
18 */
19 */
20 */
21 */
22 */
23 */
"PASS",
"Got 0x%08X @ 0x%08X. Expected 0x%08X",
"Cannot perform task while chip is running",
"Invalid NIC device",
"Read-only bit %s got changed after writing zero
0x%X",
"Read-only bit %s got changed after writing one
0x%X",
"Read/Write bit %s did not get cleared after writing zero
0x%X",
"Read/Write bit %s did not get set after writing one
0x%X",
"BIST failed",
"Could not generate interrupt",
"Aborted by user",
"TX DMA:Got 0x%08X @ 0x%08X. Expected 0x%08X",
"Rx DMA:Got 0x%08X @ 0x%08X. Expected 0x%08X",
"TX DMA failed",
"Rx DMA failed",
"Data error, got 0x%08X at 0x%08X, expected 0x%08X",
"Second read error, got 0x%08X at 0x%08X, expected 0x%08X",
"Failed writing EEPROM at 0x%04X",
"Failed reading EEPROM at 0x%04X",
"EEPROM data error, got 0x08X at 0x04X, expected 0x%08X",
"Cannot open file %s",
"Invalid CPU image file %s",
"Invalid CPU image size %d",
"Cannot allocate memory",
Hardware Diagnostics
Loopback diagnostic tests are available for testing the adapter hardware. These tests provide access to the adapter
internal/external diagnostics, where packet information is transmitted across the physical link. For Windows environments, see
Running Diagnostic Tests in Windows).
Troubleshooting Checklist
CAUTION! Before you open the case of your system, review Safety Precautions.
The following checklist provides recommended actions to take to resolve problems installing the Broadcom NetXtreme Gigabit
Ethernet adapter or running it in your system.
Inspect all cables and connections. Verify that the cable connections at the network adapter and the switch are
attached properly. Verify that the cable length and rating comply with the requirements listed in Connecting the
Network Cables.
Check the adapter installation by reviewing Installing the Hardware. Verify that the adapter is properly seated in the
slot. Check for specific hardware problems, such as obvious damage to board components or the PCI edge connector.
Check the configuration settings and change them if they are in conflict with another device.
Verify that your system is using the latest BIOS.
Try inserting the adapter in another slot. If the new position works, the original slot in your system may be defective.
Replace the failed adapter with one that is known to work properly. If the second adapter works in the slot where the
first one failed, the original adapter is probably defective.
Install the adapter in another functioning system and run the tests again. If the adapter passed the tests in the new
system, the original system may be defective.
Remove all other adapters from the system and run the tests again. If the adapter passes the tests, the other adapters
may be causing contention.
NetWare
Linux
To verify that the TG3 Linux driver is loaded properly, run:
lsmod | grep tg3
If the driver is loaded, a line similar to the one below is displayed, where size is the size of the driver in bytes, and n is the
number of adapters configured.
Table 1: Linux Driver
Module Size Used by
TG3
size n
Windows
Use the ping command to determine if the network connection is working.
NOTE: Network connectivity can also be tested using the Testing the Network feature in Broadcom Advanced Control
Suite2.
1. Click Start, and then click Run.
2. Type cmd in the Open box, and then click OK.
3. Type ipconfig /all to view the network connection to be tested.
4. Type ping IP address, and then press ENTER.
The ping statistics that are displayed indicate whether the network connection is working or not.
Typically, network test failures are the result of a configuration problem on the network or with the IP addresses. Below are
some common steps to check when troubleshooting the network:
1. Verify that the cable is attached and you have a proper link.
2. Verify that the drivers are loaded and enabled.
3. Replace the cable that is attached to the NIC/LOM.
4. Verify that the IP address is assigned correctly using the command line ipconfig or by checking the OS IP assigning
NetWare
Ping an IP host on the network to verify connection has been established:
From the command line, type ping IP address, and then press ENTER.
The ping statistics that are displayed indicate whether the network connection is working or not.
Linux
To verify that the Ethernet interface is up and running, run ifconfig to check the status of the Ethernet interface. It is
possible to use netstat -i to check the statistics on the Ethernet interface. Go to Linux Driver Software for information on
ifconfig and netstat.
Ping an IP host on the network to verify connection has been established:
From the command line, type ping IP address, and then press ENTER.
The ping statistics that are displayed indicate whether the network connection is working or not.
Miscellaneous
Problem: Although installled, the Broadcom Advanced Control Suite (BACS) application does not start.
Solution: .NET Framework 2.0 is required for BACS to operate. Install .NET Framework 2.0.
Problem: When the bus on the system is operating in PCI mode, the Broadcom NetXtreme Gigabit Ethernet adapter performs
at PCI mode if it is added after the system has booted.
Solution: When the system is booted up without any adapter, the bus operates at the lowest mode, which is PCI mode.
Reboot the system after the adapter has been added.
Problem: The Broadcom NetXtreme Gigabit Ethernet adapter may not perform at optimal level on some systems if it is added
after the system has booted.
Solution: The system BIOS in some systems does not set the cache line size and the latency timer if the adapter is added
after the system has booted. Reboot the system after the adapter has been added.
Problem: Large Send Offload (LSO) and Checksum Offload are not working on my team.
Solution: If one of the adapters on a team does not support LSO, LSO does not function for the team. Remove the adapter
that does not support LSO from the team, or replace it with one that does. The same applies to Checksum Offload.
Problem: When using the TG3 driver and Red Hat 4 (with any release prior to release 6), after setting the speed and duplex
with ethtool, autonegotiation is disabled and cannot be restored.
Solution: Unload or reload the TG3 driver, or upgrade to the latest version of ethtool.
Problem: A DCOM error message (event ID 10016) appears in the System Even Log during the installation of the Broadcom
adapter drivers.
Solution: This is a Microsoft issue. For more information, see Microsoft knowledge base KB913119 at
http://support.microsoft.com/kb/913119.
Please read all Restrictions and Disclaimers.
Back to Contents Page
Overview
The Broadcom Advanced Control Suite version 4 (BACS4) is a management application for configuring the NetXtreme I
families of adapters. BACS4 software operates on Windows and Linux server and client operating systems.
This chapter describes how to install the BACS4 management application on Linux systems. For Windows systems,
an installation program is provided which installs both the Windows drivers and the management applications, including
BACS4 (see Windows Driver and Management Application Installation for instructions).
There are two main components of the BACS4 utility: the provider component and the client software. A provider is installed
on a server, or "managed host", that contains one or more CNAs. The provider collects information on the CNAs and makes it
available for retrieval from a management PC on which the client software is installed. The client software enables viewing
information from the providers and configuring the CNAs.The BACS client software includes a graphical user interface (GUI)
and a command line interface (CLI).
Communication Protocols
A communication protocol enables exchanging information between provider and the client software. These are proprietary or
open-source implementations of the Web-Based Enterprise Management (WBEM) and Common Information Model (CIM)
standards from the Distributed Management Task Force (DMTF). Network administrators can choose the best option based on
the prevailing standard on their network.
The following table shows the available options based on the operating systems installed on the managed host and the client.
If the client uses:
Windows
Windows
WMI
WS-MAN (WinRM)
Windows
Linux
CIM-XML (OpenPegasus)
WS-MAN (OpenPegasus)
Linux
Windows
WS-MAN (WinRM)
Linux
Linux
CIM-XML (OpenPegasus)
WS-MAN (OpenPegasus)
Description
PEGASUS_ROOT
PEGASUS_HOME
PATH
$PATH:$PEGASUS_HOME/bin
PEGASUS_ENABLE_CMPI_PROVIDER_MANAGER True
PEGASUS_CIM_SCHEMA
"CIM222"
PEGASUS_PLATFORM
PEGASUS_HAS_SSL
PEGASUS_ENABLE_PROTOCOL_WSMAN
Additional Settings
The $PEGASUS_HOME variable must be set up in the shell environment, and $PEGASUS_HOME/bin needs to be appended to
Description
cimconfig -l
cimconfig -l -c
Enable Authentication
The following OpenPegasus properties have to be set as described in this section. Otherwise, the Broadcom CIM Provider will
not work properly. Ensure the following are set before launching BACS and connecting to the provider.
Start CIM server if it is not already started. Then, set the following:
cimconfig -s enableAuthentication=true -p
cimconfig -s enableNamespaceAuthorization=false -p
cimconfig -s httpAuthType=Basic -p
cimconfig -s passwordFilePath=cimserver.passwd -p
cimconfig -s forceProviderProcesses=false -p
If you want root user to connect remotely:
cimconfig -s enableRemotePrivilegedUserAccess=true -p
User configuration with privilege: The Linux system users are used for OpenPegasus authentication. The systems users have
to be added to OpenPegasus using cimuser to connect via BACS:
cimuser -a -u <username> -w <password>
Example: cimuser -a -u root -w linux1
Enable HTTP
1. If CIM server is not started, start it.
2. Use the following command to set up an HTTP port (optional):
cimconfig -s httpPort=5988 -p
This property is not available for Inbox OpenPegasus.
3. Use the following command to enable HTTP connection:
cimconfig -s enableHttpConnection=true -p
4. Use the cimserver -s and cimserver commands, respectively, to stop and restart CIM server for the new
configuration to take effect.
Enable HTTPS
1. If CIM server is not started, start it.
2. Set up HTTPS port with the following command (optional):
cimconfig -s httpsPort=5989 -p
This property is not available for inbox OpenPegasus.
1. Enable HTTPS connection with 'the following command:
cimconfig -s enableHttpsConnection=true -p
2. Use the cimserver -s and cimserver commands, respectively, to stop and restart CIM server for the new
configuration to take effect.
Install
Enter following command to install Broadcom CMPI Provider.
% rpm -i BRCM_CMPIProvider-{version}.{arch}.rpm
Uninstall
Enter following command to uninstall Broadcom CMPI Provider:
% rpm -e BRCM_CMPIProvider
RedHat
1. Click System, select Administration, and then select Firewall.
2. Select Other Ports.
3. In the Port and Protocol Dialog box, select User Defined.
4. In the Port/Port Range field, add the port number.
5. In the Protocol field, add the protocol as TCP or UDP, etc.
6. Click Apply for the firewall rules to take effect.
Example:
For CIM-XML over HTTP, the port number is 5988 and protocol is TCP.
For CIM-XML over HTTPs, the port number is 5989 and protocol is TCP.
SuSE
1. Click Compute and then click YaST.
2. Select Security & Users on the left pane.
3. On the right pane, double-click Firewall.
4. Select Custom Rules on the left pane.
5. On the right pane click Add.
6. Enter the following values:
Source Network: 0/0 (means all)
Protocol: TCP (or the appropriate protocol)
Destination Port: <Port Number> or <Range of Port Numbers>
Source Port: Leave blank.
7. Click Next and then click Finish for the firewall rules to take effect.
Example:
For CIM-XML, use the following values:
Source Network: 0/0 (means all)
Protocol: TCP
Destination Port: 5988:5989
Source Port: Leave blank.