Lab Testing: Key Findings and Conclusions
Lab Testing: Key Findings and Conclusions
Lab Testing: Key Findings and Conclusions
C
iscos Integrated Services Router Generation 2 platform models
Product Category: 1941W, 2911, 2951 3925E, 3945 and 3945E were evaluated by
Miercom for performance feature validation while being used in
Integrated branch office scenarios. We also evaluated the ASR 1002 Fixed
Router on some of these tests to show upward scalability of the next
Services Router family of routers. The objective was to validate increased levels of
Generation 2 voice, video, security, wireless, mobility and data service integration.
While testing the features and services offered by the ISR G2 branch
routers, performance and throughput was observed, to ascertain if the
Vendor Tested: activated services affected the throughput.
Todays branch routers must have the ability to serve the requirements
of the current branch, and scale to the evolving needs of integrated
services. This includes increased bandwidth requirements, supporting
advanced security features and WAN and VPN technologies, with
multi-media collaboration. (continued next page)
Products Tested:
Cisco 1941W ISR Figure 1: Throughput Compared to Target Branch Bandwidth
Cisco 2911 ISR
Cisco 2951 ISR
Cisco 3925E ISR
Cisco 3945E ISR
Cisco ASR 1002
Higher throughput was delivered than was needed to meet the bandwidth
requirements for the branches, while integrated services and features including
IPsec and NAT were activated. While maintaining 75% CPU utilization, no frame
loss was observed. Routers are listed in order of progressively increasing
performance and capabilities.
Figure 2: Throughput Compared to Target Branch Bandwidth
Router models 3925E and
3945E have an enhanced
SPE. The Cisco ASR 1002
Fixed Router is included in
the mix to show upward
migration to the next family
of routers. Higher throughput
was delivered than was
needed to meet the
bandwidth requirements
while integrated services and
features including IPsec and
NAT were activated. While
maintaining 75% CPU
utilization, no frame loss was
observed. Routers are listed
in order of progressive
Source: Miercom, May 2010 performance and capabilities.
The Cisco ISR G2 branch routers - 3925E, 3945E and the Cisco ASR 1002 Fixed Router - delivered
performance throughput values greater than the target positioning bandwidth requirements while
integrated services features were activated. By adding the new Service Performance Engines (SPE-
200 and SPE-250) to the 3925E, we observed 5.5 times increased throughput. When the SPE was
added to the 3945E, we observed 3.5 times increased throughput. All test results observed were
recorded at zero frame loss while maintaining the CPU utilization at 75%.
A solution that provides security, wireless Branch offices are classified as small, medium,
capabilities, UC, and WAN optimization in a large and regional, with unique requirements for
single box, while maintaining existing each category. All test cases described were
performance and throughput is a true integrated based on specific branch requirements. A typical
solution. Integrated network based services are small branch, such as a retail store with 10-25
an effective way to reduce costs, while users, would use the ISR 1941W. A bank with 25-
delivering new services to the end user. The 40 users would use the ISR 2911. A corporate
ISR G2 platforms deliver these services and branch or large branch accommodating 40-75
performance, while deployed in small to large users would use the ISR 2951. While a regional
remote branch office topologies. office with 75-150 users would use the ISR 3945.
Branches with greater bandwidth requirements
Miercom tests the performance and validated
would use 3925E, 3945E or an ASR 1002.
the integrated features of the Cisco ISR G2
platforms. Tests were conducted with feature Performance Tests
intensive branch office deployments, and
Each class of branch routers was tested based on
considered actual branch settings currently
the typical branch configuration of the model with
used by ISR customers.
Dynamic Multipoint Virtual Private Network
Branch Architecture (DMVPN), Zone Based Policy Firewall (ZBF) and
Intrusion Prevention System (IPS), and
The architecture of the ISR G2 platforms
Hierarchical Quality of Service (HQOS) features
enables richer media experience with video,
enabled. We subjected the Cisco ISR G2 router
voice and WAN evolution. Additions to the
platforms, with integrated services and features
services available on the ISR G2 platforms
enabled, to a traffic mix consisting of the data,
include integrated security, unified
voice and/or Telepresence traffic recommended
communications, wireless, and application
for each deployment scenario.
optimization services. Pay-As-You-Grow software
licensing and Cisco IOS software packaging also All of the Cisco ISR G2 branch routers delivered
reduces branch office expenses. throughput which exceeded by 102% to 214% the
NAT
31.24 38.28 104.38 248.81 207.02 366.66
(Mbps)
IPsec (AES)
4.67 5.51 5.93 100.95 5.80 135.24
(Mbps)
IPv4 Forwarding
IMIX packets
(Mbps)
packets
NAT
692.18 713.28 1,908.59 4,000.00 2,000.00+ 4,000.00
(Mbps)
IPsec (AES)
146.56 164.84 195.78 1,528.90 739.64 1,542.78
(Mbps)
The branch offices have been categorized into small, medium, large and regional categories with unique
requirements for each size office. The configurations shown above represent feature intensive branch
deployments today and are likely to be closer to the norm in the future. All testing and test cases
described in this document w conducted based on the branch office requirements for the respective Cisco
ISR G2 routers.
Device Under
Test (DUT)
Avalanche
Cisco ISR
Internet
How We Did It
Test Bed 1
Similar network topologies were used for all branch office deployments with variations added to reflect the typical
needs of the different branch sizes. Spirent TestCenter, Spirent Avalanche and Reflector were used for traffic
generation. Spirent TestCenter was used to generate RTP voice traffic. A custom Cisco traffic generation tool was
used to simulate Telepresence sessions. This tool sends actual Telepresence video streams and monitors for any
frame loss, jitter and latency. All traffic used for performance testing of the Cisco ISR G2 Device Under Test (DUT)
were set to meet Service Level Agreement (SLA) of zero frame loss and acceptable latency or jitter values.
To evaluate the effectiveness of the security features of the Cisco ISR G2 platforms, offensive security tests were
conducted using Miercom Offensive Security Testing Suite, the Ixia IxDefend platform (www.ixiacom.com) and the Mu
Dynamics Test Suite (www.mudynamics.com). The Ixia IxDefend advanced security assessment tool analyzed quality,
resiliency, and security exposures across a broad array of protocols and applications. The IxDefend test included 40
protocols from link layer to application protocols. Each protocol in each bundle included thousands of tests. IxDefends
tests provide the deepest possible protocol coverage.
The Mu Test Suite distills information from the most recently discovered root-cause vulnerabilities into test cases that
target the vulnerabilities that lie behind tens of thousands unique exploit vectors. The security assessment was
conducted with Ixias (www.ixiacom.com) IxDefend which was used to generate exploits and attacks. Miercom
recommends customers conduct their own needs analysis and test for the specific environment for product
deployment before making a selection. Contact [email protected] for additional details on the configurations
applied to the system under test and test tools used in this evaluation.
Telepresence Telepresence
Simulator Simulator
ASR 1002
HeadEnd
Telepresence Telepresence
Simulator Simulator
Test Bed 2
The Device Under Test (DUT) is only connected to the different traffic generation tools depicted. Throughput
measurements were taken with and without features enabled such as IPsec VPN and NAT. Tests for throughput
performance of RFC 2544 were conducted primarily using this test scenario.
Test Bed 3
The second test bed has the DUT connected to a HeadEnd router in addition to the traffic generation tools as shown
in Figure 2. This is normally used for those cases, like IPSec, that require a connection to a peer router. The WAN
interface of the ISR G2 DUT utilized a secure VPN connection as well as a backup connection specific to the size of
the branch. For our tests, headquarters was represented by a Cisco 7200 Series router, terminating at the other side
of the VPN connection and providing the core side connections for traffic generation and monitoring. To simulate the
various streams found in a typical branch setting, generic stateful HTTP data traffic consisting of 25K, 30K and 130K
objects, with 10 objects per TCP session, were used. All performance testing was conducted with router CPU
maintained in the 50-70% range, which is the recommended load that allows sufficient overhead and tolerance for
real-world network activity. Tests involving IPsec performance were conducted primarily using this test scenario.