PDF Switch Router Architectures Systems With Crossbar Switch Fabrics 1St Edition James Aweya Author Ebook Full Chapter

Download as pdf or txt
Download as pdf or txt
You are on page 1of 53

Switch Router Architectures Systems

with Crossbar Switch Fabrics 1st


Edition James Aweya (Author)
Visit to download the full and correct content document:
https://textbookfull.com/product/switch-router-architectures-systems-with-crossbar-swi
tch-fabrics-1st-edition-james-aweya-author/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Switch Router Architectures Review of Shared Bus and


Shared Memory Based Systems 1st Edition Dr. James Aweya

https://textbookfull.com/product/switch-router-architectures-
review-of-shared-bus-and-shared-memory-based-systems-1st-edition-
dr-james-aweya/

The Internet In Everything Freedom And Security In A


World With No Off Switch Laura Denardis

https://textbookfull.com/product/the-internet-in-everything-
freedom-and-security-in-a-world-with-no-off-switch-laura-
denardis/

Linear and Switch-Mode RF Power Amplifiers: Design and


Implementation Methods 1st Edition Abdullah Eroglu

https://textbookfull.com/product/linear-and-switch-mode-rf-power-
amplifiers-design-and-implementation-methods-1st-edition-
abdullah-eroglu/

The Leadership Mind Switch: Rethinking How We Lead in


the New World of Work 1st Edition D. A. Benton

https://textbookfull.com/product/the-leadership-mind-switch-
rethinking-how-we-lead-in-the-new-world-of-work-1st-edition-d-a-
benton/
Summary of Switch On Your Brain The Key To Peak
Happiness Thinking And Health 1st Edition Dr. Caroline
Leaf

https://textbookfull.com/product/summary-of-switch-on-your-brain-
the-key-to-peak-happiness-thinking-and-health-1st-edition-dr-
caroline-leaf/

Warp Knitted Fabrics Construction 1st Edition Kyosev

https://textbookfull.com/product/warp-knitted-fabrics-
construction-1st-edition-kyosev/

Number Systems for Deep Neural Network Architectures


1st Edition Alsuhli

https://textbookfull.com/product/number-systems-for-deep-neural-
network-architectures-1st-edition-alsuhli/

Flow Architectures: The Future of Streaming and Event-


Driven Integration 1st Edition James Urquhart

https://textbookfull.com/product/flow-architectures-the-future-
of-streaming-and-event-driven-integration-1st-edition-james-
urquhart/

Designing Elixir Systems with OTP Write Highly Scalable


Self Healing Software with Layers 1st Edition James
Edward Gray Ii

https://textbookfull.com/product/designing-elixir-systems-with-
otp-write-highly-scalable-self-healing-software-with-layers-1st-
edition-james-edward-gray-ii/
Switch/Router
Architectures
Switch/Router
Architectures
Systems with Crossbar Switch Fabrics

James Aweya
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2020 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
International Standard Book Number-13: 978-0-367-40785-8 (Hardback)
This book contains information obtained from authentic and highly regarded sources.
Reasonable efforts have been made to publish reliable data and information, but the author and
publisher cannot assume responsibility for the validity of all materials or the consequences
of their use. The authors and publishers have attempted to trace the copyright holders of all
material reproduced in this publication and apologize to copyright holders if permission
to publish in this form has not been obtained. If any copyright material has not been
acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted,
reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means,
now known or hereafter invented, including photocopying, microfilming, and recording, or in
any information storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access
www.copyright.com (www.copyright.com/) or contact the Copyright Clearance Center, Inc.
(CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit
organization that provides licenses and registration for a variety of users. For organizations that
have been granted a photocopy license by the CCC, a separate system of payment has been
arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks,
and are used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
www.taylorandfrancis.com
and the CRC Press Web site at
www.crcpress.com
Contents
Preface���������������������������������������������������������������������������������������������������������������� xv
Author���������������������������������������������������������������������������������������������������������������xvii

PART 1  haracteristics of Switch/Routers


C
with Crossbar Switch Fabrics

Chapter 1 The Switch/Router: Integrated OSI Layers 2 and 3


Forwarding on a Single Platform���������������������������������������������������� 3
1.1 Introduction�������������������������������������������������������������������������� 3
1.2 Flow-Based Layer 3 Forwarding������������������������������������������ 4
1.3 Network Topology-Based Layer 3 Forwarding�������������������� 5
1.4 Using Centralized or Distributed Forwarding Engines�������� 8
1.4.1 Forwarding Using a Centralized Forwarding
Engine����������������������������������������������������������������������� 8
1.4.2 Forwarding Using Distributed Forwarding
Engines��������������������������������������������������������������������� 9
1.5 Building the Layer 2 Forwarding Tables���������������������������� 10
1.5.1 MAC Address Aging���������������������������������������������� 10
1.5.2 Synchronizing MAC Address Tables in a
Distributed Forwarding Architecture��������������������� 11
1.5.3 History of MAC Address Changes������������������������� 11
1.6 Memory Architectures for Storing Forwarding
Databases���������������������������������������������������������������������������� 12
1.6.1 Content Addressable Memory (CAM)������������������� 12
1.6.2 Ternary Content Addressable Memory (TCAM)��� 14
1.7 Physical and Logical Interfaces on a Switch/Router���������� 16
1.7.1 Layer 2 (Switch) Interfaces������������������������������������� 17
1.7.1.1 Access Ports�������������������������������������������� 17
1.7.1.2 Access Ports and Tagged Packets������������ 18
1.7.1.3 Trunk Ports���������������������������������������������� 19
1.7.1.4 Port Channels or Port Groups����������������� 20
1.7.2 Layer 3 (Routed) Interfaces������������������������������������ 21
1.7.2.1 Physical Layer 3 (Routed) Interfaces������ 21
1.7.2.2 Logical Layer 3 (Routed) Interfaces
(VLAN Interfaces or Switch Virtual
Interfaces (SVIs))������������������������������������� 23
1.7.3 Subinterfaces���������������������������������������������������������� 25

v
vi Contents

1.7.4 Loopback Interfaces����������������������������������������������� 27


1.7.4.1 Local Communications��������������������������� 28
1.7.4.2 Testing and Performance Analysis���������� 28
1.7.4.3 Device Identification������������������������������� 28
1.7.4.4 Routing Information Maintenance���������� 29
1.7.5 Tunnel Interfaces���������������������������������������������������� 31
1.7.5.1 Layer 3 Tunnel Interfaces/Ports
Examples������������������������������������������������� 32
1.7.5.2 Layer 2 Tunnel Interfaces/Ports
Examples������������������������������������������������� 36
1.7.6 Port Channels��������������������������������������������������������� 40
1.7.6.1 Layer 2 and Layer 3 Port Channels��������� 41
1.7.6.2 Load Balancing��������������������������������������� 43
1.7.6.3 Dynamic Configuration of Port
Channels�������������������������������������������������� 44
1.8 Challenges�������������������������������������������������������������������������� 44
References������������������������������������������������������������������������������������� 46

Chapter 2 Understanding Crossbar Switch Fabrics��������������������������������������� 47


2.1 Introduction������������������������������������������������������������������������ 47
2.2 The Crossbar Switch Fabric����������������������������������������������� 49
2.2.1 Implementing a Crossbar Switch���������������������������� 50
2.2.2 Building Multistage Switches��������������������������������� 51
2.2.3 Challenges in Building Larger Crosspoint Arrays� 55
2.2.4 Designing Today’s System Interconnects��������������� 58
2.3 Logical Architecture and Improving Data Transfer
Throughput������������������������������������������������������������������������� 61
2.4 Components of a Practical Crossbar Switch Fabric
System�������������������������������������������������������������������������������� 65
2.5 Traffic Scheduling in the Crossbar Switch������������������������� 67
2.5.1 The Scheduling Problem���������������������������������������� 69
2.5.2 Parallel Iterative Matching (PIM)�������������������������� 70
2.5.3 Round-Robin Matching (RRM)����������������������������� 72
2.5.4 Iterative Round-Robin Matching with Slip
(iSLIP)�������������������������������������������������������������������� 73
2.5.5 Other Considerations in the Design of
Schedulers��������������������������������������������������������������� 75
2.6 Handling Multicast Traffic������������������������������������������������� 75
2.6.1 Multicast Packet Replication���������������������������������� 77
2.6.2 Multicast Traffic Scheduling���������������������������������� 79
2.6.2.1 Full versus Partial Multicast
Scheduling across the Crossbar
Switch Fabric������������������������������������������� 79
2.6.2.2 Scheduling in Internally Unbuffered
Crossbar Switches����������������������������������� 80
Contents vii

2.6.2.3 Scheduling in Internally Buffered


Crossbar Switches����������������������������������� 82
2.6.2.4 Special Focus: The ESLIP
Scheduling Algorithm����������������������������� 83
2.7 Data Transfer Process over the Switch Fabric�������������������� 86
2.8 System Monitoring and Control����������������������������������������� 87
2.8.1 Main MBus Functions�������������������������������������������� 89
2.8.2 Alarm Functionality����������������������������������������������� 90
2.9 Scalability��������������������������������������������������������������������������� 92
2.10 Fault Tolerance������������������������������������������������������������������� 95
2.11 Generic Switch/Router with Crossbar
Switch Fabric���������������������������������������������������������������������� 99
References����������������������������������������������������������������������������������� 104

Chapter 3 Introduction to Switch/Routers with Crossbar


Switch Fabrics����������������������������������������������������������������������������� 109
3.1 The Crossbar Switch Fabric��������������������������������������������� 109
3.2 Architectures with Crossbar-Based Switch Fabrics
and Centralized Forwarding Engines��������������������������������110
3.2.1 Architectures with Forwarding Using a Flow/
Route Cache in Centralized Processor������������������110
3.2.2 Architectures with Forwarding Using an
Optimized Lookup System in Centralized
Processor���������������������������������������������������������������111
3.3 Architectures with Crossbar-Based Switch Fabrics
and Distributed Forwarding Engines��������������������������������112
3.3.1 Architectures with Forwarding Engine and
Flow/Route Cache in Line Cards��������������������������113
3.3.2 Architectures with Fully Distributed
Forwarding Engines in Line Cards�����������������������114
3.4 Relating Architectures to Switch/Router Types����������������116

PART 2 Design Examples and Case Studies

Chapter 4 Cisco Catalyst 6500 Series Switches with Supervisor


Engines 1A and 2������������������������������������������������������������������������ 123
4.1 Introduction���������������������������������������������������������������������� 123
4.2 Main Architectural Features of the Catalyst
6500 Series����������������������������������������������������������������������� 124
4.3 Catalyst 6500 Switch Fabric Architecture����������������������� 128
4.4 Catalyst 6500 Line Cards Architectures�������������������������� 132
4.4.1 Fabric-Enabled Line Cards����������������������������������� 132
4.4.2 Fabric-Only Line Cards���������������������������������������� 133
viii Contents

4.5 Catalyst 6500 Control Plane Implementation and


Forwarding Engines—Supervisor Engines���������������������� 133
4.5.1 Supervisor Engine 1A Architecture��������������������� 133
4.5.1.1 Supervisor Engine 1A with Only a
PFC1������������������������������������������������������ 133
4.5.1.2 Supervisor Engine 1A with a PFC1
and MSFC1/MSFC2������������������������������ 135
4.5.2 Supervisor Engine 2 Architecture������������������������ 137
4.5.2.1 Supervisor Engine 2 with a PFC2��������� 137
4.5.2.2 Supervisor Engine 2 with a PFC2
and MSFC2������������������������������������������� 140
4.5.3 Supporting High Availability with Dual
Supervisor Engines������������������������������������������������141
4.5.4 Distributed Forwarding Card������������������������������� 142
4.5.4.1 Packet Forwarding in the DFC���������������143
4.6 Packet Flow in the Catalyst 6500������������������������������������� 145
4.6.1 Packet Flow in the Catalyst 6500 with
Centralized Forwarding���������������������������������������� 145
4.6.2 Packet Flow in the Catalyst 6500 with
Distributed Forwarding���������������������������������������� 148
References����������������������������������������������������������������������������������� 156

Chapter 5 Avaya P580 and P882 Routing Switch Architecture with


80-Series Media Module������������������������������������������������������������� 157
5.1 Introduction���������������������������������������������������������������������� 157
5.2 Basic Architecture������������������������������������������������������������ 157
5.3 Data Flow through the Avaya 80-Series Switch��������������� 158
5.4 Quality of Service (QoS) Mechanisms������������������������������161
5.4.1 Classification Precedence��������������������������������������161
5.4.2 DiffServ Mapping Table��������������������������������������� 163
5.4.3 Queuing and Scheduling�������������������������������������� 164
5.4.4 Traffic Management��������������������������������������������� 164
5.4.5 IEEE 802.1p/Q and DSCP or ToS Standards in
the Avaya 80-Series���������������������������������������������� 167
5.4.5.1 DiffServ’s Per-Hop
Behaviors (PHB)����������������������������������� 167
5.4.5.2 Packet Loss Priority (PLP)�������������������� 168
5.4.5.3 Avaya 80-Series Recommendations
for IEEE 802.1p/Q, DSCP Code
Point, and Queues���������������������������������� 169
5.5 Designing the High-Performance Switch/Router�������������170
References����������������������������������������������������������������������������������� 172
Contents ix

Chapter 6 Foundry Networks Multilayer Switches with IronCoreTM


Network Interface Module�����������������������������������������������������������175
6.1 Introduction�����������������������������������������������������������������������175
6.2 Switch Chassis Overview��������������������������������������������������175
6.3 Chassis Crossbar Switch Fabric��������������������������������������� 177
6.4 IronCoreTM Network Interface Module
Architecture���������������������������������������������������������������������� 179
6.4.1 Network Interface Module Components�������������� 180
6.4.1.1 Physical Ports���������������������������������������� 180
6.4.1.2 Multi-Port MAC������������������������������������ 182
6.4.2 Packet Processor—The Forwarding Engine of
IronCore���������������������������������������������������������������� 182
6.4.3 Route Processor Components������������������������������� 183
6.4.3.1 System Management Module/
Board—The Route Processor
Module�������������������������������������������������� 183
6.4.3.2 System Management
Interface/CPU���������������������������������������� 183
6.4.3.3 CPU Path and Data Transfer
Engine Path������������������������������������������� 184
6.4.3.4 Management Bus����������������������������������� 184
6.4.4 Shared Memory and Switch Fabric Interface
Components���������������������������������������������������������� 184
6.4.4.1 Shared-Memory Switch Fabric and
Buffer Pool�������������������������������������������� 184
6.4.4.2 Data Transfer Engine����������������������������� 185
6.4.4.3 Crossbar Backplane Connection—
Module Connection to Crossbar
Switch Fabric����������������������������������������� 185
6.4.4.4 Multiple Destination Output Priority
Queues��������������������������������������������������� 185
6.4.4.5 Multiple Input Source Buffers per
Output Port�������������������������������������������� 186
6.5 BigIron 4000 Complete Crossbar Switch System������������ 187
6.6 Packet Processing Overview�������������������������������������������� 188
6.6.1 Role of the Data Transfer Engine and Packet
Flow for Unicast and Multicast Traffic����������������� 189
6.6.1.1 Forwarding Unicast Traffic������������������� 190
6.6.1.2 Forwarding Multicast Traffic���������������� 190
6.7 Important Attributes of the Shared-Memory Based
IronCore Architecture�������������������������������������������������������191
References����������������������������������������������������������������������������������� 192
x Contents

Chapter 7 Foundry Networks Multilayer Switches with JetCoreTM


Network Interface Module���������������������������������������������������������� 193
7.1 Introduction���������������������������������������������������������������������� 193
7.2 JetCoreTM Network Interface Module Architecture��������� 193
7.2.1 Network Interface Components���������������������������� 197
7.2.1.1 Physical Ports���������������������������������������� 197
7.2.1.2 Media Access Controller (MAC)���������� 197
7.2.2 JetCore Forwarding Engine Components������������� 197
7.2.2.1 Packet Classifier—The Forwarding
Engine of JetCore���������������������������������� 197
7.2.2.2 Content Addressable Memory (CAM)�� 199
7.2.2.3 Parameter Random Access Memory
(PRAM)������������������������������������������������� 200
7.2.2.4 VLAN Multicast Assist Module����������� 200
7.2.2.5 Transmit Pipeline���������������������������������� 201
7.2.3 Memory Components of the Port Group
Switching Logic and Switch Fabric���������������������� 201
7.2.3.1 Shared-Memory Switch Fabric������������� 201
7.2.3.2 Buffer Manager������������������������������������� 202
7.2.3.3 Shared-Memory Buffer Pool����������������� 203
7.2.4 Route Processor Components������������������������������� 203
7.2.4.1 Command Bus��������������������������������������� 203
7.2.4.2 System Management Interface�������������� 203
7.2.5 Module Crossbar Switch Fabric��������������������������� 204
7.2.5.1 Backplane Connectivity������������������������ 204
7.2.5.2 Replication of Multicast Packets����������� 204
7.2.6 Multiple Destination Output Priority Queues������ 205
7.3 Other JetCore Features����������������������������������������������������� 207
7.4 Packet Processing Overview�������������������������������������������� 208
7.4.1 Packet Flow for Unicast and Multicast Traffic����� 208
7.4.1.1 Forwarding Unicast Traffic to
Destination Port(s) on a Different
Module Crossbar Switch Fabric������������ 209
7.4.1.2 Forwarding Multicast Traffic to
Destination Port(s) on Different
Module Crossbar Switch Fabric(s)�������� 209
7.5 Foundry IronWareTM Software Architecture���������������������211
7.6 Switching and Routing Architecture�������������������������������� 213
7.6.1 Foundry Switching and Routing Architecture����� 213
7.6.2 Packet Handling Mechanisms in the
Multilayer Switch������������������������������������������������� 215
7.6.3 FastIron Multilayer Switch Architecture���������������216
7.6.4 BigIron, NetIron, and TurboIron Multilayer
Switch Architecture�����������������������������������������������217
Reference������������������������������������������������������������������������������������� 218
Contents xi

Chapter 8 Cisco Catalyst 6500 Series Switches with Supervisor


Engine 720������������������������������������������������������������������������������������219
8.1 Introduction�����������������������������������������������������������������������219
8.2 Cisco Catalyst 6500 Backplane���������������������������������������� 221
8.3 Cisco Catalyst 6500 Crossbar Switch Fabric������������������� 222
8.4 Supervisor Engine 720����������������������������������������������������� 225
8.4.1 Multilayer Switch Feature Card 3 (MSFC3)�������� 228
8.4.2 Policy Feature Card 3 (PFC3)������������������������������ 228
8.5 Supervisor Engine 720–3B����������������������������������������������� 229
8.5.1 Policy Feature Card 3B (PFC3B)������������������������� 230
8.6 Supervisor Engine 720–3BXL����������������������������������������� 230
8.6.1 Policy Feature Card 3BXL (PFC3BXL)��������������� 230
8.7 Packet Forwarding in Supervisor Engines 720,
720–3B, and 720–3BXL��������������������������������������������������� 230
8.8 Catalyst 6500 Line Cards Supported by Supervisor
Engine 720������������������������������������������������������������������������ 231
8.8.1 dCEF256 Line Card Architecture������������������������ 232
8.8.2 CEF720 Line Card Architecture�������������������������� 234
8.8.3 dCEF720 Line Card Architecture������������������������ 238
8.9 Functional Elements of Distributed Forwarding Card
(DFC) and Policy Feature Card (PFC)����������������������������� 238
8.9.1 A Note on NetFlow����������������������������������������������� 242
8.9.2 Access Control Lists for QoS and Security
Processing������������������������������������������������������������� 242
8.9.3 Distributed Forwarding Operations in Catalyst
6500 with PFC or DFC����������������������������������������� 243
8.10 Packet Flow in the Catalyst 6500 with Supervisor
Engine 720������������������������������������������������������������������������ 244
8.10.1 Centralized Forwarding���������������������������������������� 244
8.10.2 Distributed Forwarding���������������������������������������� 247
8.10.3 Flow Cache-Based Packet Forwarding—
Accelerated Cisco Express
Forwarding (aCEF)����������������������������������������������� 250
References����������������������������������������������������������������������������������� 253

Chapter 9 Multicast Routing and Multicast Forwarding Information


Base (MFIB) Architecture���������������������������������������������������������� 255
9.1 Introduction���������������������������������������������������������������������� 255
9.2 Benefits of the MFIB Architecture���������������������������������� 256
9.3 Protocol-Independent Multicast (PIM)���������������������������� 257
9.4 Types of Multicast Table Entries�������������������������������������� 258
9.4.1 Multicast Table Context���������������������������������������� 259
9.5 Types of Multicast Tables������������������������������������������������� 259
9.5.1 IGMP Cache��������������������������������������������������������� 260
xii Contents

9.5.2 Reverse-Path Forwarding (RPF) Table���������������� 261


9.5.3 PIM Dense Mode (PIM-DM) Table Entries��������� 262
9.5.4 PIM Sparse Mode (PIM-SM) Table Entries��������� 263
9.5.4.1 Concept of Rendezvous Point���������������� 264
9.5.4.2 PIM-SM Router Architecture���������������� 265
9.5.4.3 Sending Multicast Data������������������������� 265
9.5.4.4 Receiving Multicast Data���������������������� 267
9.5.4.5 PIM Assert Mechanism������������������������� 268
9.5.4.6 Electing the PIM Forwarder—PIM
Assert Winner��������������������������������������� 269
9.5.5 PIM Sparse-Dense Mode������������������������������������� 270
9.5.5.1 Auto-RP������������������������������������������������� 271
9.5.5.2 Using PIM Sparse-Dense with
Auto-RP������������������������������������������������� 273
9.5.6 Multicast Source Discovery Protocol (MSDP)
Cache�������������������������������������������������������������������� 273
9.5.6.1 MSDP Peer-RPF Checks����������������������� 274
9.5.7 PIM Source Specific Multicast (PIM-SSM)
Table Entries��������������������������������������������������������� 275
9.5.8 Bidirectional PIM (BIDIR-PIM) Table Entries���� 276
9.5.8.1 Designated Forwarder Election������������� 278
9.5.8.2 Building the Bidirectional Group
Tree and Packet Forwarding����������������� 279
9.6 Multicast Reverse Path Forwarding (RPF)���������������������� 280
9.6.1 Using the RPF Table��������������������������������������������� 281
9.6.2 Data-Plane versus Control-Plane RPF Check������ 283
9.6.3 RPF Check������������������������������������������������������������ 284
9.7 MFIB Components����������������������������������������������������������� 285
9.7.1 Protocols and Tables Used in IP Multicast
Routing����������������������������������������������������������������� 288
9.7.2 Layers 2 and 3 Multicast Tables Entries in a
Switch/Router������������������������������������������������������� 290
9.7.2.1 Benefits of IGMP Snooping������������������ 292
9.7.3 Multicast Fast Drop���������������������������������������������� 292
9.7.4 Using the Multicast Routing Information Base
(MRIB)����������������������������������������������������������������� 293
9.7.5 Using the Multicast Forwarding Information
Base (MFIB)��������������������������������������������������������� 294
9.7.6 Using the Distributed MFIB�������������������������������� 295
9.8 Multicast Packet Forwarding Using the MFIB���������������� 297
9.8.1 Process Switching������������������������������������������������� 297
9.8.2 Fast-Path Processing Using Flow/Route
Caching����������������������������������������������������������������� 299
Contents xiii

9.8.3Fast-Path Processing Using Optimized


Forwarding Table Lookup Structures (without
Flow/Route Caching)�������������������������������������������� 299
9.9 PIM-SM Tunnel Interfaces����������������������������������������������� 301
References����������������������������������������������������������������������������������� 302

Chapter 10 Unicast versus Multicast Packet Forwarding: A Case Study������� 305


10.1 Introduction���������������������������������������������������������������������� 305
10.2 Unicast Forwarding Table and TCAM Lookup
Architecture���������������������������������������������������������������������� 305
10.3 Unicast Forwarding Examples����������������������������������������� 308
10.3.1 Catalyst 6500 Supervisor Engine 2���������������������� 309
10.3.2 Supervisor Engine 32������������������������������������������� 309
10.3.3 Catalyst 6500 Supervisor Engine 720�������������������314
10.4 Multicast Forwarding Tables and TCAM Lookup
Architecture�����������������������������������������������������������������������319
10.5 Multicast Packet Replication���������������������������������������������319
10.5.1 Centralized Packet Replication���������������������������� 323
10.5.2 Packet Replication in the Line Cards������������������� 323
10.5.2.1 Ingress Multicast Packet Replication���� 324
10.5.2.2 Egress Multicast Packet Replication����� 324
10.5.3 Combined Replication Methods��������������������������� 325
10.5.4 Packet Replication at Layer 3 versus Layer 2������� 325
10.5.5 Packet Replication in the Switch Fabric:
Preferred Method�������������������������������������������������� 326
10.6 Multicast Forwarding Architecture Example:
Catalyst 6500 Supervisor Engine 720������������������������������ 326
10.6.1 Ingress Packet Replication Mode Example:
Catalyst 6500 Supervisor Engine 720������������������ 333
10.6.2 Egress Packet Replication Mode Example:
Catalyst 6500 Supervisor Engine 720������������������ 335
References����������������������������������������������������������������������������������� 342

Index����������������������������������������������������������������������������������������������������������������� 343
Preface
The continuous growth of the Internet is still creating increasing demands for
bandwidth and new services from service providers’ networks. The growing reli-
ance on the Internet has also created the demand for value-added services that
require faster connectivity, higher quality of service, and more advanced mobile
user services. There is little doubt that the rising demand for Internet connectivity
and new services will continue to task the performance of network infrastructures
and the individual network devices that make those networks. Networks are also
under pressure to deliver these advanced services cost-effectively and at reduced
operational costs.
Switches, switch/routers, routers, and network devices, in general, have tradi-
tionally been based on the following basic interconnect architectures: shared bus,
shared memory, ring fabric (using point-to-point interconnects), and the crossbar
switch. Shared-bus architectures are simple in design and are easy to implement.
They can be constructed from readily available standard commercial products.
However, despite these advantages, designers have discovered over the years that
shared-bus architectures have practical performance, scalability, and fault toler-
ance limitations. On the issue of signal integrity, the bus architecture has practical
limits on the operating frequency that can be supported, signal propagation delays
(or path lengths) that are tolerable, and electrical loading on the signal lines.
The shared bus has serious bus bandwidth utilization limitation because it
allows only one data transfer on the bus at a time. On the shared-bus fabric, only
one transmission (time-slot) can be carried/propagated on the bus at any given
time, which can result in limited throughput, scalability, and low number of net-
work interfaces supported. Although multiple buses can be used to increase the
throughput and improve the reliability of bus-based architectures, its inability to
scale cost-effectively with higher data rates, number of network interfaces, and
bus clock speeds is a serious limitation.
The main disadvantage of a shared-memory architecture is that bandwidth
scalability is limited by the memory access speed (bandwidth). The access speeds
of memories have a physical limit, and this limit prevents the shared-memory
switch architecture from scaling to very high bandwidths and port speeds.
Another factor is that the shared-memory bandwidth has to be at least two times
the aggregate system port speeds for all the ports to run at full line rate.
Ring architectures overcome the one-at-a-time data-transfer limitations of the
shared bus by allowing multiple concurrent data transfers on the ring in order to
achieve higher aggregate data throughput. However, high data transfer latencies
and ring reliability issues such as single points of failure are major concerns in
ring architectures. The ring is susceptible to single-point-of-failures because it
supports only one path between any two adjacent nodes. One malfunctioning node
or link on the ring can disrupt communication on the entire fabric. The capacity
of the ring architectures can be improved by implementing multiple parallel rings.

xv
xvi Preface

These rings are usually controlled in a distributed manner, but Medium Access
Control (MAC) implementation on the multiple rings can be difficult to realize.
Given the limitations of the other interconnect architectures, designers
have long considered the use of crossbar switch fabrics, especially for high-
performance, high-capacity systems. Crossbar switch fabrics offer several design
benefits over the traditional shared-media fabrics. High-performance crossbar
switches can support multiple data transfers over the fabric at the same time and
can easily be made nonblocking, avoiding the bandwidth limitations seen in the
shared-media fabrics with one-at-a-time data transfer type of operations.
Crossbar switch fabrics have the flexibility of connecting any input to any
output, allowing for multiple concurrent data transfers. They can be designed
to have relatively higher bandwidth, scalability, and fault tolerance. In addition
to its basic architectural advantages, the crossbar switch can be implemented in
ASICs and FPGAs. Bigger switches can be constructed from smaller crossbar
switch chips. There is an abundance of literature from the commercial sector and
academia on how to design crossbar switch fabrics of different types and capabili-
ties. For this reason, a comprehensive list of references will be nearly impossible
in this book. More so, crossbar switch fabric design has been a very well-studied
area, especially during the days of Asynchronous Transfer Mode (ATM) switch
development.
This book discusses the various switch/router architectures that use crossbar
switch fabric as their internal interconnects. The book also discusses the main
issues involved in the design of these switch/routers that use crossbar switch fab-
rics. The issues discussed include performance, implementation complexity, and
scalability to higher speeds. To enhance reader understanding of switch/routers,
the book begins by describing the basics of switch/routers and then the most com-
mon crossbar switch fabric-based architectures in today’s networks.
After presenting a detailed discussion of centralized versus distributed for-
warding architectures, the book discusses the processing considerations that led
to the design of distributed forwarding systems using crossbar-based switch fab-
rics. The discussion includes the hardware and software designs in select but dif-
ferent switch/router architectures, to allow the reader to appreciate the different
designs out there and how crossbar-based switch/routers have evolved over the
years. The book uses case studies to examine the inner workings of switch/router
design in a comprehensive manner with the goal of enhancing reader understand-
ing. The example architectures discussed are selected such that they cover the
majority of features found in switch/routers used in today’s networks. It is written
in a simple style and language to allow readers to easily understand and appreci-
ate the material presented.
Author
James Aweya, PhD, is a chief research scientist at Etisalat British Telecom
Innovation Center (EBTIC), Khalifa University, Abu Dhabi, UAE. He has been
granted 64 US patents and has published over 54 journal papers, 39 conference
papers, and 43 Nortel technical reports. He has authored one book and is a senior
member of the Institute of Electrical and Electronics Engineers (IEEE).

xvii
Part 1
Characteristics of Switch/Routers
with Crossbar Switch Fabrics
1 Integrated OSI Layers
The Switch/Router

2 and 3 Forwarding on
a Single Platform

1.1 INTRODUCTION
This chapter introduces switch/routers and the architectures and methods they
support to perform multilayer switching. An understanding of multilayer switch-
ing is necessary for the reader to better appreciate the various architectures dis-
cussed in this book. A switch/router, sometimes referred to as a multilayer switch,
is a device that supports the forwarding of packets at both Layers 2 and 3 of the
Open Systems Interconnection (OSI) model (Figure 1.1).
The switch/router also supports the relevant Layers 2 and 3 control plane proto-
cols needed to create the forwarding databases used in the forwarding of packets
[AWEYA1BK18]. In addition to features for system management and configura-
tion, the switch/router may support quality-of-service (QoS) and security filtering
and control mechanisms that use parameters within the Layers 4 to 7 fields of
packets being forwarded.
Network devices attached to different virtual local area networks (VLANs)
can only communicate with one another through a router (or Layer 3 forwarding
device) as illustrated in Figure 1.1. In a Layer 2 network, VLANs can be used to
define distinct and separate broadcast domains. Each defined broadcast domain
(or VLAN) is the set of devices attached to that domain that can receive Layer 2
broadcast packets (e.g., Ethernet frames) sent from any device within that domain.
Routers are typically used to bound the distinct broadcast domains (VLANs)
because routers do not forward broadcast frames—all broadcast traffic within a
VLAN is confined to that VLAN and cannot cross any attached router. VLANs
are designed to mimic the behavior of legacy shared-medium LAN segments
where broadcasts from any one device are seen by all other devices on that LAN
segment.
Layer 2 forwarding is relatively simple compared to Layer 3 forwarding
[AWEYA1BK18]. To perform Layer 3 forwarding, the switch/router needs a route
processor (also referred to as Layer 3 control engine) that runs the routing and
management protocols for the system. The routing protocols build and maintain
the routing tables required for Layer 3 forwarding. In addition to creating the rout-
ing and forwarding databases (tables), the route processor also performs all the
non-data transfer housekeeping functions for the system (system configuration,

3
4 Switch/Router Architectures

Switch/Router

Layer 3 Control Plane


Router
Layer 3 Data Plane
Layer 3 Forwarding

Layer 2 Control Plane

Layer 2 Data Plane


Layer 2
Forwarding

A4 B1

VLAN A VLAN B
A1

A3 B3
A2 B2

FIGURE 1.1 Layers 2 and 3 Forwarding in the Switch/Router.

monitoring, download of software to other modules, management of QoS and


security access control lists (ACL), and other tasks for packet processing).
A switch/router can use either flow-based or network topology-based informa-
tion for Layer 3 packet forwarding. The advantages and disadvantages of both
methods have been discussed in greater detail in [AWEYA1BK18]. Nonetheless,
we give a brief overview of these methods later to set the proper context for the
discussions that follow in the later chapters.

1.2 FLOW-BASED LAYER 3 FORWARDING


In flow-based Layer 3 forwarding, the switch/router maintains a flow/route cache
of the forwarding information of recently processed packets (e.g., destination IP
address, next-hop IP node (and its receiving interface MAC address), egress port,
and any other relevant forwarding information). To populate the flow/route cache,
the switch/router forwards the first packet in any flow to the route processor for
software-based processing and forwarding using the route processor’s master for-
warding table.
After the first packet of the flow is forwarded, the forwarding information used
by the route processor is used to populate the flow/route cache so that subsequent
packets of the same flow can be forwarded using the simpler and faster flow/route
The Switch/Router 5

Centralized Processing Module Centralized Processing Module

Route Routing Route Routing


Processor Table Processor Table

Forwarding Forwarding
Table Table

Flow/Route
Cache

Switch Fabric Switch Fabric

Network
Network Network Network Flow/ Interfaces
Interface Interface Interface Route Output
Cache Processing
First Packet Subsequent Packets
of a Flow of a Flow First Packet Subsequent Packets
of a Flow of a Flow

FIGURE 1.2  ayer 3 Forwarding Using Flow-Based Forwarding Table (or Flow/Route
L
Cache).

cache. The basic concepts of flow-based Layer 3 forwarding are illustrated in


Figure 1.2.
To enable high-speed Layer 3 packet forwarding, the switch/router typically
employs specialized application-specific integrated circuits (ASIC) to perform
the forwarding and all the relevant Layers 2 and 3 packet rewrite operations of the
forwarded packets. The main Layer 3 (or IP) packet rewrites include updating the
time-to-live (TTL) value and recalculating the IP checksum. The basic Layer 2
rewrites (assuming Ethernet is used) include rewriting the source MAC address
in the outgoing packet to be that of the egress interface, rewriting the destination
MAC address to be that of the receiving interface of the next-hop node, and recal-
culating the Ethernet checksum.
The Ethernet checksum recalculation is necessary because the source and des-
tination MAC addresses of the packet change as the packet traverses the switch/
router when forwarded at Layer 3. The switch/router is required to recalculate
the Ethernet checksum as these new MAC addresses are written in the outgoing
packet. The packet forwarding may include more rewrites such as adding VLAN
tags, updating IP and/or Ethernet packet class-of-service information, and so on.

1.3 NETWORK TOPOLOGY-BASED LAYER 3 FORWARDING


In topology-based forwarding, the route processor runs the routing protocols to
create the routing tables. Entries in the routing table can also be created manually
as static routes. The most relevant information needed for packet forwarding is
distilled from the routing table to generate the more compact forwarding table.
The forwarding table contains the same information needed to forward packets as
the routing table, the only difference is it contains only the information that can
6 Switch/Router Architectures

be used directly by a forwarding engine in forwarding packets—it excludes all


other information not needed for forwarding.
In topology-based forwarding, the forwarding engine performs Layer 3 for-
warding using the Layer 3 forwarding table (also called the forwarding information
base (FIB)) and Layer 2 rewrites using information maintained in the adjacency
table which is dynamically updated by Layer 2 address discovery protocols such
as the Address Resolution Protocol (ARP). Using the Layer 3 forwarding and
Layer 2 adjacency tables, the forwarding engine can quickly perform lookups for
forwarding information such as a packet’s next-hop IP address, egress port, and
MAC address of the receiving interface of the next-hop IP node (Figure 1.3).
For topology-based forwarding, the following two main databases are used by
the forwarding engine:

• Layer 3 Forwarding Table: The forwarding engine performs lookups


in a Layer 3 forwarding table for the forwarding information of a packet.
Each lookup is performed by extracting the IP destination address of
the packet and then making a longest prefix matching (LPM) search in
the forwarding table. LPM is more complex than lookups in a flow/route
cache and can be time consuming and processing intensive when per-
formed in software. High-speed, high-performance LPM searches are
generally done in hardware. Conceptually, the Layer 3 forwarding table
is similar to the routing table albeit rather compact and smaller. The
forwarding table contains the same forwarding information maintained
in the routing table. The routing table is updated (dynamically by the

Centralized Route Processor Module Integrated or Separate


Tables

Master
Routing Adjacency
Forwarding
Table Table
Table

Routing Route
Protocols Processor

Integrated Switch Fabric


or
Separate
Tables

Forwarding Forwarding Forwarding Forwarding


Table Engine Table Engine

Adjacency Adjacency
Table Input Output Table Input Output
Processing Processing Processing Processing

Line Card Line Card

Layer 3 Layer 3
Packets Packets

FIGURE 1.3 Layer 3 Forwarding Using Network Topology-Based Forwarding Table.


The Switch/Router 7

routing protocols) whenever topology or routing changes occur in the


network. The forwarding table is then immediately updated to reflect
these changes. The routing table and forwarding table must always be
kept synchronized as much as possible. The forwarding table maintains
information such as the next-hop IP address information and the corre-
sponding egress port on the switch/router.
• Adjacency Table: Two nodes in a network are considered adjacent
if they can reach each other over a single Layer 2 protocol hop (e.g.,
Ethernet, Point-to-Point Protocol (PPP), Asynchronous Transfer Mode
(ATM), IEEE 802.11, etc.). The adjacency table is used to maintain
Layer 3 address to Layer 2 address mapping. The adjacency table main-
tains a Layer 2 (e.g., MAC) address for every next-hop IP address in the
forwarding table. Before a packet is transmitted out its egress port, the
Layer 2 destination address in the outgoing packet is rewritten using
the Layer 2 address information read from the adjacency table. This
Layer 2 address is that of the receiving interface of the next-hop IP node
(the Layer 2 adjacency of the current node). The adjacency table can be
integrated into the Layer 3 forwarding table or implemented as a sepa-
rate table. However, integrating it into the forwarding table allows one
lookup to be performed to retrieve all forwarding information including
the Layer 2 adjacencies.

The topology-based forwarding model allows the separation (or decoupling) of the
control-plane functions (i.e., running routing and control protocols) from the data-
plane functions (i.e., forwarding table lookups and packet rewrites). Nevertheless,
the control-plane functions (running software in the route processor) are still
responsible for creating and maintaining the master forwarding and adjacency
tables and then downloading these to the data-plane functions (running in the
forwarding engine(s)).
A switch/router may support multiple route processors for redundancy pur-
poses (e.g., primary and secondary route processors running in active-active or
active-standby mode). A switch/router may also support multiple forwarding
engines running in a distributed manner each using a copy of the master forward-
ing table maintained by the route processor.
The forwarding engine may not be able to forward all packets it receives. These
special packets have to be forwarded to the route processor for further processing.
Examples of these special (or exemption) packets are:

• Control packets from routing protocols


• IP packets with IP header options
• IP packets requiring fragmentation
• Packets with IP time-to-live (TTL) expired
• Packets carrying ICMP echo requests (used to ping IP devices)
• IP packets coming from or destined to tunnel interfaces
• Packets requiring encryption, network address translation, etc.
8 Switch/Router Architectures

1.4 USING CENTRALIZED OR DISTRIBUTED


FORWARDING ENGINES
Packet forwarding can be done using one of two methods, centralized or
distributed.

1.4.1 Forwarding Using a Centralized Forwarding Engine


In centralized forwarding, a single centralized forwarding engine or a pool of
them perform all packet forwarding decisions for all packets received from all
network interfaces in the system. In addition to making forwarding decisions, the
centralized forwarding engine(s) perform the QoS and security ACL processing
and filtering as well as other data-path functions required in the system.
All packets entering the switch/router must pass through the centralized for-
warding engine to be processed. Incoming packets are passed from the network
interfaces over the switch fabric (which can be a shared-bus, shared-memory or
crossbar switch) to the central forwarding engine. Figure 1.4 illustrates the logical
architectures of centralized forwarding.
Some centralized forwarding architectures offload some amount of the data-
path processing to the line cards or interface modules by allowing them to for-
ward only the packet headers to the centralized forwarding engine [BRYAN93]
[CISCCAT6000]. The storage of the packet payloads and some packet rewrite
operations are carried out in the line cards. Examples of these kinds of architec-
ture are described in detail in [AWEYA1BK18] and also in Chapters 4 and 8 of
this book.

Centralized
Forwarding
Centralized Processing Module Route Processor Module Engine Module

Routing Routing Forwarding Routing Routing Forwarding


Protocols Table Table Protocols Table Table

Route Forwarding Route Forwarding


Processor Engine Processor Engine

Switch Fabric Switch Fabric

Network Network Network Network Network Network


Interface Interface Interface Interface Interface Interface

Layer 3 Layer 3 Layer 3 Layer 3


Packets Packets Packets Packets

FIGURE 1.4 Centralized Forwarding.


The Switch/Router 9

1.4.2 Forwarding Using Distributed Forwarding Engines


In a distributed forwarding architecture, multiple independent forwarding engines
are spread out in the system, located typically in the line cards (or network inter-
face modules). This allows the interfaces or line cards to make forwarding deci-
sions independently using their local copies of the forwarding tables downloaded
from the route processor (Figure 1.5). In this architecture, the centralized route
processor generates the master forwarding table but also ensures that the distrib-
uted forwarding tables are kept synchronized to the master table.
The route processor runs the routing protocols to create both the routing table
and the master Layer 3 forwarding table. The route processor then copies the
contents of its master tables to local forwarding tables used by the distributed for-
warding engines located on the line cards. This allows each line card to make for-
warding decisions independently without direct assistance from the centralized
route processor. An incoming packet is processed by its ingress line card and then
transferred directly across the switch fabric to its destination egress line card.
Each line card in the distributed architecture uses its copy of the master for-
warding table and adjacency table for forwarding packets. Some architectures
with local ARP processing capabilities on the line cards may allow the line card
to main a local ARP (or adjacency) table which is created and maintained by the
local ARP module. Other architectures may relegate all ARP processing to the
centralized route processor, which creates all adjacencies for the entire system.
Designers who aim to keep the cost and complexity of the line card low (by not

Centralized Route Processor Module

Master
Routing
Forwarding
Table
Table

Routing Route
Protocols Processor

Switch Fabric

Forwarding Forwarding Forwarding Forwarding


Table Engine Table Engine

Input Output Input Output


Processing Processing Processing Processing
Line Card Line Card

Layer 3 Layer 3
Packets Packets

FIGURE 1.5 Distributed Forwarding.


10 Switch/Router Architectures

including more processing beyond pure data-path processing) adopt this central-
ized ARP processing approach.
High-capacity, high-performance routing systems are generally based on dis-
tributed forwarding architectures. The forwarding performance and throughput
with distributed forwarding is equal to the aggregate throughput of all the dis-
tributed forwarding engines as long as the switch fabric is not a bottleneck. The
distributed forwarding architecture also allows each line card to be specifically
designed to support its own unique set of local functions and interfaces (encryp-
tion, network address translation capabilities, tunneling protocols, different types
of Layer 2 protocols and encapsulations, mix of interface types and speeds, etc.).
Each line card can be tailor made to meet specific design objectives without being
unnecessarily constrained by issues in other system modules.
The Catalyst 6500 with a Supervisor Engine 720 (discussed in Chapter 8)
supports distributed forwarding using a built-in crossbar switch fabric module
and line cards that have an integrated Distributed Forwarding Card (DFC). This
Catalyst 6500 switch/router still maintains a centralized forwarding engine (in a
module called the Policy Feature Card (PFC)) even when using line cards with
DFCs for backward-compatibility with older line cards that do not support the
DFC feature.

1.5 BUILDING THE LAYER 2 FORWARDING TABLES


This section describes how the Layer 2 forwarding engine constructs and main-
tains the Layer 2 forwarding table (also referred to as the MAC address table).
The MAC address table is constructed by reading the source MAC address of
arriving Ethernet frames. If the source MAC address and its associated switch
port are not listed in the address table, the forwarding engine enters that MAC
address, switch port, and VLAN in the table.
When the forwarding engine receives a frame with a destination MAC address
not already listed/entered in its MAC address table, it floods the frame to all switch
ports associated with the same VLAN (as the source MAC address) except the port
through which the frame was received. When the device associated with this desti-
nation MAC address eventually replies with its own frames, the forwarding engine
adds the source MAC address in the received frames plus the switch port through
which they were received to the MAC address table. The forwarding engine is then
able to forward subsequent frames with this destination MAC address to the switch
port just learned without flooding to all switch ports in the same VLAN.

1.5.1 MAC Address Aging


In the Policy Feature Cards (e.g., PFC3C and PFC3CXL) used in some Cisco
Catalyst 6500 switch/router Supervisor Engines, the MAC address table can
maintain up to 96,000 MAC address entries (64,000 address entries for other
PFC3 models), a space large enough to hold many addresses without flooding
frames belonging to unknown or unlisted MAC address entries. The Layer 2
Another random document with
no related content on Scribd:
,

645

in epilepsy,

500

in thermic fever,

397

Digiti mortui,

1252

1254

Diplopia in disseminated sclerosis,

875

876
in nervous diseases,

40

in tabes dorsalis,

829

in tumors of the brain,

1042

Dipsomania,

147

636

637

Disease of one lateral half of the spinal cord,


1165

Thomsen's,

461

Diseases, mental,

99

of nervous system, general semeiology of,

19

of peripheral nerves,

1176

of the membranes of the brain and spinal cord,

703

Dislocation in infantile paralysis,


1130

Disorders of sleep,

364

of speech,

568

Dizziness in general paralysis of the insane,

187

195

Double consciousness in nervous diseases,

28

vision in nervous diseases,


40

Doubting insanity,

170

Douche, cold, in catalepsy,

338

in ecstasy,

338

in spinal sclerosis,

903

Dreams (see

Sleep, and its Disorders

).
Dropsy of the head (see

Hydrocephalus, Chronic

),

740

sleeping,

383

Dura mater, cerebral congestion,

704

cerebral hæmatoma of (hemorrhagic pachymeningitis),

707

Definition, etiology, and symptoms,

707
Diagnosis,

709

Duration and pathology,

708

Prognosis,

709

Treatment,

710

cerebral inflammation of (pachymeningitis),

703

external pachymeningitis,

704

Diagnosis and prognosis,

706
Etiology and symptoms,

704

Pathological anatomy,

705

Treatment,

706

internal pachymeningitis,

706

Treatment,

707

spinal, acute inflammation of,

747
Duration of acute mania,

162

of acute myelitis,

821

of acute spinal meningitis,

718

of amyotrophic spinal sclerosis,

868

of catalepsy,

334

of chorea,

449

of chronic hydrocephalus,

743
of delirium tremens,

629

of ecstasy,

343

of family form of tabes dorsalis,

871

of hæmatoma of the dura mater,

708

of hysteria,

258

of hystero-epilepsy,

307

of progressive unilateral facial atrophy,


699

of spina bifida,

759

of symmetrical gangrene,

1261

of tabes dorsalis,

839

840

of tubercular meningitis,

729

of tumors of the brain,

1045

of the spinal cord,


1106

of vertigo,

418

of writers' cramp,

521

Dyskinesis, definition, in nervous diseases,

47

Dyslalia,

569

572

Dyspepsia, headache from,


405

Dyspeptic symptoms of chronic alcoholism,

601

607

Dysphagia, hysterical,

239

245

E.

Ear affections, hysterical,

249
influence on causation of acute pachymeningitis,

716

of external pachymeningitis,

704

disorders of, as a cause of abscess of the brain,

474

of epilepsy,

474

of vertigo,

421

in progressive unilateral facial atrophy,

698

middle, disease of, as a cause of thrombosis of cerebral veins and


sinuses,
985

Eclampsia,

464

Definition, etiology, and symptoms,

464

CSTASY

339

Definition,

339

Course, diagnosis, and duration,

343
Etiology,

341

History and synonyms,

339

Prognosis,

343

Symptoms,

342

Treatment,

344

Eczema as a cause of chorea,

444
Education, improper, as a cause of hysteria,

218

220

of speech in aphasia of hemiplegia,

979

relation of, to hysteria,

274

Electrical reactions in amyotrophic lateral sclerosis,

868

in Bell's palsy,

1205

1206
in diffuse sclerosis,

890

in infantile paralysis,

1125

in spastic spinal paralysis,

864

of injured and divided nerves,

1184

1188

Electricity, use of, in Bell's palsy,

1207
in catalepsy,

338

in chronic lead-poisoning,

691

in hysteria,

281

286

in hystero-epilepsy,

311-313

in infantile spinal paralysis,

1156

in labio-glosso-laryngeal paralysis,

1175
in migraine,

415

1232

in multiple neuritis,

1198

in myelitis, acute,

824

in myxœdema,

1273

in nerve injuries,

1189

in neuralgia,

1225

You might also like