Whitepaper

Large surveillance systems

System Overview Document

Table of contents
Overview

1. Purpose and motivation

2. Overview of the networks

2.1 Outdoor network

2.1.1 The Core

2.1.2 The Outdoor Stations

2.1.3 Milestone VMS

2.2 Indoor network

2.2.1 The Control Center

2.2.2
2.2.3
2.2.4
2.2.5

The Core
The floors
Allied Telesis switch network configuration
Genetec VMS

3. Further reading

Appendix I
Appendix II

4
6
6

7
7
7
9
9
10

10
11
14

Overview
During May 27-31, 2013, Axis Communications AB hosted a global technical conference with participation
and cooperation from partners including ABB, Allied Telesis, Firetide, Genetec, HP, NetApp, Microsemi,
Milestone, Moodifier, Raytec, Veracity, and Weidmuller. The conference theme was Large Surveillance
Installations. Two separate and fully operational Video Surveillance Systems (VSS), deploying different
technologies, was set up and tested; one indoor network and one outdoor network, each with 1000
cameras.
It is important to know that the naming of the 2 systems depends only on the cameras. In the outdoor
system both indoor and outdoor Axis cameras were used, whereas in the indoor system only indoor
cameras were used. The other components in the VSS works well with both indoor and outdoor
installations.
This paper describes the purpose and goal of the conference and provides an overview of the two video
surveillance systems. More detailed papers on the configuration and setup of each system will also be
made available.

1. Purpose and motivation

Video surveillance has moved to the forefront globally to serve the security needs within military, government, utilities, corporate, medical, retail, hospitality, transportation, education, and more.
Today, large IP Video Surveillance Systems (VSS) consisting of over 1000 cameras are becoming more
common. In a large campus network, or Metro-Area Network (MAN), the IP video surveillance system is
typically overlaid on an infrastructure that has been designed to carry multiple applications and services.
Such a network, supporting a significant number of end users, needs to be very reliable, manageable, and
scalable.
The demand from users of a system that delivers high-resolution, high-definition, real-time results also
put higher requirements on a reliable high-speed IP network infrastructure. To ensure this, network equipment, cameras, the server/data center, the monitoring center, as well as the Video Management Systems
(VMS) all must be working together effectively in harmony. This presents great challenges for the system
integrators and engineers. There is no one solution that fits all.
During the Global Technical Conference, a state-of-the-art, large scale digital video surveillance system
with 1000 cameras was deployed. The purpose of the deployment was to highlight the critical components
of this kind of system through testing and troubleshooting to identify bottlenecks, limitations for network
design, configuration, reliability and scalability.
Another objective was to clearly identify building blocks that can be easily copied in order to design a
highly reliable, manageable, as well as scalable system. During the conference, tests to identify the minimum memory and storage configuration to support such large networks were conducted. As two separate
technical solutions were set up, we had the ability to compare the performance of Blade versus Rack
servers. We also tested how to make better use of the servers by adding on Hypervisors to increasing the
number of VMS recording servers per physical server. These findings are presented in two seperate white
papers. Please refer to Large survaillance systems - An Allied Telesis, Axis, Genetec and NetApp goint
solution guide and Large survaillance systems - An Axis, HP and Milestone solution guide for more
details. The system integrators and engineers will be able to refer to these papers as solution guidelines.
The stakeholders of the event were experts of leading companies within the following fields: IP cameras,
network switching, computer servers, fiber interfaces, VMS providers, wireless manufacturers, structured
cabling and network storage.

2. Overview of the networks

During the conference, two completely separate networks supporting two VSS was configured and tested
with different technologies. The two VSS setups varied in both hardware and software. One network was
installed for outdoor - and the other one for indoor environment. Each network consisted of 1000 cameras. It was not physically possible to test with 1000 cameras during the one week conference period.
To build the 1000 camera solution a combination of real- and simulated cameras were used. Axis Virtual
Camera (AVC), a software application that generates multiple virtual cameras streams from an actual
physical camera was used to replicate. About 50 to 70 physical cameras were installed in each network;
the rest were virtual cameras generated to simulate the actual data traffic and bandwidth requirement for
such a large system.

2.1

Outdoor Network
The network was divided in different outdoor stations connected via fiber, wireless and copper. The core
of the network was located in a customized 40ft shipping container. The container contained servers,
storage, core switch, power supply, workstation, video screens, cameras, lighting, cooling and heating
system, i.e. all the necessary equipment to support the 1000 camera network. The network had 52 physical cameras installed. Of these, 22 of them were installed inside the container and 30 were distributed
among the four outside stations. Please see the diagram in Appendix II for details. The rest of the cameras were simulated through Axis Virtual Cameras (AVC).

2.1.1

The Core
The core of the network consisted of two racks with the equipment illustrated in Figure 1
Figure 1. The core of the network consisted of two racks with the equipment.

1. HP BL460c Gen8 E5-2660 2P 96GB blade servers

2. HP BLc7000 Blade enclosure, the enclosure supports up to 16 blade servers.

At the conference, seven HP BL460c Gen8 E5-2660 2P 96GB blade servers was mounted in the blade
enclosure. On each server, Hypervisor from Microsoft HyperV was used to manage the virtual ma
chines. Each server ran 2 virtual machines with Windows Server 2008 and Milestone XProtect
Corporate software VMS on top to manage the video data. Each virtual machine supported 150 cameras.

3. HP 3PAR P7400 SAN Storage System

4. HP R12000/3 UPS
5. Microsemi PD-9524G/ACDC/M-1 Midspan Rack Power injector 60w per port X 24 ports
6. Veracity rack with 2 Longspan Long Range Ethernet and PoE extended Ethernet range up to 600
meters with PoE
7. HP 3800-48G-PoE+-4SFP+ Core Switch connecting camera data streams to the servers
8. HP Z420 Workstation
9. AXIS Q7900 Rack with 2 T8647 coax converters for PoE over coax, and 1 Q7406 encoder
blade for coax connection to analog cameras
10. Power supply for coax conversion
11. 3 HP DL360e Gen8 E5-2430 8SFF Perf EU Servers for AVC
The outdoor network had a centralized virtualization of cameras. 52 cameras were connected though
access switches to the system and the virtualization was managed at a single location from these
three AVC servers.
Figure 2 provides a rear view of the HP network, server and storage solution in the rack.

Figure 2. Rear view of the HP network.

1. The HP 3800-48G-PoE+-4SFP+ core Switch is connected to 4 HP 2530-24G-PoE+ networks switches

coming from the 4 outdoor stations
2. The HP Blade System c7000 enclosure with 7 server blades and 2 Virtual Connect FlexFabric modules,
installed in I/O Bay 1 & 2
3. HP 3PAR P7400 SAN array using the FlexFabric feature of the HP Blade system, eliminating the need
for SAN switches in the configuration
The configuration provided a highly reliable solution that used a small number of 10Gb uplinks into the
network from the blade system and four fiber cables to connect the storage array holding the video data.
As can be seen in the configuration shown in Figure 2, using this technology, the performance required by
1000 cameras can be contained within a single rack. Only four cables were coming from the rack to the
network, as the uplinks to the cameras and all other network and SAN cables required by the multiple rack
servers were configured inside the blade server through a single web-based interface. As a consequence,
the power and cooling required, as well as the floor space, are radically decreased. This makes the setup a
much greener solution for a typical customer environment.

2.1.2

The Outdoor Stations

A total of four outdoor stations with different cameras were set up to run different workshops for testing
the Axis camera products. Each of the stations was connect through 1 Gbps copper cable from a HP
2530-24G-PoE+ Switch to the core switch in the container. The stations were also daisy-chained together via fiber cable. A total of 30 cameras were installed in the four outdoor stations.

2.1.3

Milestone VMS
For the outdoor network the Milestone XProtect Corporate suggested solution was deployed to manage
and operate 1000+ cameras. Figure 3 illustrates this solution.

Figure 3. The outdoor network solution by Milestone XProtect

As the foundation for this, we provided a standard installation, with one physical Management Server /
SQL Server, five physical Recording Servers with two virtual machines of each server. In addition, the
setup included one Fail-over server with two virtual machines.
We even installed and configured a Mobile Server on the same machine as the Fail-over Server in order to
provide a wireless network (FireTide). This allowed for clients connecting to the Mobile Server.
To demonstrate the options for centralized solutions, we also will installed Milestones Smart-wall solution
which addressed the market for tactical/situation rooms, monitoring centers, etc.
Once operational we were able to demonstrate features such as:
> Easy-to-use daily operation using XProtect Smart Client and Smart Wall
> Scenarios for using SD card in cameras
> Secure connections and exports using HTTPS
> How to change settings for multiple cameras in one shot!
> Cost-efficient solutions using 64-bit technology and multi-stage storage
> Alarm management

2.2

Indoor Network
In order to recreate the processing and bandwidth demands of a 1000 camera indoor network, 68 physical cameras were installed and distributed over four Axis buildings in Lund. AVCs were installed on one HP
DL360p server and laptops in the network to replicate streams from actual cameras. The indoor network
was managed from the control center on the first floor of the J building in Axis headquarters at Emdalavgen (J1). This network used multicasting and all switches supported IGMP.

2.2.1

The Control Center

The control center was located in J1. There was a video wall with five HP monitors, eight cameras and
eight AT-8000GS/24PoE or AT-x610-24Ts-PoE+ in two lab rooms. Two HP Z420 workstations with Genetec Security Center client software were also installed in the control center.

2.2.2

The Core
The core of the indoor network was located in the H building (H0) and consisted of five high-end servers
by HP, running Windows Server 2008. There was a storage system by NetApp, with raw capacity of 180
TB, and a core switch with two SwitchBlade x908s switches in stack configuration by Allied Telesis. The
switches provided connectivity to the system elements, hosts and cameras. Figure 4 illustrates the rack
view.
1. One HP DL360p Gen8 Server. The server ran Windows server 2008 with AVC and NetApp SANtricity
10.84 on top. The SANtricity management client configured and administrated the NetApp E-5460
storage system installed in the core of the indoor network.
2. Five HP DL 360p Gen8 E5-2650 servers. On each server VMware ESXi 5.1.0 U1 HP branded release was
installed as the hypervisor. Each server was virtualized into four virtual machines running Windows
Server 2008 with Genetec Security Center 5.2 VMS on top. Each virtual machine had four file archivers
with 240 Mb/s throughput per archiver handling 50 cameras per archiver.

Figure 4. The rack view of the core of the indoor network.

3. The core switch in stacking configuration with two Allied Telesis SwitchBlade x908. This powerful Allied
Telesis core switch was an extremely feature-rich Quality of Service switch that can manage the
characteristics of over 1,000 separate data streams simultaneously. This made the servers ideal for
the provisioning of shared service networks involving real-time applications at low-latency switching
capability. The two AT-SBx908 switches formed the virtual network core. 16 access network swit-
ces were connected to the core with two aggregated links, one to each member of the stack. The
use of link aggregation was made possible because the two switches were in a single virtual chassis.
Such a topology creates a loop-free network with link, and core switch carrier-class resiliency. From
the core switch, four fiber cables connected towards NetApp Storage System E5460. Each port
could work up to 10Gb/s and supports iSCSI protocol. The core switches could easily be configured
by CLI (Command Line Interface) or Web browser with industry standard syntax command configuration.
4. NetApp Storage System E5460, the NetApp E-Series storage array, is targeted for the video surveil
lance market. It is a sixth-generation storage array that includes patented mechanical engineering,
providing dense, scalable, and highly reliable bandwidth and capacity. The disk controller firmware
supports an optimal mix of high bandwidth, large-block streaming and small-block random I/O.

The E5460 is targeted at Fiber Channel deployments. The solution deploys dual controllers for high
availability. All components of the E-Series are hot swappable; firmware upgrades can be completed
while the system is operational. The E5460 also features dual-active controller with automated I/O
path failover. Both controllers have a data path to all shelves and drives in the array. Both controller
models deploy cache memory for read and write buffering. In the rack, four Twinax Direct Attach SFP
+ cables connect the 10 Gb/s fiber ports in the two controller cards back to the core switchs optical
interfaces. A total of 60 disk drives are installed in the enclosure, 58x HDD spindles with 3TB capacity
each and two 800 GB Solid State Drives each.

2.2.3

The floors
A total of 68 physical cameras and 20 laptops ran AVC to generate additional cameras streams to meet
the 1000 camera network bandwidth usage. Cameras and laptops were placed out on the 14 floors of the
Axis Communications HQ, in the F, G, H and J buildings. On each floor, one ATGS950/16PS network
switch was installed to connect to the cameras and laptops though copper cables. These switches were
then connected via 1Gbs fiber cables to the patch panels of each building and further connected to the
core two SwitchBlade x908s through 1Gbs fiber cable.

2.2.4

Allied Telesis switch network configuration

To support the IP video network and enable remote camera control and operation, a high-speed IP/Ethernet network with its design and functionality tailored to IP video applications was required.
The requirements for this kind of network infrastructure are best met by a network design in which different services are split into different VLANs, and transported over resilient links or rings protected by an
extremely fast failover mechanism.
Figure 5 illustrates the network configuration proposed by Allied Telesis.

2x1 Gbps
BD fiber
LAG

20x1 Gbps

8x1 Gbps
BD fiber

HP servers
DL380e Gen8
Genetec Server

x90

ERA

CAM

5x

5x

5x
ERA

CAM

5x

ERA

CAM

5x

ild

Bu

5x

in

ERA

CAM

5x

4x10 Gbps

ERA

CAM

5x

5x

H
g
in r
ild lla
Bu ce

5x

5x
ERA

CAM

5x

ERA

CAM

5x

in

ild

Bu

5x

NetApp E5400
Storage system

5x

ERA

CAM

ERA

CAM

5x

5x

5x

5x
ERA

CAM

5x

ERA

CAM

5x

in

ild

Bu

5x

5x

ERA

CAM

5x

5x

ERA

CAM

5x

5x

H
g
in
ild oor
Bu fl

5x

ERA

CAM

5x

5x

ERA

CAM

5x

ERA

CAM

5x

Figure 5. The network configuration by Allied Telesis

As real-time video is sensitive to latency and buffering, no service downtime is allowed. The Allied Telesis
fast failover network protection solution is the Ethernet Protected Switched Ring (EPSR, a ring protection
protocol) and Virtual Chassis Stack (VCStack). Please see more information in WP3. These are extremely
reliable, high-performance protection mechanisms that can restore connectivity within <50ms of a link
failure being detected. Services such as IP video surveillance can each be provisioned with one or more
VLANs running over the EPSR rings or Aggregated Links (LAGs) with data on Layer 2 or Layer 3 switched
between the rings, the links and the central site facility.
A reliable, scalable design was achieved by subtending multiple rings of two SwitchBlade x908s with
VCStack providing the gateway between the rings and central site.

Robust access as AT-GS950/16PS WebSmart network switches or AT-x610/14TsPoE+, both featuring PoE+
for connectivity with security cameras integrating simplicity with the performance and reliability of managed switch; thus providing an inexpensive yet secure and reliable solution for users to integrate management at the edge networks.

2.2.5

Genetec VMS
The VMS chosen for the indoor surveillance system was Genetec Security Center 5.2. It was dimensioned
to manage about 1000 cameras by 20 archivers distributed over five physical servers. Please see Appendix
I for more information of Genetecs system design.

3. Further reading
The purpose of this paper has been to provide an overview of the actual setup of the two systems at the
conference. To learn more about installation, configurations, and best practices, please refer to the white
papers Large survaillance systems - An Allied Telesis, Axis, Genetec and NetApp goint solution guide and
Large survaillance systems - An Axis, HP and Milestone solution guide.

10

Appendix I

Staying
one step
ahead

Follow the

Secure your goals

with the world leader in network video

leading expert

Enit am ationsed tionsequat

> Dolore t ip ea faci bla
> Lobore digna fac umm odigna
> Henisl ut niamet augait luptatet
1. HP BL460c Gen8 E5-2660 2P 64GB servers
> 7 x Blade servers
> Handling 150 cameras per server
> Milestone XProtect Corporate
> Microsoft Hyper-V
2. HP BLc7000 Blade enclosure
3. HP 3PAR P7400 SAN Storage
4. HP R12000/3 UPS
5. Microsemi PD-9524G/ACDC/M-1
> Midspan Rack Power injector 60w per port X 24 ports
6. HP 3800-48G-PoE+-4SFP+ Core Switch
> Connecting camera data streams to the servers
7. HP Z420 Workstation
8. AXIS Q7900 Rack with
> 2 x T8647 Coax converters, PoE over coax
> 1 x Q7406 encoder blade for coax connection to analog cameras
9. HP DL360e Gen8 E5-2430 8SFF Perf EU Server for AVC

Figure 1. Outdoor Network Container view

Figure 2. Outdoor Network Container Rack View

11

Figure 3. Outdoor Network Milestone VMS Configuration

Figure 4. Outdoor Network Station View

12

Figure 5. Indoor Network Diagram

13

Appendix II

Axis Global Conference

Genetec Security Center system

The Security Center is a unified security platform. It seamlessly blends Genetec s IP license plate
recognition, video surveillance, and access control systems into one innovative yet simple solution.
With an intuitive interface, the Security Center facilitates the seamless management of multiple security
and safety systems, regardless of installation size.

Omnicast is the IP video surveillance system of the Security Center

platform that provides seamless management of digital video,
audio and metadata. Its unique design enables the management of
cameras, monitors, recorders and peripherals through any existing
IP network. In contrast to a DVR or matrix system, the Omnicast
system is massively distributed over the network ensuring layout
flexibility and high fault tolerance. Manipulating video in digital
formats offers tremendous flexibility in all aspects of video
management, including but not limited to, available video quality,
multi-streaming, and archive management.

Server & Client Architecture

Directory manages the system configurations, the
events, alarms and audit trails, events
Media router role manages the optimal route
used to send a stream from its source (Archiver or
camera) to its destination (Security Desk)
Archiver manages the command and control
of video units, the management of the video
archives, motion detection and Video redirection
when necessary.
Security center is an open storage platform.
Netapp will be used for this deployment.

Client applications
Video Architecture for Axis Global Conference
Security Center Omnicasts unique design offers the freedom to
design a system that truly matches your security needs. Omnicast
supports a very wide range of IP units and CCTV equipment.

Security desk manages: live video and events

monitoring, video and events investigation, alarm
management and report generation
Config tool allows to configure the video system.

Innovative Solutions

14

System Design Assumptions

Network

The Archiver Server

A fully IGMPv2 & v3-compliant multicast network backbone is

required for this system architecture.

The Archiver server and bandwidth requirements have been

calculated with the following assumptions:
All cameras shall record continuously and display the same
video quality: H264, 30 fps, maximum resolution.

Archiver SQL Databases

Archiver SQL databases are to be stored locally on each server
to improve performance. SQL Express will be used for each
archiver.

The maximum network throughput is 5 Mbps per camera.

21 archivers are necessary to handle the 1000 cameras in a
virtualized environment. An additional server will be used for
the Directory.

Server Requirements

Client Workstations Requirements

High-End server hardware is required for each Genetec server.

Virtualization will be used in this system design. Vmware ESXi
5.1 will be used. Dedicated NIC cards are necessary for each
virtual machine set up.

A number of 30 clients workstations will be deployed and they

shall meet the system requirements shown below.

Each Genetec server shall meet the minimum system

requirements shown below.

8 GB of RAM or better

Intel Core i7 2600 @ 3.4 Ghz or better

80 GB SATA II hard drive for OS and Security applications

Quad core Intel Xeon E5640 2.66 Ghz or better

1 GB PCI-Express X 16 dual head video adapter

8 GB of RAM or better

1600X1200 or higher screen resolution

80 GB SATA II hard drive or better for the OS and Security

Center applications

100/1000 Ethernet network interface card

Standard SVGA video card

1024x768 or higher screen resolution
100/1000 Ethernet network interface card
DVD ROM drive
Windows Server 2008 SP2/R2 (64-bit)

Genetec
2280 Alfred-Nobel Blvd., Suite 400,
Montreal, QC, Canada H4S 2A4
T 514.332.4000
F 514.332.1692
genetec.com
[email protected]

2013 Genetec. All rights reserved. Synergis, Omnicast, Security Center, AutoVu, Genetec, and the Genetec
logo are either registered trademarks or trademarks of Genetec. All other trademarks contained herein are
the property of their respective owners.

15

53062/EN/R2/1409

About Axis Communications

Axis offers intelligent security solutions that enable a smarter, safer world. As the global market
leader in network video, Axis is driving the industry by continually launching innovative network
products based on an open platform - delivering high value to customers through a global
partner network. Axis has long-term relationships with partners and provides them with
knowledge and ground-breaking network products in existing and new markets.
Axis has more than 1,600 dedicated employees in more than 40 countries around the world,
supported by a network of over 65,000 partners across 179 countries. Founded in 1984, Axis is a
Sweden-based company listed on NASDAQ OMX Stockholm under the ticker AXIS.
For more information about Axis, please visit our website www.axis.com.

2014 Axis Communications AB. AXIS COMMUNICATIONS, AXIS, ETRAX, ARTPEC and VAPIX are registered trademarks or trademark applications of
Axis AB in various jurisdictions. All other company names and products are trademarks or registered trademarks of their respective companies. We
reserve the right to introduce modifications without notice.