VPLEX Installation and Config Guide
VPLEX Installation and Config Guide
VPLEX Installation and Config Guide
Copyright 1996, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012 EMC Corporation. All
Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is
subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR
WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS
IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
EMC2, EMC, Data Domain, RSA, EMC Centera, EMC ControlCenter, EMC LifeLine, EMC OnCourse, EMC Proven, EMC Snap,
EMC SourceOne, EMC Storage Administrator, Acartus, Access Logix, AdvantEdge, AlphaStor, ApplicationXtender,
ArchiveXtender, Atmos, Authentica, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap,
AVALONidm, Avamar, Captiva, Catalog Solution, C-Clip, Celerra, Celerra Replicator, Centera, CenterStage, CentraStar,
ClaimPack, ClaimsEditor, CLARiiON, ClientPak, Codebook Correlation Technology, Common Information Model,
Configuration Intelligence, Configuresoft, Connectrix, CopyCross, CopyPoint, Dantz, DatabaseXtender, Direct Matrix
Architecture, DiskXtender, DiskXtender 2000, Document Sciences, Documentum, elnput, E-Lab, EmailXaminer,
EmailXtender, Enginuity, eRoom, Event Explorer, FarPoint, FirstPass, FLARE, FormWare, Geosynchrony, Global File
Virtualization, Graphic Visualization, Greenplum, HighRoad, HomeBase, InfoMover, Infoscape, Infra, InputAccel, InputAccel
Express, Invista, Ionix, ISIS, Max Retriever, MediaStor, MirrorView, Navisphere, NetWorker, nLayers, OnAlert, OpenScale,
PixTools, Powerlink, PowerPath, PowerSnap, QuickScan, Rainfinity, RepliCare, RepliStor, ResourcePak, Retrospect, RSA, the
RSA logo, SafeLine, SAN Advisor, SAN Copy, SAN Manager, Smarts, SnapImage, SnapSure, SnapView, SRDF, StorageScope,
SupportMate, SymmAPI, SymmEnabler, Symmetrix, Symmetrix DMX, Symmetrix VMAX, TimeFinder, UltraFlex, UltraPoint,
UltraScale, Unisphere, VMAX, Vblock, Viewlets, Virtual Matrix, Virtual Matrix Architecture, Virtual Provisioning, VisualSAN,
VisualSRM, Voyence, VPLEX, VSAM-Assist, WebXtender, xPression, xPresso, YottaYotta, the EMC logo, and where
information lives, are registered trademarks or trademarks of EMC Corporation in the United States and other countries.
All other trademarks used herein are the property of their respective owners.
Copyright 2012 EMC Corporation. All rights reserved. Published in the USA.
This course covers how to perform VPLEX Cluster hardware installations. It is intended for
anyone that will perform rack and stack of VPLEX Clusters. Also covered are, system
initialization, Management IP configuration, pre -configuration upgrades, and setting up ESRS
in the system. This course supports VPLEX GeoSynchrony 5.1.
This course is intended for rack and stack and system initialization of VPLEX Clusters.
The objectives for this course are shown here.
Upon completion of this course, you should be able to perform a basic installation of a VPLEX
Cluster.
This module focuses on how to validate the components required for installation.
VPLEX release notes provide information on current documentation available with VPLEX. It
also lists the documents in order that should be used to install and setup a new VPLEX. The
release notes also display the current software versions that are installed with the product. It
also displays the product interoperability with storage arrays, host operating systems and
software, High-availability, and switches. The release notes also list known problems and
limitations and configuration limits.
The VPLEX Site Preparation guide describes the requirements and steps to prepare a
customer site for installation of a VPLEX cluster.
The EMC 40U-C Cabinet Unpacking and Setup guide is an illustrated guide that shows
how to remove EMCs 40U-C cabinet from the shipping pallet and prepare it for
installation.
The VPLEX Procedure Generator is a Windows program that can be run to generate
the current Installation and Preparation Guide, the VPLEX NDU guide, and VPLEX
Hardware replacement procedures.
The Security Configuration guide list VPLEX user accounts. It also explains how to
access a VPLEX cluster and where the VPLEX log files are located.
The Installation and Preparation guide describes how to install and configure a VPLEX
cluster and VPLEX Metro. It also displays basic diagrams of a small, medium, and large
VPLEX configuration. It also provides a brief description of hardware components.
Copyright 2012 EMC Corporation. Do not copy - All Rights Reserved.
Listed here are the configuration tasks for a VPLEX Metro or Geo. These steps are also listed
in the VPLEX Configuration Guide. It is very important to follow these tasks in order when
configuring a VPLEX cluster.
Configuration flow for a Local VPLEX is similar to the steps listed in Cluster 1. However, in this
course, we will review the more complex VPLEX Metro configuration.
Please note that the BE ports do not log into the switch until you start and progress through
the EZ Setup wizard. This is important to understand since you would typically try to prestage the exposure of back-end storage. One way to get around this issue is to utilize the
VPLEX WWN Calculator. This is covered later in this module.
It is best practice to always refer to the VPLEX Configuration Guide for the latest installation
information.
The VPLEX Configuration worksheet identifies information that can be gathered before
installing an EMC VPLEX Local or VPLEX Metro implementation. Availability of this
information at the start of an installation helps to ensure smooth and efficient installation
and setup procedures. This document is a supplement to the EMC Installation and
Preparation Guide, and is provided in Microsoft Word format to provide the ability for
anyone with Word to type the information into the tables. Complete the worksheet prior to
starting the Installation and Preparation.
Shown here is a list of hardware and software that is useful when installing VPLEX.
The VPLEX Procedure Generator provides a way to create various VPLEX procedures. Once
the VPLEX PG is installed, it never needs to be updated. The VPLEX PG automatically
downloads and installs new repositories when they are available.
The VPLEX equipment needs time to become acclimated to a data center. Temperature and
humidity values must ensure that condensation does not occur on any VPLEX components.
The altitude and atmospheric pressure specs are referenced to a standard day at 57F
(14.8C). Maximum wet bulb is 82F (28C).
If components are loose, the orange captive screws should be loosened, the part should be
pushed firmly into the backplane and the thumb screws should be tightened. The required
tools are listed in the EMC 40U-C Cabinet Unpacking and Setup guide.
Listed here are general external cabling requirements for a VPLEX installation.
This table shows the port layout of a VPLEX engine. This pictures also displays the way each
director should be connected to the fabric. Here, front-end port A0-FC00 of Director A is
connected to Fabric A and front-end port on Director B, B0-FC00 is also connected to Fabric
A.
The next two ports on each director, A0-FC01 and B0-FC01 are connected to Fabric B.
Connecting VPLEX to the customers fabrics as shown helps to evenly distribute the paths
between the directors. The front-end and back-end connections should also be spread across
I/O modules. This prevents data unavailability if a single I/O module fails.
If configuring a VPLEX Metro, each director in each cluster should be connected to the intercluster SAN as shown.
If configuring a VPLEX Metro, each director in each cluster should be connected to the intercluster SAN as shown. This example shows a fully populated VPLEX Metro with four engines
in each cluster.
This table shows the port layout of a VPLEX engine. This pictures also displays the way each
director should be connected to the fabric. Here front-end port A0-FC00 of Director A is
connected to Fabric A and front-end port on Director B, B0-FC00 is also connected to Fabric
A.
The next two ports on each director, A0-FC01 and B0-FC01 are connected to Fabric B.
Connecting VPLEX to the customers fabrics as shown helps to evenly distribute the paths
between the directors. The front-end and back-end connections should also be spread across
I/O modules. This prevents data unavailability if a single I/O module fails.
If configuring a VPLEX Metro, each director in each cluster should be connected to the intercluster SAN as shown.
If configuring a VPLEX Geo, each director in each cluster should be connected to the intercluster WAN as shown.
The fabric cables should be routed horizontally to the sides of the cabinet as shown. This
prevents blocking other components in the cabinet. The cables should be routed down the
sides of the cabinet through the plastic cable holders. The cables should be secured with the
Velcro straps that are attached to the cabinet frame.
Review the EMC VPLEX Simple Support Matrix (available on Powerlink) for specific array requirements.
Download and complete the VPLEX with GeoSynchrony 5.1 Pre-installation Configuration Worksheet,
which is available with other VPLEX documentation on EMC Powerlink.
Prior to installation there are several items that need to be verified or completed:
Redundant fabrics need to be in place for connection to the VPLEX I/O ports. Each
director should connect to both fabrics with both the front-end and back-end ports.
Review the Implementation and Planning Best Practices for EMC VPLEX Technical
Notes.
The Management Server has four Ethernet ports. The diagram shows the location of the
Service LAN port and the Public LAN port. The two other ports are shipped pre-cabled for
internal connection to the directors. The service LAN port is always 128.221.252.2.
Therefore, connecting a laptop to the service LAN port allows access to administer a VPLEX.
To initially connect to the VPLEX, the user must use SSH utility and the protocol version must
be set to 2. The user must also login using the service account. It is also important to disable
any other network ports on your server laptop when connecting to the Management Server
as it could cause issues. The public LAN port, eth3 should be connected to the customers
IP network. This IP address is set at a later time in the installation.
After connecting to the Management Server, the VPlexcli process should be verified that it is
running. The Management Server hostname can then be set using the command listed.
To set the Management Server IP address, the user must login to the VPlexcli. Each time the
VPLEX Management Console is started, a new set of firmware logs are created. Each log has
the filename, firmware.log_x. This identifies the exact time the Management
Console started. The Management Server should be configured with a public IP on the
customers network. This allows the cluster to remotely manage VPLEX over the network.
This slide lists the high level steps for running the EZ-Setup Wizard. The EZ-Setup Wizard
performs several tasks on a cluster.
To begin the VPLEX installation, the EZ-Setup Wizard is used. This wizard is invoked by using
the configuration system-setup command. Notice the wizard starts by asking you to read the
VPLEX Installation Guide and Release Notes before you proceed with the installation.
To continue with the configuration type Yes.
The VPLEX Setup Wizard needs to be run to perform initial configuration of a VPLEX single
cluster, VPLEX Metro, and Geo. The VPLEX Setup Wizard will take the installer through a
series of questions about the VPLEX cluster. It will then automatically configure the system
according to the answers supplied. Since we will be configuring a VPLEX metro, select option
2.
The next option is to select the first or second cluster to configure. Option 1 must be
selected. If for some reason the EZ-Setup Wizard fails, the manual installation procedures
must be used to finish the VPLEX configuration.
The number of engines in the system must be specified. This example shows a small
configuration and therefore the number of engines specified is 1. VPLEX System Events and
Reports can be configured to be sent to an ESRS Gateway during initial configuration of a
VPLEX cluster.
The next step is to configure the CA and host certificate. These certificates are used to
configured the VPN between two Management Servers in a VPLEX Metro. They are also used
for GUI web server access. By default, the host certificate is set to expire in two years and the
CA certificate is set to expire in 5 years. These default values can be modified if required
during these steps.
The passphrases may be the same for all certificates in a VPLEX Metro configuration however
for best security practice you should use the same CA passphrase in both clusters and unique
Host passphrases in each cluster.
After configuring the certificates, the VPLEX EZ-Setup Wizard will run through the rest of the
configuration. It will set the directors names, configure the cluster ID, enable VPLEX back-end
ports, commission the directors, and many other configuration tasks. At the end of the
configuration, the automated configuration steps are done for a single cluster install.
To zone storage to VPLEX, you need to know the port WWNs of the back-end VPLEX ports,
which are the ports that connect to the storage arrays.
The WWN calculator is installed. The installer also can create a desktop icon.
The VPLEX Port WWN Calculator encodes the VPLEX seed number to WWPN's. All WWPN's
and WWNN's can be reported in Microsoft Excel from the GUI.
The VPLEX seed number can be located from the front of the cabinet by removing the
engines bezel.
Locate the Seed Number Location from the VPlexcli command displayed.
Zone the storage to the VPLEX back-end ports, following the recommendations in the
Implementation and Planning Best Practices for EMC VPLEX Technical Notes.
Notice in this step, VPLEX has only been configured on one cluster, therefore only the
director WWNs for that cluster will be visible from the switch. You can manually zone the
second cluster but the VPLEX directors will not log in until you run the wizard.
When the VPLEX back-end ports are zoned to the storage array, perform LUN Masking
operations. When LUNs have been provisioned to the VPLEX, continue the setup. This part of
the EZ-Setup finishes by re-discovering the back-end arrays. Wait several minutes for the rediscovery to complete before you proceed to the next task.
Each VPLEX Engine requires 4 meta-data volumes where two work in a mirror and the other
two are used for backup. When possible, it is recommended that the meta-data volumes are
located in different storage arrays. It is required to have two 78 GB LUNs ready for the
Metadata volume. These LUNs are used to create the Metadata volume.
Two more volume are required for backup volumes.
From the VPlexcli prompt, type the following command to list available storage volumes that
meet the requirements for a Metadata volume that are 78 GB or larger and unclaimed.
From the output, identify four volumes to be used for the Metadata volume, a mirror of the
Metadata volume, and backups for both the Metadata volume and the mirror.
Unless only one back-end array is visible, the Metadata volume and mirror must be on
separate arrays, and should be in separate failure domains. This requirement also applies to
the mirror volume and its backup volume.
You must backup the Metadata mirrored volume with two more devices, then schedule
regular backups.
You must register the cluster with EMC in order to receive service.
Once the EMC Cluster is registered, enable the front-end ports. At this point, if you log into
your fabric switch, you should see the front-end ports log in. We will zone the front-end
ports to our hosts in the next modules.
Establish a PuTTY session to the management server in cluster 2 and log in with username
service and password Mi@Dim7T.
Enter the vplecli and log in with username service and password Mi@Dim7T.
In a VPLEX Metro or Geo configuration, the second cluster synchronizes the time between
the local management server and a remote management server using NTP.
Use this command only on the second cluster to be configured in a VPLEX Metro
configuration. Use this command before any set-up has been performed on the second
cluster of a VPLEX Metro configuration. Use this command during initial system configuration
before using the configuration system-setup command. Use this command only
on the second cluster to be configured in a VPLEX Metro configuration. This command
synchronizes Cluster 2 with Cluster 1 even if the time was correct on Cluster 2 but incorrect
on Cluster 1. The command prompts for the public IP address of the first cluster's
management server.
The synchronization might cause your VPlexcli or shell session to end. If this happens, log
in again and proceed.
Since we will be connecting two VPLEX together in a cluster we need to verify they are both
the same version. Additionally, check the cluster health to make sure it is capable to join with
cluster 1.
Connect to the second cluster with the VPlexcli. Since this is the second cluster configured,
Option 2 must be selected. If for some reason the EZ-Setup Wizard fails, the manual
installation procedures must be used to finish the VPLEX configuration.
The configuration will guide you through the same steps as the previous cluster. You have the
option of configuring authentication, EMC notification, and call home functionality. Once
those parameters are configured the Certificate Authority must be configured. This will be
used to connect to the first cluster. Notice it is recommended that the values for the CA
should be the same on both clusters to establish a VPN connection.
The next part of EZ-Setup will request the connectivity information of the first cluster. The
setup asks for the IP address, service account password and a local certificate. After this is
created, the EZ-Setup asks for the certificate key to establish the VPN connection with cluster
1. If the information is correct, the VPN connection will get configured and back-end ports
will be enabled.
From the VPlexcli prompt, type the displayed command to list all director port WWNs.
Wait several minutes for array rediscovery to complete before you proceed to the next task.
VPLEX creates two backup volumes, each named. Note that this process takes up to ten
minutes.
Several of the remaining tasks require moving back and forth between VPlexcli sessions to
both clusters. You will find it easier to leave two sessions open.
Once the EMC Cluster is registered, enable the front-end ports. At this point, if you log into
your fabric switch, you should see the front-end ports log in. We will zone the front-end
ports to our hosts in the next modules.
The EZ-Setup Wizard completes and performs the following steps on the cluster:
This module covered how to perform the Easy Setup for VPLEX.
Follow the recommendations in the Implementation and Planning Best Practices for EMC
VPLEX Technical Note to zone the WAN COM ports.
Each FC WAN port in each cluster will be zoned to the FC WAN ports on the same fabric at
the remote site. This is roughly equivalent to one initiator zoned to multiple targets.
(Although COM ports are both initiators and targets, COM ports will not log into each other
using the WAN ports).
This zoning provides additional fault tolerance and error isolation in the event of
configuration error or a rogue fabric device (when compared to a single large zone).
EMC recommends single initiator zoning for the FC-WAN fabrics. On each fabric designate
one of the clusters as having initiator ports. The other cluster would then have target ports
on that WAN fabric. This configuration provides the best protection and fault isolation while
minimizing the number of zones.
Each WAN COM initiator port should have a logical path to the targets that it can see. Ports
A4-FC02 and B4-FC02 should connect to fabric 1 and A4-FC03 and B4-FC03 should connect to
fabric 2.
Best practice state that inter-cluster Metro-Plex traffic should be isolated from other traffic
using VSANs, VFs, or LSANs.
Each FC WAN port in each cluster will be zoned to the FC WAN ports on the same fabric at
the remote site. This is roughly equivalent to one initiator zoned to multiple targets.
Although COM ports are both initiators and targets, COM ports will not log into each other
using the WAN ports.
This zoning provides additional fault tolerance and error isolation in the event of
configuration error or a rogue fabric device (when compared to a single large zone). Though
this requires more setup than a single zone, it is worth the effort and should not be
considered out of the norm for a SAN administrator.
There would be a maximum of 16 zones on each fabric for the large configuration. 32 zones
in total must be created in a Quad Engine configuration.
For the single fabric case, zoning should be maintained as if there were two fabrics (so only
one corresponding port from each target director in each zone).
The EZ-Setup Wizard completes and performs the following steps on the cluster:
From the VPlexcli prompt on management server 2, type the configuration joinclusters command, which verifies connectivity between cluster 1s directors and cluster
2s directors, and joins the directors into a unified configuration.
A logging volume should be created on storage that is extremely fast because it demands
high levels of performance during link outages and cluster failures. EMC also recommends
mirroring the logging volume between two different storage arrays.
A depth of 32 means that 128 K (32 x 4 K) is written to the first disk, the next 128 K is written
to the next disk, and so on.
Repeat these actions to create logging volumes for both clusters.
This module focuses on how to install and configure the VPLEX Witness.
For VPLEX Metro and Geo, an optional component called the VPLEX Witness can be deployed
at a third location to improve data availability in the presence of cluster failures and intercluster communication loss. The VPLEX Witness is implemented as a virtual machine and
requires a VMware ESX server for its operation. The customer host must be deployed in a
separate failure domain from either VPLEX cluster to eliminate the possibility of a single fault
affecting both a cluster and VPLEX Witness. VPLEX Witness connects to both VPLEX clusters
over the management IP network.
VPLEX Witness observes the state of the clusters, and thus can distinguish between a outage
of the inter-cluster link and a cluster failure. VPLEX Witness uses this information to guide
the clusters to either resume or suspend I/O.
VPLEX Witness capabilities vary depending on whether the VPLEX is a Metro (synchronous
consistency groups) or Geo (asynchronous consistency groups). In Metro systems, VPLEX
Witness provides seamless zero RTO fail-over for storage volumes in synchronous
consistency groups.
In Geo systems, VPLEX Witness automates fail-over for asynchronous consistency groups and
provides zero RTO and zero RPO fail-over in all cases that do not result in data rollback.
These are the configuration tasks for a VPLEX Metro or Geo. These steps are also listed in the
VPLEX Configuration Guide. It is very important to follow these tasks in order when
configuring a VPLEX cluster.
Configuration flow for a Local VPLEX is similar to the steps listed in Cluster 1. However in this
course we will review the more complex VPLEX Metro configuration.
Please note that the BE ports do not log into the switch until you start and progress through
the EZ Setup wizard. This is important to understand since you would typically try to prestage the exposure of back-end storage. One way to get around this issue is to utilize the
VPLEX WWN Calculator. This is covered later in this module.
It is best practice to always refer to the VPLEX Configuration Guide for the latest installation
information.
The Cluster Witness Server VM must be deployed so any faults that may affect individual
clusters do not affect the Cluster Witness Server VM. This includes possible faults related to
physical location, power, and networking connectivity.
If desired, the Cluster Witness Server VM can be deployed as a fault-tolerant (FT) VM in a
VMware FT-enabled cluster. Currently this deployment is only supported with VMware-FT; it
is not supported with VMware-HA.
Refer to the Procedure Generator for specific requirements for the host, network ,and overall
environment.
You can deploy the Cluster Witness Server OVA from a vCenter or directly from an ESX host.
Ensure that the target ESX Server is managed by the vCenter Server. If not, ask your vCenter
administrator to add this ESX Server to the inventory of "Hosts and Clusters" managed by
vCenter server.
If you are logging in directly from a vSphere client into an ESX Server, ensure that the ESX
Server is not managed by a vCenter server.
From vSphere Client, deploy the ODF and configure where to save it on the ESX server and
the network. There are two network connections required so disregard VMwares
recommendation and configure both connections on the same virtual network.
Once the VM is deployed, verify the status of the deployment to make sure the VM is
configured correctly. It is important to make sure the Mac Address Change option of the
virtual switch where the VM is connected is set to Accept.
From the ESXi Server, start the VM. The VM console is shown above, it gives the user the
opportunity to accept the license terms, set the time zone, and configure the public IP.
Through the console you can also access the witness CLI.
Select configure network and set a public IP address for the console. The configuration is
displayed here. Once you establish the configuration, verify your settings and if they are
correct continue.
When the configuration is accepted, wait at least ten minutes so the IP addresses can be set.
In the vSphere client UI, in the Summary tab in the General section, the IP addresses field
displays the configured public IP address of the VM. Other details of the VM are also
included in that tab.
Log into the management server for cluster 1 and run the command to configure a cluster
witness VPN configuration. You will be prompted for a certificate authority passphrase and
key. This should match your previous keys for the VPLEX.
From Cluster 1, the management server will try to ping the second cluster to complete the
VPN connection configuration. Once the connection is completed, wait 10 minutes for the
VPN to be established before verifying connectivity.
Once the VPN configuration is completed, verify VPN connectivity. Use the VPN status
command for this. Notice the command output now includes a line for the Cluster Witness
Server. If the command cannot contact the Cluster Witness Server, try again. If the problem
persists, check your network configuration on the ESXi server.
This course covered VPLEX installation and initial configuration. The course also covered the
steps to prepare a site for installation and what utilities to use during the installation. Backend ports were provisioned by the storage array. Front-End ports were connected to the
hosts and then FC-WAN ports were used to connect two VPLEX clusters together into a
Metro or Geo implementation. This course guided you through using VPLEX EZ-Setup Wizard
for the installation and VPLEX witness installation.