Info Scale

Veritas InfoScale 7.4.
2
Fundamentals for UNIX/Linux:
Administration (Lessons)
Not for Distribution.

Veritas InfoScale 7.4.2 Fundamentals for UNIX/Linux: Administration
(Lessons)
THIS PUBLICATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS,
REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE
EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. VERITAS TECHNOLOGIES LLC
SHALL NOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE
FURNISHING, PERFORMANCE, OR USE OF THIS PUBLICATION. THE INFORMATION CONTAINED HEREIN
IS SUBJECT TO CHANGE WITHOUT NOTICE.
No part of the contents of this book may be reproduced or transmitted in any form or by any means
without the written permission of the publisher.
Course Developer Lead Subject Matter Experts Technical Contributors and

Reviewers
Aditya Konde Pratik Boharapi Gayatri Deshpande

Ashlesha Shinde Sreejith Nair Mangilal Daravath
Swati Joshi Harshwardhan Dandwate Subodh Jain
Ranvir Mankoo Abdulhafiz Sheikh Jayant Chopade
Raj Kiran Prasad Thota Srinivasu Vetcha Vineet Mishra
Shoaib Shekh Kiran Salunkhe
Manish Arya Kajal Agrawal
Shyam Gorkhe Syed faizan Ali
Asaf Sabir
Viswanathan R
Sheerin Sheikh
Ramesh Babu Damodharan
Copyright @ 2020 Veritas Technologies LLC. All rights reserved.
For specific country offices Veritas World Headquarters © 2020 Veritas Technologies LLC. All
rights reserved. Veritas and the Veritas
and contact numbers, please 500 East Middlefield Road
Logo are trademarks or registered
visit our website at Mountain View, CA 94043 USA
trademarks of Veritas Technologies LLC
www.veritas.com. +1 (650) 933 1000 or its affiliates in the U.S. and other
www.veritas.com countries. Other names may be
trademarks of their respective owners.
ii
Table of Contents
PART 1: Veritas InfoScale Storage 7.4.2 for UNIX/Linux: Administration
Course Introduction
About this course ................................................................................................................. Intro-2
Education and support resources ........................................................................................ Intro-7
InfoScale Storage Basics
Lesson 01: Installing and Licensing InfoScale

Introducing the Veritas InfoScale product suite .........................................................................1-4
Tools for installing InfoScale products......................................................................................1-17
InfoScale Cloud offerings ..........................................................................................................1-36
Installing Veritas InfoScale Storage ..........................................................................................1-40
Installing Veritas InfoScale Availability .....................................................................................1-45
Upgrading Veritas InfoScale Enterprise ....................................................................................1-50
Lesson 02: Virtual Objects

Operating system storage devices and virtual data storage ......................................................2-4
Volume Manager (VxVM) storage objects ...............................................................................2-14
VxVM volume layouts and RAID levels .....................................................................................2-17
Lesson 03: Creating a Volume and File System

Preparing disks and disk groups for volume creation ................................................................3-4
Creating a volume and adding a file system .............................................................................3-13
Displaying disk and disk group information .............................................................................3-18
Displaying volume configuration information ..........................................................................3-26
Removing volumes, disks, and disk groups ..............................................................................3-30
Lesson 04: Working with Volumes with Different Layouts

Volume layouts ...........................................................................................................................4-4
Creating volumes with various layouts.....................................................................................4-13
Allocating storage for volumes .................................................................................................4-17
Lesson 05: Making Configuration Changes

Administering mirrored volumes................................................................................................5-4
Resizing a volume and a file system .........................................................................................5-15
Moving data between systems .................................................................................................5-21
Renaming VxVM objects ...........................................................................................................5-27
Lesson 06: Administering File System

Benefits of using Veritas File System ..........................................................................................6-4
Using Veritas File System commands .......................................................................................6-10
Table of Contents iii

© 2020 Veritas Technologies LLC. All Rights Reserved

Logging in VxFS .........................................................................................................................6-15
Controlling file system fragmentation ......................................................................................6-18
Using thin provisioning disk arrays ...........................................................................................6-24
PART 2: Veritas InfoScale Availability 7.4.2 for UNIX/Linux: Administration
InfoScale Availability Basics
Lesson 01: High Availability Concepts

High availability concepts ...........................................................................................................1-4
Clustering concepts ....................................................................................................................1-7
High availability application services ........................................................................................1-12
Clustering prerequisites............................................................................................................1-17
Lesson 02: VCS Building Blocks

VCS terminology .........................................................................................................................2-4
Cluster communication ............................................................................................................2-14
VCS architecture .......................................................................................................................2-21
Multi version cluster .................................................................................................................2-24
Lesson 03: VCS Operations

Common VCS tools and operations ............................................................................................3-4
Service group operations ............................................................................................................3-9
Resource operations .................................................................................................................3-15
Core VCS enhancements in 7.4.2..............................................................................................3-18
Lesson 04: VCS Configuration Methods

Starting and stopping VCS ..........................................................................................................4-4
Overview of configuration methods .........................................................................................4-11
Online configuration .................................................................................................................4-13
Controlling access to VCS .........................................................................................................4-20
VCS Password encryption .........................................................................................................4-26
Lesson 05: Preparing Services for VCS

Preparing applications for VCS ...................................................................................................5-4
Performing one-time configuration tasks ..................................................................................5-7
Testing the application service .................................................................................................5-13
Stopping and migrating a service .............................................................................................5-23
Collecting configuration information .......................................................................................5-26
Lesson 06: Online Configuration

Online service group configuration ............................................................................................6-4
Adding resources ........................................................................................................................6-9
Solving common configuration errors ......................................................................................6-19
Testing the service group .........................................................................................................6-22
iv Veritas InfoScale 7.4.2 Fundamentals for UNIX/Linux: Administration (Lessons)


Lesson 07: Offline Configuration
Offline configuration examples ..................................................................................................7-4
Offline configuration procedures ...............................................................................................7-8
Solving offline configuration problems ....................................................................................7-18
Testing the service group .........................................................................................................7-23
Lesson 08: Configuring Notification

Notification overview .................................................................................................................8-4
Configuring notification ............................................................................................................8-11
Overview of triggers .................................................................................................................8-29
InfoScale Availability Additions
Lesson 09: Handling Resource Faults

VCS response to resource faults .................................................................................................9-4
Determining failover duration ..................................................................................................9-10
Controlling fault behavior .........................................................................................................9-15
Recovering from resource faults ..............................................................................................9-20
Fault notification and event handling.......................................................................................9-22
Lesson 10: Intelligent Monitoring Framework

IMF overview ............................................................................................................................10-4
IMF configuration .....................................................................................................................10-8
Faults and failover with intelligent monitoring ......................................................................10-14
Lesson 11: Cluster Communications

VCS communications review ....................................................................................................11-4
Cluster interconnect configuration ..........................................................................................11-9
Cluster startup ........................................................................................................................11-15
System and cluster interconnect failure .................................................................................11-19
Changing the interconnect configuration...............................................................................11-26
Table of Contents v

Veritas InfoScale 7.4.2 Fundamentals for
UNIX/Linux: Administration
Course introduction
© 2020 Veritas Technologies LLC. All rights reserved. Veritas and the Veritas Logo are trademarks or registered trademarks of Veritas Technologies LLC
or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners.
This is the Course Introduction lesson in the Veritas InfoScale 7.4.2 Fundamentals for
UNIX/Linux: Administration course.

Intro-1
Topic: About this course
This topic describes the contents of this course.
This topic describes the contents of this course.


Intro-2
Intended audience
This course is intended for the following personnel who will install, deploy, configure, manage, and integrate
InfoScale Storage and InfoScale Availability:
• UNIX/Linux system administrators
• System engineers
• Technical support personnel
• Network/SAN administrators
• Systems integration/development staff
This course is designed for UNIX/Linux system administrators, system engineers, technical
support personnel, network/SAN administrators, and systems integration/development staff,
who will install, configure, manage and integrate InfoScale Storage and InfoScale Availability.

Intro-3
Course objectives
• Install and configure Veritas InfoScale Enterprise.
• Configure and manage disks, disk groups, and volumes.
• Administer file systems.
• Create a cluster.
• Configure service groups and resources.
• Implement and verify failover and failback capability for application, storage, and network services.
This course will NOT prepare you for the certification exams or the Advanced courses of both the products.
After completing this course, you will be able to perform the tasks listed on this slide.

Intro-4
Course prerequisites
• Knowledge of and hands-on experience with UNIX/Linux systems administration is required.
Knowledge of and hands-on experience with UNIX/Linux systems administration is required.


Intro-5
Lessons in this course
PART 1: Veritas InfoScale Storage 7.4.2 for UNIX/Linux: • Lesson 02: VCS Building Blocks
Administration
• Lesson 03: VCS Operations
• Lesson 04: VCS Configuration Methods
• Lesson 01: Installing and Licensing InfoScale Storage
• Lesson 02: Virtual Objects
• Lesson 05: Preparing Services for VCS
• Lesson 03: Creating a Volume and File System • Lesson 06: Online Configuration
• Lesson 04: Working with Volumes with Different Layouts • Lesson 07: Offline Configuration
• Lesson 05: Making Configuration Changes
• Lesson 08: Configuring Notification
• Lesson 06: Administering File System
PART 2: Veritas InfoScale Availability 7.4.2 for UNIX/Linux: InfoScale Availability Additions
Administration
• Lesson 09: Handling Resource Faults
InfoScale Availability Basics • Lesson 10: Intelligent Monitoring Framework
• Lesson 01: High Availability Concepts
• Lesson 11: Cluster Communications
The lessons in this course are displayed on the slide. This five-day class is a condensed version
of the five-day Veritas InfoScale Storage 7.4.2 for UNIX/Linux: Administration course and the
five-day Veritas InfoScale Availability 7.4.2 for UNIX/Linux: Administration course. This course
is a subset of the two courses, and it covers the absolute basics of the two products -
InfoScale Storage 7.4.2 and InfoScale Availability 7.4.2.

Intro-6
Topic: Education and support resources
This topic describes Veritas Education offerings and other Veritas resources
available to help you design, configure, operate, monitor, and support
Veritas InfoScale 7.4.2.
This topic describes Veritas Education offerings and other Veritas resources available to
help you design, configure, operate, monitor, and support Veritas InfoScale 7.4.2.

Intro-7
Veritas Open eXchange
https://vox.veritas.com/
• The latest technology articles from industry

experts
• Easy and fast access to technical content and
product information
• Access to premium content, such as book
previews and free sample chapters
• Peer-to-peer discussion forums
• Training and education resources
Find out more. It’s free!
The Veritas Open eXchange allows customers and users of Veritas products to network, get
help, and learn more about the industry-leading solutions. Veritas Open eXchange is a
customer-focused resource, intended to help you design and implement a utility computing
strategy to provide availability, performance, and automation for your storage, servers, and
applications. Veritas Open eXchange provides the following resources:
• Technical documents, such as articles, white papers, and product specs.
• Interactive services, such as the discussion forum, where members can discuss current
topics, share tips and tricks, and help one another troubleshoot problems.
Best of all, it is free. Sign up to become a member: https://vox.veritas.com/

Intro-8
MyVeritas
https://www.veritas.com/support/en_US.html
MyVeritas is single destination that allows you to access all of your Veritas enterprise services
and information. Visit https://www.veritas.com/support/en_US.html to view this page.

Intro-9
Veritas Education Services: Links
https://www.veritas.com/services/education-services.html
• Certification Paths: Backup & Recovery,

Information Governance,
Storage & Availability
• APTARE Training: Learn About APTARE training
• View FAQs about Education Services
• Manage your training transcript and print certificates of

completion by signing in to the Veritas Learning Portal
10
Visit the Veritas Education Services page to learn more about Veritas product training and
certification at: https://www.veritas.com/services/education-services.html
This slide displays links related to curriculum paths, Veritas certification, and other training
related information.
• Curriculum Paths: Backup & Recovery, Information Governance, Storage & Availability.
• Get Certified in InfoScale and other Veritas products.
• View FAQs about Education Services.
• Manage your training transcript and print certificates of completion by signing in to the
Veritas Learning Portal.

Intro-10
End of presentation
Intro-11
Lesson 01: Installing and Licensing InfoScale
This is the Installing and Licensing InfoScale lesson in the Veritas InfoScale 7.4.2
Fundamentals for UNIX/Linux: Administration course.

01-1
Administration
Administration
The lessons in this course are displayed on the slide.


01-2
Lesson objectives
Topic Objective
Introducing the Veritas InfoScale • Provide an overview of the InfoScale product suite.
product suite • Name the products in the InfoScale product suite.
• List features of the InfoScale products.
Tools for installing InfoScale products • List the checks to perform.
• Identify installation tools.
• List the user interfaces to manage InfoScale products.
InfoScale Cloud offerings Provide an overview of InfoScale support for cloud environments
and solutions.
Installing Veritas InfoScale Storage Perform basic installation and configuration of InfoScale Storage
components.
Installing Veritas InfoScale Availability Perform basic installation and configuration of InfoScale
Availability.
Upgrading Veritas InfoScale Enterprise Perform a basic upgrade of the SFHA component of InfoScale
Enterprise.
The table on this slide lists the topics and objectives for this lesson.

01-3
Topic: Introducing the Veritas InfoScale product suite
After completing this topic, you will be able to:

• Provide an overview of the InfoScale product suite
• Name the products in the InfoScale product suite
• List features of the InfoScale products
This is the Introducing the Veritas InfoScale product suite topic.


01-4
Introducing Veritas InfoScale
InfoScale family of products draws on Veritas' long heritage of

world-class storage management and availability solutions.
The InfoScale family of products draws on Veritas' long heritage of world-class storage
management and availability solutions.

01-5
Overview of the InfoScale product suite
Value added graphical management tool
InfoScale Operations Manager
InfoScale Enterprise
SFCFSH Extensive
DMP vDMP SF SFHA VVR SFRAC SFSYBCE VCS HA DR
A application
availability
InfoScale Storage InfoScale Availability

SDS with Extensive capability
comprehensive DMP vDMP SF SFCFS VVR VCS HA DR (including DR) for
storage management application availability
InfoScale Foundation
Base offering targeting DMP vDMP SF*
storage management
* Subset of features available in Storage Foundation Standard
This slide displays information about the InfoScale product suite.


01-6
Mapping to previous SFHA offerings: Tabular representation
The new InfoScale Family

Earlier SFHA offerings InfoScale
InfoScale InfoScale InfoScale InfoScale
Operations
Foundation Storage Availability Enterprise
Manager
Cluster File System  
Cluster Server  
Dynamic Multi-Pathing   
Dynamic Multi-Pathing for VMware   
Operations Manager 
Replicator Option  
Storage Foundation  
Storage Foundation Basic   
Storage Foundation for Oracle RAC 
Storage Foundation for Sybase ASE CE 
Storage Foundation High Availability 
The earlier SFHA offerings mapped to the new InfoScale family are displayed on the slide. For
additional information about the Veritas InfoScale family, refer to:
https://sort.veritas.com/infoscale/intro
Note: While Operations Manager maps to the InfoScale Operations Manager, it is not a stand-
alone product. Instead, it is a graphical management interface for the InfoScale family and is
available free of charge to InfoScale customers.

01-7
InfoScale Foundation features: Linux
Features included in InfoScale Foundation
• 1-256 TB File System • Encryption of data at rest • Mount lock • Quality of Service (QoS) for
• ALUA and generic ALUA array (FIPS Certified) • Named data streams applications
support • Fault tolerant file system • Online file system • SSD device exploitation
• Data Management APIs • File Change Log defragmentation • Site awareness with remote
• Device names consistent • File system migration from • Online file system grow and mirrors
across cluster nodes EXT4 to VxFS shrink • SmartIO Support for vMotion
• Device names using Array • File system snapshots • Online patching of selected with DMP for VMware with
Volume IDs Volume Manager packages SmartPools
• Import cloned LUN
• Dirty region logging • Online relay out • Support for Erasure Coded
• Install and configure InfoScale volumes
• Dynamic LUN expansion enterprises through VIOM • Online volume grow and
shrink • Support for native 4k sector
• Dynamic Multi-pathing with • Installer - Required patches size storage devices
intelligent pathing for EMC for OS updates • Operations Manager
VPLEX • SmartTier
• Installer-SORT integration • Partitioned directories
• Enclosure-based naming • Thin storage reclamation
• Keyless licensing • Public cloud support - AWS using UNMAP
infrastructure
• iSCSI device support
This slide displays information about the InfoScale Foundation features for Linux. For
additional information, refer to: https://sort.veritas.com/product_features

01-8
InfoScale Storage features: Linux
Features included in InfoScale Storage
• Application isolation • Flashsnap: Fast • ODM and Cached ODM • SmartIO - Write back caching
• Atomic I/O resynchronization (CODM) • SmartMove
• CFS Multi -transaction • FSS -64 node support • Remote mirrors for campus • Space-optimized instant
server • Full 64-bit support without clusters snapshot
• Cluster File System additional packages • Replicator option (formerly • Split-mirror snapshot
• CVM • Full size instant snapshots VVR) • Storage management plugin for
• Compression • H/w assisted copy • SCSI-3 based I/O fencing Docker
• Concurrent I/O • Heartbeats: LLT over UDP • SCSI3-PGR fencing with • Thin storage reclamation
• Deduplication • Heartbeats: Low-priority coordinator disks • User and group quotas
• Direct I/O • Heartbeats: Dedicated • SmartAssist - SmartIO • Database storage checkpoints
• File Replicator option Private Network analysis and enablement and rollback (Oracle only)
• File system storage connections tool • FlashSnap: Database (Oracle
checkpoints • Heartbeats: Infiniband and • SmartIO - Volume Manager only)
• Flashsnap: Disk group split 10GB Ethernet with RDMA and File System read • SmartTier for Oracle
and join • Hot-relocation caching
• Large cluster deployments • Non-SCSI3 fencing with CPS
(up to 128 nodes) • Plus InfoScale Foundation
features
This slide displays information about the InfoScale Storage features for Linux. For additional
information, refer to: https://sort.veritas.com/product_features

01-9
InfoScale Availability features: Linux
Features included in InfoScale Availability

• 128 node cluster support • Heartbeats: Dedicated • Keyless licensing • Replication agents (if they
• Adaptive HA Private Network connections • Manual campus cluster change direction upon
• Application agents • Heartbeats: Infiniband and failover failover)
• Auto-Clear faults 10GB Ethernet with RDMA • Monitor Only resource • SCSI3-PGR fencing with
• Bundled agents • Installer - Required patches • Non-SCSI3 fencing with coordinator disks
• FireDrill for OS updates Coordination Point Servers • Support for Oracle Flex-ASM
• Full 64-bit support without • Intelligent Monitoring • Operations Manager and Oracle PDB
additional packages Framework (IMF) • Preferred application or • VBS cross site recovery plan
• Global Cluster Option (GCO) • Just In Time(JIT) target VM service group I/O fencing • VCS - Online upgrade
• Heartbeats: LLT over UDP Availability (VMware) for • Preferred node I/O fencing • Virtual Business Services
• Heartbeats: Low-priority Failover • Preferred site I/O fencing
• Priority based failover
10
This slide displays information about the InfoScale Availability features for Linux. For
additional information, refer to: https://sort.veritas.com/product_features

01-10
InfoScale Enterprise features: Linux
Features included in InfoScale Enterprise
Installer - One click Rolling upgrade Sybase ASE CE Oracle RAC integration
default install for fresh integration
installs
Plus InfoScale Storage features Plus InfoScale Availability features
11
This slide displays information about the InfoScale Enterprise features for Linux. For additional
information, refer to: https://sort.veritas.com/product_features

01-11
Transitions and co-deploying InfoScale products
InfoScale Operations Manager

InfoScale Enterprise
SFCFSH
DMP vDMP SF SFHA VVR SFRAC SFSYBCE VCS HA DR
A
InfoScale Storage InfoScale Availability
DMP vDMP SF SFCFS VVR VCS HA DR
InfoScale Foundation
Transitions
DMP vDMP SF*
Co-Deploy
* Subset of features available in Storage Foundation Standard
12
This slide displays the information about the flow of transitions and how InfoScale products
are co-deployed.

01-12
Supported upgrade methods
Full upgrade
Online upgrade (VCS only)
Phased upgrade
Rolling upgrade
Manual upgrade
13
InfoScale supports the following upgrade methods:

• Full upgrade
• Online upgrade (VCS only)
• Phased upgrade
• Rolling upgrade
• Manual upgrade

01-13
Native installer support
• AI (Automated Installer)
• ZFS BEs (Boot Environments)
• Kickstart (RHEL)
• YUM (RHEL)
• Satellite server (RHEL)
• NIM
• NIMADM
• ADI
14
This slide displays information about the native installer support for InfoScale.

01-14
Supported configuration management tools
Deploy the Veritas InfoScale product using the following automation tools:
NOTE:
• Using the Ansible automation tool, you can deploy, configure, and administer the InfoScale product and it’s
features.
• For more information, refer to the links provided in the Notes section.
15
You can deploy the Veritas InfoScale product using the automation tools such as Ansible, Chef,
and Puppet.
• For additional information about Ansible Guides, refer to:
https://sort.veritas.com/public/infoscale/ansible/docs/Infoscale7.4.1_Ansible_Support_Li
nux_Guide.pdf
https://sort.veritas.com/utility/ansible
https://www.veritas.com/content/support/en_US/doc/109864724-141543589-
0/v135387664-141543589
• For additional information about Chef Guides, refer to:
https://www.veritas.com/support/en_US/doc/infoscale_chef_deploy_unix
https://www.veritas.com/content/support/en_US/doc/infoscale_chef_deploy_unix

01-15
Veritas Services and Operations Readiness Tools (SORT)
SORT website: https://sort.veritas.com/
SORT mobile application:

https://sort.veritas.com/mobile_apps
16
You can use the Veritas SORT Web page to download the InfoScale product guides and
patches related to latest and previous releases.
For additional information about SORT, refer to:
https://sort.veritas.com/
https://sort.veritas.com/mobile_apps

01-16
Topic: Tools for installing InfoScale products

• List the checks to perform
• Identify installation tools
• List the user interfaces to manage InfoScale products
17
This is the Tools for installing InfoScale products topic.


01-17
Pre-installation tasks
Product Features
• Products and Platforms Lookups
Installations or • Documents
upgrades review • InfoScale and SFHA Future Platform and Feature Plans
product information
• Installation and Upgrade general checklists

• Installation and Upgrade custom reports by SORT data collectors
• License and Deployment assessment custom report by SORT data collectors
Use the following • Veritas InfoScale Entitlement Calculator
tools when planning
for installations or • System Performance Value Unit (SPVU) Calculator
upgrades
18
This slide displays information about the pre-installation tasks you need to perform before
installing InfoScale.

01-18
Hardware and software requirements
• Refer the HCL and SCL

sort.veritas.com
• Minimum configurations:
– Memory
– Disk space
• Systems installed and verified
• Obtain appropriate licenses
• Determine supported software:
• veritas.com/licensing
• Sales representative – Operating system, Patch level, and Applications
• Technical Support – Volume management, File system
19
The details about InfoScale SCL, HCL, and Licensing information are listed on the slide.
For additional information, refer to:
• InfoScale 742 SCL (Linux):
https://www.veritas.com/content/support/en_US/doc/infoscale_scl_742_lin
• InfoScale 742 Windows: https://sort.veritas.com/DocPortal/pdf/infoscale_scl_742_win
• InfoScale 742 HW list:
https://www.veritas.com/content/support/en_US/doc/infoscale_hcl_73_731_74_unix
• InfoScale 742 docs list: https://sort.veritas.com/sitemap/document/28

01-19
Additional considerations
• Eliminate single points of failure
• Provide redundancy for:
– Public network interfaces and infrastructures
– HBAs for shared storage (Fibre or SCSI)
• Configure systems’ hardware identically for HA solutions:
– System hardware
– Network interface cards
– Storage HBAs
– Use shared storage with SCSI 3 support for data protection
• Configure systems’ software identically for HA solutions:
– Operating system version and patch level
– Kernel, networking and configuration files
Some hardware variability may be appropriate, for example, to meet

 different workload requirements among HA services.
20
Some additional considerations that you should consider are as follows:

• Eliminate single points of failure.
− Public network interfaces and infrastructures
− HBAs for shared storage (Fibre or SCSI)
− System hardware
− Network interface cards
− Storage HBAs
− Use shared storage with SCSI 3 support for data protection
• Configure system’s software identically for HA solutions:
− Operating system version and patch level
− Kernel, networking and configuration files
Note that some hardware variability may be appropriate, for example, to meet different
workload requirements among HA services.

01-20
InfoScale pre-installation checks
Automatically
using SORT Data
Collectors
Manually using
the checklist Using the Installer
downloaded from pre-check option
SORT
21
You can perform the InfoScale pre-installation checks by:

• Automatically using SORT Data Collectors
• Using the Installer pre-check option
• Manually using the checklist downloaded from SORT
Reference links:
• https://sort.veritas.com/data_collectors
• https://sort.veritas.com/checklist/install

01-21
Pre-checks using SORT Data Collectors
SORT Downloads page
1
1. Download the data collection tool for your environment.

2 3
2. Install the data collection tool.
3. Run the data collection tool. The data collection tool
analyzes your systems and stores the results in an XML file.
4. Upload the XML file to generate a report.
22
To perform pre-checks using SORT Data Collectors, perform the following steps:
1. Download the data collection tool for your environment.
2. Install the data collection tool.
3. Run the data collection tool. The data collection tool analyzes your systems and stores the
results in an XML file.
4. Upload the XML file to generate a report.


01-22
Pre-checks using checklists
SORT Assessments page
23
This slide displays information about the pre-checks using checklists.


01-23
Pre-checks using the installer
Information required
• Product to install
• Product component to install
./installer -precheck sys1 sys2
Veritas InfoScale Storage and Availability Solutions Precheck Program
sys1 sys2
1) Veritas InfoScale Foundation
2) Veritas InfoScale Availability
3) Veritas InfoScale Storage
4) Veritas InfoScale Enterprise
Select a product to perform pre-installation check for: [1-4,q]
24
The slide displays an example of the InfoScale installer procedure list.


01-24
Planning for an InfoScale installation
Keep the following information handy before you start the installation:
• The system name with the fully-qualified domain name

• The product license key if there are no plans to use keyless licensing
• The cluster name and cluster ID (HA products only)
• The public NIC device names (HA products only)
• The private heartbeat NIC device names (HA products only)
25
Before installing the InfoScale product, keep the following information ready:
• The cluster name and cluster ID (HA products only)
• The public NIC device names (HA products only)
• The private heartbeat NIC device names (HA products only)

01-25
InfoScale licensing options
Managed by
Keyless
VOM
No key After 60 days Non compliant Alerts
Select product
Key (Enterprise
Add valid key Valid key
keys, .SLF file)
All hosts must be licensed (legally entitled) to run InfoScale products.
Keyless After 60 days, warning messages are written to syslog every four hours.
licensing:
Hosts must be managed by Veritas InfoScale Operations Manager to be license-compliant.
Use vxkeyless for setting and changing product level.
In previous releases 7.3.x text-based license keys are used. From 7.4 (or later releases), .SLF license key certificates are issued in
the form of license key files. For more information refer to the 7.4.2 InfoScale Install and License Guides:
Linux: https://www.veritas.com/content/support/en_US/doc/109508799-141543583-0/v23679202-141543583
Windows: https://www.veritas.com/content/support/en_US/doc/109356204-141924754-0/wxrt-tot_v55825087-141924754
26
For Enterprise Licensing: .SLF key: You need to purchase the .SLF key for Enterprise
installation.
For Keyless licensing:
• After 60 days, warning messages are written to syslog every four hours.
• Hosts must be managed by Veritas InfoScale Operations Manager to be license-compliant.
• Use vxkeyless for setting and changing product level.
In previous releases 7.3.x text-based license keys are used. From 7.4 (or later releases), .SLF
license key certificates are issued in the form of license key files. For more information about
the 7.4.2 InfoScale Install and License Guides, refer to:
Linux: https://www.veritas.com/content/support/en_US/doc/109508799-141543583-
0/v23679202-141543583
Windows: https://www.veritas.com/content/support/en_US/doc/109356204-141924754-
0/wxrt-tot_v55825087-141924754

01-26
Adding licenses
Manually using For fresh installs:

the licensing /opt/VRTS/bin/vxlicinst –k <.slf license file path>
tools vxdctl license init
For upgrades:
/opt/VRTS/bin/vxlicinstupgrade –k <.slf license
file path>
Automatically ./installer –license sys1 sys2

using the Veritas InfoScale Storage License Program
sys1
installer To comply with the terms of our End User License Agreement, you have
60 days to either:
* Enter a valid license key matching the functionality in use on the
systems
* Enable keyless licensing and manage the systems with a Management
Server. For more details visit
...
Reference link: https://www.veritas.com/content/support/en_US/doc/109508799-141543583-0/v110725623-

141543583
27
You can add licenses by:

• Manually using the licensing tools
• Automatically using the installer
For additional information about registering Veritas InfoScale using a permanent license key
file, refer to: https://www.veritas.com/content/support/en_US/doc/109508799-141543583-
0/v110725623-141543583

01-27
Licensing tools on SORT
https://sort.veritas.com/license_calc
28
This slide displays information about the Licensing tools on SORT.

For additional information, refer to:
https://sort.veritas.com/license_calc
https://sort.veritas.com/assessment/licenses

01-28
Licensing Monitoring using Telemetry: UNIX/Linux
• From 7.4.1 release (or later) InfoScale supports Licensing Monitoring using Telemetry system [Veritas Edge
Receiver (VCR)].
• Benefits:
– Compliance requirements of license usage by customer are met.
– Customer will be able to proactively plan for the renewals of licenses.
– Customer will be able to see the consolidated report on one console related to platform wise deployment and license
information.
29
If your licenses are monitored using Telemetry, you will avail the following benefits:
• Compliance requirements of license usage (by customer) are met.
• You will be able to proactively plan for the renewals of licenses.
• You will be able to see the consolidated report on one console related to platform wise
deployment and license information.

01-29
Tools to perform post-install checks
SORT Data Collectors Similar to performing the pre-install checks
Installer
./installer –version sys1
The -version option is used to check the status of installed products on the system. The installer also
provides an option to perform a detailed post-installation check.
./installer -version sys1

Checking communication on sys1................. Done
Checking installed products on sys1............ Done
Platform of sys1:
Linux RHEL 7.0 x86_64
Installed product(s) on sys1:
…
30
This slide displays information about the tools required to perform post-install checks.

01-30
User interfaces for managing InfoScale products
VIOM CLI
• Veritas InfoScale Operations Manager • Command-line interface

• Suited for large numbers of clusters • Suited for local system or local cluster
• Management for all Storage Foundation management
products
For custom solution package and InfoScale deployments using the VIOM Distribution Manager add-on,
refer to:
https://www.veritas.com/content/support/en_US/doc/120571146-141764001-1
https://www.veritas.com/content/support/en_US/doc/120571146-141764001-0/v121755920-141764001
31
The user interfaces for managing InfoScale products are Veritas InfoScale Operations Manager
and the Command-line interface. VIOM is suitable for large numbers of clusters and the
management for all Storage Foundation products. The CLI is suitable for local system or local
cluster management.
For custom solution package and InfoScale deployments using the VIOM Distribution Manager
add-on, refer to:
https://www.veritas.com/content/support/en_US/doc/120571146-141764001-1
https://www.veritas.com/content/support/en_US/doc/120571146-141764001-
0/v121755920-141764001

01-31
Operating System Support for 7.4.2
• LINUX:
– RHEL 7.7, 8.1
– OL 7.7
– SLES12 SP5, SLES 15 SP1
– CENTOS 7.7, 8.1
• SOLARIS:
– SPRAC 11 Update 4
– Solaris x64 (Availability only) 11 Update 4
• AIX:
– AIX 7.1 TL5
– AIX 7.2 TL3 and TL4
• WINDOWS:
– Windows 2016 (with Latest Update) Datacenter, Standard and
Storage Server
– Windows 2019 (with Latest Update) Datacenter and Standard
32
This slide displays InfoScale 7.4.2 support for operation systems.


01-32
Application Support for 7.4.2
• Oracle Database: 19c, 12cR2

• SQL Server: 2016, 2017, 2019
• DB2: 11.1, 11.5
• SCOM: 2016,2019
33
This slide displays InfoScale 7.4.2 support for applications.


01-33
Virtualization Support for 7.4.2
• ESXi:
– v.6.5 U3
– v.6.7 U3
• HyperV: 2016
• Nutanix: AOS 5.10.5
34
This slide displays InfoScale 7.4.2 support for virtualization environments.


01-34
Supported Upgrade Paths
InfoScale for Linux/Unix

v.6.2.1 v.7.2
v.7.3.1 v.7.4.1
InfoScale
InfoScale for Windows

v.7.2 v.7.3.1 v.7.4.1 v.7.4.2
35
This slide displays the InfoScale upgrade path from the previous releases to the current
InfoScale 7.4.2 release.

01-35
Topic: InfoScale Cloud offerings
After completing this topic, you will be able to explain the InfoScale support
for cloud environments and solutions.
36
This is the InfoScale Cloud offerings topic.


01-36
InfoScale solutions support for Cloud Environments
• Helps you make your applications highly available
and resilient to disruptions.
• Applications may reside in your on-premises data
centers or in public, private, or hybrid cloud
environments.
• With InfoScale - combine cost-effective platform
independence with consistent high application
performance and availability.
• Veritas supports InfoScale configurations in the
following cloud environments:
– Amazon Web Services (AWS)
– Google Cloud Platform (GCP)
– Microsoft Azure
– OpenStack
– VMware ESXi, Nutanix
37
The Veritas InfoScale product suite helps you make your applications highly available and
resilient to disruptions. The applications may reside in your on-premises data centers or in
public, private, or hybrid cloud environments. With InfoScale, you can combine cost-effective
platform independence with consistent high application performance and availability. Veritas
supports InfoScale configurations in the following cloud environments:
• Amazon Web Services (AWS)
• Microsoft Azure
• Google Cloud Platform (GCP)
• OpenStack
• Nutanix

01-37
InfoScale support for Cloud Storage connectors
• Veritas InfoScale Storage components allows you to manage various kinds of storage, which includes SAN, local flash,
SSD, DAS, and cloud S3 targets.
• Veritas InfoScale Availability agents provide monitoring and failover capabilities for the networking, storage, and
replication resources that are associated with an application in the cloud.
• Using these components you can configure an application for:
– Migration from on-premises to cloud or from cloud to cloud.
– High availability or disaster recovery within a cloud.
– Disaster recovery from on-premises to cloud or across clouds.
Reference link for InfoScale 7.4.2 support for Cloud Environments:

https://www.veritas.com/support/en_US/doc/130803809-141542355-0/index
38
Veritas InfoScale Storage components allows you to manage various kinds of storage, which
includes SAN, local flash, SSD, DAS, and cloud S3 targets. Veritas InfoScale Availability agents
provide monitoring and failover capabilities for the networking, storage, and replication
resources that are associated with an application in the cloud. Using these components you
can configure an application for:
• Migration from on-premises to cloud or from cloud to cloud
• High availability or disaster recovery within a cloud
• Disaster recovery from on-premises to cloud or across clouds
For additional information about the InfoScale 7.4.2 support for cloud environments, refer to:
https://www.veritas.com/support/en_US/doc/130803809-141542355-0/index

01-38
InfoScale deployment in AWS and Azure Marketplace
39
Veritas InfoScale Enterprise enables organizations to provision and manage storage and
provides high availability (HA) for business-critical applications. Storage provisioning is
independent of hardware types or locations with predictable quality-of-service by identifying
and optimizing critical workloads. It increases storage agility and enables you to work with
and manage multiple types of storage to achieve better ROI without compromising on
performance and flexibility.
For application HA, InfoScale Enterprise monitors an application to detect any application or
node failure and brings the application services up on a target system in case of a failure. You
can deploy the InfoScale solution in AWS and Azure marketplace, for more information refer
to the following links:
AWS deployment:
• https://aws.amazon.com/marketplace/pp/Veritas-Technologies-LLC-Veritas-InfoScale-
Enterpr/B07CYGD14V
• https://s3.amazonaws.com/veritas-infoscale-
7.3.1/CFTs/Veritas_Infoscale731_CFT_Deployment_Guide.pdf
Azure deployment:
• https://www.veritas.com/content/support/en_US/doc/infoscale_azure_deploy_armst_lin
• https://azuremarketplace.microsoft.com/en-us/marketplace/eca/3909
− For InfoScale Storage (only): https://azuremarketplace.microsoft.com/en-
us/marketplace/eca/3908
− For other InfoScale solutions in Azure: https://azuremarketplace.microsoft.com/en-
us/marketplace/eca?page=1&search=veritas%20InfoScale

01-39
Topic: Installing Veritas InfoScale Storage
After completing this topic, you will be able to perform basic installation
and configuration of InfoScale Storage.
40
This is the Installing Veritas InfoScale Storage topic.


01-40
Introducing Veritas InfoScale Storage
Combines industry-leading File System and Volume Manager

technology
Provides heterogeneous online storage management
Increases storage utilization
Enhances storage I/O path availability
Delivers predictable Quality-of-Service
Maximizes storage efficiency, data availability, operating system

agility, and performance
41
Veritas InfoScale Storage, provides the following benefits and features:

• Combines industry-leading File System and Volume Manager technology
• Provides heterogeneous online storage management
• Increases storage utilization
• Enhances storage I/O path availability
• Delivers predictable Quality-of-Service
• Maximizes storage efficiency, data availability, operating system agility, and performance

01-41
InfoScale Storage product components
Includes Veritas File System (VxFS) and Veritas Volume Manager (VxVM).
Storage Foundation In addition to all InfoScale Foundation functionality, this component unlocks
(SF) features such as:
• Compression • Replicator Option

• Deduplication • SmartIO
• FlashSnap • SmartMove
• Hot-relocation • Docker support
• ODM and CODM • User and group quotas.
Storage Foundation Cluster Includes Cluster File System (CFS) and Cluster Volume Manager (CVM). In addition to
File System (SFCFS). all InfoScale Storage functionality, features unlocked by this component are:
• Application isolation (technology preview)

• Concurrent I/O
• Flexible Storage Sharing
• Heartbeat networks
• I/O fencing
42
Storage Foundation (SF) - Includes Veritas File System (VxFS) and Veritas Volume Manager
(VxVM). In addition to all InfoScale Foundation functionality, this component unlocks the
features that are displayed on the slide. Storage Foundation Cluster File System (SFCFS) -
Includes Cluster File System (CFS) and Cluster Volume Manager (CVM). In addition to all
InfoScale Storage functionality, the features unlocked by this component are displayed on the
slide.

01-42
Inputs required to install and configure InfoScale Storage SF
component
• The system name with the fully-qualified domain name.

• The product license key if there are no plans to use keyless licensing.
43
The following information is required to install and configure the InfoScale Storage SF
component:

01-43
Inputs required to install and configure InfoScale Storage SFCFS
component

• The cluster name and cluster ID
• Network interfaces for cluster interconnect heartbeat links
• Disks or Coordination Point Server information
44
The following information is required to install and configure the InfoScale Storage SFCFS
component:
• The cluster name and cluster ID.
• Network interfaces for cluster interconnect heartbeat links.
• Disks or Coordination Point Server information.

01-44
Topic: Installing Veritas InfoScale Availability
After completing this topic, you will be able to perform basic installation
and configuration of InfoScale Availability.
45
This is the Installing Veritas InfoScale Availability topic.


01-45
Introducing Veritas InfoScale Availability
Keeps critical business services up and running
Provides resiliency across physical and virtual environments
Delivers high availability with a robust software-defined

approach
Maximizes IT service continuity
46
Veritas InfoScale Availability keeps critical business services up and running. It provides
resiliency across physical and virtual environments and delivers high availability with a robust
software-defined approach. It also maximizes IT service continuity.

01-46
Key features of InfoScale Availability
InfoScale Availability
• Adaptive HA
• FireDrill
• Global Cluster Option (GCO)
• Intelligent Monitoring Framework (IMF)
• Just In Time(JIT) target VM Availability (VMware) for
Planned Failover
• Replication agents
• Virtual Business Service
47
The Key features of InfoScale Availability are:

• Adaptive HA
• FireDrill
• Global Cluster Option (GCO)
• Intelligent Monitoring Framework (IMF)
• Just In Time(JIT) target VM Availability (VMware) for Planned Failover
• Replication agents
• Virtual Business Service

• Application HA

01-47
Hardware and software recommendations
– System hardware
– Storage HBAs

48
The Hardware and software recommendations are as follows:

• Eliminate single points of failure.
− System hardware
− Storage HBAs

01-48
Inputs required to install InfoScale Availability

• The product license key if there are no plans to use keyless
licensing
Before you start • The cluster name and cluster ID
the installation • Network interfaces for cluster interconnect heartbeat links
• The cluster virtual IP address, NIC, and netmask

• Setting cluster secure mode communication
• VCS user names and passwords (Default admin / password)
Optional inputs • SMTP server host name
• SNMP console host name, trap port, and message levels
49
The following information is required to install InfoScale Availability successfully:

• Optional inputs
− The cluster virtual IP address, NIC, and netmask
− Setting cluster secure mode communication
− VCS user names and passwords (Default admin / password)

− SMTP server host name
− SNMP console host name, trap port, and message levels

01-49
Topic: Upgrading Veritas InfoScale Enterprise
After completing this topic, you will be able to perform a basic upgrade of
the SFHA component of InfoScale Enterprise.
50
This is the Upgrading Veritas InfoScale Enterprise topic.


01-50
Introducing Veritas InfoScale Enterprise
Addresses enterprise IT service continuity needs
Provides resiliency and software defined storage for critical

services
Realizes better ROI and unlocks high performance by

integrating next-generation storage technologies
Provides high availability and disaster recovery
Protects complex multi-tiered applications across any

distance in physical and virtual environments
51
Veritas InfoScale Enterprise Addresses enterprise IT service continuity needs. It provides

resiliency and software defined storage for critical services, realizes better ROI, and unlocks
high performance by integrating next-generation storage technologies. It provides high
availability and disaster recovery and also protects complex multi-tiered applications across
any distance in physical and virtual environments.

01-51
InfoScale Enterprise product components
Product Components Key features
• Cluster Server (VCS) • Sybase ASE CE integration

• Storage Foundation (SF) • Oracle RAC integration
• Storage Foundation and High Availability • Plus InfoScale Storage features
(SFHA) • Plus InfoScale Availability features
• Storage Foundation Cluster File System HA
(SFCFSHA)
• Storage Foundation for Oracle RAC (SF
Oracle RAC)
52
The InfoScale Enterprise product components are as follows:

• Cluster Server (VCS)
• Storage Foundation (SF)
• Storage Foundation and High Availability (SFHA)
• Storage Foundation Cluster File System HA (SFCFSHA)
• Storage Foundation for Oracle RAC (SF Oracle RAC)

01-52
Hardware and software recommendations
– System hardware
– Storage HBAs

53
The hardware and software recommendations for InfoScale Enterprise are as follows:
− System hardware
− Storage HBAs

01-53
Inputs required to install InfoScale Enterprise
Keep the following information handy before

Optional inputs
you start the installation:
• The system name with the fully-qualified • The cluster virtual IP address, NIC, and
domain name netmask
• The product license key if there are no plans • Setting cluster secure mode communication
to use keyless licensing • VCS user names and passwords (Default
• The cluster name and cluster ID admin / password)
• Network interfaces for cluster interconnect • SMTP server host name
heartbeat links • SNMP console host name, trap port, and
message levels
54
The information required to install InfoScale Enterprise is as follows:

• Optional inputs
− The cluster virtual IP address, NIC, and netmask
− Setting cluster secure mode communication
− VCS user names and passwords (Default admin / password)

− SMTP server host name
− SNMP console host name, trap port, and message levels

01-54
Preparing for an upgrade to Veritas InfoScale
Tasks to complete before upgrading to InfoScale
• Review the Veritas InfoScale Release Notes

• Review the Veritas Technical Support website for additional information
• Review product documentation that includes but is not limited to the InfoScale Configuration
and Upgrade Guide, the HCL, the SCL, and other product guides.
• Make sure that all users are logged off and that all major user applications are properly shut
down
• Make sure that you have created a valid backup
• Ensure that you have enough file system space to upgrade
• For any startup scripts in /etc/init.d/, comment out any application commands or
processes that are known to hang if their file systems are not present
• Schedule sufficient outage time and downtime for the upgrade
• Make sure that the file systems are clean before upgrading
55
Ensure that you complete the following tasks before upgrading to InfoScale:
• Review the Veritas InfoScale Release Notes
• Review the Veritas Technical Support website for additional information
• Review product documentation that includes but is not limited to the InfoScale
Configuration and Upgrade Guide, the HCL, the SCL, and other product guides.
• Make sure that all users are logged off and that all major user applications are properly
shut down
• Make sure that you have created a valid backup
• Ensure that you have enough file system space to upgrade
• For any startup scripts in /etc/init.d/, comment out any application commands or processes
that are known to hang if their file systems are not present
• Schedule sufficient outage time and downtime for the upgrade
• Make sure that the file systems are clean before upgrading

01-55
Inputs required to upgrade InfoScale Enterprise
Details of the existing Veritas deployment such as:

• The system names with the fully-qualified domain name
• The cluster name
56
• The system names with the fully-qualified domain name
• The cluster name

01-56
Lesson summary
• Key points
– In this lesson, you learned about the InfoScale product suite.
– You also learned about the products of the InfoScale family.
– In addition, you learned about the features of the InfoScale product family and InfoScale cloud offerings.
– Finally, you learned about the Installation procedure, InfoScale Licensing pattern, and deployment tools used for
InfoScale products.
• Reference materials
– Veritas InfoScale Enterprise Administrator’s Guide
– https://sort.veritas.com
– http://www.veritas.com/support
– InfoScale 7.4.2 Document list:
• https://sort.veritas.com/sitemap/document/25
• https://www.veritas.com/content/support/en_US/doc/79618328-141543577-0/v89676408-141543577
– InfoScale support for Cloud environments: https://www.veritas.com/support/en_US/doc/130803809-141542355-
0/index
57
For more information about the topics discussed in this lesson, refer to the resources listed on
the slide and remember to check the Veritas Support Web site frequently.

01-57
Lab 00 (Lab Intro): Setting up the Lab Environment
• Exercise A: Viewing the virtual machine configuration
• Exercise B: Displaying networking information
58

01-58
Lab 01: Installing InfoScale Storage for using Storage Foundation
• Exercise A: Verifying that the system meets installation requirements

• Exercise B: Installing InfoScale Storage and configuring Storage Foundation
• Exercise C: Performing post-installation and version checks
59

01-59
What did you learn?
You are about to be asked a series
of questions related to the current
lesson.
60
The next section is a quiz. In this quiz, you are asked a series of questions related to the
current lesson.

01-60
Question 1: Operating system support
Which Linux operating systems are supported by InfoScale 7.4.2?
A. RHEL 7.7
B. RHEL 8.1
C. CentOS 7.7
D. RHEL: 7.7 & 8,1 and CentOS 7.7 & 8.1
61

01-61
Answer 1: Operating system support
Which Linux operating systems are supported by InfoScale 7.4.2?
A. RHEL 7.7
B. RHEL 8.1
C. CentOS 7.7
D. RHEL: 7.7 & 8,1 and CentOS 7.7 & 8.1
The correct answer is D. InfoScale 7.4.2 supports both RHEL and CentOS for both 7.7 and 8.1 releases.
62

01-62
Question 2: Cloud solution support
Which cloud environment(s) are supported by InfoScale?
A. AWS, Azure
B. GCP, OpenStack
C. VMware (ESXi), Nutanix
D. All the above
63

01-63
Answer 2: Cloud solution support
Which cloud environment(s) are supported by InfoScale?
A. AWS, Azure
B. GCP, OpenStack
C. VMware (ESXi), Nutanix
D. All the above
The correct answer is D. InfoScale product family supports all the above cloud platforms.
64

01-64
End of presentation
01-65
Lesson 02: Virtual Objects
This is the Virtual Objects lesson in the Veritas InfoScale 7.4.2 Fundamentals for UNIX/Linux:
Administration course.

02-1
Administration
Administration


02-2
Lesson objectives
Topic Objective
• Recall basic storage terminology.
Operating system storage devices and
• Explain the structural characteristics of a disk placed under
virtual data storage
VxVM control.
Identify the virtual objects that are created by VxVM to manage
Volume Manager (VxVM) storage objects data storage, including disk groups, VxVM disks, subdisks,
plexes, and volumes.
Identify virtual storage layout types used by Volume Manager
VxVM volume layouts and RAID levels
(VxVM) to remap address space.

02-3
Topic: Operating system storage devices and virtual data
storage

• Recall basic storage terminology.
• Explain the structural characteristics of a disk placed under VxVM control.
This is the Operating system storage devices and virtual data storage topic.

02-4
Disks at the operating system level
• The basic physical storage device that ultimately

stores your data is the hard disk.
• Each operating system has its own way of
identifying disks and structuring data on disks.
• The operating system must detect the disk first
before Volume Manager can detect and use it.
Each UNIX flavor supported by Storage Foundation has a unique way of detecting and using
storage devices. Some platforms, such as Solaris and Linux, use a partition table and disk
partitions to organize data on the physical disks. Other platforms such as AIX and HP-UX, use
OS-native logical volume management software to detect disks as physical volumes.
Storage Foundation hides the complexity of the device management layer by introducing a
virtual data layer that works the same on all the supported UNIX platforms. The way Volume
Manager uses disks to organize data is explained in detail later in this lesson.
However, the key point to note is that the Volume Manager can only use a device if it is
recognized by the operating system on the Storage Foundation host. Therefore, if a disk
device is not visible in the Volume Manager, you first have to ensure that the operating system
detects it correctly.

02-5
Operating system disk naming
Operating systems have different naming conventions for disks:
Operating system Device naming convention example

Solaris /dev/[r]dsk/c1t9d0s2
HP-UX /dev/[r]dsk/c3t2d0 (no slice)

/dev/[r]disk/disk22
AIX /dev/hdisk2 (no slice)
SCSI disks:
/dev/sdf (no slice)
/dev/sdf3
Linux IDE disks:
/dev/hda (no slice)
/dev/hda8
Use the following OS-specific commands to list storage devices on individual platforms. Refer
to manual pages for specific command syntax.
Solaris
You locate and access the data on a physical disk by using a device name that specifies the
controller, target ID, and disk number. A typical device name uses the format: c#t#d#.
• c# is the controller number.
• t# is the target ID.
• d# is the logical unit number (LUN) of the drive attached to the target.
If a disk is divided into partitions, you also specify the partition number in the device name:
s# is the partition (slice) number. For example, the device name c0t0d0s1 is connected to
controller number 0 in the system, with a target ID of 0, physical disk number 0, and partition
number 1 on the disk.
HP-UX
Traditionally, you locate and access the data on a physical disk by using a device name that
specifies the controller, target ID, and disk number. A typical traditional device name uses the
format: c#t#d#.
• c# is the controller number.
• t# is the target ID.
• d# is the logical unit number (LUN) of the drive attached to the target.

02-6
For example, the c0t0d0 device name is connected to the controller number 0 in the system,
with a target ID of 0, and the physical disk number 0.
With HP-UX 11iv3, a new method called agile view has been introduced. The new convention
uses the /dev/[r]disk/diskN name where N is the decimal instance number for the
disk. This is called a persistent device special file name. The persistent device special file
names are not available before HP-UX 11iv3.
AIX
Every device in AIX is assigned a location code that describes its connection to the system.
The general format of this identifier is AB-CD-EF-GH, where the letters represent decimal
digits or uppercase letters. The first two characters represent the bus, the second pair identify
the adapter, the third pair represent the connector, and the final pair uniquely represent the
device. For example, a SCSI disk drive might have a location identifier of 04-01-00-6,0. In this
example, 04 means the PCI bus, 01 is the slot number on the PCI bus occupied by the SCSI
adapter, 00 means the only or internal connector, and the 6,0 means SCSI ID 6, LUN 0.
However, this data is used internally by AIX to locate a device. The device name that a system
administrator or software uses to identify a device is less hardware dependent. The system
maintains a special database called the Object Data Manager (ODM) that contains essential
definitions for most objects in the system, including devices. Through the ODM, a device
name is mapped to the location identifier. The device names are referred to by special files
found in the /dev directory. For example, the SCSI disk identified previously might have the
device name hdisk3 (the fourth hard disk identified by the system). The device named hdisk3
is accessed by the file name /dev/hdisk3.
If a device is moved so that it has a different location identifier, the ODM is updated so that it
retains the same device name, and the move is transparent to users. This is facilitated by the
physical volume identifier stored in the first sector of a physical volume. This unique 128-bit
number is used by the system to recognize the physical volume wherever it may be attached
because it is also associated with the device name in the ODM.
Linux
On Linux, device names are displayed in the format:
• sdx[N]
• hdx[N]
In the syntax, sd refers to a SCSI disk, and hd refers to an EIDE disk.
• x is a letter that indicates the order of disks detected by the operating system. For
example, sda refers to the first SCSI disk, sdb refers to the second SCSI disk, and so on.
• N is an optional parameter that represents a partition number in the range 1 through 16.
For example, sda7 references partition 7 on the first SCSI disk.
Primary partitions on a disk are 1, 2, 3, 4; logical partitions have numbers 5 and up. If the
partition number is omitted, the device name indicates the entire disk.

02-7
Physical data storage
Users Applications Databases
• Reads and writes to

physical disks can be slow.
• Disk arrays can improve
I/O throughput and
redundancy.
Physical disks/LUNs
• Disk array: A system containing multiple physical disks used to create LUNs
• LUN: A logical representation of storage space that is recognized by the
operating system as a uniquely identifiable storage device
• Multipathing: Multiple access routes to LUNs in a disk array to achieve
performance and redundancy
Note: Throughout this course, the term disk is used to mean either a physical
disk or LUN.
Disk arrays
Reads and writes on unmanaged physical disks can be a relatively slow process, because disks
are physical devices that require time to move the heads to the correct position on the disk
before reading or writing. If all the read and write operations are performed to individual
disks, one at a time, the read-write time can become unmanageable.
A disk array is a collection of physical disks. Performing I/O operations on multiple disks in a
disk array can improve I/O speed and throughput.
Hardware arrays present disk storage to the host operating system as LUNs. A LUN can be
made up of a single physical disk, a collection of physical disks, or even a portion of a physical
disk. From the operating system point of view, a LUN corresponds to a single storage device.
Multipathing
Some disk arrays provide multiple ports to access disk devices. These ports, coupled with the
host bus adaptor (HBA) controller and any data bus or I/O processor local to the array,
compose multiple hardware paths to access the disk devices. This is called multipathing.
In a multipathing environment, a single storage device may appear to the operating system as
multiple storage devices. Special multipathing software is usually required to administer
multipathed storage devices. Veritas Dynamic Multi-Pathing (DMP) product which is part of
the Storage Foundation software provides seamless management of multiple access paths to
storage devices in heterogeneous operating systems and storage environments.

02-8
Example array structure
LUNs are a virtual presentation.
Array with 12 physical disks
Array-based mirrored RAID

group composed of two
physical disks.
Six RAID groups created
Twelve
array-based
LUNs
In an array, the LUNs are a virtual presentation. Therefore, you cannot know where in the
array the actual data will be put. That means you have no control over the physical conditions.
The array in the slide contains slots for 14 physical disks, and the configuration places 12
physical disks in the array. These physical disks are paired together into 6 mirrored RAID
groups. In each RAID group, 12 logical units, or LUNs, are created. These LUNs appear to hosts
as SAN-based SCSI disks. The remaining two disks are used as spares in case one of the active
disks fails.

02-9
Virtual data storage
• Volume Manager volumes appear to Operating
applications as disk devices. system
• Volumes accessed by OS device files:

/dev/vx/[r]dsk/… Application
• Volume Manager supports:
– Online administration
– Various multi-disk configurations
VxVM
– High availability volume
– Dynamic multipathing
– Hot relocation
Physical
disks/
LUNs
10
Virtual storage management

Veritas Volume Manager creates a virtual level of storage management above the physical
device level by creating virtual storage objects. The virtual storage object visible to users and
applications is called a volume.
What is a volume?
A volume is a virtual object, created by the Volume Manager, that stores data. A volume
consists of space from one or more physical disks on which the data is physically stored.
How do you access a volume?
Volumes created by VxVM appear to the operating system as physical disks, and applications
that interact with volumes work in the same way as with physical disks. All users and
applications access volumes as contiguous address space using special device files in a
manner similar to accessing a disk partition.
Volumes have block and character device nodes in the /dev tree. You can supply the name of
the path to a volume in your commands and programs, in your file system and database
configuration files, and in any other context where you would otherwise use the path to a
physical disk partition.

02-10
Volume Manager control
• Entire disk under VxVM control
• Cross-platform data sharing (CDS) format
‒ Accessible on different platforms
‒ Independent of where the disk was initialized
CDS disk
(default) Offset (128K)
Private region (32 MB)
OS-reserved areas: (metadata)
• Platform blocks
Metadata
• VxVM ID blocks
• AIX and HP-UX Public region
coexistence labels (user data)
User
data
11
Volume Manager-controlled disks

With Volume Manager, you can enable virtual data storage by bringing a disk under Volume
Manager control. By default, Volume Manager uses a cross-platform data sharing (CDS) disk
layout. A CDS disk is consistently recognized by all VxVM-supported UNIX platforms and
consists of:
• OS-reserved area: To accommodate platform-specific disk usage, 128K is reserved for disk
labels, platform blocks, and platform-coexistence labels.
• Private region: The private region stores information, such as disk headers, configuration
copies, and kernel logs, in addition to other platform-specific management areas that
VxVM uses to manage virtual objects. The private region represents a small management
overhead.
• Public region: The public region consists of the remainder of the space on the disk. The
public region represents the available space that Volume Manager can use to assign to
volumes and is where an application stores data. Volume Manager never overwrites this
area unless specifically instructed to do so.

02-11
Comparing CDS and other VxVM disk formats
Sliced disk Simple disk

CDS disk
VxVM sliced format (AIX: aixdisk format)
VxVM cdsdisk format (Default)
(Solaris and Linux) (HP-UX: hpdisk format)
• Private and public regions on • Private and public regions • Private and public regions at
a single disk slice on separate disk slices specific disk offsets
• Portable between different • Not portable between • Not portable between
operating systems different operating systems different operating systems
• Unsuitable for OS boot • Suitable for OS boot • Suitable for HP-UX boot
partitions partitions partitions (not AIX)
12
In addition to the default CDS disk format, Volume Manager supports other platform-specific
disk formats. These disk formats are used for bringing the boot disk under VxVM control on
operating systems that support that capability.
On platforms that support bringing the boot disk under VxVM control, CDS disks cannot be
used for boot disks. CDS disks have specific disk layout requirements that enable a common
disk layout across different platforms, and these requirements are not compatible with the
particular platform-specific requirements of boot disks. Therefore, when placing a boot disk
under VxVM control, you must use a non-default disk format (sliced on Solaris and Linux,
hpdisk on HP-UX).
For non-boot disks, you can convert CDS disks to other disk layout formats and vice versa by
using VxVM utilities.

02-12
Topic: Volume Manager (VxVM) storage objects
After completing this topic, you will be able to identify the virtual objects
created by VxVM to manage data storage, including disk groups, VxVM
disks, subdisks, plexes, and volumes.
13
This is the Volume Manager (VxVM) storage objects topic.


02-13
Volume Manager storage objects
Disk group: acctdg
acctdg
expvol payvol
Volume:
• expvol
• payvol
acctdg01-01 acctdg01-02
Plex: acctdg02-01 acctdg03-02
acctdg02-02
• expvol-01 acctdg03-01
• payvol-01|02 expvol-01 payvol-01 payvol-02
Subdisks:
• acctdg01-01|02 acctdg01 acctdg02 acctdg03
• acctdg02-01|02
• acctdg03-01|02 acctdg01-01 acctdg02-01 acctdg03-01
acctdg01-02 acctdg02-02 acctdg03-02
VxVM disks:
• acctdg01
• acctdg02
• acctdg03 Disk1 Disk2 Disk3
OS disks
14
Disk groups
A disk group is a collection of VxVM disks that share a common configuration. Disks are
grouped into disk groups for management purposes, such as to hold the data for a specific
application or set of applications. For example, data for accounting applications can be
organized in a disk group called acctdg. A disk group configuration is a set of records with
detailed information about related Volume Manager objects in a disk group, their attributes,
and their connections.
Volume Manager objects cannot span disk groups. For example, a volume’s sub disks, plexes,
and disks must be derived from the same disk group as the volume. You can create additional
disk groups as necessary. Disk groups enable you to group disks into logical collections. Disk
groups and their components can be moved as a unit from one host machine to another.
Volume Manager disks
A Volume Manager (VxVM) disk represents the public region of a physical disk that is under
Volume Manager control. Each VxVM disk corresponds to one physical disk. Each VxVM disk
has a unique virtual disk name called a disk media name. The disk media name is a logical
name used for Volume Manager administrative purposes. Volume Manager uses the disk
media name when assigning space to volumes. A VxVM disk is given a disk media name when
it is added to a disk group.

02-14
Default disk media name: diskgroup##
You can supply the disk media name or allow Volume Manager to assign a default name. The
disk media name is stored with a unique disk ID to avoid name collision. After a VxVM disk is
assigned a disk media name, the disk is no longer referred to by its physical address. The
physical address (for example, c#t#d# or hdisk#) becomes known as the disk access record.
Subdisks
A VxVM disk can be divided into one or more subdisks. A subdisk is a set of contiguous disk
blocks that represent a specific portion of a VxVM disk, which is mapped to a specific region
of a physical disk. A subdisk is a subsection of a disk’s public region. A subdisk is the smallest
unit of storage in Volume Manager. Therefore, subdisks are the building blocks for Volume
Manager objects.
A subdisk is defined by an offset and a length in sectors on a VxVM disk.
Default subdisk name: DMname-##
A VxVM disk can contain multiple subdisks, but subdisks cannot overlap or share the same
portions of a VxVM disk. Any VxVM disk space that is not reserved or that is not part of a
subdisk is free space. You can use free space to create new subdisks.
Conceptually, a subdisk is similar to a partition. Both a subdisk and a partition divide a disk
into pieces defined by an offset address and length. Each of those pieces represent a
reservation of contiguous space on the physical disk. However, while the maximum number of
partitions to a disk is limited by some operating systems, there is no theoretical limit to the
number of subdisks that can be attached to a single plex. This number has been limited by
default to a value of 4096. If required, this default can be changed, using the vol_subdisk_num
tunable parameter. For more information on tunable parameters, see the Veritas Storage
Foundation and High Availability Solutions Tuning Guide.
Plexes
Volume Manager uses subdisks to build virtual objects called plexes. A plex is a structured or
ordered collection of subdisks that represents one copy of the data in a volume. A plex
consists of one or more subdisks located on one or more physical disks. The length of a plex is
determined by the last block that can be read or written on the last subdisk in the plex.
Default plex name: volume_name-##
Volumes
A volume is a virtual storage device that is used by applications in a manner similar to a
physical disk. Due to its virtual nature, a volume is not restricted by the physical size
constraints that apply to a physical disk. A VxVM volume can be as large as the total of
available, unreserved free physical disk space in the disk group. A volume consists of one or
more plexes.

02-15
Topic: VxVM volume layouts and RAID levels
After completing this topic, you will be able to identify virtual storage
layout types used by VxVM to remap address space.
16
This is the VxVM volume layouts and RAID levels topic.


02-16
Volume layouts
Disk spanning Data redundancy (parity)
Striped RAID-5
Concatenated
(RAID-0) (RAID-5)
Data redundancy with mirroring

Striped and Mirrored and
Mirrored
mirrored striped (Layered)
(RAID-1)
(RAID-0+1) (RAID-1+0)
17
VxVM volume layouts and RAID levels

RAID
RAID is an acronym for Redundant Array of Independent Disks. RAID is a storage management
approach in which an array of disks is created, and part of the combined storage capacity of
the disks is used to store duplicate information about the data in the array. By maintaining a
redundant array of disks, you can regenerate data in the case of disk failure.
RAID configuration models are classified in terms of RAID levels, which are defined by the
number of disks in the array, the way data is spanned across the disks, and the method used
for redundancy. Each RAID level has specific features and performance benefits that involve a
trade-off between performance and reliability.
Volume layouts
RAID levels correspond to volume layouts. A volume’s layout refers to the organization of
plexes in a volume. Volume layout is the way plexes are configured to remap the volume
address space through which I/O is redirected at run-time. Volume layouts are based on the
concepts of disk spanning, redundancy, and resilience.
Disk spanning
Disk spanning is the combination of disk space from multiple physical disks to form one logical
drive. Disk spanning has two forms:
• Concatenation: Concatenation is the mapping of data in a linear manner across two or
more disks.

02-17
In a concatenated volume, subdisks are arranged both sequentially and contiguously
within a plex. Concatenation allows a volume to be created from multiple regions of one
or more disks if there is not enough space for an entire volume on a single region of a
disk.
• Striping: Striping is the mapping of data in equally-sized chunks alternating across multiple
disks. Striping is also called interleaving.
In a striped volume, data is spread evenly across multiple disks. Stripes are equally-sized
fragments that are allocated alternately and evenly to the subdisks of a single plex. There
must be at least two subdisks in a striped plex, each of which must exist on a different
disk. Configured properly, striping not only helps to balance I/O but also to increase
throughput.
Data redundancy
To protect data against disk failure, the volume layout must provide some form of data
redundancy. Redundancy is achieved in two ways:
• Mirroring: Mirroring is maintaining two or more copies of volume data.
A mirrored volume uses multiple plexes to duplicate the information contained in a
volume. Although a volume can have a single plex, at least two are required for true
mirroring (redundancy of data). Each of these plexes should contain disk space from
different disks for the redundancy to be useful.
• Resilience: A resilient volume, also called a layered volume, is a volume that is built on
one or more other volumes. Resilient volumes enable the mirroring of data at a more
granular level. For example, a resilient volume can be concatenated or striped at the top
level and then mirrored at the bottom level.
A layered volume is a virtual Volume Manager object that nests other virtual objects
inside of itself. Layered volumes provide better fault tolerance by mirroring data at a more
granular level.
• Parity: Parity is a calculated value used to reconstruct data after a failure by doing an
exclusive OR (XOR) procedure on the data. Parity information can be stored on a disk. If
part of a volume fails, the data on that portion of the failed volume can be re-created from
the remaining data and parity information.
A RAID-5 volume uses striping to spread data and parity evenly across multiple disks in an
array. Each stripe contains a parity stripe unit and data stripe units. Parity can be used to
reconstruct data if one of the disks fails. In comparison to the performance of striped
volumes, write throughput of RAID-5 volumes decreases, because parity information
needs to be updated each time data is accessed. However, in comparison to mirroring,
the use of parity reduces the amount of space required.

02-18
Erasure-coded Volume layout
19
Erasure coding is a volume layout with an advantage of storage saving. Mirroring can provide
fault tolerance just like Erasure coding but will have huge storage overhead. So, it is treated as
a feature rather than a new area.
Erasure coding is a new feature available as a technology preview in Veritas InfoScale for
configuration and testing in non-production environments. It is supported in DAS, SAN, FSS,
and standalone environments.
As storage systems expand and become more complex, traditional data protection
mechanisms are proving to be inadequate against failures. Erasure coding offers a more
robust solution in redundancy and fault tolerance for critical storage archives. In erasure
coding, data is broken into fragments, expanded and encoded with redundant data pieces and
stored across different locations or storage media. When one or more disks fail, the data on
failed disks is reconstructed using the parity information in the encoded disks and data in the
surviving disks. Erasure coded volumes must be created using disk group version 230 or later.
When you create an erasure coded volume, Veritas InfoScale, by default, runs asynschronous
initialization on the volume ensuring that data and parities are synchronized for all regions.
The operation runs in the background allowing the volume to be available for use to
applications immediately after creation. The volumes display the SYNC state after creation
until all the regions are synchronized. This functionality is supported on both private and
shared/FSS disk groups.
You can manually initialize an erasure coded volume by setting init=zero at the time of
creating the volume. The initialization zeroes out all the regions and the volume is not
available for use until the initialization process completes.

02-19
Lesson summary
• Key points
– In this lesson, you learned about the basic storage terminology, such as physical disks, LUNs, disk arrays,
and multipathing.
– You also learned about the structural characteristics of a disk after placing it under Volume Manager
control.
– In addition, you learned about the virtual objects created by the Volume Manager to manage data
storage, including disk groups, Volume Manager disks, subdisks, plexes, and volumes.
– Finally, you learned about the VxVM volume layout types and the corresponding RAID levels that are
used by Volume Manager to remap address space.
– Storage Foundation Administrator’s Guide
– http://www.veritas.com/support
20

02-20
Lab 02: Virtual Objects
• Exercise A: Text-based VxVM menu interface

• Exercise B: Accessing CLI commands
• Exercise C: Adding managed hosts to the VIOM Management Server
21

02-21
What did you learn?
lesson.
22
current lesson.

02-22
Question 1: Volume Manager objects
Which Volume Manager object is a set of contiguous disk blocks that represent a specific
portion of a VxVM disk?
A. Volume
B. Plex
C. Partition
D. Subdisk
23

02-23
Answer 1: Volume Manager objects
Which Volume Manager object is a set of contiguous disk blocks that represent a specific
portion of a VxVM disk?
A. Volume
B. Plex
C. Partition
D. Subdisk
The correct answer is D. A subdisk is the smallest unit of storage in Volume Manager, which is mapped to a specific
region of a physical disk. A subdisk is defined by an offset and a length in sectors on a VxVM disk.
24

02-24
Question 2: Volume Manager regions
When a disk is brought under Volume Manager control, the _________ stores information that
VxVM uses to manage virtual objects and represents a small management overhead.
A. Public region
B. TOC region
C. Virtual region
D. Private region
25

02-25
Answer 2: Volume Manager regions
When a disk is brought under Volume Manager control, the _________ stores information that
VxVM uses to manage virtual objects and represents a small management overhead.
A. Public region
B. TOC region
C. Virtual region
D. Private region
The correct answer is D. The private region stores information, such as disk headers, configuration copies, and kernel
logs, in addition to other platform-specific management areas that VxVM uses to manage virtual objects.
26

02-26
Question 3: Characteristic of CDS disks
Select the characteristic that describes CDS disks.
A. Can only be accessed from the platform on which it was initialized

B. Is suitable for boot partitions
C. Is suitable for moving between different operating systems
D. Is the default disk type in Volume Manager 3.5 and earlier
27

02-27
Answer 3: Characteristic of CDS disks
Select the characteristic that describes CDS disks.
A. Can only be accessed from the platform on which it was initialized

B. Is suitable for boot partitions
C. Is suitable for moving between different operating systems
D. Is the default disk type in Volume Manager 3.5 and earlier
The correct answer is C. By default, Volume Manager uses a cross-platform data sharing (CDS) disk layout. A CDS disk is
consistently recognized by all VxVM-supported UNIX platforms .
28

02-28
Question 4: Management and configuration boundaries
____________ represent management and configuration boundaries and enable you to
organize disks into logical collections.
A. Volumes
B. Plexes
C. Subdisks
D. Disk groups
29

02-29
Answer 4: Management and configuration boundaries
____________ represent management and configuration boundaries and enable you to
organize disks into logical collections.
A. Volumes
B. Plexes
C. Subdisks
D. Disk groups
The correct answer is D. A disk group is a collection of VxVM disks that share a common configuration.
30

02-30
Question 5: Volume layout
The volume layout that allows a volume to be created from multiple regions of one or more
disks if there is not enough space on a single region of a disk is called:
A. Concatenated
B. Striped
C. RAID-5
D. Mirrored
31

02-31
Answer 5: Volume layout
The volume layout that allows a volume to be created from multiple regions of one or more
disks if there is not enough space on a single region of a disk is called:
A. Concatenated
B. Striped
C. RAID-5
D. Mirrored
The correct answer is A. Concatenation is the mapping of data in a linear manner across two or more disks.
Concatenation allows a volume to be created from multiple regions of one or more disks if there is not enough space for
an entire volume on a single region of a disk.
32

02-32
End of presentation
02-33
Lesson 03: Creating a Volume and File System
This is the Creating a Volume and File System lesson in the Veritas InfoScale 7.4.2

03-1
Administration
Administration


03-2
Lesson objectives
Topic Objective
Preparing disks and disk groups for Initialize an OS disk as a VxVM disk and create a disk group by using
volume creation command-line utilities.
Creating a volume and adding a file Create a concatenated volume, add a file system to an existing volume, and
system mount the file system.
Displaying disk and disk group

View disk and disk group information and identify disk status.
information
Displaying volume configuration

Display volume layout information.
information
Removing volumes, disks, and disk Remove a volume, evacuate a disk, remove a disk from a disk group,
groups destroy a disk group, and shred a disk.

03-3
Topic: Preparing disks and disk groups for volume
creation
After completing this topic, you will be able to initialize an OS disk as a

VxVM disk and create a disk group by using command-line utilities.
This is the Preparing disks and disk groups for volume creation topic.

03-4
Selecting a disk naming scheme
Host
Fibre Channel Types of naming schemes:
hub/switch
• OS-based device naming (OSN)
• Enclosure-based naming (EBN) (default )
enc3
enc2
Disk
enc1
enclosures
EBN OSN OS device file Description

emc0_dd1 sdf /dev/sdf LUN from an EMC Symmetrix array
3pardata0_49 sdd /dev/sdd LUN from a 3PAR array
disk_1 sdb /dev/sdb Disk from a JBOD array
Physical disk that does not return a
sda sda /dev/sda
path-independent identifier to VxVM
An enclosure, or disk enclosure, is an intelligent disk array, which permits hot swapping of
disks. With Storage Foundation, disk devices can be named for enclosures rather than for the
controllers through which they are accessed as with standard disk device naming (for
example, c0t0d0 or hdisk2).
Enclosure-based naming allows Storage Foundation to access enclosures as separate physical
entities. By configuring redundant copies of your data on separate enclosures, you can
safeguard against failure of one or more enclosures. This is especially useful in a storage area
network (SAN) that uses Fibre Channel hubs or fabric switches and when managing the
dynamic multipathing (DMP) feature of Storage Foundation. For example, if two paths
(c1t99d0 and c2t99d0) exist to a single disk in an enclosure, VxVM can use a single DMP
metanode, such as enc0_0, to access the disk.

The table shows some examples of naming schemes.

03-5
Note: With SF 5.1 and later, the enclosure-based naming scheme is the default naming
scheme for all fresh SF installations. Upgrades to a new version of SF preserve the naming
scheme set by the user in previous versions of SF.
Benefits of enclosure-based naming include:
• Easier fault isolation: Storage Foundation can more effectively place data and metadata to
ensure data availability.
• Device-name independence: Storage Foundation is independent of arbitrary device
names used by third-party drivers.
• Improved SAN management: Storage Foundation can create better location identification
information about disks in large disk farms and SANs.
• Improved cluster management: In a cluster environment, disk array names on all hosts in
a cluster can be the same. Storage Foundation 5.0 MP3 and later provide consistent
enclosure-based device names across systems in a cluster; that is each LUN in a disk array
is indexed using the same number on different systems sharing storage in the same
cluster.
• Improved dynamic multipathing (DMP) management: With multipathed disks, the name
of a disk is independent of the physical communication paths, avoiding confusion and
conflict.
You can use the vxddladm command to determine the current naming scheme as follows::
vxddladm get namingscheme
NAMING_SCHEME PERSISTENCE LOWERCASE USE_AVID
======================================================
OS Native No Yes Yes
You can change the naming scheme from the command line using the vxddladm set
command with the following options:
vxddladm set namingscheme=<osn|ebn> \
[persistent=<yes|no>] [lowercase=<yes|no>] \
[use_avid=<yes|no>]
If you set the use_avid option to yes, the LUNs are numbered based on the array volume
ID instead of the traditional indexing method.
You can also change the device naming scheme using the Change the disk naming scheme
option in the vxdiskadm menu.

03-6
Disk configuration stages
emc0_dd1
Stage 1:
Initialize disk.
emc0_dd1
Stage 2:
Assign disk
to disk group.
appdg01
Disk name (OSN): sdf
Disk name (EBN): emc0_dd1 Stage 3:
Disk group: appdg Assign disk space to
Disk media name: appdg01 volumes.
DMP disk device file:
/dev/vx/[r]dmp/emc0_dd1 Volume
Disk access name
Before configuring a disk for use by VxVM

In order to use the space of a physical disk to build VxVM volumes, you must place the disk
under Volume Manager control. Before a disk can be placed under Volume Manager control,
the disk media must be formatted outside of VxVM using standard operating system
formatting methods. SCSI disks are usually preformatted. After a disk is formatted, the disk
can be initialized for use by Volume Manager. In other words, disks must be detected by the
operating system, before VxVM can detect the disks.
Stage one: Initialize a disk
A formatted physical disk is considered uninitialized until it is initialized for use by VxVM.
When a disk is initialized, the public and private regions are created, and VM disk header
information is written to the private region. Any partitions (other than slice 2 on the Solaris
platform) that may have existed on the disk are removed.
These disks are under Volume Manager control but cannot be used by Volume Manager until
they are added to a disk group.
Note: Encapsulation is another method of placing a disk under VxVM control in which existing
data on the disk is preserved.
Changing the disk layout
To display or change the default values that are used for initializing disks, select the
Change/display the default disk layouts option in vxdiskadm:

03-7
• For disk initialization, you can change the default format and the default length of the
private region. If the attribute settings for initializing disks are stored in the user-created
file, /etc/default/vxdisk, they apply to all disks to be initialized
• On Solaris for disk encapsulation, you can additionally change the offset values for both
the private and public regions. To make encapsulation parameters different from the
default VxVM values, create the user-defined
/etc/default/vxencap file and place the parameters in this file.
• On HP-UX when converting LVM disks, you can change the default format and the default
private region length. The attribute settings are stored in the
/etc/default/vxencap file.
Stage two: Assign a disk to a disk group
When you add a disk to a disk group, VxVM assigns a disk media name to the disk and maps
this name to the disk access name.
• Disk media name: A disk media name is the logical disk name assigned to a drive by
VxVM. VxVM uses this name to identify the disk for volume operations, such as volume
creation and mirroring.
• Disk access name: A disk access name represents all paths to the device. A disk access
record maps the physical location to the logical name and represents the link between the
disk media name and the disk access name. Disk access records are dynamic and can be
re-created when vxdctl enable is run.
The disk media name and disk access name, in addition to the host name, are written to the
private region of the disk. The disk name field in the private region is used to hold the disk
media name and the devicetag field is used to hold the disk access name. Space in the public
region is made available for assignment to volumes. Whenever the VxVM configuration
daemon is started (or vxdctl enable is run), the system reads the private region on every
disk and establishes the connections between disk access names and disk media names.
After disks are placed under Volume Manager control, storage is managed in terms of the
logical configuration. File systems mount to logical volumes, not to physical partitions. Logical
names, such as
/dev/vx/[r]dsk/diskgroup/volume_name, replace physical locations, such as
/dev/[r]dsk/device_name.
The free space in a disk group refers to the space on all disks within the disk group that has
not been allocated as subdisks. When you place a disk into a disk group, its space becomes
part of the free space pool of the disk group.
Stage three: Assign disk space to volumes
When you create volumes, space in the public region of a disk is assigned to the volumes.
Some operations, such as removal of a disk from a disk group, are restricted if space on a disk
is in use by a volume.

03-8
Disk group purposes
Disk groups enable you to: sysdg webdg
• Group disks for a set of users or applications.

• Easily move data from one host to another.
• Ease administration of high availability environments.
• Support large configurations.
appdg oradg
What is a disk group?

A disk group is a collection of physical disks, volumes, plexes, and subdisks that are used for a
common purpose. A disk group is created when you place at least one disk in the disk group.
When you add a disk to a disk group, a disk group entry is added to the private region header
of that disk. Because a disk can only have one disk group entry in its private region header,
one disk group does not “know about” other disk groups, and therefore disk groups cannot
share resources, such as disk drives, plexes, and volumes.
A volume with a plex can belong to only one disk group, and subdisks and plexes of a volume
must be stored in the same disk group. You can never have an “empty” disk group, because a
disk group with no disks would have no private region available in which to store the disk
group definition. Therefore, you cannot remove all disks from a disk group without destroying
the disk group.
Why are disk groups needed?
Disk groups assist disk management in several ways:
• Disk groups enable the grouping of disks into logical collections for a particular set of users
or applications.
• Disk groups enable data, volumes, and disks to be easily moved from one host machine to
another.
• Disk groups ease the administration of high availability environments. Disk drives can be
shared by two or more hosts, but they can be accessed by only one host at a time. If one
host crashes, the other host can take over its disk groups and therefore its disks.
• A disk group provides the configuration boundary for VxVM objects.

03-9
System-wide reserved disk groups
sysdg webdg
System A
bootdg = sysdg
Reserved defaultdg = acctdg
names:
• bootdg
• defaultdg System B
appdg
• nodg bootdg = nodg

defaultdg = nodg
nodg is the default value for
bootdg and defaultdg.
To display what is set as bootdg or defaultdg:

vxdg bootdg
vxdg defaultdg To set the default disk group:
vxdctl defaultdg diskgroup
10
VxVM has reserved three disk group names that are used to provide boot disk group and
default disk group functionality. The names bootdg, defaultdg, and nodg are system-wide
reserved disk group names and cannot be used as names for any of the disk groups that you
set up.
If you choose to place your boot disk under VxVM control, VxVM assigns bootdg as an alias for
the name of the disk group that contains the volumes that are used to boot the system.
The main benefit of creating a default disk group is that SF commands default to that disk
group if you do not specify a disk group on the command line. defaultdg is an alias for the disk
group name that should be assumed if the -g option is not specified to a command. You can
set defaultdg when you install Veritas Volume Manager (pre-SF 5.1) or anytime after
installation.
By default, both bootdg and defaultdg are set to nodg.
Notes
• The definitions of bootdg and defaultdg are written to the volboot file. The definition of
bootdg results in a symbolic link from the named bootdg in
/dev/vx/dsk and /dev/vx/rdsk.
• The rootdg disk group name is no longer a reserved name for VxVM versions after 4.0. If
you are upgrading from a version of Volume Manager earlier than 4.0 where the system
disk is encapsulated in the rootdg disk group, the bootdg is assigned the value of rootdg
automatically.

03-10
Creating a disk group
• A disk group must contain at least one disk.

• You cannot add a disk to more than one disk group.
• Default disk media names vary with interface used.
‒ In general, diskgroup##
For example: appdg01
‒ Must be unique within a disk group
• You must add a disk to a disk group before you can use it for
creating volumes.
11
A disk must be placed into a disk group before it can be used by VxVM. A disk group cannot
exist without having at least one associated disk. When you create a new disk group, you
specify a name for the disk group and at least one disk to add to the disk group. The disk
group name must be unique for the host machine.
Adding disks
To add a disk to a disk group, you select an uninitialized disk or a free disk. If the disk is
uninitialized, you must initialize the disk before you can add it to a disk group.
Disk naming
When you add a disk to a disk group, the disk is assigned a disk media name. The disk media
name is a logical name used for VxVM administrative purposes.

Notes on disk naming
You can change disk media names after the disks have been added to disk groups. However, if
you must change a disk media name, it is recommended that you make the change before
using the disk for any volumes. Renaming a disk does not rename the subdisks on the disk,
which may be confusing.
Assign logical media names, rather than use the device names, to facilitate transparent logical
replacement of failed disks.

03-11
Creating a disk group: CLI and vxdiskadm
Create a disk group or add disks using vxdiskadm:
Add or initialize one or more disks
Initialize disks:
vxdisksetup -i accessname [attributes]
vxdisksetup -i hds9500-alua0_0 (EBN)
vxdisksetup -i hdisk5 (OSN: AIX)
vxdisksetup -i sdf (OSN:Linux)
Initialize the disk group by adding at least one disk:

vxdg init diskgroup dm_name=accessname
vxdg init appdg appdg01=hds9500-alua0_0
Add more disks to the disk group:

vxdg -g diskgroup adddisk dm_name=accessname
vxdg -g appdg adddisk appdg02=hds9500-alua0_1
12
From the vxdiskadm main menu, select the Add or initialize one or more disks option.
Specify the disk group to which the disk should be added. To add the disk to a new disk group,
you type a name for the new disk group. You use this same menu option to add additional
disks to the disk group.
To verify that the disk group was created, you can use vxdg list.
When you add a disk to a disk group, the disk group configuration is copied onto the disk, and
the disk is stamped with the system host ID.

03-12
Topic: Creating a volume and adding a file system
After completing this topic, you will be able to create a concatenated

volume, add a file system to an existing volume, and mount the file system.
13
This is the Creating a volume and adding a file system topic.


03-13
Creating a volume from the command line
vxassist -g diskgroup make volume_name length
For example:
vxassist -g appdg make appvol 1g
• Block device file:

/dev/vx/dsk/diskgroup/volume_name
• Raw device file:
/dev/vx/rdsk/diskgroup/volume_name
• For example:
/dev/vx/dsk/appdg/appvol
/dev/vx/rdsk/appdg/appvol
14
When you create a volume, you indicate the desired volume characteristics, and VxVM
creates the underlying plexes and subdisks automatically. The VxVM interfaces require
minimal input if you use default settings. For experienced users, the interfaces also enable
you to enter more detailed specifications regarding all aspects of volume creation.
Before you create a volume
Before you create a volume, ensure that you have enough disks to support the layout type.
• A striped volume requires at least two disks.
• A mirrored volume requires at least one disk for each plex. A mirror cannot be on the
same disk that other plexes of the same volume are using.
To create a volume from the command line, you use the vxassist command. In the syntax:
• Use the -g option to specify the disk group in which to create the volume.
• make is the keyword for volume creation.
• volume_name is a name you give to the volume. Specify a meaningful name which is
unique within the disk group.
• length specifies the number of sectors in the volume. You can specify the length by
adding an m, k, g, or t to the length.

03-14
Adding a VxFS file system after volume creation
Step CLI
1 Create the file system on the volume.
2 Create a mount point directory.
3 Mount the file system on the mount point.
15
A file system provides an organized structure to facilitate the storage and retrieval of files. You
can add a file system to a volume when you create a volume or any time after you create the
volume initially.
− When a file system has been mounted on a volume, the
data is accessed through the mount point directory.
− When data is written to files, it is actually written
to the block device file:
/dev/vx/dsk/diskgroup/volume_name.
− When fsck is run on the file system, the raw device
file is checked:
/dev/vx/rdsk/diskgroup/volume_name.
To add a file system to a volume from the command line, you must create the file system,
create a mount point for the file system, and then mount the file system.
Solaris
• To create and mount a VxFS file system:
− mkfs -F vxfs /dev/vx/rdsk/datadg/datavol
− mkdir /data
− mount -F vxfs /dev/vx/dsk/datadg/datavol /data
• To create and mount a UFS file system:
− newfs /dev/vx/rdsk/datadg/datavol
− mkdir /data
− mount /dev/vx/dsk/datadg/datavol /data
03-15
HP-UX
• To create and mount a VxFS file system:
• mkfs -F vxfs /dev/vx/rdsk/datadg/datavol
• mkdir /data
• mount -F vxfs /dev/vx/dsk/datadg/datavol /data
• To create and mount an HFS file system:
• newfs -F hfs /dev/vx/rdsk/datadg/datavol
• mkdir /data
• mount -F hfs /dev/vx/dsk/datadg/datavol /data
AIX
• To create and mount a VxFS file system using mkfs:
• mkfs -V vxfs /dev/vx/rdsk/datadg/datavol
• mkdir /data
• mount -V vxfs /dev/vx/dsk/datadg/datavol /data
• To create and mount a VxFS file system using crfs:
crfs -v vxfs -d /dev/vx/rdsk/datadg/datavol -m /data -A yes
Notes:
• An uppercase V is used with mkfs; a lowercase v is used with crfs (to avoid conflict
with another crfs option).
• crfs creates the file system, creates the mount point, and updates the file systems file
(/etc/filesystems). The -A yes option requests mount at boot.
• If the file system already exists in /etc/filesystems, you can mount the file system
by simply using the syntax: mount mount_point.
Linux
To create and mount a VxFS file system using mkfs:
mkfs -t vxfs /dev/vx/rdsk/datadg/datavol
mkdir /data
mount -t vxfs /dev/vx/dsk/datadg/datavol /data

03-16
Mounting a file system at boot
Add an entry to the OS-specific file system table.
Solaris /etc/vfstab example:
/dev/vx/dsk/appdg/appvol /dev/vx/rdsk/appdg/appvol /app vxfs 2 yes -
Information Entry
Device to mount: /dev/vx/dsk/appdg/appvol
Device to fsck: /dev/vx/rdsk/appdg/appvol
Mount point: /app
File system type: vxfs
fsck pass: 2
Mount at boot: yes
Mount options: -
17
Using CLI, if you want the file system to be mounted at every system boot, you must edit the
file system table file by adding an entry for the file system. If you later decide to remove the
volume, you must remove the entry in the file system table file.
AIX
In AIX, you can use the following commands when working with the file system table file,
/etc/filesystems:
• To view entries: lsfs mount_point
• To change details of an entry, use chfs. For example, to turn off mount at boot: chfs -
A no mount_point

03-17
Topic: Displaying disk and disk group information
After completing this topic, you will be able to view disk and disk group
information and identify disk status.
18
This is the Displaying disk and disk group information topic.


03-18
Viewing basic disk information
vxdisk -o alldgs list
VxVM
DEVICE TYPE DISK GROUP STATUS disks
emc0_dd1 auto:cdsdisk appdg01 appdg online
emc0_dd3 auto:cdsdisk - - online Free disk
emc0_dd4 auto:none - - online invalid
emc0_dd8 auto:cdsdisk - (oradg) online
Uninitialized
disks
Deported disk group
19
Displaying basic disk information

By viewing disk information, you can determine if a disk has been initialized and added to a
disk group, verify the changes that you make to disks, and keep track of the status and
configuration of your disks.
Use the vxdisk -o alldgs list command to display basic information about all
disks attached to the system. The vxdisk list command displays the:
• Device names for all recognized disks
• Type of disk, that is, how a disk is placed under VxVM control
• Disk names
• Disk group names associated with each disk

• Status of each disk
In the output:
• A status of online, in addition to entries in the Disk and Group columns indicates that the
disk has been initialized or encapsulated, assigned a disk media name, and added to a disk
group. The disk is under Volume Manager control and is available for creating volumes.
• A status of online without entries in the Disk and Group columns indicates that the drive
has been initialized or encapsulated but is not currently assigned to a disk group. Note
that if there is a disk group name in parentheses without any disk media name, it indicates
that the disk belongs to a deported disk group.

03-19
• A status of online invalid indicates that the disk has neither been initialized nor
encapsulated by VxVM. The disk is not under VxVM control.
• A status of error (not shown on the slide) indicates that Volume Manager can no longer
access the disk device, possibly due to a failure.
Notes:
• On the HP-UX platform, LVM disks have a type of auto:LVM and a status of LVM.
• On the Solaris platform, ZFS/SVM disks have a type of auto:ZFS or auto:SVM and a status
of ZFS or SVM respectively.

03-20
Viewing detailed disk information
vxdisk list [accessname] Summary information

vxdisk –s list [accessname]
vxdisk –p list [accessname]
vxdisk –p list emc0_dd1
…
VID : EMC
…
LUN properties PID : SYMMETRIX
MEDIA_TYPE : hdd
LUN_TYPE : std
…
ATYPE : A/A
ARRAY_VOLUME_ID : DD1
ARRAY_PORT_PWWN : 10.10.5.3:3260
ANAME : EMC
TRANSPORT : iSCSI
ENCLOSURE_NAME : emc0
NUM_PATHS : 2
21
To display detailed information about a disk, you use the vxdisk list command with the
name of the disk. With this command, you can either use the disk access name or the disk
media name together with the disk group name as shown in the following syntax:
vxdisk -g diskgroup list dm_name
vxdisk -g appdg list appdg01
Device: emc0_dd5
devicetag: emc0_dd5
type: auto
hostid: train12
disk: name=appdg01 id=1000753057.1114.train12
group: name=appdg id=1000753077.1117.train12
...
In the example output:
• Device is the VxVM name for the device path.
• devicetag is the name used by VxVM to refer to the physical disk.
• type is how a disk was discovered by VxVM. auto is the default type.
• hostid is the name of the system that currently manages the disk group to which the
disk belongs; if blank, no host is currently controlling this group.

03-21
• disk is the VM disk media name and internal ID.
• group is the disk group name and internal ID.
To view a summary of information for all disks, you use the -s option with the vxdisk
list command.
To display discovered properties of a disk, such as vendor ID, array port ID/WWN and so on,
you use the -p option with the vxdisk list command.
vxdisk -p list emc0_dd1
DISK : emc0_dd1
DISKID : 1320849045.92.sym1
VID : EMC
UDID : EMC%5FSYMMETRIX%5F313635323300%5FDD0DD1
SCSI_VERSION : 3
SCSI3_VPD_ID : 5123456000000000
REVISION : 5671
PORT_SERIAL_NO : 1a-a
PID : SYMMETRIX
PHYS_CTLR_NAME : c3
NR_DEVICE : Y
MEDIA_TYPE : hdd
LUN_TYPE : std
LUN_SNO_ORDER : 0
LUN_SERIAL_NO : DD0DD1
LIBNAME : libvxemc.so
HARDWARE_MIRROR: no
DMP_DEVICE : emc0_dd1
DDL_DEVICE_ATTR: lun
CAB_SERIAL_NO : 313635323300
ATYPE : A/A
ARRAY_VOLUME_ID: DD1
ARRAY_PORT_PWWN: 10.10.5.3:3260
ANAME : EMC
TRANSPORT : iSCSI

03-22
ENCLOSURE_NAME : emc0
NUM_PATHS : 2
Notes: The disk name and the disk group name are changeable. The disk ID and disk group ID
are never changed as long as the disk group exists or the disk is initialized.

03-23
Viewing disk group information
vxdg list
NAME STATE ID
appdg enabled,cds 969583613.1025.cassius
oradg enabled,cds 971216408.1133.cassius
To display free space in a disk group:

vxdg -g diskgroup –u h free
Human-friendly units for size
24
To display disk group information:

• Use vxdg list to display disk group names, states, and IDs for all imported disk groups
in the system.
• Use vxdg free to display free space on each disk. This command displays free space on
all disks in all disk groups that the host can detect. Add -g diskgroup to restrict the
output to a specific disk group.
Note: This command does not show space on spare disks. Reserved disks are displayed
with an “r” in the FLAGS column.
• Use vxdisk -o alldgs list to display all disk groups, including deported disk
groups. For example: vxdisk -o alldgs list

03-24
Using vxlist to display disk and disk group information
vxlist disk [accessname]
vxlist dg [diskgroup]
vxlist disk
vxlist dg
25
The vxlist command is a new display command that provides a consolidated view of the SF
configuration.
To display the vxlist command output, the vxdclid daemon must be running. If this
daemon is not running, execute the following command as the root user
/opt/VRTSsfmh/adm/dclisetup.sh
For more information on using the vxlist command, refer to the manual pages.

03-25
Topic: Displaying volume configuration information
After completing this topic, you will be able to display volume layout
information by using the command line.
26
This is the Displaying volume configuration information topic.


03-26
Displaying volume information
To display volume layout information use,
vxprint [–g diskgroup] [options]
Option Description
-h Hierarchical listing
-r Related records (Layered volumes)
-t Tabular listing
-A Active disk groups
-u unit Unit of measure (for size)
27
To display volume layout information use, the vxprint command

You can use the vxprint command to display information about how a volume is
configured. This command displays records from the VxVM configuration database.
vxprint -g diskgroup [options]
The vxprint command displays information about disk groups, disk media, volumes, plexes,
and subdisks. You can specify a variety of options with the command to expand or restrict the
information displayed. Only some of the options are presented in this training. For more
information about additional options, see the vxprint(1m) manual page.

03-27
Interpreting the vxprint output
vxprint -g appdg –htr –u h | more
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME NVOLUME KSTATE STATE
V NAME RVG /VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
…
dg appdg default default 16000 1000753077.1117.train12
dm appdg01 emc0_dd1 auto 32.00m 1.96g - To interpret the output,

dm appdg02 emc0_dd2 auto 32.00m 1.96g - match header lines with
dm appdg03 emc0_dd3 auto 32.00m 1.96g -
dm appdg04 emc0_dd4 auto 32.00m 1.96g - output lines.
v appvol - ENABLED ACTIVE 1.00g SELECT - fsgen

pl appvol-01 appvol ENABLED ACTIVE 1.00g CONCAT - RW
sd appdg01-01 appvol-01 appdg01 0.00 1.00g 0.00 emc0_dd1 ENA
28
To display the volume, plex, and subdisk record information for a disk group:
vxprint -g diskgroup -htr - u h
In the output, the top few lines indicate the headers that match each type of output line that
follows. Each volume is listed along with its associated plexes and subdisks and other VxVM
objects.
• dg is a disk group.
• st is a storage pool (used in Intelligent Storage Provisioning).
• dm is a disk.
•
rv is a replicated volume group (used in Veritas Volume Replicator).

• rl is an rlink (used in Veritas Volume Replicator).
• co is a cache object.
• vt is a volume template (used in Intelligent Storage Provisioning).
• v is a volume.
• pl is a plex.
• sd is a subdisk.
• sv is a subvolume.
• sc is a storage cache.
• dc is a data change object.
• sp is a snap object.
For more information, see the vxprint (1m) manual page.

03-28
Interpreting the vxprint output
vxlist vol[ume] [volume_name]
vxinfo –g diskgroup [-p]
vxlist volume
TY VOLUME DISKGROUP SIZE STATUS LAYOUT LINKAGE
vol appvol appdg 1.00g healthy concat -
vol mirvol appdg 500.00m healthy concat -
vol oravol oradg 3.00g healthy striped -
vxinfo –g appdg -p
vol appvol fsgen Started
plex appvol-01 ACTIVE
vol mirvol fsgen Started
plex mirvol-01 ACTIVE
plex mirvol-02 ACTIVE
29
The vxlist command is useful in summarizing the volume information on the system. You
can also use this command to display the disks and the plexes associated with a specific
volume, using the following command options:
vxlist –s disk vol volume_name
vxlist -s disk vol appvol
disks
TY DEVICE DISK NPATH ENCLR_NAME ENCLR_SNO
STATUS
disk emc0_dd1 appdg01 2 emc0 ... imported
vxlist –s plexes vol volume_name

vxlist -s plexes vol appvol
plexes
TY NAME TYPE STATUS
plex appvol-01 simple attached
The vxinfo command prints the accessibility and the usability information on VxVM
volumes. The -p option with vxinfo also reports the name and status of each plex within
the volume.

03-29
Topic: Removing volumes, disks, and disk groups
After completing this topic, you will be able to remove a volume, evacuate
a disk, remove a disk from a disk group, destroy a disk group, and shred a
disk.
30
This is the Removing Volumes, Disks, and Disk Groups topic.


03-30
Removing a volume
• Before removing a volume:

‒ Application must be stopped
‒ File system must be unmounted
• When you remove a volume, volume disk space is freed.
vxassist remove volume:
vxassist -g diskgroup remove volume volume_name
vxassist -g appdg remove volume appvol
vxedit:
vxedit -g diskgroup -rf rm volume_name

vxedit -g appdg -rf rm appvol
31
Only remove a volume if you are sure that the data in the volume is not needed, or the data is
backed up elsewhere. A volume must be closed before it can be removed. For example, if the
volume contains a file system, the file system must be unmounted. You must edit the OS-
specific file system table file manually in order to remove the entry for the file system and
avoid errors at boot. If the volume is used as a raw device, the application, such as a database,
must close the device.

03-31
Evacuating a disk
Before removing a disk, you may need to evacuate data from the disk to another disk in the disk group.
vxdiskadm:
Move volumes from a disk
CLI:
vxevac -g diskgroup from_dm_name [to_dm_name]

vxevac -g appdg appdg01 appdg02
32
Evacuating a disk moves the contents of the volumes on a disk to another disk. The contents
of a disk can be evacuated only to disks in the same disk group that have sufficient free space.
To evacuate to any disk except for appdg03:
vxevac -g appdg appdg02 !appdg03

03-32
Removing a disk from a disk group
Remove a disk
vxdiskadm:
CLI:
vxdg -g diskgroup rmdisk dm_name
Example:
vxdg –g appdg rmdisk appdg04
33
You can verify the removal by using the vxdisk list command to display disk
information. A disk that has been taken out of a disk group no longer has a disk media name
or disk group assignment but still shows a status of online.
Before the disk is taken out of the disk group:
DEVICE TYPE DISK GROUP STATUS
...
After the disk is taken out of the disk group using the vxdg -g appdg rmdisk
appdg01 command:
emc0_dd1 auto:cdsdisk -- online ...

03-33
Uninitializing and shredding a disk
vxdiskunsetup accessname
vxdiskunsetup [-f] -o shred[=1|3|7] accessname
vxdiskunsetup emc_dd4
vxdiskunsetup -o shred hds9970v0_14

disk_shred: Shredding disk hds9970v0_14 with type 1
disk_shred: Disk raw size 2461040640 bytes
disk_shred: Writing 37552 (65536 byte size) pages and 32768
bytes to disk
disk_shred: Wipe Pass 0: Pattern 0x3a
disk_shred: Shred passed random verify of 131072 bytes at
offset 1828888576
34
After the disk has been removed from its disk group, you can remove it from Volume Manager
control completely by using the vxdiskunsetup command. This command reverses the
configuration of a disk by removing the public and private regions that were created by the
vxdisksetup command. The vxdiskunsetup command does not operate on disks that
are active members of an imported disk group. This command does not usually operate on
disks that appear to be imported by some other host—for example, a host that shares access
to the disk. You can use the -C option to force deconfiguration of the disk, removing host
locks that may be detected.
Before the disk is uninitialized:

emc0_dd1 auto:cdsdisk - - online ...
After the disk is uninitialized using the vxdiskunsetup emc0_dd1 command:
emc0_dd1 auto:none - - online invalid ...

03-34
You can use the -o shred option with the vxdiskunsetup command to shred a disk.
Shredding a disk destroys the data stored on the disk by overwriting the disk with a digital
pattern in one of three ways:
• One-pass: VxVM overwrites the disk with a randomly selected pattern. This option takes
the least amount of time. This is the default behavior if the user does not specify a type.
• Three-pass: The disk is overwritten a total of 3 times. In the first pass, it is overwritten
with a pre-selected digital pattern. The second time, it is overwritten with the binary
complement of the pattern. In the last pass, the disk is overwritten with a randomly
selected digital pattern. This algorithm is based on the US DoD.5200.22-M standard for
the sanitization of sensitive data.
• Seven-pass: Disk is overwritten a total of 7 times. Each pass consists of overwriting the
disk with a randomly selected digital pattern or with the binary complement of the
previous pattern. This algorithm is based on the US DoD.5200.28-STD standard.
Note: Use the -f option to force a shred operation on a Solid State Drive (SSD) disk.

03-35
Destroying a disk group
• Disk group no longer exists. olddg
• All disks are freed.
• Before destroying a disk group:
‒ Applications must be stopped.
‒ File systems must be unmounted.
Destroy
vxdg destroy diskgroup

vxdg destroy olddg
Free disk pool
36
Destroying a disk group permanently removes a disk group from Volume Manager control,
and the disk group ceases to exist. When you destroy a disk group, all of the disks in the disk
group are made available as empty disks. Volumes and configuration information including
the automatic configuration backups of the disk group are removed. Disk group configuration
backups are discussed later in this course. Because you cannot remove the last disk in a disk
group, destroying a disk group is the only method to free the last disk in a disk group for
reuse. A disk group cannot be destroyed if any volumes in that disk group are in use or contain
mounted file systems. The bootdg disk group cannot be destroyed.
Caution: Destroying a disk group can result in data loss. Only destroy a disk group if you are
sure that the volumes and data in the disk group are not needed.
To destroy a disk group from the command line, use the vxdg destroy command.
Note: You can bring back a destroyed disk group by importing it with its dgid if its disks had
not been re-used for other purposes.

03-36
Lesson summary
• Key points
– In this lesson, you learned how to create a volume with a file system.
– You also learned about device-naming schemes, how to add a disk to a disk group, and how to view
configuration information for volumes, disk groups, and disks.
– In addition, you learned how to remove a volume, disk, and disk group.
– Veritas InfoScale Installation Guide
– Veritas Storage Foundation Administrator’s Guide
– Veritas Services and Operations Readiness Tools (SORT):
https://sort.veritas.com
37

03-37
Lab 03: Creating a Volume and File System
• Exercise A: Creating disk groups, volumes and file systems: CLI

• Exercise B: Removing volumes and disks: CLI
• Exercise C: Destroying disk data using disk shredding: CLI
• Exercise D: (Optional) Creating disk groups, volumes, and file systems: VIOM
• Exercise E: (Optional) Removing volumes, disks, and disk groups: VIOM
38

03-38
What did you learn?
lesson.
39
current lesson.

03-39
Question 1: Logical name of an enclosure
By default, the logical name of an enclosure is typically based on the:
A. Controller number
B. Target ID
C. Disk media name
D. Vendor ID
40

03-40
Answer 1: Logical name of an enclosure
By default, the logical name of an enclosure is typically based on the:
A. Controller number
B. Target ID
C. Disk media name
D. Vendor ID
The correct answer is D. By default, the logical name of an enclosure is typically based on the Vendor ID.
41

03-41
Question 2: Configuring a disk for VxVM
In the first stage of configuring a disk for use by VxVM:
A. A disk is initialized.
B. A disk is assigned to a disk group.
C. The space on a disk becomes part of the free space pool default disk group.
D. Disk space is assigned to volumes.
42

03-42
Answer 2: Configuring a disk for VxVM
In the first stage of configuring a disk for use by VxVM:
A. A disk is initialized.
B. A disk is assigned to a disk group.
C. The space on a disk becomes part of the free space pool default disk group.
D. Disk space is assigned to volumes.
The correct answer is A. SCSI disks are usually preformatted. After a disk is formatted, the disk must be initialized for use
by Volume Manager. In other words, disks must be detected by the operating system, before VxVM can detect the disks.
43

03-43
Question 3: Write a disk header command
To configure a disk for use by VxVM and write a disk header to the disk, you use the
command:
A. vxdisk
B. vxdiskconfig
C. vxdisksetup
D. vxdgsetup
44

03-44
Answer 3: Write a disk header command
To configure a disk for use by VxVM and write a disk header to the disk, you use the
command:
A. vxdisk
B. vxdiskconfig
C. vxdisksetup
D. vxdgsetup
The correct answer is C. To configure a disk for use by VxVM and write a disk header to the disk, you use the
vxdisksetup -i accessname [attributes] command.
45

03-45
Question 4: Display detailed information command
Select the appropriate command to display detailed information about disk01.
A. vxdg list disk01

B. vxdisk –g appdg list disk01
C. vxdisplay list disk01
D. vxvm list disk01
46

03-46
Answer 4: Display detailed information command
Select the appropriate command to display detailed information about disk01.
A. vxdg list disk01

B. vxdisk –g appdg list disk01
C. vxdisplay list disk01
D. vxvm list disk01
The correct answer is B.
47

03-47
Question 5: Display all disk groups command
Select the appropriate command to display all disk groups in the system, including
deported disk groups.
A. vxdg list
B. vxdg -a alldgs list
C. vxdisk list alldgs
D. vxdisk -o alldgs list
48

03-48
Answer 5: Display all disk groups command
Select the appropriate command to display all disk groups in the system, including
deported disk groups.
A. vxdg list
B. vxdg -a alldgs list
C. vxdisk list alldgs
D. vxdisk -o alldgs list
The correct answer is D.
49

03-49
Question 6: Create a volume command
To create a volume from the command line, you use the command:
A. vxdg make vol

B. vxvol new
C. vxassist make
D. vxdisk new vol
50

03-50
Answer 6: Create a volume command
To create a volume from the command line, you use the command:
A. vxdg make vol

B. vxvol new
C. vxassist make
D. vxdisk new vol
The correct answer is C.
51

03-51
Question 7: Display volume configuration command
The command that displays volume configuration information is:
A. vxdisplay
B. vxvol
C. vxconfig
D. vxprint
52

03-52
Answer 7: Display volume configuration command
The command that displays volume configuration information is:
A. vxdisplay
B. vxvol
C. vxconfig
D. vxprint
The correct answer is D. The vxprint command can display information about disk groups, disk media, volumes,
plexes, and subdisks.
53

03-53
Question 8: Remove the expvol volume command
Select the appropriate command to remove the expvol volume. The volume exists in the
acctdg disk group.
A. vxassist -g acctdg remove volume expvol
B. vxremove -g acctdg remove volume expvol
C. vxassist -g acctdg destroy volume expvol
D. vxdg -g acctdg destroy volume expvol
54

03-54
Answer 8: Remove the expvol volume command
Select the appropriate command to remove the expvol volume. The volume exists in the
acctdg disk group.
A. vxassist -g acctdg remove volume expvol
B. vxremove -g acctdg remove volume expvol
C. vxassist -g acctdg destroy volume expvol
D. vxdg -g acctdg destroy volume expvol
The correct answer is A.
55

03-55
Appendix
This appendix contains slides that are platform-specific, and they may be
reviewed per the viewer’s discretion and interest, or you may optionally
end the presentation now.
56
This is the Appendix topic.


03-56
AIX: Adding a file system: CLI
Step CLI
Create the file system on the volume.

1
mkfs -V vxfs /dev/vx/rdsk/appdg/appvol
Create a mount point directory.
2
mkdir /app
Mount the file system on the mount point.
3
mount -V vxfs /dev/vx/dsk/appdg/appvol /app
57
On AIX, to create and mount a file system, use the mkfs -V vxfs command, then the
mkdir command, and then the mount -V command.
To create and mount a file system using crfs, use the crfs -v vxfs command.
To return to the Adding a File System After Volume Creation slide, click the back button at
the bottom of the screen.

03-57
HP-UX: Adding a file system: CLI
Step CLI

1
mkfs -F vxfs /dev/vx/rdsk/appdg/appvol
2
mkdir /app
3
mount -F vxfs /dev/vx/dsk/appdg/appvol /app
58
On HP, to create and mount a file system, use the mkfs -F vxfs command, then the
mkdir command, and then the mount command.
To create and mount a HFS file system, it is the same, except use the newfs –F hfs
command rather than mkfs and use the –F hfs flag in the mount command.

03-58
Linux: Adding a file system: CLI
Step CLI

1
mkfs -t vxfs /dev/vx/rdsk/appdg/appvol
2
mkdir /app
3
mount -t vxfs /dev/vx/dsk/appdg/appvol /app
59
On Linux, to create and mount a file system use the mkfs -t vxfs command, then
mkdir, and then mount -t vxfs.

03-59
Solaris: Adding a file system: CLI
Step CLI

1
mkfs -F vxfs /dev/vx/rdsk/appdg/appvol
2
mkdir /app
3
mount -F vxfs /dev/vx/dsk/appdg/appvol /app
60
On Solaris, to create and mount a file system, use the mkfs -F vxfs command, then the
mkdir command, and then the mount –F vxfs command.
To create and mount a UFS file system, it is almost the same, except use the newfs
command rather than mkfs.
For the mount command, do not use the –F flag.

03-60
End of presentation
03-61
Lesson 04: Working with Volumes with Different

Layouts
This is the Working with Volumes with Different Layouts lesson in the Veritas InfoScale 7.4.2

04-1
Administration
Administration


04-2
Lesson objectives
Topic Objective
Identify the features, advantages, and disadvantages of VxVM

Volume layouts
volume layouts.
Creating volumes with various Create concatenated, striped, and mirrored volumes from the
layouts command line.
Allocating storage for volumes Allocate storage for a volume by specifying storage attributes.

04-3
Topic: Volume layouts
After completing this topic, you will be able to identify the features,
advantages, and disadvantages of volume layouts supported by VxVM.
This is the Volume layouts topic.


04-4
Concatenated layout
Disk group: appdg
appdg
appvol
Volume:
appvol
appdg01-01
12 GB
Plex:
appvol-01 appdg02-02
appvol-01
Subdisks:
• appdg01-01
• appdg02-02 appdg01 appdg02
VxVM disks: appdg02-01

8 GB appdg01-01
• appdg01 appdg02-02 4 GB
• appdg02
Volume layouts
Each volume layout has different advantages and disadvantages. For example, a volume can
be extended across multiple disks to increase capacity, mirrored on another disk to provide
data redundancy, or striped across multiple disks to improve I/O performance. The layouts
that you choose depend on the levels of performance and availability required by your
system.
Concatenated layout
A concatenated volume layout maps data in a linear manner onto one or more subdisks in a
plex. Subdisks do not have to be physically contiguous and can belong to more than one VM
disk. Storage is allocated completely from one
subdisk before using the next subdisk in the span. Data is accessed in the remaining subdisks
sequentially until the end of the last subdisk. For example, if you have 12 GB of data then a
concatenated volume can logically
map the volume address space across subdisks on different disks. The addresses 0 GB to 8 GB
of volume address space map to the first 8-gigabyte subdisk, and addresses 9 GB to 12 GB
map to the second 4-gigabyte subdisk. An address offset of 10 GB, therefore, maps to an
address offset of 2 GB in the second subdisk.

04-5
Striped layout
Disk group: appdg Veritas recommends <=8 columns
appdg
appvol SU = Stripe unit

Volume:
appvol
64K (default)
Plex:
appvol-01 SU1 SU2 SU3 SU4 Stripes
Columns
SU5 SU6 SU7 SU8
Subdisks: SU9 SU10 SU11 SU12
• appdg01-01
SU13 SU14 SU15 SU16
• appdg02-01 appvol-01
• appdg03-01
• appdg04-01
VxVM disks: appdg01 appdg02 appdg03 appdg04

• appdg01
• appdg02
• appdg03 appdg01-01 appdg02-01 appdg03-01 appdg04-01
• appdg04
A striped volume layout maps data so that the data is interleaved, or allocated in stripes,
among two or more subdisks on two or more physical disks. Data is allocated alternately and
evenly to the subdisks of a striped plex.
The subdisks are grouped into “columns.” Each column contains one or more subdisks and can
be derived from one or more physical disks. To obtain the maximum performance benefits of
striping, you should not use a single disk to
provide space for more than one column.
All columns must be the same size. The size of a column is equal to the size of the volume
divided by the number of columns. The default number of columns in a striped volume is
based on the number of disks in the disk group.
Data is allocated in equal-sized units, called stripe units, that are interleaved between the
columns. Each stripe unit is a set of contiguous blocks on a disk. The stripe unit size can be in
units of sectors, kilobytes, megabytes, or gigabytes. The default stripe unit size is 64K, which
provides adequate performance for most general purpose volumes. Performance of an
individual volume may be improved by matching the stripe unit size to the I/O characteristics
of the application using the volume.

04-6
Mirrored layout
Disk group: appdg
appdg
appvol
Volume:
appvol
Plexes:
• appvol-01
• appvol-02 appdg02-02
appdg01-02
appdg03-02
Subdisks:
• appdg01-02 appvol-01 appvol-02
• appdg02-02
• appdg03-01
appdg01 appdg02 appdg03
VxVM disks: appdg01-01 appdg02-01 appdg03-01
• appdg01 appdg02-02 appdg03-02
• appdg02 appdg01-02
appdg02-03 appdg03-03
• appdg03
By adding a mirror to a concatenated or striped volume, you create a mirrored layout. A

mirrored volume layout consists of more than one plex that duplicate the information
contained in a volume. Each plex in a mirrored layout contains an identical copy of the volume
data. In the event of a physical disk failure and when the plex on the failed disk becomes
unavailable, the system can continue to operate using the unaffected mirrors.
Although a volume can have a single plex, at least two plexes are required to provide
redundancy of data. Each of these plexes must contain disk space from different disks to
achieve redundancy.
Volume Manager uses true mirrors, which means that all copies of the data are the same at all
times. When a write occurs to a volume, all plexes must receive the write before the write is
considered complete.
Distribute mirrors across controllers to eliminate the controller as a single point of failure.

04-7
RAID-5 layout Disk group: appdg
appdg
P = Parity;
appvol SU = Stripe unit
a calculated value
Volume: used to reconstruct 16K (default)
appvol
app after disk
failure
Plex:
appvol-01 SU1 SU2 SU3 P Stripes
Columns
SU5 SU6 P SU4
Subdisks: SU9 P SU7 SU8
• appdg01-01 P SU10 SU11 SU12
• appdg02-01 appvol-01
• appdg03-01
• appdg04-01
VxVM disks: appdg01 appdg02 appdg03 appdg04

• appdg01
• appdg02 appdg01-01 appdg02-01 appdg03-01 appdg04-01
• appdg03
• appdg04
A RAID-5 volume layout has the same attributes as a striped plex, but one column in each
stripe is used for parity. Parity provides redundancy. Parity is a calculated value used to
reconstruct data after a failure. While data is
being written to a RAID-5 volume, parity is calculated by performing an exclusive OR (XOR)
procedure on the data. The resulting parity is then written to the volume. If a portion of a
RAID-5 volume fails, the data that was on that portion of the failed volume can be re-created
from the remaining data and parity information.
RAID-5 volumes keep a copy of the data and calculated parity in a plex that is striped across
multiple disks. Parity is spread equally across columns. Given a five-column RAID-5 where
each column is 1 GB in size, the RAID-5 volume size is 4 GB. An amount of space equivalent to
one column is devoted to parity; the remaining space is used for data. The default stripe unit
size for a RAID-5 volume is 16K. Each column must be the same length but may be made from
multiple subdisks of variable length. Subdisks used in different columns must not be located
on the same physical disk. RAID-5 requires a minimum of three disks for data and parity.
When implemented as recommended, an additional disk is required for the log. RAID-5
cannot be mirrored.

04-8
Erasure-coded layout
Erasure coding is a volume layout with an advantage of storage saving. Mirroring can provide
any fault tolerance just like Erasure coding but will have huge storage overhead. So, it is
treated as a feature rather than new area.
Erasure coding is a new feature available as a technology preview in Veritas InfoScale for
configuration and testing in non-production environments. It is supported in DAS, SAN, FSS,
and standalone environments.
As storage systems expand and become more complex, traditional data protection
mechanisms are proving to be inadequate against failures. Erasure coding offers a more
robust solution in redundancy and fault tolerance for critical storage archives. In erasure
coding, data is broken into fragments, expanded and encoded with redundant data pieces and
stored across different locations or storage media. When one or more disks fail, the data on
failed disks is reconstructed using the parity information in the encoded disks and data in the
surviving disks. Erasure coded volumes must be created using disk group version 230 or later.
When you create an erasure coded volume, Veritas InfoScale, by default, runs asynschronous
initialization on the volume ensuring that data and parities are synchronized for all regions.
The operation runs in the background allowing the volume to be available for use to
applications immediately after creation. The volumes display the SYNC state after creation
until all the regions are synchronized. This functionality is supported on both private and
shared/FSS disk groups.
You can manually initialize an erasure coded volume by setting init=zero at the time of
creating the volume. The initialization zeroes out all the regions and the volume is not
available for use until the initialization process completes.

04-9
Comparing volume layouts
Concatenation Striping Mirroring RAID-5 Erasure Coding
• Redundancy • Fault tolerance
through parity with storage
• Redundant • Less space saving.
Advantages
• No disk space • Load-balancing • Improved read requirements • Complementary

restrictions • Improved performance than mirroring to mirroring
• Simplest to use performance • Fast recovery • Improved read • High availability
through logging performance of data with
• Fast recovery efficient storage
through logging utilization.
• Disk space
Disadvantages
• Poor write • Inconsistent in

• No redundancy • No redundancy requirements
performance handling node
• No performance • Disk space • Potentially slower failure
• Poor performance
improvements restrictions write
after a disk failure • Poor reclamation
performance
10
Concatenation: Advantages
• Better utilization of free space: Concatenation removes the restriction on size of storage
devices imposed by physical disk size. It also enables better utilization of free space on
disks by providing for the ordering of available discrete disk space on multiple disks into a
single addressable volume.
• Simplified administration: System administration complexity is reduced because making
snapshots and mirrors uses any size space, and volumes can be increased in size by any
available amount.
Concatenation: Disadvantages
• No protection against disk failure: Concatenation does not protect against disk failure. A
single disk failure results in the failure of the entire volume.
Striping: Advantages
• Improved performance through parallel data transfer: Improved performance is obtained
by increasing the effective bandwidth of the I/O path to the data. This may be achieved by
a single volume I/O operation spanning across a number of disks or by multiple concurrent
volume I/O operations to more than one disk at the same time.
• Load-balancing: Striping is also helpful in balancing the I/O load from multiuser
applications across multiple disks.

04-10
Striping: Disadvantages
• No redundancy: Striping alone offers no redundancy or recovery features.
• Disk failure: Striping a volume increases the chance that a disk failure results in failure of
that volume. For example, if you have three volumes striped across two disks, and one of
the disks is used by two of the volumes, then if that one disk goes down, both volumes go
down.
Mirroring: Advantages
• Improved availability: With concatenation or striping, failure of any one disk makes the
entire plex unusable. With mirroring, data is protected against the failure of any one disk.
Mirroring improves the availability of a striped or concatenated volume.
• Improved read performance: Reads benefit from having multiple places from which to
read the data.
Mirroring: Disadvantages
• Requires more disk space: Mirroring requires twice as much disk space, which can be
costly for large configurations. Each mirrored plex requires enough space for a complete
copy of the volume’s data.
• Slightly slower write performance: Writing to volumes is slightly slower, because multiple
copies have to be written in parallel. The overall time the write operation takes is
determined by the time needed to write to the slowest disk involved in the operation. The
slower write performance of a mirrored volume is not generally significant enough to
decide against its use. The benefit of the resilience that mirrored volumes provide
outweighs the performance reduction.
RAID-5: Advantages
• Redundancy through parity: With a RAID-5 volume layout, data can be recreated from
remaining data and parity in case of the failure of one disk.
• Requires less space than mirroring: RAID-5 stores parity information, rather than a
complete copy of the data.
• Improved read performance: RAID-5 provides similar improvements in read performance

as in a normal striped layout.
• Fast recovery through logging: RAID-5 logging minimizes recovery time in case of disk
failure.
RAID-5: Disadvantages
• Slow write performance: The performance overhead for writes can be substantial,
because a write can involve much more than simply writing to a data block. A write can
involve reading the old data and parity, computing the new parity, and writing the new
data and parity. If you have more than twenty percent writes, do not use RAID-5.

04-11
• Very poor performance after a disk failure: After one column fails, all I/O performance
goes down. This is not the case with mirroring, where a disk failure does not have any
significant effect on performance
Erasure coding: Advantages
• A robust solution in redundancy and fault tolerance: . In erasure coding, data is broken
into fragments, expanded and encoded with redundant data pieces and stored across
different locations or storage media. When one or more disks fail, the data on failed disks is
reconstructed using the parity information in the encoded disks and data in the surviving
disks.
• Complementary to mirroring: Mirroring technology which provide high availability of data
but it is not a cost effective solution where one needs to store huge amount of data. To
ensure that data is available after 3 faults of disks/nodes, in mirroring data needs to be
replicated 4 times. With Erasure coding one can get same fault tolerance with just 1.5
times of the storage. It To provide high availability of data with efficient storage utilization.
Erasure coding : Disadvantages
• Very poor performance after a node failure: Volumes may go into an inconsistent state If a
node fails when an erasure coded volume is online and operational. The write operations
that were active at the time of node failure can cause inconsistencies in the parity
information for the corresponding region.
• Poor reclamation compliance: Erasure coded volumes cannot be created on thin
provisioned/reclaimable (TP/R) disks. Reclamation may cause corruption of erasure coded
volumes.
• Poor administrative operations support: Adding or removing mirrors, Snapshots and
related operations, Replication and related operations, Disk group split and move,
perations on erasure coded volumes, Relayout of erasure coded volumes are not yet
supported on erasure coded volumes

04-12
Topic: Creating volumes with various layouts
After completing this topic, you will be able to create concatenated, striped,
and mirrored volumes from the command line.
13
This is the Creating volumes with various layouts topic.


04-13
Using CLI to create volumes with various layouts
Concatenated:
vxassist -g diskgroup make volume_name length \ [layout=concat]

[dm_names...]
Striped:
vxassist -g diskgroup make volume_name length \

layout=stripe [ncol=n] [stripeunit=size] \
[dm_names...]
Mirrored:
vxassist -g diskgroup [-b] make volume_name \

length layout=mirror-concat|mirror-stripe \
[nmirror=n] [dm_names...]
14
To specify different volume layouts while creating a volume from the command line using the
vxassist make command, you use the layout attribute. If you do not specify the layout
attribute, by default, vxassist creates a concatenated volume that uses one or more
sections of disk space. The layout=striped attribute designates a striped layout and the
layout=mirror-concat or the layout=mirror-stripe attributes designate a
mirrored volume layout. Note that you can also use the layout=mirror attribute to create
a mirrored volume. However, layout=mirror may result in the creation of layered volumes.
Layered volumes are covered in detail later in this lesson.
Note: To guarantee that a concatenated volume is created, include the layout=nostripe
attribute in the vxassist make command. Without the layout attribute, the default layout
is used that may have been changed by the creation of the /etc/default/vxassist
file. The following additional attributes are used with the striped volume layout:
• ncol=n designates the number of stripes, or columns, across which the volume is
created. This attribute has many aliases. For example, you can also use nstripe=n or
stripes=n.

04-14
The minimum number of stripes in a volume is 2 and the maximum is 8. You can edit these
minimum and maximum values in /etc/default/vxassist using the min_columns
and max_columns attributes.
• stripeunit=size specifies the size of the stripe unit to be used. The default is 64K.
The following additional attributes are used with the mirrored volume layout:
• To specify more than two mirrors, you add the nmirror attribute.
• When creating a mirrored volume, the volume initialization process requires that the
mirrors be synchronized. The vxassist command normally waits for the mirrors to be
synchronized before returning to the system prompt. To run the process in the background,
you add the -b option.
Estimating volume size
The vxassist maxsize command can determine the largest possible size for a volume
that can currently be created with a given set of attributes. This command does not create the
volume but returns an estimate of the maximum volume size. The output value is displayed in
sectors, by default.
vxassist -g appdg maxsize layout=stripe ncol=2
Maximum volume size: 14389248 (7026Mb)
If the volume with the specified attributes cannot be created, an error message is returned:
VxVM vxassist ERROR V-5-1-752 No volume can be created within
the given constraints
Creating volumes with the maximum possible size
With SF 6.x, you can also create a volume of the maximum possible size using a
single command:
vxassist –g diskgroup make volume_name maxsize[=length] \
attributes
You can also provide an upper limit for the maximum size by specifying
maxsize=length parameter. If the maximum possible size is higher than this upper limit,
the volume is created using the upper limit as the volume length. If the maximum possible
size is smaller than this limit, the volume is created with the maximum possible size.

04-15
Volume creation examples
Concatenated:
vxassist –g appdg make appvol 10g
Striped across four disks:

vxassist -g appdg make appvol 2g layout=stripe ncol=4
Striped, specifying stripe unit size and disks:

vxassist -g appdg make appvol 8g layout=stripe ncol=4 \
stripeunit=256k appdg01 appdg02 appdg03 appdg04
Concatenated and mirrored with three mirrors:

vxassist -g appdg make appvol 5g layout=mirror-concat nmirror=3
Striped and mirrored :

vxassist -g appdg make appvol 2g layout=mirror-stripe ncol=4
16
While creating a concatenated volume, the vxassist command attempts to locate

sufficient contiguous space on one disk for the volume. However, if necessary, the volume is
spanned across multiple disks. VxVM selects the disks on
which to create the volume unless you designate the disks by adding the disk media names to
the end of the command. To stripe the volume across specific disks, you can specify the disk
media names at
the end of the command. The order in which disks are listed on the command line does not
imply any ordering of disks within the volume layout. To exclude a disk or list of disks, add an
exclamation point (!) before the disk
media names. For example, !appdg01 specifies that the disk appdg01 should not be used
to create the volume.

Creating a mirrored and logged volume
When you create a mirrored volume, you can add a dirty region log by adding the
logtype=drl attribute:
vxassist -g diskgroup [-b] make volume_name length \
layout=mirror-concat logtype=drl [nlog=n]
• A log plex that consists of a single subdisk is created.
• If you plan to mirror the log, you can add more than one log plex by specifying
a number of logs using the nlog=n attribute, where n is the number of logs.
vxassist -g appdg make appvol 5m layout=mirror-concat
logtype=drl

04-16
Topic: Allocating storage for volumes
After completing this topic, you will be able to allocate storage for a volume
by specifying storage attributes and ordered allocation.
17
This is the Allocating storage for volumes topic.


04-17
Specifying storage attributes
vxassist -g diskgroup make volume_name length [layout=layout] \
[mirror=ctlr|enclr|target] [!][storage_attributes...]
Attribute Purpose Examples
appdg02
Define storage devices used by ctlr:c2
storage_attributes
the volume enclr:emc1
mediatype:ssd
mirror=ctlr
mirror= Mirror across device classes
mirror=enclr
! Used to exclude an attribute !enclr:emc2
vxassist –g appdg make appvol 2g mediatype:ssd

vxassist –g appdg make appvol 10g layout=mirror-concat \
mirror=enclr enclr:emc1 enclr:emc2
vxassist –g appdg make appvol 5g !ctrl:c2
18
VxVM selects the disks on which each volume resides automatically, unless you specify
otherwise. To create a volume on specific disks, you can designate those disks when creating a
volume. By specifying storage attributes when you create a volume, you can:
• Include specific disks, controllers, enclosures, targets, or trays to be used for the volume.
• Exclude specific disks, controllers, enclosures, targets, or trays from being used for the
volume.
• Mirror volumes across specific controllers, enclosures, targets, or trays. (By default, VxVM
does not permit mirroring on the same disk.)
By specifying storage attributes, you can ensure a high availability environment. For example,
you can only permit mirroring of a volume on disks connected to different controllers and
eliminate the controller as a single point of failure. To exclude a disk, controller, enclosure,
target, or tray, you add the exclusion symbol (!) before the storage attribute. For example, to
exclude appdg02 from volume creation, you use the format: !appdg02.
Note: When creating a volume, all storage attributes that you specify for use must belong to
the same disk group. Otherwise, VxVM does not use these storage attributes to create a
volume.

04-18
Lesson summary
• Key points
– In this lesson, you learned about the advantages and disadvantages of volume layouts supported by
VxVM.
– You also learned how to create concatenated, striped, and mirrored volumes.
– Finally, you learned how to allocate storage for a volume by specifying storage attributes.
– Veritas InfoScale Release Notes
19

04-19
Lab 04: Working with Volumes with Different Layouts
• Exercise A: Creating volumes with different layouts: CLI

• Exercise B: Creating erasure coded volume for object store use case: CLI
• Exercise C: (Optional) Creating volumes with user defaults: CLI
20

04-20
What did you learn?
lesson.
21
current lesson.

04-21
Question 1: Comparing volume layouts
The volume layout that provides redundancy at the cost of at least twice as much disk
space is called:
A. Concatenation
B. Striping
C. Mirroring
D. RAID-5
22

04-22
Answer 1: Comparing volume layouts
The volume layout that provides redundancy at the cost of at least twice as much disk
space is called:
A. Concatenation
B. Striping
C. Mirroring
D. RAID-5
The correct answer is C. A mirrored volume layout consists of more than one plex that duplicate the information
contained in a volume. In the event of a physical disk failure and when the plex on the failed disk becomes unavailable,
the system can continue to operate using the unaffected mirrors.
23

04-23
Question 2: Comparing volume layouts
The volume layout that appends space from multiple disks if necessary based on the size
of the volume but provides no redundancy is called:
A. Striping
B. Concatenation
C. Mirroring
D. RAID-5
24

04-24
Answer 2: Comparing volume layouts
The volume layout that appends space from multiple disks if necessary based on the size
of the volume but provides no redundancy is called:
A. Striping
B. Concatenation
C. Mirroring
D. RAID-5
The correct answer is B. A concatenated volume layout maps data in a linear manner onto one or more subdisks in a
plex. Concatenation does not protect against disk failure. A single disk failure results in the failure of the entire volume.
25

04-25
Question 3: Types of volume layouts
The mirrored volume layout in which the top-level volume contains a striped plex and
the component subvolumes are mirrored is ______________.
A. mirror-concat
B. concat-mirror
C. mirror-stripe
D. stripe-mirror
26

04-26
Answer 3: Types of volume layouts
The mirrored volume layout in which the top-level volume contains a striped plex and
the component subvolumes are mirrored is ______________.
A. mirror-concat
B. concat-mirror
C. mirror-stripe
D. stripe-mirror
The correct answer is D. In the stripe-mirror volume layout, the top-level volume contains a striped plex and the
component subvolumes are mirrored
27

04-27
Question 4: Volume creation examples
The oradg disk group is made up of disks, each 2 GB in length. You want to create a 6 GB
2-way mirrored volume that is striped across 2 columns.
What is the minimum number of disks you need to have in the oradg disk group to be
able to create this volume?
A. 4
B. 6
C. 8
D. 12
28

04-28
Answer 4: Volume creation examples
The oradg disk group is made up of disks, each 2 GB in length. You want to create a 6 GB
2-way mirrored volume that is striped across 2 columns.
What is the minimum number of disks you need to have in the oradg disk group to be
able to create this volume?
A. 4
B. 6
C. 8
D. 12
The correct answer is C
29

04-29
Question 5: Specifying storage attributes
Select the appropriate command to create a 1-GB volume called expvol striped across 4
columns in the acctdg disk group.
A. vxassist -g acctdg make expvol 1g layout=stripe ncol=4
B. vxvol -g acctdg make expvol 1g layout=stripe ncol=4
C. vxassist -g acctdg make expvol 1g
D. vxassist -g acctdg make expvol layout=stripe ncol=4 1g
30

04-30
Answer 5: Specifying storage attributes
Select the appropriate command to create a 1-GB volume called expvol striped across 4
columns in the acctdg disk group.
A. vxassist -g acctdg make expvol 1g layout=stripe ncol=4
B. vxvol -g acctdg make expvol 1g layout=stripe ncol=4
C. vxassist -g acctdg make expvol 1g
D. vxassist -g acctdg make expvol layout=stripe ncol=4 1g
31

04-31
End of presentation
04-32
Lesson 05: Making Configuration Changes
This is the Making Configuration Changes lesson in the Veritas InfoScale 7.4.2 Fundamentals
for UNIX/Linux: Administration course.

05-1
Administration
Administration


05-2
Lesson objectives
Topic Objective
Add a mirror to and remove a mirror from an existing volume, add a

Administering mirrored volumes
log, and change the volume read policy.
Resizing a volume and

Resize an existing volume and a file system from the command line.
a file system
Deport a disk group from one system and import it on another

Moving data between systems
system.
Renaming VxVM objects Rename VxVM objects, such as disks and disk groups.

05-3
Topic: Administering mirrored volumes
After completing this topic, you will be able to add a mirror to and remove a
mirror from an existing volume, add a log, and change the volume read
policy.
This is the Administering mirrored volumes topic.


05-4
Adding a mirror to a volume
• Provides redundancy
Plex • Improves concurrent read performance
• Requires plex synchronization (in background)
• Limitations:
Plex Plex ‒ Only volumes with concatenated or striped plexes
‒ By default, mirror created with the same layout as original plex
‒ Each mirror on separate disks
‒ All disks in the same disk group
If a volume was not originally created as a mirrored volume, or if you want to add additional
mirrors, you can add a mirror to an existing volume.
By default, a mirror is created with the same plex layout as the plex already in the volume. For
example, assume that a volume is composed of a single striped plex. If you add a mirror to the
volume, VxVM makes that plex striped, as well. However, you can specify a different layout.
A mirrored volume requires at least two disks. You cannot add a mirror to a disk that is
already being used by the volume. A volume can have multiple mirrors, as long as each mirror
resides on separate disks.
Only disks in the same disk group as the volume can be used to create the new mirror. Unless
you specify the disks to be used for the mirror, VxVM automatically locates and uses available
disk space to create the mirror.

A volume can contain up to 32 plexes (mirrors); however, the practical limit is 31. One plex
should be reserved for use by VxVM for background repair operations.
Removing a mirror
When a mirror (plex) is no longer needed, you can remove it. You can remove a mirror to
provide free space, to reduce the number of mirrors, to remove a temporary mirror.
Caution: Removing a mirror results in loss of data redundancy. If a volume only has two
plexes, removing one of them leaves the volume unmirrored.

05-5
Migrating data to a new array
1. Set up LUNs on the new array.

2. Get the OS to detect the LUNs.
3. Get VxVM to recognize the LUNs.
4. Initialize the new LUNs.
5. Add the LUNs to the same disk group as the
original volume.
6. Mirror the volume using the new LUNs.
SAN 7. Wait for the synchronization to complete.
8. Set volume read policy to the plex on the
new LUNs.
9. If satisfied, remove old plex.
10. Remove old LUNs from the disk group.
Use VIOM to automate migration
Without Storage Foundation, moving data from one array to another requires downtime.
Using Storage Foundation, you can mirror to a new array, ensure it is stable, and then remove
the plexes from the old array. No downtime is necessary. This is useful in many situations, for
example, if a company purchases a new array.
The high level steps for migrating data using Storage Foundation are listed on the slide. Note
that if you have multiple volumes on the old array, you would need to repeat steps 6 to 9 for
each volume. The following steps illustrate the commands you need to use to perform the
migration using a simple example where the appvol volume in the appdg disk group is moved
from the emc0 enclosure to the emc1 enclosure. To keep the example simple, only one LUN is
used to mirror the simple volume.
1. Set up LUNs on the new array.

2. Get the OS to detect the LUNS. For example, type devfsadm on a Solaris system.
3. vxdisk scandisks new (for VxVM to recognize LUNS from the new emc1
enclosure)
4. vxdisksetup -i emc1_dd1 (Repeat for each new LUN to be used in the volume.)
5. vxdg -g appdg adddisk appdg02=emc1_dd1
6. vxassist -g appdg mirror appvol appdg02

05-6
7. Wait for the synchronization to complete.
8. vxvol -g appdg rdpol prefer appvol appvol-02 (appvol-02 is the new
plex in the volume that is configured on the emc1 enclosure. Note that setting read
policies for mirrored volumes is explained in more detail later in this lesson.)
9. After the testing period: vxplex -g appdg -o rm dis appvol-01 (appvol-01
is the original plex in the volume that was configured on the emc0 enclosure.)
10.vxdg -g appdg rmdisk appdg01 (appdg01 is the disk media name of the old
LUN from the emc0 enclosure.)
Note that the steps after you get Storage Foundation to recognize the LUNs in the new array
can be automated using the Move Volumes functionality that is available with the Veritas
Operations Manager Storage Provisioning add-on. This wizard moves all VxVM volumes from
one enclosure to another in a single operation.

05-7
Adding/removing mirrors
vxassist -g diskgroup mirror volume_name \

[layout=layout_type] [dm_name]
Adding a mirror:
vxassist -g appdg mirror appvol
vxassist -g diskgroup remove mirror volume_name

[!dm_name]
vxplex -g diskgroup –o rm dis plex_name
Removing a mirror:
Plex Plex
vxassist -g appdg remove mirror appvol
vxplex -g appdg –o rm dis appvol-02
Adding a mirror: CLI

To add a mirror onto a specific disk, you specify the disk name in the command:
vxassist -g appdg mirror appvol appdg03
Removing a mirror: CLI

To remove a mirror, use vxassist remove mirror as shown on the slide. If you specify
a disk media name with an exclamation mark in front, the plex that contains a subdisk on that
disk is removed. To remove a specific plex, you can also use the following vxplex command
specifying the name of the plex you want to remove:
vxplex -g diskgroup -o rm dis plex_name

05-8
How dirty region logging works
1 The DRL updates dirty region bits.
Read I/O 0
0
Write I/O 0
1
Plex Plex DRL
2 If a system crashes during I/O… …volume recovery is performed for

3 those regions only.
Write I/O 10
0 Recovery 1
Write I/O 10 0
Plex Plex DRL Recovery 1
Plex Plex DRL
Logging in VxVM
By enabling logging, VxVM tracks changed regions of a volume. Log information can then be
used to reduce plex synchronization times and speed the recovery of volumes after a system
failure. Logging is an optional feature, but is highly recommended, especially for large
volumes.
Dirty region logging
Dirty region logging (DRL) is used with mirrored volume layouts. DRL keeps track of the
regions that have changed due to I/O writes to a mirrored volume. Prior to every write, a bit is
set in a log to record the area of the disk that is being changed. In case of system failure, DRL
uses this information to recover only the portions of the volume that need to be recovered.
If DRL is not used and a system failure occurs, all mirrors of the volumes must be restored to a
consistent state by copying the full contents of the volume between its mirrors. This process
can be lengthy and I/O intensive.
When you enable logging on a mirrored volume, one log plex is created by default. The log
plex uses space from disks already used for that volume, or you can specify which disk to use.
To enhance performance, you should consider placing the log plex on a disk that is not already
in use by the volume.

05-9
Logging in VxVM
By enabling logging, VxVM tracks changed regions of a volume. Log information can then be
used to reduce plex synchronization times and speed the recovery of volumes after a system
failure. Logging is an optional feature, but is highly recommended, especially for large
volumes.
Dirty region logging
Dirty region logging (DRL) is used with mirrored volume layouts. DRL keeps track of the
regions that have changed due to I/O writes to a mirrored volume. Prior to every write, a bit is
set in a log to record the area of the disk that is being changed. In case of system failure, DRL
uses this information to recover only the portions of the volume that need to be recovered.
If DRL is not used and a system failure occurs, all mirrors of the volumes must be restored to a
consistent state by copying the full contents of the volume between its mirrors. This process
can be lengthy and I/O intensive.
When you enable logging on a mirrored volume, one log plex is created by default. The log
plex uses space from disks already used for that volume, or you can specify which disk to use.
To enhance performance, you should consider placing the log plex on a disk that is not already
in use by the volume.
How does DRL work?

In the dirty region log:
• A small number of bytes of the DRL are reserved for internal use. The remaining bytes are
used for the DRL bitmap.
• The bytes are divided into two bitmaps: an active bitmap and a recovery bitmap.
• Each bit in the active bitmap maps to a single region of the volume.
• A maximum of 2048 dirty regions per system is allowed by default.
How the bitmaps are used in dirty region logging

Both bitmaps are zeroed when the volume is started initially, after a clean shutdown. As
regions transition to dirty, the corresponding bits in the active bitmap are set before the
writes to the volume occur.
If the system crashes, the active map is OR’d with the recovery map.
• Mirror resynchronization is now limited to the dirty bits in the recovery map.
• The active map is simultaneously reset, and normal volume I/O is permitted.
Usage of two bitmaps in this way allows VxVM to handle multiple system crashes.

05-10
Adding a dirty region log (DRL) to a volume
• For mirrored volumes only

• Not enabled by default
• Multiple logs for redundancy
vxassist -g diskgroup addlog volume_name [logtype=drl] \

[nlog=n] [attributes]
vxassist -g diskgroup remove log volume_name
vxassist -g appdg addlog appvol logtype=drl

vxassist -g appdg remove log appvol
11
Adding a log to a volume

To create a volume that is mirrored and logged:
vxassist -g appdg make appvol 5m layout=mirror-concat \
logtype=drl
Dirty region log considerations:
• Multiple logs can be added to mirror the DRL, up to a maximum of one log per data plex in
the volume.
• The size of the DRL is determined by Volume Manager based on the length of the volume.
•
DRL adds a small I/O overhead for most write access patterns.
• DRL should not be used for:
• Mirrored boot disks
• Volumes that have a data change object (DCO)
• Data change objects are used with the FastResync feature.
• Data volumes for databases that support the SmartSync feature of Volume Manager
Redo log volumes and other volumes that are used primarily for sequential writes
may benefit from using a sequential DRL instead of a standard DRL (logtype=drlseq).

05-11
Volume read policies
Round robin Preferred plex
Volume Volume
Read I/O Read I/O Read I/O
Preferred
Read I/O
Selected plex Site read

Read I/O
Volume from plex at Volume
Read I/O local site
Is there
a striped plex?
Default
12
One of the benefits of mirrored volumes is that you have more than one copy of the data
from which to satisfy read requests. The read policy for a volume determines the order in
which plexes are accessed during read I/O operations.
• Round robin: VxVM reads each plex in turn in “round-robin” manner for each
nonsequential I/O detected. Sequential access causes only one plex to be accessed in
order to take advantage of drive or controller read-ahead caching policies. If a read is
within 256K of the previous read, then the read is sent to the same plex.
• Preferred plex: VxVM reads first from a plex that has been named as the preferred plex.
Read requests are satisfied from one specific plex, presumably the plex with the highest
performance. If the preferred plex fails, another plex is accessed. For example, if you are
mirroring across disk arrays with significantly different performance specifications, setting
the plex on the faster array as the preferred plex would increase performance.
• Selected plex: This is the default read policy. Under the selected plex policy, Volume
Manager chooses an appropriate read policy based on the plex configuration to achieve
the greatest I/O throughput. If the mirrored volume has exactly one enabled striped plex,
the read policy defaults to that plex; otherwise, it defaults to a round-robin read policy.
• Siteread: VxVM reads preferentially from plexes at the locally defined site. This is the
default policy for volumes in disk groups where site consistency has been enabled.
• Split: Divides the read requests and distributes them across all the available plexes.

05-12
One of the benefits of mirrored volumes is that you have more than one copy of the data
from which to satisfy read requests. The read policy for a volume determines the order in
which plexes are accessed during read I/O operations.
• Round robin: VxVM reads each plex in turn in “round-robin” manner for each
nonsequential I/O detected. Sequential access causes only one plex to be accessed in
order to take advantage of drive or controller read-ahead caching policies. If a read is
within 256K of the previous read, then the read is sent to the same plex.
• Preferred plex: VxVM reads first from a plex that has been named as the preferred plex.
Read requests are satisfied from one specific plex, presumably the plex with the highest
performance. If the preferred plex fails, another plex is accessed. For example, if you are
mirroring across disk arrays with significantly different performance specifications, setting
the plex on the faster array as the preferred plex would increase performance.
• Selected plex: This is the default read policy. Under the selected plex policy, Volume
Manager chooses an appropriate read policy based on the plex configuration to achieve
the greatest I/O throughput. If the mirrored volume has exactly one enabled striped plex,
the read policy defaults to that plex; otherwise, it defaults to a round-robin read policy.
• Siteread: VxVM reads preferentially from plexes at the locally defined site. This is the
default policy for volumes in disk groups where site consistency has been enabled.
• Split: Divides the read requests and distributes them across all the available plexes.

05-13
Setting the volume read policy
vxvol -g diskgroup rdpol policy volume_name [plex]
round | select | prefer | siteread | split
vxvol -g appdg rdpol prefer appvol appvol-02

vxlist vol appvol | grep –i policy
Read Policy: PREFER
14
Setting the volume read policy: CLI

vxvol -g diskgroup rdpol round volume_name
vxvol -g diskgroup rdpol prefer volume_name preferred_plex
vxvol -g diskgroup rdpol select volume_name
Note: Before configuring the siteread policy, the Site Awareness feature must be configured
by assigning hosts and LUNs to different sites. Note that setting the siteread policy on a
volume has no impact if the site name has not been set for the host.
You can also use the vxprint command to observe the read policy of a mirrored volume as
shown in the following output extracts. Note that the fields related to the read policy are
displayed in bold font for emphasis:

The vxprint output with the default read policy:
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL
PREFPLEX
UTYPE
v appvol - ENABLED ACTIVE 2097152 SELECT – fsgen
The vxprint output after the read policy is changed to preferred plex:
v appvol - ENABLED ACTIVE 2097152 PREFER appvol-02 fsgen

05-14
Topic: Resizing a volume and a file system
After completing this topic, you will be able to resize an existing volume and
file system from the command line.
15
This is the Resizing a volume and a file system topic.


05-15
Resizing a volume and a file system
Volume resize operations:

• Performed online
• Increase or decrease volume size
‒ Disk group must have available space.
‒ VxVM rules on space allocation apply.
VxFS file system resize operations:

• Performed online (mounted)
• Increase or decrease file system size
Volume must have available space.
Order of volume and file system resize is important.
16
Resizing a volume
If users require more space on a volume, you can increase the size of the volume. If a volume
contains unused space that you need to use elsewhere, you can shrink the volume.
When the volume size is increased, sufficient disk space must be available in the disk group to
support extending the existing volume layout. A volume with concatenated layout can be
grown by any amount on any disk within the disk group whereas a volume with striped layout
can be grown only if subdisks remain the same length and an equal number of disks as stripes
are available. When increasing the size of a volume, VxVM assigns the necessary new space
from available disks. By default, VxVM uses space from any disk in the disk group, unless you
define specific disks.
Resizing a volume with a file system

Volumes and file systems are separate virtual objects. When a volume is resized, the size of
the raw volume is changed. If a file system exists that uses the volume, the file system must
also be resized. When you resize a volume using VIOM or the vxresize command, the file
system is also resized.
Resizing volumes with other types of data
For volumes containing data other than file systems, such as raw database data, you must
ensure that the data manager application can support the resizing of the data device with
which it has been configured.

05-16
Resizing a volume and a file system: Methods
Method What is resized?
VIOM Both volume and file system
vxresize Both volume and file system
vxassist Volume only
fsadm VxFS file system only
17
To resize a volume from the command line, you can use either the vxassist command or
the vxresize command. Both commands can expand or reduce a volume to a specific size
or by a specified amount of space, with one significant difference:
• vxresize automatically resizes a volume’s file system.
• vxassist does not resize a volume’s file system.
When using vxassist, you must resize the file system separately by using the fsadm
command.
When you expand a volume, both commands automatically locate available disk space unless
you designate specific disks to use. When you shrink a volume, the unused space becomes
free space in the disk group.

When you resize a volume, you can specify the length of a new volume in sectors, kilobytes,
megabytes, or gigabytes. The unit of measure is added as a suffix to the length (s, k, m, or g).
If no unit is specified, the default unit is sectors.

05-17
Resizing a volume and a file system with vxresize
vxresize [-bxs] [fstype] -g diskgroup volume_name \
[+|-]new_length
vxassist -g diskgroup maxgrow volume_name
1 vxresize –g mydg myvol 5g

5 GB 6 GB 4 GB 3 GB
Original 1 GB
2 vxresize –g mydg myvol +1g volume size
1 2 3 4
3 vxresize –g mydg myvol 4g
4 vxresize –g mydg myvol -1g
18
Resizing a volume and file system: CLI

The new_length operand can begin with a plus sign (+) to indicate that the new length is
added to the current volume length. A minus sign (-) indicates that the new length is
subtracted. -b runs the process in the background. The -x switch restricts the change to an
expand operation and the -s switch restricts the change to a shrink operation.
The vxassist maxgrow command can be used to get an estimate of how much an
existing volume can be expanded. The output indicates the amount by which the volume can
be increased and the total size to which the volume can grow. The output is displayed in
sectors, by default.
vxassist -g datadg maxgrow datavol
Volume datavol can be extended by 366592 to 1677312 (819Mb).

Note that this command does not change the size of the volume.
The ability to expand or shrink a file system depends on the file system type and whether the
file system is mounted or unmounted.

05-18
Consider the following examples:
Example: The size of the volume myvol is 1 GB. To extend myvol to 5 GB:
vxresize -g mydg myvol 5g
To extend myvol by an additional 1 GB:
vxresize -g mydg myvol +1g
To shrink myvol back to a length of 4 GB:
vxresize -g mydg myvol 4g
To shrink myvol by an additional 1 GB:
vxresize -g mydg myvol -1g
Resizing a volume only: vxassist
The vxassist command can be used to resize a volume only as follows:
vxassist -g diskgroup {growto|growby|shrinkto|shrinkby} \
volume_name size
• growto increases volume to specified length
• growby increases volume by specified amount
• shrinkto reduces volume to specified length
• shrinkby reduces volume by specified amount
You should use this command only if the volume does not include a file system or if you are
resizing the volume and the file system separately for a specific purpose.
Resizing a file system only: fsadm
You may need to resize a file system to accommodate a change in use—for example, when
there is an increased need for space in the file system. You may also need to resize a file
system as part of a general reorganization of disk
usage—for example, when a large file system is subdivided into several smaller file systems.
You can resize a VxFS file system while the file system remains mounted by using the fsadm
command:
fsadm [-b newsize] [-r rawdev] mount_point
Using fsadm to resize a file system does not automatically resize the underlying volume.
When you expand a file system, the underlying device must be large enough to contain the
new larger file system.

05-19
Resizing a Volume Manager disk to match a resized LUN
• Intended for LUNs that are part of an imported

3 disk group
2
1 • Required if the LUN is resized in the hardware
LUN IDs: 1|2|3
vxdisk [-f] -g diskgroup resize dm_name
vxdisk -g appdg resize appdg01
20
When you resize a LUN in the hardware, you should resize the VxVM disk corresponding to
that LUN. You can use vxdisk resize to update disk headers and other VxVM structures
to match a new LUN size. This command does not resize the underlying LUN itself.

05-20
Topic: Moving data between systems
After completing this topic, you will be able to deport a disk group from one
system and import it on another system.
21
This is the Moving data between systems topic.


05-21
Example: HA environment
Computer sysdg osdg Computer
rootvol rootvol
sys1 sys2
Boot disks Boot disks
appdg dbdg
appvol dbvol
Additional disks
22
The slide illustrates a high availability environment.

In the example, computers sys1 and sys2 each have their own bootdg on their own private
SCSI bus. The two hosts are also on a shared SCSI bus. On the shared bus, each host has a disk
group, and each disk group has a set of VxVM disks and volumes. There are additional disks
on the shared SCSI bus that have not been added to a disk group.
If sys1 fails, then sys2, which is on the same SCSI bus as the appdg disk group, can take
ownership or control of the disk group and all of its components.

05-22
Deporting a disk group
appdg
Volume Before deporting:
• Stop applications.
• Unmount file systems.
VM disks
When you deport a disk group,
Deport you can optionally:
• Specify a new host.
• Rename the disk group.
appdg
Volume
Before deporting:
• Stop applications.
• Unmount file systems.
VM disks
23
A deported disk group is a disk group over which management control has been surrendered.
The objects within the disk group cannot be accessed, its volumes are unavailable, and the
disk group configuration cannot be changed. (You cannot access volumes in a deported disk
group because the directory containing the device nodes for the volumes are deleted upon
deport.) To resume management of the disk group, it must be imported.
A disk group cannot be deported if any volumes in that disk group are in use. Before you
deport a disk group, you must unmount file systems and stop any application using the
volumes in the disk group.
Deporting and specifying a new host
When you deport a disk group using CLI commands, you have the option to specify a new host
to which the disk group is imported at reboot. If you know the name of the host to which the
disk group will be imported, specify the new host during the operation. If you do not specify
the new host, the disks could accidentally be added to another disk group, resulting in data
loss. You cannot specify a new host using the vxdiskadm utility.
Deporting and renaming
When you deport a disk group using InfoScale Operations Manager or CLI commands, you also
have the option to rename the disk group. Note that the disk cannot be renamed when
deporting using the vxdiskadm utility.

05-23
Importing a disk group
appdg
Volume After import:
• Start volumes (if necessary)
• Mount file systems
• Start applications
VM disks
Import
When you import a disk group, you can:
appdg
• Specify a new disk group name.
Volume
• Clear host locks.
• Import as temporary.
• Force an import.
VM disks
24
During a disk group import operation, the volume device files

(/dev/vx/[r]dsk/diskgroup/volume_name) are created and with SF versions 5.1
SP1 and later, the volumes are automatically started.
Importing and renaming
A deported disk group cannot be imported if another disk group with the same name has
been created since the disk group was deported. You can import and rename a disk group at
the same time.
Importing and clearing host locks
When a disk group is created, the system writes a lock on all disks in the disk group. The lock
ensures that dual-ported disks (disks that can be accessed simultaneously by two systems) are
not used by both systems at the same time. If a system crashes, the locks stored on the disks
remain, and if you try to import a disk group containing those disks, the import fails.
Importing as temporary
A temporary import does not persist across reboots. A temporary import can be useful, for
example, if you need to perform administrative operations on the temporarily imported disk
group.
Forcing an import
A disk group import fails if the VxVM configuration daemon cannot find all of the disks in the
disk group. If the import fails because a disk has failed, you can force the import. Forcing an
import should always be performed with caution.

05-24
How to deport and import a disk group
vxdiskadm:
Remove access to (deport) a disk group
CLI:
vxdg [-n new_dg_name] [-h hostname] deport diskgroup
vxdiskadm:
Enable access to (import) a disk group

Pre-5.1 SP1
CLI:
vxdg [-ftC] [-n new_name] import diskgroup
vxvol –g diskgroup startall
25
How to deport a disk group

Before deporting a disk group, unmount all file systems used within the disk group that is to
be deported. You can also stop all volumes in the disk group to verify that they are not being
used:
umount mount_point
vxvol -g diskgroup stopall
After you deport a disk group, disks that were in the disk group have a state of Deported. If
the disk group was deported to another host, the disk state is Foreign.
How to import a disk group:
With SF 5.1 SP1 and later, all volumes in the disk group are started automatically during a disk
group import by default. However, with earlier versions of SF or if the autostartvolumes
parameter is modified to off, you must manually start all volumes after you import a disk
group from the command line.
A disk group must be deported from its previous system before it can be imported to the new
system. During the import operation, the system checks for host import locks. If any locks are
found, you are prompted to clear the locks.
To temporarily import a disk group, you use the -t option. This option does not set the
autoimport flag, which means that the import cannot survive a reboot.

05-25
To display all disk groups, including deported disk groups:
DEVICE TYPE DISK GROUP
STATUS
emc0_dd1 auto:cdsdisk appdg01 appdg
online

05-26
Topic: Renaming VxVM objects
After completing this topic, you will be able to rename VxVM objects, such
as disks and disk groups.
27
This is the Renaming VxVM objects topic.


05-27
Renaming VxVM objects
• Online operation
‒ Object names must be unique within the DG.
‒ Related objects must also be renamed.
• Possible impact on file system or application
vxedit -g diskgroup rename old_name new_name
vxedit -g appdg rename appdg01 appdg03

vxedit -g appdg rename appvol appdatavol
28
Changing the disk media name

VxVM creates a unique disk media name for a disk when you add a disk to a disk group.
Sometimes you may need to change a disk name to reflect changes of ownership or use of the
disk. Renaming a disk does not change the physical disk device name. The new disk name
must be unique within the disk group.
Before you rename a VxVM object
Before you rename a VxVM object, you should carefully consider the change. For example,
VxVM names subdisks based on the disks on which they are located. A disk named appdg01
contains subdisks that are named appdg01-01, appdg01-02, and so on. Renaming a disk does
not automatically rename its subdisks. Similarly, renaming a volume does not automatically
rename its plexes. Volumes are not affected when subdisks are named differently from the
disks.

05-28
Renaming a disk group
dbserver
olddgname newdgname
Deport Import
vxdg -n new_dg_name deport old_dg_name

vxdg import new_dg_name
or
vxdg deport old_dg_name
vxdg -n new_dg_name import old_dg_name
29
You cannot import or deport a disk group when the target system already has a disk group of
the same name. To avoid name collision or to provide a more appropriate name for a disk
group, you can rename a disk group.
• To rename a disk group when moving it from one system to another, you specify the new
name during the deport or during the import operations.
• To rename a disk group without moving the disk group, you must still deport and reimport
the disk group on the same system.
Note that renaming a disk group:
• does not change the disk group ID (dgid).
• may require modifying the file system table (For example, /etc/vfstab for Solaris).
• may require modifying applications, such as databases, using the volumes.
Using the CLI, for example, to rename the disk group appdg to oradg:
vxdg -n oradg deport appdg or vxdg deport appdg
vxdg import oradg vxdg -n oradg import appdg
From the command line, if you need to restart all volumes in the disk group:
vxvol -g new_dg_name startall
vxvol -g oradg startall

05-29
Support for disk group level encryption key management and the
re-key operation
• InfoScale supports the use of a single KMS key for all the volumes in a disk group.
• You can maintain a common KMS key at the disk group level instead of maintaining an individual KMS key for
each volume.
• When you start an encrypted volume with a common KMS key with the disk group, VxVM needs to fetch only
one key to enable access to the volume. This reduces the network load sent to the KMS in the form of multiple
requests based on the number of volumes.
• A single request to KMS allows to start all the volumes in a single operation.
To use the key, InfoScale provides the option to re-key the volumes that change the KMS key when needed. This
option is also known as key rotation.
You can use an external scheduler based on your policy to schedule the re-key operation.
30
InfoScale supports the use of a single KMS key for all the volumes in a disk group.
Consequently, you can maintain a common KMS key at the disk group level instead of
maintaining an individual KMS key for each volume. When you start an encrypted volume that
has a common KMS key with the disk group, VxVM needs to fetch only one key to enable
access to the volume. Thus, a common KMS key reduces the network load that is sent to the
KMS in the form of multiple requests based on the number of volumes. A single request to
KMS lets you to start all the volumes in a single operation.
To make the use of this single key more secure, InfoScale provides the option to re-key the
volumes that change the KMS key when needed. This option is also known as key rotation.
You can use an external scheduler based on your policy to schedule the re-key operation.

05-30
Support for disk group level encryption key management and the
re-key operation
To use a single key for all the encrypted volumes in a disk group, set the value of the same_enckey tunable to yes as
follows:
At the time of disk group creation, set:
vxdg -o same_enckey=yes init DiskGroupName diskName1 diskName2 ... diskNameN
The re-key operation behaves as follows:

• The volume encryption key remains unchanged.
• The encrypted volume encryption key is retrieved using the existing KMS key, and then encrypted again with the
new KMS key.
• The newly encrypted volume encryption key and the new KMS identifier of the changed KMS key are stored in the
volume record.
The disk group level encryption key management and key rotation feature does not support VVR configuration and
disk group operations like join, split, move.
31
To use a single key for all the encrypted volumes in a disk group, set the value of the
same_enckey tunable to yes as follows:
At the time of disk group creation, set:
vxdg -o same_enckey=yes init DiskGroupName diskName1 diskName2 ... diskNameN
The re-key operation behaves as follows:

• The volume encryption key remains unchanged.
• The encrypted volume encryption key is retrieved using the existing KMS key, and then
encrypts it again with the new KMS key.

• The newly encrypted volume encryption key and the new KMS identifier of the changed
KMS key are stored in the volume record.
Note: The disk group level encryption key management and key rotation feature does not
support VVR configuration and disk group operations like join, split, move.

05-31
Lesson summary
• Key points
– In this lesson, you learned how to add a mirror to and remove a mirror from an existing volume.
– You also learned how to change the volume read policy and resize an existing volume.
– In addition, you learned to deport a disk group from one system and import it on another system.
– Finally, you learned how to rename VxVM objects, such as disks and disk groups, and upgrade disk
groups.
32

05-32
Lab 05: Making Configuration Changes
• Exercise A: Administering mirrored volumes

• Exercise B: Resizing a volume and file system
• Exercise C: Renaming a disk group
• Exercise D: Moving data between systems
• Exercise E: (Optional) Resizing a file system
33

05-33
What did you learn?
lesson.
34
current lesson.

05-34
Question 1: mirror expvol volume command
Select the appropriate command to mirror the expvol volume that exists in the acctdg
disk group, using any available disk space.
A. vxassist -g acctdg mirror expvol
B. vxdisk -g acctdg addmirror expvol
C. vxassist -g acctdg newmirror expvol
D. vxdg -g acctdg nmirror expvol
35

05-35
Answer 1: mirror expvol volume command
Select the appropriate command to mirror the expvol volume that exists in the acctdg
disk group, using any available disk space.
A. vxassist -g acctdg mirror expvol
B. vxdisk -g acctdg addmirror expvol
C. vxassist -g acctdg newmirror expvol
D. vxdg -g acctdg nmirror expvol
36

05-36
Question 2: Add dirty region log command
Select the appropriate command to add a dirty region log to the expvol mirrored volume
that exists in the acctdg disk group.
A. vxdisk -g acctdg addlog expvol logtype=drl
B. vxlog -g acctdg newlog expvol logtype=drl
C. vxassist -g acctdg addlog expvol logtype=drl
D. vxassist -g acctdg newlog expvol logtype=nlog
37

05-37
Answer 2: Add dirty region log command
Select the appropriate command to add a dirty region log to the expvol mirrored volume
that exists in the acctdg disk group.
A. vxdisk -g acctdg addlog expvol logtype=drl
B. vxlog -g acctdg newlog expvol logtype=drl
C. vxassist -g acctdg addlog expvol logtype=drl
D. vxassist -g acctdg newlog expvol logtype=nlog
38

05-38
Question 3: Set read policy command
Select the appropriate command to set the read policy for the expvol volume, which
exists in the acctdg disk group, to round robin.
A. vxassist -g acctdg rdpol round expvol
B. vxvol -g acctdg rdpol round expvol
C. vxvol -g acctdg rdpol robin expvol
D. vxassist -g acctdg rdpol robin expvol
39

05-39
Answer 3: Set read policy command
A. vxassist -g acctdg rdpol round expvol
B. vxvol -g acctdg rdpol round expvol
C. vxvol -g acctdg rdpol robin expvol
D. vxassist -g acctdg rdpol robin expvol
40

05-40
Question 4: Extend volume and resize file system command
Select the appropriate command to extend the payvol volume by an additional 30 MB and resize
the file system at the same time. The volume is in the hrdg disk group.
A. vxassist -g hrdg growby payvol 30m

B. vxresize -g hrdg payvol 30m
C. vxresize -g hrdg payvol +30m
D. vxassist -g hrdg growto payvol 30m
41

05-41
Answer 4: Extend volume and resize file system command
Select the appropriate command to extend the payvol volume by an additional 30 MB and resize
the file system at the same time. The volume is in the hrdg disk group.
A. vxassist -g hrdg growby payvol 30m

B. vxresize -g hrdg payvol 30m
C. vxresize -g hrdg payvol +30m
D. vxassist -g hrdg growto payvol 30m
42

05-42
Question 5: Renaming a disk
To rename a disk from the command line, you use the command:
A. vxedit rename
B. vxdisk rename
C. vxdg rename
D. vxdisk newname
43

05-43
Answer 5: Renaming a disk
To rename a disk from the command line, you use the command:
A. vxedit rename
B. vxdisk rename
C. vxdg rename
D. vxdisk newname
44

05-44
Question 6: Deport/rename command
Select the appropriate command to deport the acctdg disk group and rename it as paydg
at the same time.
A. vxdg -r acctdg deport paydg
B. vxdg -d paydg deport acctdg
C. vxdg -n acctdg deport paydg
D. vxdg -n paydg deport acctdg
45

05-45
Answer 6: Deport/rename command
Select the appropriate command to deport the acctdg disk group and rename it as paydg
at the same time.
A. vxdg -r acctdg deport paydg
B. vxdg -d paydg deport acctdg
C. vxdg -n acctdg deport paydg
D. vxdg -n paydg deport acctdg
46

05-46
End of presentation
05-47
Lesson 06: Administering File Systems
This is the Administering File Systems lesson in the Veritas InfoScale 7.4.2 Fundamentals for

06-1
Administration
Administration


06-2
Lesson objectives
Topic Objective
Benefits of using Veritas File System Explain Veritas File System features, such as extent-based allocation.
Using Veritas File System commands Apply the appropriate VxFS commands from the command line.
Logging in VxFS Perform logging in VxFS by using the intent log and the file change
log.
Controlling file system Perform logging in VxFS by using the intent log and the file change
fragmentation log.
Using thin provisioning disk arrays Use SF features that optimize storage with thin provisioning disk
arrays, such as the SmartMove feature and thin reclamation.

06-3
Topic: Benefits of using Veritas File System
After completing this topic, you will be able to explain Veritas File System
features, such as extent-based allocation.
This is the Benefits of using Veritas File System topic.


06-4
Veritas File System features (1/2)
• Intent log
Fast file system recovery
• Extent-based allocation
Extent attributes for files
• Online administration
– Online resize
– Online defragmentation
– Online backup with file system snapshots
• Storage checkpoints
• Multi-Volume File System support
• SmartTier (previously known as Dynamic Storage Tiering)
• Improved database performance
– Quick I/O for databases
– Veritas extension for ODM
A file system is simply a method for storing and organizing computer files and the data they
contain to make it easy to find and access them.
Veritas File System includes the following features:
• Intent log
− Veritas File System (VxFS) was the first commercial journaling file system. With
journaling, metadata changes are first written to a log (or journal) then to disk. Since
changes do not need to be to be written in multiple places, throughput is much faster
as the metadata is written asynchronously.
− VxFS provides fast recovery of a file system from system failure because the recovery
usually involves only a log replay.

• Extent-based allocation
− Extents allow disk I/O to take place in units of multiple blocks if storage is allocated in
consecutive blocks. This topic is analyzed in more detail in the following pages.
− Extent attributes
Extent attributes are the extent allocation policies associated with a file.
• Online administration
A lot of the file system administration tasks, such as backing the file system up or resizing
the file system, can be performed while the file system is still mounted. Online file system
defragmentation is discussed later in this lesson.

06-5
Veritas File System features (2/2)
• Performance tuning options
– Extended mount options
– Large file and file system support
– Improved synchronous writes
– Delayed allocations for extending writes
• Cross-platform data sharing
• Access control lists (ACLs)
• Quotas
• File change log
• SmartMove feature
• Thin provisioning support
• File system data compression
• File system deduplication
• File replication
• Cluster File System
• Storage checkpoints
Backup and restore applications can leverage Storage Checkpoint, a disk- and I/O-efficient
copying technology for creating periodic frozen images of a file system.
• Multi-volume file system support
The multi-volume support feature allows several volumes to be represented by a single
logical object. This feature is used with the SmartTier feature.
• SmartTier (previously known as Dynamic Storage Tiering)
The SmartTier feature allows you to configure policies that automatically allocate storage
from specific volumes for certain files, or relocate files by running file relocation
commands, which can improve performance for applications that access specific types of
files.
• Improved database performance
Databases can be created on the character devices to achieve the same performance as
databases created on raw disks.

06-6
• Performance tuning options
− The VxFS file system supports extended mount options to specify enhanced data
integrity modes, enhanced performance modes, temporary file system modes. For
more information on these modes of operation, refer to the Veritas Storage
Foundation Administrator’s Guide.
− VxFS provides superior performance for synchronous write applications.
− VxFS supports files larger than two gigabytes and large file systems up to 256
terabytes.
• Cross-platform data sharing
Cross-platform data sharing allows data to be serially shared among heterogeneous
systems where each system has direct access to the physical devices that hold the data.
• Access control lists (ACLs)
An access control list (ACL) stores a series of entries that identify specific users or groups
and their access privileges for a directory or file.
• Quotas
VxFS supports quotas, which allocate per-user and per-group quotas and limit the use of
two principal resources: files and data blocks.
• File change log
The VxFS file change log tracks changes to files and directories in a file system.
• The SmartMove feature
The information stored by Veritas File System about used and unused blocks is used by
Veritas Volume Manager to optimize mirror synchronization operations.
• Storage Foundation thin reclamation
The thin reclamation feature allows you to release free data blocks of a VxFS file system to
the free storage pool of a thin storage LUN. This feature is only supported on file systems
mounted on a VxVM volume.

Note: The Storage Foundation thin reclamation feature is not supported on the Solaris
x64 operating environment.
• File system data compression
The file system data compression feature with SF 6.x aims to reduce the space used by
files, while retaining the accessibility of the files and being transparent to applications.

06-7
• File system deduplication
The Veritas file system deduplication feature is another new feature with SF 6.x that aims
to maximize storage utilization. This feature scans the file system, identifies the duplicate
data and eliminates it without any continuous cost.
• File replication
Veritas File Replicator (VFR) is included in the Veritas Replicator license, supports file-level
replication of application data. VFR tracks all updates to the file system and periodically
replicates these updates at the end of a configured time interval.
• Cluster File System
Clustered file systems are an extension of VxFS that support concurrent direct media
access from multiple systems.

06-8
VxFS extent-based allocation
inode for n, 18
File system /app/datafile n+37, 6
block
Address-length pair: n, 18 Address-length pair: n+37, 6
n n+1 n+2 n+3 n+4 n+5 n+6 n+7 n+8 n+37 n+38 n+39
n+9 n+10 n+11 n+12 n+13 n+14 n+15 n+16 n+17 n+40 n+41 n+42
First data extent Second data extent
• Inode: A unique identifier for a file

• Block: Smallest amount of disk space allocated to a file
• Extent: A set of contiguous blocks
‒ Initial extent size based on size of I/O write request
‒ Additional extents allocated progressively larger as file is expanded
VxFS extent-based allocation

Similar to other file systems on UNIX platforms, VxFS uses index tables to store information
and location information about blocks used for files. VxFS allocation is extent-based as
opposed to block-based.
• Block-based allocation: File systems that use block-based allocation assign disk space to a
file one block at a time.
• Extent-based allocation: File systems that use extent-based allocation assign disk space in
groups of contiguous blocks, called extents. Veritas File System selects a contiguous range
of file system blocks, called an extent, for inclusion in a file.
Extent-based allocation enables larger I/O operations to be passed to the underlying drivers,
which results in good performance and less metadata overhead.
Each file is associated with an index block, called an inode. In an inode, an extent is
represented as an address-length pair, which identifies the starting block address and the
length of the extent in logical blocks. This enables the file system to directly access any block
of the file.
VxFS automatically selects an extent size by using a default allocation policy that is based on
the size of I/O write requests. The default allocation policy attempts to balance two goals:
• Optimum I/O performance through large allocations.
• Minimal file system fragmentation through allocation from space available in the file
system that best fits the data The first extent allocated is large enough for the first write to
the file. Typically, the first extent is the smallest power of 2 that is larger than the size of
the first write, with a minimum extent allocation of 8K. Additional extents are
progressively larger, doubling the size of the file with each new extent.

06-9
Topic: Using Veritas File System commands
After completing this topic, you will be able to apply the appropriate VxFS
commands from the command line.
10
This is the Using Veritas File System commands topic.


06-10
Using VxFS commands
• PATH environment variable:
• /opt/VRTS/bin
• Other optional directories
• MANPATH environment variable: /opt/VRTS/man
man command_vxfs (for standard OS commands)
• VxFS uses standard file system management syntax:
command [fstype] [generic_options] \
[-o VxFS_options][special|mount_point]
mount –t vxfs –r –o noatime

/dev/vx/dsk/appdg/appvol /app
AIX HP-UX Linux Solaris

11
You can generally use Veritas File System (VxFS) as an alternative to other disk-based, OS-
specific file systems, except for the file systems used to boot the system. File systems used to
boot the system are mounted read-only in the boot process, before the VxFS driver is loaded.
VxFS can be used in place of:
• UNIX File System (UFS) on Solaris, except for root, /usr, /var, and /opt.
• Hierarchical File System (HFS) on HP-UX, except for /stand.
• Journaled File System (JFS) and Enhanced Journaled File System (JFS2) on
• AIX, except for root and /usr.
•
Extended File System Version 2 (EXT2) and Version 3 (EXT3) on Linux, except for root,
/boot, /etc, /lib, /var, and /usr.
Location of VxFS commands
Most Veritas file system commands are located in /opt/VRTS/bin, which must be
included in the PATH environment variable.

06-11
General file system command syntax
To access VxFS-specific versions, or wrappers, of standard commands, you use the Virtual File
System switchout mechanism followed by the file system type, vxfs. The switchout
mechanism directs the system to search the appropriate directories for VxFS-specific versions
of commands.
Platform File System Switchout

Solaris -F vxfs
HP-UX -F vxfs
AIX -V vxfs (or -v vxfs when used with crfs)
Linux -t vxfs
Note: The Linux platform includes a native fsadm command in the /usr/sbin directory. If
this path is listed before the /opt/VRTS/bin directory in the PATH environment variable,
provide the full pathname of the fsadm command (/opt/VRTS/bin/fsadm) to use the
VxFS-specific version of this command.
Using VxFS commands by default

If you do not use the switchout mechanism, then the file system type is taken from the
default specified in the OS-specific default file system file.
If you want Veritas File System to be your default file system type, then you change the
default file system file to contain vxfs.
Platform Default file system file
Solaris /etc/default/fs
HP-UX /etc/default/fs
AIX /etc/vfs
Linux /etc/default/fs

06-12
VxFS-specific mkfs options
mkfs [fstype] [generic_options] [-o VxFS_options] special
Option Description
• Displays how fs would be

-o N • Does not create the file system
created
-m • Displays how fs was created • File system must already exist.
• Valid values: 7, 8, and 9
-o version=n • Specifies layout version
• Default: Version 9
• Cannot be changed after creation

• Valid values:
-o bsize=n • Sets block size for files
1024 bytes (1K) to 8192 bytes (8K).
• Default: 1K (<=1 TB FS), 8K (>1 TB FS)
13
Using mkfs command options

You can set a variety of file system properties when you create a Veritas file system by adding
VxFS-specific options to the mkfs command.
Here are some examples of outputs from a Linux platform before and after the creation of a
Veritas file system:
mkfs -t vxfs -o N /dev/vx/rdsk/appdg/appvol
version 9 layout
4194304 sectors, 2097152 blocks of size 1024, log size 16384
blocks rcq size 1024 blocks largefiles supported
mkfs -t vxfs /dev/vx/rdsk/appdg/appvol

mkfs -m /dev/vx/rdsk/appdg/appvol
mkfs -t vxfs -o
bsize=1024,version=9,inosize=256,logsize=16384,rcqsize=1024,lar
gefiles /dev/vx/rdsk/appdg/appvol 4194304

06-13
Other file system commands
Option Description
options
mount [-v] Displays mounted file systems
Mount
mount -a Mounts all in file system table
mount … -r…or mount…-o ro … Mounts as read-only

Unmount
umount /mydata Unmounts a file system

options
umount –a Unmounts all mounted file systems
fstyp /dev/vx/dsk/appdg/appvol Displays file system type

options
Display
df -h Displays free space
14
Identifying the file system type

If you do not know the file system type of a particular file system, you can determine the file
system type by using the fstyp command. You can use the fstyp command to describe
either a mounted or unmounted file system.
Identifying free space
To report the number of free disk blocks and inodes for a VxFS File System, you use the df
command. The df command displays the number of free blocks and free inodes in a file
system or directory by examining the counts kept in the superblocks. Extents smaller than 8K
may not be usable for all types of allocation, so the df command does not count free blocks
in extents below 8K when reporting the total number of free blocks.

06-14
Topic: Logging in VxFS
After completing this topic, you will be able to perform logging in VxFS by
using the intent log and the file change log.
15
This is the Logging in VxFS topic.


06-15
Intent log
The intent log records pending
1 file system changes before
metadata is changed.
Intent log Structural files Allocation units
Data
Metadata
3
If the system
crashes, the
intent log is 2
The intent log is written before
replayed by Disk
file system updates are made.
VxFS fsck.
fsck
16
Intent log
A file system may be left in an inconsistent state after a system failure. Recovery of structural
consistency requires examination of file system metadata structures. Veritas File System
provides fast file system recovery after a system failure by using a tracking feature called
intent logging, or journaling. Intent logging is the process by which intended changes to file
system metadata are written to a log before changes are made to the file system structure.
Once the intent log has been written, the other updates to the file system can be written in
any order. In the event of a system failure, the VxFS fsck utility replays the intent log to
nullify or complete file system operations that were active when the system failed.
Traditionally, the length of time taken for recovery using fsck was proportional to the size of
the file system. For large disk configurations, running fsck is a time-consuming process that
checks, verifies, and corrects the entire file system.
The VxFS version of the fsck utility performs an intent log replay to recover a file system
without completing a full structural check of the entire file system. The time required for log
replay is proportional to the log size, not the file system size. Therefore, the file system can be
recovered and mounted seconds after a system failure. Intent log recovery is not readily
apparent to users or administrators, and the intent log can be replayed multiple times with no
adverse effects.
Note: Replaying the intent log may not completely recover the damaged file system structure
if the disk suffers a hardware failure. Such situations may require a complete system check
using the VxFS fsck utility.

06-16
Maintaining VxFS consistency
To check file system consistency by using the intent log for the VxFS file system on the appvol
volume:
fsck [fstype] /dev/vx/rdsk/appdg/appvol
To perform a full check and report status at regular intervals:

fsck [fstype] -o full,status,interval=N \
/dev/vx/rdsk/appdg/appvol
17
You use the VxFS-specific version of the fsck command to check the consistency of and
repair a VxFS file system. The fsck utility replays the intent log by default, instead of
performing a full structural file system check, which is usually sufficient to set the file system
state to CLEAN. You can also use the fsck utility to perform a full structural recovery in the
unlikely event that the log is unusable.
The syntax for the fsck command is:
/opt/VRTS/bin/fsck [fstype] [generic_options] [-y|-Y] [-n|-N]
\
[-o full,nolog] special
For a complete list of generic options, see the fsck(1m) manual page. Some options include:
-o p can only be run with log fsck, not with full fsck.

06-17
Topic: Controlling file system fragmentation
After completing this topic, you will be able to defragment a VxFS file
system.
18
This is the Controlling file system fragmentation topic.


06-18
Fragmentation
Initial Allocation Fragmented Defragmented
• Degree of fragmentation depends on:

‒ File system usage
‒ File system activity patterns
• Fragmentation types:
‒ Directory
‒ Extent
19
Controlling file system fragmentation

In a Veritas file system, when free resources are initially allocated to files, they are aligned in
the most efficient order possible to provide optimal performance. On an active file system,
the original order is lost over time as files are created, removed, and resized. As space is
allocated and deallocated from files, the available free space becomes broken up into
fragments. This means that space has to be assigned to files in smaller and smaller extents.
This process is known as fragmentation. Fragmentation leads to degraded performance and
availability.
VxFS provides online reporting and optimization utilities to enable you to monitor and
defragment a mounted file system. These utilities are accessible through the file system
administration command, fsadm.

Types of fragmentation
VxFS addresses two types of fragmentation:
• Directory fragmentation: As files are created and removed, gaps are left in directory
inodes. This is known as directory fragmentation. Directory fragmentation causes directory
lookups to become slower.
• Extent fragmentation: As files are created and removed, the free extent map for an
allocation unit changes from having one large free area to having many smaller free areas.
Extent fragmentation occurs when files cannot be allocated in contiguous chunks and
more extents must be referenced to access a file. In a case of extreme fragmentation, a
file system may have free space, none of which can be allocated.

06-19
Monitoring fragmentation A high total in the Dirs to Reduce
column indicates fragmentation.
To monitor directory fragmentation:
fsadm –D /mnt1
Dirs Total Immed Immeds Dirs to Blocks
Searched Blocks Dirs to Add Reduce to Reduce
total 486 99 388 6 6 6
To monitor extent fragmentation:
fsadm –E /home
…
Free Space Fragmentation Index : 80
File Fragmentation Index : 60
# Files Fragmented by Fragmentation Index
0 1-25 26-50 51-75 76-100
7838181 3774 82836 281144 8080449
% Free blocks in extents smaller than 64 blks: 8.35
% Free blocks in extents smaller than 8 blks: 4.16
% blks allocated to extents 64 blks or larger: 45.81
Output displays percentages of free and
allocated blocks per extent size.
20
Running fragmentation reports

You can monitor fragmentation in a Veritas file system by running reports that describe
fragmentation levels. You use the fsadm command to run reports on both directory and
extent fragmentation. The df command, which reports on file system free space, also
provides information useful in monitoring fragmentation.
Interpreting fragmentation reports
In general, for optimum performance, the percentage of free space in a file system should not
fall below 10 percent. A file system with 10 percent or more free space has less fragmentation
and better extent allocation. A badly fragmented file system will have one or more of the
following characteristics:
• Greater than 5 percent of free space in extents of less than 8 blocks in length
• More than 50 percent of free space in extents of less than 64 blocks in length
• Less than 5 percent of the total file system size available as free extents in lengths of 64 or
more blocks
Fragmentation can also be determined based on the fragmentation index. The fragmentation
report displays fragmentation indices for both the free space and the files in the file system. A
value of 0 for the fragmentation index means that the file system has no fragmentation, and a
value of 100 means that the file system has the highest level of fragmentation. The
fragmentation index is new with SF 6.x and enables you to determine whether you should
perform extent defragmentation or free space defragmentation.

06-20
Defragmenting a file system
fsadm [-de] [-DEH] [-t time] [-p passes] mount_point|-f dir
During extent reorganization:

• Small files are made contiguous.
• Large files are built from large extents.
Examples:
fsadm –eEH /mnt1
fsadm –eEH –f /mnt1/largedir
During directory reorganization:

• Directories are packed into the inode area.
• Directories are placed before other files.
• Entries are sorted by access time.
Example: fsadm –dDH /mnt1
21
Defragmenting a file system

You can use the online administration utility fsadm to defragment, or reorganize, file system
directories and extents. The fsadm utility defragments a file system mounted for read/write
access by:
• Removing unused space from directories
• Making all small files contiguous
• Consolidating free blocks for file system use
• Sorting entries by the time of last access
Only a privileged user can reorganize a file system.

The fsadm defragmentation options
If you specify both -d and -e, directory reorganization is always completed before extent
reorganization. If you use the -D and -E with the -d and -e, options, fragmentation reports
are produced both before and after the reorganization.
You can use the -t and -p options to control the amount of work performed by fsadm,
either in a specified time or by a number of passes. By default, fsadm runs five passes. If
both -t and -p are specified, fsadm exits if either of the terminating conditions is reached.
On the Linux platform, the -T time option is used instead of the
-t time option because the -t switch is used for file system switchout mechanism.

06-21
Free space defragmentation for a file system
fsadm [fstype] –C [-v] mount_point
Free space reorganization aims to create bigger chunks of free space in the file system.
df -os /mnt3
/mnt3 (/dev/vx/dsk/testdg/vol3): … blocks … files
Free Extents by Size
1: 2077988 2: 2104073 4: 1371895
8: 2226679 16: 1618029 32: 1000385
64: 53134 128: 1667 256: 480
512: 352 1024: 302 2048: 244
4096: 172 8192: 107 16384: 76
32768: 122 65536: 5 131072: 0
262144: 0 524288: 0 1048576: 0
2097152: 0 4194304: 0 8388608: 0
16777216: 0 33554432: 0 67108864: 0
134217728: 0 268435456: 0 536870912: 0
1073741824: 0 2147483648: 0
22
Free space defragmentation for a file system

The free space defragmentation option is new with SF 6.x. It attempts to get bigger chunks of
free space in the file system by:
• Freeing as many fragmented allocation units as possible
• Filling as many allocation units completely as possible
• Never breaking any file extent during data movement to ensure that file extent
fragmentation does not get worse during the process
Note that you can observe the available free extents by size using the VxFS-specific df -os
command as shown on the slide.

06-22
Scheduling defragmentation
Frequency depends on usage and activity patterns
Run on demand or as a cron job
Adjust intervals based on reports
23
The best way to ensure that fragmentation does not become a problem is to defragment the
file system on a regular basis. The frequency of defragmentation depends on file system
usage, activity patterns, and the importance of file system performance.
In general, follow these guidelines:
• Schedule defragmentation during a time when the file system is relatively idle.
• For frequently used file systems, you should schedule defragmentation daily or weekly.
• For infrequently used file systems, you should schedule defragmentation at least monthly.
• Full file systems tend to fragment and are difficult to defragment. You should consider
expanding the file system.
To determine the defragmentation schedule that is best for your system, select what you think
is an appropriate interval for running extent reorganization and run the fragmentation reports
both before and after the reorganization. If the degree of fragmentation is approaching the
bad fragmentation figures, then the interval between fsadm runs should be reduced. If the
degree of fragmentation is low, then the interval between fsadm runs can be increased.
You should schedule directory reorganization for file systems when the extent reorganization
is scheduled. The fsadm utility can run on demand and can be scheduled regularly as a cron
job. The defragmentation process can take some time. You receive an alert when the process
is complete.

06-23
Topic: Using thin provisioning disk arrays
After completing this topic, you will be able to use SF features that optimize
storage with thin provisioning disk arrays, such as the SmartMove feature
and thin reclamation.
24
This is the Using thin provisioning disk arrays topic.


06-24
What is a thin provisioning disk array?
Traditional Thin provisioned
storage utilization storage utilization
Actual file system data: 100 GB 100 GB
Volume on host: 1 TB 1 TB
Physical storage
allocated for LUN in 1 TB 100 GB
array:
Allocate at LUN Allocate on

creation write
25
Thin provisioning is a hardware-based storage solution that enables system administrators to

configure storage space for a server without pre allocating the space on storage array. A thin
provisioning disk array creates virtual disk drives (LUNs) that appear to be one size, but whose
actual physical storage only covers a fraction of their claimed size. If a LUN needs more
storage, the storage array allocates more physical storage to it, without changing the
presented size.
So for example, a project may require up to a 1TB of storage space over the life of the project.
The actual data that currently exists may be only 100GB. In the standard method of
provisioning, 1TB of space needs to be preallocated. A majority of that space may never be
used and is therefore wasted space.
When using a thin provisioning capable array, a virtual container (virtual volume) is created for
the 1TB. The array then creates/resizes LUNs as actual data is written to the virtual container.
The administrator is not involved after the initial virtual container is created unless the
amount of actual physical storage is used up.
To truly benefit from thin storage, you need the right stack on all hosts:
• A multi-pathing driver that supports the thin hardware
• A file system optimized not to waste storage on thin volumes
• A stack to reclaim space as you migrate to thin storage
• A stack to continually optimize utilization of thin storage
SF unlocks thin provisioning’s full potential with DMP and VxFS which is the only cross-
platform thin storage-friendly file system.

06-25
Displaying information on thin disks
vxdisk list
3pardata0_0034 auto:cdsdisk 3pardata0_0034 thindg online thin
3pardata0_0035 auto:cdsdisk 3pardata0_0035 thindg online thinrclm
vxdisk –e list
3pardata0_0034 auto 3pardata0_0034 thindg online hdisk49 tp std
3pardata0_0035 auto 3pardata0_0035 thindg online hdisk50 tprclm std
vxdisk –o thin,fssize list

DEVICE SIZE(MB) PHYS_ALLOC(MB) FS_SIZE(MB) GROUP TYPE
3pardata0_003a 51200 49 49 thindg thinrclm
3pardata0_0034 51200 16555 6400 thindg thinrclm
3pardata0_0035 51200 32 32 thindg thin
26
SF automatically controls the applicability of features such as SmartMove and thin

reclamation based on known device attributes. If SmartMove is enabled only for thin LUNs
and a device is known to be thin by Storage Foundation, then mirroring operations are
optimized to keep the device thin. If a device is known to be ‘thinrclm’, then SF allows thin
reclamation commands to be issued to it.
SF 5.0 MP3 and later automatically discover thin LUNs and their attributes. If a thin LUN is not
automatically discovered as thin, you can use the following command to manually inform SF
that the LUN is thin or thin reclaim:
vxdisk -g diskgroup set dm_name thin=[on|reclaim]
The vxdisk -e list command prints the extended device attributes (EXT_ATTR) as the
last column to indicate the type of the device.

06-26
To display properties of the devices that support thin provisioning, use the vxdisk -o
thin list command. This command also indicates whether the LUN supports thin
reclamation. Thin reclamation is the process of reclaiming unused storage that is a result of
deleted files and volumes back to the available free pool of the thin provisioning capable
array. Not all thin provisioning arrays support thin reclamation. Use the vxdisk -o
thin,fssize list command to display and compare the physically allocated storage
size to the storage size used by the file system. If there is a big difference between the two
sizes, it is time to initiate a thin reclamation process on the corresponding device.
The vxdisk -p list command displays the discovered properties of the disks including
the attributes related to thin provisioning and thin reclamation.
This tunable is system-wide and persistent, so it only needs to be set once per server. Setting
this tunable parameter to none completely disables the SmartMove feature. You can also use
the vxdefault command to change the value of this tunable parameter. The vxdefault
command is explained in more detail later in this topic.
Note: The Veritas file system must be mounted to get the benefits of the SmartMove feature.
This feature can be used for faster plex creation and faster array migration.

06-27
Introducing SmartMove
Server
Used block
Empty block
VxFS file
system
API
Volume
plex 1 plex 2
(new mirror)
28
This tunable is system-wide and persistent, so it only needs to be set once per server. Setting
this tunable parameter to none completely disables the SmartMove feature. You can also use
the vxdefault command to change the value of this tunable parameter. The vxdefault
command is explained in more detail later in this topic.
Note: The Veritas file system must be mounted to get the benefits of the SmartMove feature.
This feature can be used for faster plex creation and faster array migration.

06-28
Administering thin provisioning parameters
vxdefault list
KEYWORD CURRENT-VALUE DEFAULT-VALUE
autostartvolumes on on
fssmartmovethreshold 100 100
reclaim_on_delete_start_time 22:40 22:10
reclaim_on_delete_wait_period -1 1
same_key_for_alldgs off off
sharedminorstart 33000 33000
usefssmartmove all all
• To modify: vxdefault set tunable value

• Non-default parameters stored in: /etc/default/vxsf
29
The vxdefault command is used to modify and display the tunable parameters that are
stored in the /etc/vx/vxsf file as shown on the slide.
The sharedminorstart tunable parameter is used with the dynamic disk group reminoring
feature. This feature is used to allocate minor numbers dynamically to disk groups based on
their private or shared status. Shared disk groups are used with Cluster Volume Manager and
are not covered in this course.
The fssmartmovethreshold defines a threshold value; only if the file system %usage is less
than this threshold, then the SmartMove feature is used. By default, the
fssmartmovethreshold is set to 100 which means that SmartMove is used with all vxfs file
systems with less than 100% usage.
The autostartvolumes tunable parameter turns on or off automatic volume recovery. If

this parameter is set to on, VxVM automatically recovers and starts disabled volumes when
you import, join, move or split a disk group.

06-29
Migrating to thin provisioning using SmartMove
1 2 3
Verify SmartMove Add thin LUNs to Mirror volumes to
turned on. disk group. thin LUNs.
4 5 6
Remove original If necessary,
Test performance
mirrors and LUNs expand (if larger
of new plexes.
from disk group. thin LUNs).
Storage Provisioning
add-on in VIOM
30
On the slide, the procedure displays the migration from a traditional disk array to a disk array
that supports thin provisioning. This is done assuming that the total space provided by the
thin provisioning array is larger in size than the traditional LUNs used to build the volume and
file system.
Here is an example of an implementation:
1. Turn the SmartMove feature on if necessary.
vxdefault list
vxdefault set usefssmartmove all (if necessary)
2. Add the new, thin LUN, called thinarray0_01 in this example, to the existing disk group.
Note that you can use multiple LUNs although this example is showing only one.
vxdisksetup -i thinarray0_01
vxdg -g appdg adddisk thinarray0_01
3. Add the new, thin LUN as a new plex to the volume.
vxassist -g appdg mirror appvol thinarray0_01

06-30
4. Test the performance of the new LUN.
You can optionally direct all read requests to the plex on the new LUN and then use
benchmarking tools or statistic commands to test performance.
vxvol -g appdg rdpol prefer appvol appvol-02
5. Remove the original mirror and the original LUN.
vxplex -g appdg -o rm dis appvol-01
6. Optionally, grow the file system and the volume to use all of the larger thin LUN.
vxresize -g appdg -x appvol newsize thinarray0_01

06-31
Reclaiming storage with thin provisioning
The disk array contains physically allocated space.
As files are deleted; disk space is freed.
The system administrator

must reclaim disk space.
By VxFS:
fsadm –R [-P] mount_point
Multi-threaded
reclamation
By VxVM:
vxdisk reclaim disk|enclosure|diskgroup
32
Thin provisioning (TP) capable arrays allocate actual physical storage only when the
applications using the LUNs write data. However, when portions of this data is deleted,
storage is not normally reclaimed back to the available free pool of the thin provisioning
capable array.
Storage Foundation uses the VxFS knowledge of used and unused blocks at the file system
level to reclaim that unused space. This process must be manually started by the system
administrator.
Thin reclamation can only be performed on volumes with mounted VxFS file systems.
Volumes without a VxFS file system or volumes that are not currently mounted are not
reclaimed. If the volume consists of a mix of thin-provisioning disks and regular disks, the
reclamation is only performed on the thin-provisioning disks.

Thin reclamation can be triggered on one or more disks, enclosures or disk groups, or at the
file system level on a mounted VxFS file system as displayed on the slide.
When you reclaim at the file system level, the command goes through all the free extents in
the file system and issues the storage level reclaim on the regions which are free. Every time
the command is run, the complete file system is scanned. VxVM is optimized to issue the
reclaim only to the TP LUNs in the file system.
When you reclaim at the VxVM level, the reclaim command goes through the list of all TP
LUN-backed mounted file systems associated to the specified object, and issues the reclaim
on all the file systems. The output displays the list of volumes skipped and the list of volumes
reclaimed.

06-32
Aggressive reclamation
fsadm –R [-o analyze | –A] mount_point
Compacts free space first for effective reclamation
Free chunks before data compaction
Free chunks after data compaction
Space
reclaimed
33
Thin reclamation as implemented in SF 5.0 MP3 (and as described on the previous page) is a
best effort in the sense that it takes any existing contiguous free space in the file system and
reports it to Volume Manager for reclamation. If that contiguous free space is large enough to
be reclaimed in the array (based on chunk size and chunk alignment on the LUN), the space is
effectively reclaimed. Otherwise, the free space is not reclaimed.
The core benefit of this approach is that it either returns storage to the array free pool, or it
does not; the operation never triggers additional storage usage.
The main drawback is that if the free space is fragmented into small contiguous areas, it may
not get reclaimed.
InfoScale Storage has the capability to perform more aggressive reclamation by moving data
around in the file system to maximize the size of the contiguous free space. This is an
additional option for reclamation that can only be triggered at the file system level using the
fsadm -R -A mount_point command. Note that you can use the -o analyze
option first to determine if you should perform a normal reclaim operation or an aggressive
reclaim operation.
Notes:
• Aggressive reclamation can only be performed on file systems that are known to use thin
reclaim capable storage.
• Aggressive reclamation can increase the thin storage usage temporarily during the data
compaction process.

06-33
Automatic reclamation for volumes
• Volume on thin reclaimable LUNs

• Volume deleted or shrunk  Array disk space automatically reclaimed
• Related parameters:
– reclaim_on_delete_wait_period
• Number of days to wait (-1 to 366)
• Default  1 day
– reclaim_on_delete_start_time
• Time of day to start (HH:MM)
• Default  22:10
34
The commands like vxassist remove volume, and vxedit –rf rm volume, and
the volume shrink operation can trigger automatic reclamation if the released storage is on
thin provision reclaimable LUNs.
The reclaim operation is asynchronous, because the delete or shrink operations are quicker.
The reclamation of the storage released due to volume delete or shrink is performed by the
vxrelocd daemon and can be controlled by the following tunable parameters:
• reclaim_on_delete_wait_period=[-1 – 366]
A value of -1 indicates immediate reclamation and a value of 366 indicates that no
reclamation will be performed by the vxrelocd daemon.
• reclaim_on_delete_start_time=[00:00-23:59]
The vxdg destroy diskgroup command does not reclaim any storage automatically.
The thin provision reclaimable LUNs belonging to the destroyed disk group must be reclaimed
manually using the vxdisk reclaim disk command.

06-34
Default VxVM behavior with thin LUNs and SF Enterprise license
appvol
DCO Plex Plex

01 02
DCO Volume
DCO DCO
Log Log
Plex Plex • When a disk group includes thin LUNs, mirrored volumes are automatically
created with data change objects (if using SF Enterprise license).
• Updates to the original volume are recorded in DCO logs and stored on disk.
• Resynchronization involves applying only changed data, rather than
performing an entire atomic resynchronization.
35
The SF Enterprise license enables the FastResync feature of Veritas Volume Manager. The
FastResync feature is used for fast resynchronization of the plexes of a mirrored volume. This
feature is mostly used with instant volume snapshots. However, it is also used for
resynchronization of plexes that become stale with respect to the contents of the volume due
to failures.
Without FastResync, when a plex of a mirrored volume becomes stale, the resynchronization
involves an entire atomic copy from the active plexes to the stale plex. With FastResync,
Volume Manager keeps track of the changed regions of the volume and synchronizes only
those regions. This behavior helps with optimizing thin LUN usage. Therefore, FastResync is
automatically enabled on mirrored volumes if the disk group contains thin LUNs and the
feature is licensed.
When FastResync is enabled on a mirrored volume, a data change object (DCO) is created with
a DCO volume to hold the FastResync maps as well as the DRL recovery maps and other
special maps used with instant snapshot operations on disk.
Note that you cannot remove a mirrored volume using the vxassist remove volume
command if it has an associated DCO log. To remove a mirrored volume with a DCO log, use
the following vxedit command:
vxedit -g diskgroup -rf rm volume_name

06-35
Changes in VxFS Disk Layout Versions (DLV)
Veritas InfoScale 7.4.1 - Linux Veritas InfoScale 7.4.2 - Linux

• The following DLV changes are applicable: • The following DLV changes are applicable:
– Added support for DLV 15 – Added support for DLV 16
– Default DLV is DLV 15 – Default DLV is DLV 16
– Support deprecated for DLV 10 – Support deprecated for DLV 11
• With this change, you can create and mount • With this change, you can create and mount
VxFS only on DLV 11 and later. DLV 6 to 10 VxFS only on DLV 12 and later. DLV 6 to 11 can
can be used for a local mount only. be used for a local mount only.
36
The following DLV changes are applicable in Veritas InfoScale 7.4.1 - Linux
• Support added for DLV 15
• Default DLV is DLV 15
• Support deprecated for DLV 10
With this change, you can create and mount VxFS only on DLV 11 and later. DLV 6 to 10 can be
used for a local mount only.
The following DLV changes are applicable in Veritas InfoScale 7.4.2 - Linux
• Added support for DLV 16
• Default DLV is DLV 16

• Support deprecated for DLV 11
With this change, you can create and mount VxFS only on DLV 12 and later. DLV 6 to 11 can be
used for a local mount only.

06-36
Support for SELinux security extended attributes
• The SELinux policy for RHEL 7.6 and later now includes support for VxFS file system
as persistent storage of SELinux security extended attributes.
• With this support, users can use SELinux security functionalities and features on
VxFS files and directories on RHEL 7.6 and later.
37
The SELinux policy for RHEL 7.6 and later now includes support for VxFS file system as
persistent storage of SELinux security extended attributes.
With this support, users can use SELinux security functionalities and features on VxFS files and
directories on RHEL 7.6 and later.

06-37
Lesson summary
• Key points
– In this lesson, you learned how to apply the appropriate VxFS commands from the command line to
administer the file system.
– You also learned how to perform logging in VxFS by using the intent log and the file change log.
– In addition, you learned how to defragment a Veritas file system.
– Finally, you learned how to use thin provisioning disk arrays with Storage Foundation.
38

06-38
Lab 06: Administering File Systems
• Exercise A: Preparing for “Defragmenting a Veritas File System” exercise

• Exercise B: Defragmenting a Veritas File System
• Exercise C: Using SmartMove
• Exercise D: Observing thin reclamation
39

06-39
What did you learn?
lesson.
40
current lesson.

06-40
Question 1: Defragment a file system command
The command that is used to defragment a file system is:
A. fsadm
B. fsck
C. fcladm
D. mkfs
41

06-41
Answer 1: Defragment a file system command
The command that is used to defragment a file system is:
A. fsadm
B. fsck
C. fcladm
D. mkfs
42

06-42
Question 2: Unmount file systems command
To unmount all file systems except for those required by the operating system, you use
the command:
A. umount -v
B. umount -o
C. umount -p
D. umount -a
43

06-43
Answer 2: Unmount file systems command
To unmount all file systems except for those required by the operating system, you use
the command:
A. umount -v
B. umount -o
C. umount -p
D. umount -a
44

06-44
Question 3: Create a Veritas file system command
A. -o block=4096
B. -o bsize=4096
C. -o blocksize=4096
D. -b bsize=4096
45

06-45
Answer 3: Create a Veritas file system command
To create a Veritas file system that uses a block size of 4096 bytes, you use the mkfs
command option:
A. -o block=4096
B. -o bsize=4096
C. -o blocksize=4096
D. -b bsize=4096
46

06-46
Question 4: Switchout mechanism
The switchout mechanism used on Linux to access VxFS-specific versions of standard
commands is:
A. -o vxfs
B. -t vxfs
C. -o fs
D. -t vfs
47

06-47
Answer 4: Switchout mechanism
The switchout mechanism used on Linux to access VxFS-specific versions of standard
commands is:
A. -o vxfs
B. -t vxfs
C. -o fs
D. -t vfs
48

06-48
Question 5: Display free disk blocks and inodes command
To display the number of free disk blocks and inodes for the Veritas file system on a
device, you use the command:
A. vxfree
B. db
C. freedb
D. df
49

06-49
Answer 5: Display free disk blocks and inodes command
To display the number of free disk blocks and inodes for the Veritas file system on a
device, you use the command:
A. vxfree
B. db
C. freedb
D. df
50

06-50
Appendix
This appendix contains slides that are platform specific and may be
reviewed at the viewer’s discretion and interest. You may opt to end the
presentation now.
51


06-51
AIX: Using VxFS commands
Replaces Journaled File System (JFS) and Enhanced Journaled File System
(JFS2) except for / and /usr
Location of VxFS commands:

/opt/VRTSvxfs/sbin
(optional for PATH variable)
File system type switch: -V vxfs (or -v vxfs when used with crfs)
/usr/lib/fs/vxfs
Related directories:
/etc/fs/vxfs
Default file system file: /etc/vfs
Back
52
Veritas File System can be used in place of Journaled File System (JFS) and Enhanced
Journaled File System (JFS2) on AIX, except for root and /usr.
The location of Veritas File System commands on AIX is displayed on the slide.
The file system switchout for Veritas file system is accomplished using -V vxfs, or -v
vxfs when used with crfs.
The default file system file is /etc/vfs.
To return to the “Using VxFS Commands” slide, click the back button at the bottom of the
screen.

06-52
HP-UX: Using VxFS commands
Replaces Hierarchical File System (HFS) except for /stand.
Location of VxFS
commands:
/sbin/fs
(optional for PATH
variable)
File system type switch: -F vxfs
Default file system file: /etc/default/fs
Back
53
Veritas File System can be used in place of Hierarchical File System (HFS) on HP-UX, except for
/stand.
The location of Veritas File System commands on HP-UX is displayed on the slide.
The file system switchout for Veritas file system is accomplished using -F vxfs.
The default file system file is /etc/default/fs.
screen.

06-53
Linux: Using VxFS commands
Replaces Extended File System Version 2 (EXT2) and Version 3 (EXT3)
except for /, /boot, /etc, /lib, /var, and /usr

/usr/lib/fs/vxfs
File system type switch: -t vxfs
Related directories: /sbin/command.vxfs
Back
54
Veritas File System can be used in place of Extended File System Version 2 (EXT2) and Version
3 (EXT3) on Linux, except for the root, /boot, /etc, /lib, /var, and /usr directories.
The location of Veritas File System commands on Linux is displayed on the slide.
The file system switchout for Veritas file system is accomplished using -t vxfs.
The default file system file is /etc/default/fs.
screen.

06-54
Solaris: Using VxFS commands
Replaces UNIX File System (UFS) except for /, /usr, /var, and /opt

/opt/VRTSvxfs/sbin
File system type switch: -F vxfs
/usr/lib/fs/vxfs/bin
Related directories:
/etc/fs/vxfs
Back
55
Veritas File System can be used in place of UNIX File System (UFS) on Solaris, except for root,
/usr, /var, and /opt. The location of Veritas File System commands on Solaris is
displayed on the slide.
The file system switchout for Veritas file system is accomplished using -F vxfs. The default
file system file is /etc/default/fs.
screen.

06-55
End of presentation
06-56
Lesson 01: High Availability Concepts
This is the High Availability Concepts lesson in the Veritas InfoScale 7.4.2 Fundamentals for

01-1
Administration
Administration
• Lesson 01: High Availability Concepts • Lesson 11: Cluster Communications


01-2
Lesson objectives
Topic Objective
High availability concepts Define high availability.
Clustering concepts Recall clustering terminology.
Explain how applications are managed in a high availability

High availability application services
environment.
Clustering prerequisites List the key requirements for a clustering environment.

01-3
Topic: High availability concepts
After completing this topic, you will be able to define high availability.
This is the High availability concepts topic.


01-4
Range of availability levels
Business continuance
WAN clustering
Global
Cluster
LAN clustering
Synchronous Cluster Server

replication
Asynchronous
replication Volume Replicator
Shared storage
AVAILABILITY
Block level Cluster File System & Volume Manager

Incremental backups
Hot & cold

backups NetBackup
Journaled
file system Storage Foundation
Low level SLA Medium level SLA High level SLA
INVESTMENT
Data centers may implement different levels of availability depending on their requirements
for availability.
• Backup: At a minimum, all data needs to be protected using an effective backup solution,
such as Veritas NetBackup.
• Data availability: Local mirroring provides real-time data availability within the local data
center. Point-in-time copy solutions protect against corruption. Online configuration keeps
data available to applications while storage is reconfigured to meet changing IT and
business needs. DMP provides resilience against path failure.
• Shared disk groups and cluster file systems: These features minimize application failover
time because the disk groups, volumes, and file systems are available on multiple systems
simultaneously.
• Local clustering: The next level is an application clustering solution, such as Veritas Cluster
Server, for application and server availability.
• Remote replication: After implementing local availability, you can further ensure data
availability in the event of a site failure by replicating data to a remote site. Replication can
be application-, host-, or array-based.
• Remote clustering: Implementing remote clustering ensures that the applications and data
can be started at a remote site. Veritas Cluster Server supports remote clustering with
automatic site failover capability.

01-5
Costs of downtime
Actual unplanned downtime per month:

• Hours: 9
• Cost per hour: $336k to $540k
• Total cost: $3,024,000 to $4,860,000
Goal for monthly unplanned downtime:

• Hours: 3
• Cost savings: $2,016,000 to $3,240,000
A Gartner study shows that large companies experienced a loss of between $3,024,000 and
$4,860,000 (USD) per month for nine hours of unplanned downtime.
In addition to the monetary loss, downtime also results in loss of business opportunities and
reputation. Planned downtime is almost as costly as unplanned. Planned downtime can be
significantly reduced by migrating a service to another server while maintenance is
performed. Given the magnitude of the cost of downtime, the case for implementing a high
availability solution is clear.

01-6
Topic: Clustering concepts
After completing this topic, you will be able to recall clustering terminology.
This is the Clustering concepts topic.


01-7
Types of clusters
Cluster is a broadly-used term:

• High availability (HA) clusters
• Parallel processing clusters
• Load balancing clusters
• High performance computing clusters
• Fault tolerant clusters
VCS is primarily an HA cluster with support for:

• Parallel processing applications, such as Oracle RAC
• Application workload balancing
The term cluster refers to multiple independent systems connected into a management
framework.
Types of clusters
A variety of clustering solutions are available for various computing purposes.
• HA clusters: Provide resource monitoring and automatic startup and failover
• Parallel processing clusters: Break large computational programs into smaller tasks
executed in parallel on multiple systems
• Load balancing clusters: Monitor system load and distribute applications automatically
among systems according to specified criteria
• High performance computing clusters: Use a collection of computing resources to

enhance application performance
• Fault-tolerant clusters: Provide uninterrupted application availability
Fault tolerance guarantees 99.9999 percent availability, or approximately 30 seconds of
downtime per year. Six 9s (99.9999 percent) availability is appealing, but the costs of this
solution are well beyond the affordability of most companies. In contrast, high availability
solutions can achieve five 9s (99.999 percent availability—less than five minutes of downtime
per year) at a fraction of the cost.
The focus of this course is the Veritas Cluster Server, which is primarily used for high
availability, although it also provides some support for parallel processing and load balancing.

01-8
Local cluster configurations
Active/Passive or Active/Active N-to-1, N + 1, N-to-N
A/A
A/P
Utilization: Utilization:
Link to examples
Depending on your clustering solution, you may be able to implement a variety of

configurations, enabling you to deploy the clustering solution to best suit your HA
requirements and utilize existing hardware.
• Active/Passive—In this configuration, an application runs on a primary or master server
and a dedicated redundant server is present to take over on any failover.
• Active/Active—In this configuration, each server is configured to run specific applications
or services, and essentially provides redundancy for its peer.
• N-to-1—In this configuration, the applications fail over to the spare when a system
crashes. When the server is repaired, applications must be moved back to their original
systems.
• N + 1—Similar to N-to-1, the applications restart on the spare after a failure. Unlike the N-
to-1 configuration, after the failed server is repaired, it can become the redundant server.
• N-to-N—This configuration is an active/active configuration that supports multiple
application services running on multiple servers. Each application service is capable of
being failed over to different servers in the cluster.
In the example displayed on the slide, utilization is increased by reconfiguring four
active/passive clusters and one active/active cluster into one N-to-1 cluster and one N-to-N
cluster respectively. This enables a saving of four systems.
Click the link to view some examples of cluster configurations.

01-9
Campus and global cluster configurations
Campus cluster DR cluster
Replication
VVR IBM HDS

EMC NetApp HP
Oracle
10
Cluster configurations that enable data to be duplicated among multiple physical locations
protect against site-wide failures.
Campus clusters
The campus or stretch cluster environment is a single cluster stretched over multiple
locations, connected by an Ethernet subnet for the cluster interconnect and a fiber channel
SAN, with storage mirrored at each location.
Advantages of this configuration are as follows:
• It provides local high availability within each site as well as protection against site failure.
• It is a cost-effective solution; replication is not required.
• Recovery time is short.

• The data center can be expanded.
• You can leverage existing infrastructure.

01-10
Global cluster configurations
Campus cluster DR cluster
Replication
VVR IBM HDS

EMC NetApp HP
Oracle
11
Global clusters, or wide-area clusters, contain multiple clusters in different geographical

locations. Global clusters protect against site failures by providing data replication and
application failover to remote data centers.
Global clusters are not limited by distance because cluster communication uses TCP/IP.
Replication can be provided by hardware vendors or by a software solution, such as Veritas
Volume Replicator, for heterogeneous array support.

01-11
Topic: High availability applications
After completing this topic, you will be able to explain how applications are
managed in a high availability environment.
12
This is the High availability applications topic.


01-12
HA application services
• Collection of all hardware and software components required to provide a service

• All software components moved together
• Components started, stopped in order
• Examples: Web servers, databases, and applications
Application • Application requires:

− Database
DB IP
− IP address
FS FS • Database requires file systems

NIC • File systems require volumes
Vol Vol • Volumes require disk groups
DG
13
An application service is a collection of hardware and software components required to

provide a service, such as a Web site, that an end-user can access by connecting to a
particular network IP address or host name. Each application service typically requires
components of the following three types:
• Application binaries (executables)
• Network
• Storage
If an application service needs to be switched to another system, all of the components of the
application service must migrate together to re-create the service on another system. These
are the same components that the administrator must manually move from a failed server to
a working server to keep the service available to clients in a non-clustered environment.
Application service examples include:
• A Web service consisting of a Web server program, IP addresses, associated network
interfaces used to allow access into the Web site, a file system containing Web data files,
and a volume and disk group containing the file system.
• A database service may consist of one or more IP addresses, database management
software, a file system containing data files, a volume and disk group on which the file
system resides, and a NIC for network access.

01-13
Local application service failover
Clients
Servers Application
Application
DB IP
DB IP
FS FS
FS FS NIC
NIC
Vol Vol
Vol Vol
DG
DG
14
Cluster management software performs a series of tasks in order for clients to access a
service on another server in the event a failure occurs. The software must:
• Ensure that data stored on the disk is available to the new server, if shared storage is
configured (Storage).
• Move the IP address of the old server to the new server (Network).
• Start up the application on the new server (Application).
The process of stopping the application services on one system and starting it on another
system in response to a fault is referred to as a failover.

01-14
Local and global failover
Site migration
Application
Application
DB IP
DB IP
FS FS
NIC
FS FS
NIC Vol Vol
Vol Vol
DG
DG
Replication
15
In a global cluster environment, the application services are generally highly available within a
local cluster, so faults are first handled by the HA software, which performs a local failover.
When HA methods such as replication and clustering are implemented across geographical
locations, recovery procedures are started immediately at a remote location when a disaster
takes down a site.

01-15
Application requirements for clustering
Function Requirement
Start/restart Restarted to a known state after a failure
Stop Stopped using a defined procedure
Clean Cleaned up after operational failures
Monitor Monitored periodically
Not tied to a particular host due to licensing constraints or

Node-independent host name dependencies
16
The most important requirements for an application to run in a cluster are crash tolerance and
host independence. This means that the application should be able to recover after a crash to
a known state, in a predictable and reasonable time, on two or more hosts.
Most commercial applications today satisfy this requirement. More specifically, an application
is considered well-behaved and can be controlled by clustering software if it meets the
requirements shown in the slide.

01-16
Topic: Clustering prerequisites
After completing this topic, you will be able to list the key requirements for
a clustering environment.
17
This is the Clustering prerequisites topic.


01-17
Hardware and infrastructure redundancy
Multiple cluster interconnect links
Multiple LAN
components and links
Multiple storage
components and links
18
All failovers cause some type of client disruption. Depending on your configuration, some
applications take longer to fail over than others. For this reason, good design dictates that the
HA software first try to fail over within the system, using agents that monitor local resources.
Design as much resiliency as possible into the individual servers and components so that you
do not have to rely on any hardware or software to cover a poorly configured system or
application. Likewise, try to use all resources to make individual servers as reliable as possible.
Single point of failure analysis
Determine whether any single points of failure exist in the hardware, software, and
infrastructure components within the cluster environment. Any single point of failure
becomes the weakest link of the cluster. The application is equally inaccessible if a client
network connection fails, or if a server fails. In addition, consider the location of redundant
components. Having redundant hardware equipment in the same location is not as effective
as placing the redundant component in a separate location.
In some cases, the cost of redundant components outweighs the risk that the component will
become the cause of an outage. For example, buying an additional expensive storage array
may not be practical. Decisions about balancing cost versus availability need to be made
according to your availability requirements.

01-18
External dependencies
NIS master Primary DNS
Secondary DNS
NIS slave
• Avoid dependence on services outside the cluster, where possible.

• Ensure redundancy of external services required by cluster applications.
19
Whenever possible, it is good practice to eliminate or reduce reliance by high availability

applications on external services. If it is not possible to avoid outside dependencies, ensure
that those services are also highly available. For example, network name and information
services, such as DNS (Domain Name System) and NIS (Network Information Service), are
designed with redundant capabilities.

01-19
Lesson summary
• Key points
– In this lesson, you learned about high availability and clustering concepts.
– In addition, you learned how to manage applications in a high availability environment.
– Finally, you learned about the key requirements for a clustering environment.
– Veritas Cluster Server Administrator’s Guide
20

01-20
Lab 02: Installing Veritas InfoScale Enterprise for using SFHA
• Exercise A: Installing InfoScale Enterprise using the Common Product Installer (CPI)]
• Exercise B: Running a post-installation check
• Exercise C: Adding cluster systems to VIOM as managed hosts
21

01-21
What did you learn?
lesson.
22
current lesson.

01-22
Question 1: Clustering concepts
Which statement about N-to-1 clustering is true?
A. Applications can fail over to any other system if a system faults.

B. The spare system rotates among all cluster nodes.
C. Applications must be moved back after the failed system is repaired.
D. The spare system is only capable of running one failed over application.
23

01-23
Answer 1: Clustering concepts
Which statement about N-to-1 clustering is true?
A. Applications can fail over to any other system if a system faults.

B. The spare system rotates among all cluster nodes.
C. Applications must be moved back after the failed system is repaired.
D. The spare system is only capable of running one failed over application.
The correct answer is C. In N-to-1 cluster there is one failover target node for all the applications running in the
cluster and once the failed system is repaired the corresponding service group should be moved back.
24

01-24
Question 2: High availability applications
What is the minimum number of systems required to implement an active/passive
failover configuration in a local cluster?
A. One
B. Two
C. Three
D. Four
25

01-25
Answer 2: High availability applications
What is the minimum number of systems required to implement an active/passive
failover configuration in a local cluster?
A. One
B. Two
C. Three
D. Four
The correct answer is B. For an active/passive local cluster, minimum number of system required is 2.
26

01-26
A global cluster is:
A. A single cluster stretched across a wide-area network.

B. Multiple cooperating clusters geographically separated.
C. A combination of campus clusters and local clusters.
D. Two local, but separate, clusters that share data.
27

01-27
A global cluster is:
A. A single cluster stretched across a wide-area network.

B. Multiple cooperating clusters geographically separated.
C. A combination of campus clusters and local clusters.
D. Two local, but separate, clusters that share data.
The correct answer is B. Global clusters contains multiple clusters in different geographical locations.
28

01-28
Question 4: Clustering prerequisites
The goal of hardware redundancy is to:
A. Remove single points of failure.

B. Prevent failures.
C. Guarantee 100% application uptime.
D. Double bandwidth.
29

01-29
Answer 4: Clustering prerequisites
The goal of hardware redundancy is to:
A. Remove single points of failure.

B. Prevent failures.
C. Guarantee 100% application uptime.
D. Double bandwidth.
The correct answer is A. In clustering goal is to avoid any single point of failures whether hardware or software. We
achieve the same through hardware redundancy in case of hardware.
30

01-30
To cluster an application, you must have:
A. Primary and secondary DNS name servers.

B. Multiple network interfaces and IP addresses.
C. A database for managing shared storage.
D. Appropriate licensing for all failover target systems.
31

01-31
To cluster an application, you must have:
A. Primary and secondary DNS name servers.

B. Multiple network interfaces and IP addresses.
C. A database for managing shared storage.
D. Appropriate licensing for all failover target systems.
The correct answer is D. You must be licensed to use the software components of an application as well as
clustering software on all the nodes.
32

01-32
Appendix
presentation now.
33


01-33
Active/passive
Failed
Before failover After failover
34

01-34
Active/passive N-to-1
Passive f/o
node
After failover
Before failover Failed
After repair
Passive f/o
node
35

01-35
Active/passive N + 1
F/O
node
After failover
Before failover Failed
After repair
F/O
node
36

01-36
Active/active
Failed
Before failover After failover
37

01-37
N-to-N
A D G J
B E H K
C F I L
Before failover
After failover
Failed
D G J
A B C
E H K
F I L
38

01-38
End of presentation
01-39
Lesson 02: VCS Building Blocks
This is the VCS Building Blocks lesson in the Veritas InfoScale 7.4.2 Fundamentals for

02-1
Administration
Administration


02-2
Lesson objectives
Topic Objective
VCS terminology Define VCS terminology.
Cluster communication Outline the different VCS cluster communication mechanisms.
VCS architecture Summarize the implementation of high availability in the VCS architecture.
Multi-version cluster Explain the InfoScale support for multi-version clusters.

02-3
Topic: VCS terminology
After completing this topic, you will be able to define VCS terminology.
This is the VCS terminology topic.


02-4
VCS clusters
A cluster is a collection of systems working together for increased service availability.
Consisting of:
• Up to 128 systems (nodes).
• An interconnect for cluster communication.
• A public network for client connections.
• Shared storage accessible by each system.
Shared storage
Online service
Offline service
Cluster Interconnect
A VCS cluster is a collection of independent systems working together under the VCS
management framework for increased service availability.
VCS clusters have the following components:
• Up to 128 systems—sometimes referred to as nodes or servers—each running its own
operating system.
• A cluster interconnect, which enables cluster communications.
• A public network, connecting each system in the cluster to a LAN for client access.
• Shared storage (optional), accessible by each system in the cluster that needs to run the
application.

02-5
Service groups
A service group is a container that enables VCS to manage an

webapache application service as a unit.
webip webmnt
webnic webvol Defined by:

• Resources: Components required to provide the
service.
webdg
• Dependencies: Relationships between components.
• Attributes: Specified values used to define startup
websg and failure behavior.
A service group is a virtual container that enables VCS to manage an application service as a
unit. The service group contains all the hardware and software components required to run
the service. The service group enables VCS to coordinate failover of the application service
resources in the event of failure or at the administrator’s request.
A service group is defined by these attributes:
• The cluster-wide unique name of the group
• The list of the resources in the service group, usually determined by which resources are
needed to run a specific application service
• The dependency relationships between the resources
• The list of cluster systems on which the group is allowed to run

• The list of cluster systems on which you want the group to start automatically

02-6
Service group types
Failover
• Online on only one cluster system at a time.
• Referred to as active/passive.
• Most common type.
Parallel
• Online on multiple cluster systems simultaneously.
• Referred to as active/active.
• Example: Oracle Real Application Cluster (RAC).
Service groups types include

• Failover: This service group runs on one system at a time in the cluster. Most application
services, such as database and NFS servers, use this type of group.
• Parallel: This service group runs simultaneously on more than one system in the cluster.
This type of service group requires an application that can be started on more than one
system at a time without threat of data corruption.

02-7
Resources
VCS resources correspond to the hardware or software components
webapache of an application service.
webip webmnt
Resources:
• Have unique names throughout the cluster .
webnic webvol
• Are always contained within service groups.
• Are categorized as:
webdg – Persistent: Never turned off.
– Nonpersistent: Turned on and off.
websg
Recommendations:
• Match resource and service group names to easily identify all resources in a group.
• Use a naming convention that identifies the service and type of resource.
Resources are VCS objects that correspond to hardware or software components, such as the
application, the networking components, and the storage components.
VCS controls resources through these actions:
• Bringing a resource online (starting)
• Taking a resource offline (stopping)
• Monitoring a resource (probing)
Resource categories
• Persistent, never turned off
• None
VCS can only monitor persistent resources—these resources cannot be brought online
or taken offline. The most common example of a persistent resource is a network
interface card (NIC), because it must be present but cannot be stopped.
• On-only
VCS brings the resource online if required but does not stop the resource if the
associated service group is taken offline. ProcessOnOnly is a resource used to start,
but not stop a process such as daemon, for example.
• Nonpersistent, also known as on-off
Most resources fall into this category, meaning that VCS brings them online and takes
them offline as required. Examples are Mount, IP, and Process.

02-8
Resources dependencies
Resource dependencies determine online and offline order of resources.
webapache
Parent
Dependencies:
• Determine parent/child relationships.
• Are defined to be parent dependent on child.
webip • Cannot be cyclical.
Parent/child
webnic
Child Persistent resources, such as NIC, cannot be parents.

Online Offline
order order
Resources depend on other resources because of application or operating system

requirements. Dependencies are defined to configure VCS for these requirements.
Dependency rules
These rules apply to resource dependencies:
• A parent resource depends on a child resource. In the diagram, the Mount resource
(parent) depends on the Volume resource (child). This dependency illustrates the
operating system requirement that a file system cannot be mounted without the Volume
resource being available.
• Dependencies are homogenous. Resources can only depend on other resources.
• No cyclical dependencies are allowed. There must be a clearly defined starting point.

02-9
Resources attributes
webmnt Attributes define an individual resource.
Attributes:
• Specify values used by VCS to manage the resource.
• Can be required or optional, as specified by the resource
type definition.
Online webmnt
–F vxfs /dev/vx/dsk/webdatadg/webdatavol /webdata

–t
mount –V
10
Resources attributes define the specific characteristics on individual resources. As shown in

the slide, the resource attribute values for the sample resource of type Mount correspond to
the UNIX command line to mount a specific file system. VCS uses the attribute values to run
the appropriate command or system call to perform an operation on the resource. Each
resource has a set of required attributes that must be defined in order to enable VCS to
manage the resource.
For example, the Mount resource has four required attributes that must be defined for each
resource of type Mount:
• The directory of the mount point (MountPoint)
• The device for the mount point (BlockDevice)
• The type of file system (FSType)

• The options for the fsck command (FsckOpt)
The first three attributes are the values used to build the UNIX mount command shown on
the slide. The FsckOpt attribute is used if the mount command fails. In this case, VCS runs
fsck with the specified options (-y, which means answer yes to all fsck questions) and
attempts to mount the file system again.
Some resources also have additional optional attributes you can define to control how VCS
manages a resource. In the Mount resource example, MountOpt is an optional attribute you
can use to define options to the UNIX mount command. For example, if this is a read-only file
system, you can specify -ro as the MountOpt value.

02-10
Resource types and type attributes
Resource types define characteristics of resources.
Types:
• Specify the attributes needed to define a
resource.
• Are used to create a resource.
• Are similar to source code for a resource.
mount
mount [-t fstype]
mount [-V
[-F fstype] [options]
fstype] [options] block_device
[options] block_device mount_point
block_device mount_point
mount_point
11
Resources are classified by resource type. For example, disk groups, network interface cards
(NICs), IP addresses, mount points, and databases are distinct types of resources. VCS
provides a set of predefined resource types—some bundled, some add-ons—in addition to
the ability to create new resource types.
Individual resources are instances of a resource type. For example, you may have several IP
addresses under VCS control. Each of these IP addresses is, individually, a single resource of
resource type IP. A resource type can be thought of as a template that defines the
characteristics or attributes needed to define an individual resource (instance) of that type.
You can view the relationship between resources and resource types by comparing the mount
command for a resource on the previous slide with the mount syntax on this slide. The
resource type defines the syntax for the mount command. The resource attributes fill in the
values to form an actual command line.

02-11
Agents
online VCS agents manage resources.

offline
monitor
clean Each agent:
• Corresponds to a resource type.
• Manages all resources of that type.
10.1.2.3 appvol • Runs on each system with an active resource.
webvol • Has one or more entry points which perform set
OS- actions on resources.
specific
webdg
device
IP Vol
NIC DG
12
Agents are processes that control resources. Each resource type has a corresponding agent
that manages all resources of that resource type. Each cluster system runs only one agent
process for each active resource type, no matter how many individual resources of that type
are in use.
Agents control resources using a defined set of actions, also called entry points. The four entry
points common to most agents are:
• Online: Resource startup
• Offline: Resource shutdown
• Monitor: Probing the resource to retrieve status
• Clean: Killing the resource or cleaning up as necessary when a resource fails to be taken
offline gracefully
Each entry point has corresponding scripts or binaries that specify how to perform each of the
four actions. For example, the startup entry point of the Mount agent mounts a block device
on a directory, whereas the startup entry point of the IP agent uses the ifconfig (Solaris,
AIX, HP-UX) or ip addr add (Linux) command to set the IP address on a unique IP alias on
the network interface.
The difference between offline and clean is that offline is an orderly termination and clean is a
forced termination. In UNIX, this can be thought of as the difference between exiting an
application and sending the kill -9 command to the process.
VCS provides both predefined agents and the ability to create custom agents.

02-12
Veritas Cluster Server Bundled Agents Reference Guide
• Defines all VCS resource types for all bundled agents.

• Includes agents for each supported UNIX platform.
• Is downloadable from the Support Web site.
13
The Veritas Cluster Server Bundled Agents Reference Guide describes the agents that are
provided with VCS and defines the required and optional attributes for each associated
resource type. Veritas also provides additional application and database agents in an Agent
Pack that is updated quarterly. Some examples of these agents are Data Loss Prevention,
Documentum, and IBM DB2 Database.
Select Downloads > High Availability Agents at the Support Web site for a complete list of
agents available for VCS.
Note: The Veritas Cluster Administrator’s Guide provides an appendix with a complete
description of attributes for all cluster objects.
To obtain PDF versions of product documentation for VCS and agents, visit the SORT Web site.

02-13
Topic: Clustering communication
After completing this topic, you will be able to outline the different VCS
cluster communication mechanisms.
14
This is the Cluster communication topic.


02-14
Cluster communication
The cluster interconnect provides a communication channel between nodes.
The interconnect:
• Determines affiliated nodes by cluster ID.
• Uses a heartbeat mechanism.
• Maintains a single view of the cluster membership.
• Is also referred to as the private network.
15
VCS requires a cluster communication channel between systems in a cluster to serve as the
cluster interconnect. This communication channel is also sometimes referred to as the private
network because it is often implemented using a dedicated Ethernet network.
Veritas recommends that you use a minimum of two dedicated communication channels with
separate infrastructures—for example, multiple NICs and separate network hubs—to
implement a highly available cluster interconnect.
The cluster interconnect has two primary purposes:
• Determine cluster membership: Membership in a cluster is determined by systems
sending and receiving heartbeats (signals) on the cluster interconnect. This enables VCS to
determine which systems are active members of the cluster and which systems are joining
or leaving the cluster. In order to take corrective action on node failure, surviving members
must agree when a node has departed. This membership needs to be accurate and
coordinated among active members—nodes can be rebooted, powered off, faulted, and
added to the cluster at any time.
• Maintain a distributed configuration: Cluster configuration and status information for
every resource and service group in the cluster is distributed dynamically to all cluster
systems.
Cluster communication is handled by the Group Membership Services/Atomic Broadcast
(GAB) mechanism and the Low Latency Transport (LLT) protocol, as described in the next
sections.

02-15
Low-Latency Transport (LLT)
LLT is a high-performance, low-latency protocol for cluster communication.
LLT:
• Sends heartbeat messages.
• Transports cluster communication traffic.
LLT • Balances traffic load across multiple network links.
• Runs on an Ethernet network.
LLT
16
Clustering technologies from Veritas use a high-performance, low-latency protocol for

communications. LLT is designed for the high-bandwidth and low-latency needs of not only
Veritas Cluster Server, but also Veritas Cluster File System and Veritas Storage Foundation for
Oracle RAC.
LLT runs directly on top of the Data Link Provider Interface (DLPI) layer over Ethernet and has
several major functions:
• Sending and receiving heartbeats over all network links
• Monitoring network traffic over multiple network links to every active system
• Balancing the cluster communication load over multiple links
• Maintaining the state of communication

• Providing a transport mechanism for cluster communications

02-16
Group Membership Services/Atomic Broadcast (GAB)
GAB is a broadcast protocol that provides cluster membership services.
GAB:
GAB • Manages cluster membership.
• Sends and receives configuration information.
• Uses LLT as the transport mechanism.
GAB LLT
LLT
17
GAB provides the following:

• Group Membership Services: GAB maintains the overall cluster membership by way of its
group membership services function. Cluster membership is determined by tracking the
heartbeat messages sent and received by LLT on all systems in the cluster over the cluster
interconnect.
GAB messages determine whether a system is an active member of the cluster, joining
the cluster, or leaving the cluster. If a system stops sending heartbeats, GAB determines
that the system has departed the cluster.
• Atomic Broadcast: Cluster configuration and status information are distributed
dynamically to all systems in the cluster using GAB’s atomic broadcast feature. Atomic
broadcast ensures that all active systems receive all messages for every resource and
service group in the cluster.

02-17
I/O fencing
I/O fencing is a mechanism to prevent uncoordinated access to shared storage.
GAB
GAB GAB I/O fencing:

• Monitors GAB for cluster membership changes.
• Protects data:
− Prevents simultaneous access to shared storage.
− Known as fencing off nodes.
GAB LLT
LLT

18
The fencing driver implements I/O fencing, which prevents multiple systems from accessing
the same Volume Manager-controlled shared storage devices in the event that the cluster
interconnect is severed. In the example of a two-node cluster displayed in the diagram, if the
cluster interconnect fails, each system stops receiving heartbeats from the other system.
GAB on each system determines that the other system has failed and passes the cluster
membership change to the fencing module. The fencing modules on both systems contend for
control of the disks according to an internal algorithm. The losing system is forced to panic
and reboot.
The winning system is now the only member of the cluster, and it fences off the shared data
disks so that only systems that are still part of the cluster membership (only one system in this
example) can access the shared storage. The winning system takes corrective action as
specified within the cluster configuration, such as bringing service groups online that were
previously running on the losing system.

02-18
High Availability Daemon (HAD)
HAD is the VCS engine, which manages agents

Agents and tracks all configuration and state changes.
HAD
hashadow
HAD:
vxfen • Manages agents and service groups.
• Maintains resource configuration and state information.
• Is monitored by the hashadow daemon.
GAB
LLT
19
The VCS engine, also referred to as the high availability daemon (HAD), is the primary VCS
process running on each cluster system. HAD tracks all changes in cluster configuration and
resource status by communicating with GAB. HAD manages all application services (by way of
agents) whether the cluster has one or many systems. Building on the knowledge that the
agents manage individual resources, you can think of HAD as the manager of the agents. HAD
uses the agents to monitor the status of all resources on all nodes.
This modularity between HAD and the agents allows for efficiency of roles:
• HAD does not need to know how to start up Oracle or any other applications that can
come under VCS control.
• Similarly, the agents do not need to make cluster-wide decisions.
This modularity allows a new application to come under VCS control simply by adding a new
agent—no changes to the VCS engine are required. On each active cluster system, HAD
updates all the other cluster systems with changes to the configuration or status.
In order to ensure that the had daemon is highly available, a companion daemon,
hashadow, monitors had, and if had fails, hashadow attempts to restart had. Likewise,
had restarts hashadow if hashadow stops.

02-19
Dualstack IPv4 and IPv6 support
The InfoScale 7.4.2 release introduces the Dualstack IPv4 and IPv6 support for InfoScale Availability/Enterprise.
20
The InfoScale 7.4.2 release introduces the Dualstack IPv4 and IPv6 support for InfoScale
Availability/Enterprise.

02-20
Topic: VCS architecture
After completing this topic, you will be able to summarize the

implementation of high availability in the VCS architecture.
21
This is the VCS architecture topic.


02-21
Maintaining the cluster configuration
• The cluster configuration is in memory on

each node.
• Configuration changes are broadcast by
HAD to all systems.
• The configuration is preserved on disk
HAD (main.cf).
HAD
vxfen
vxfen
main.cf GAB
main.cf
GAB
LLT
LLT
22
HAD maintains configuration and state information for all cluster resources in memory on
each cluster system. Cluster state refers to tracking the status of all resources and service
groups in the cluster. When any change to the cluster configuration occurs, such as the
addition of a resource to a service group, HAD on the initiating system sends a message to
HAD on each member of the cluster by way of GAB atomic broadcast, to ensure that each
system has an identical view of the cluster. Atomic means that all systems receive updates, or
all systems are rolled back to the previous state, much like a database atomic commit.
The cluster configuration in memory is created from the main.cf file on disk in the case
where HAD is not currently running on any cluster systems, so there is no configuration in
memory. When you start VCS on the first cluster system, HAD builds the configuration in
memory on that system from the main.cf file. Changes to a running configuration (in
memory) are saved to disk in main.cf when certain operations occur. These procedures are
described in more detail later in the course.

02-22
VCS configuration files
/etc/VRTSvcs/conf/config/main.cf
include "types.cf"
cluster webvcs (
UserNames = { admin = ElmElgLimHmmKumGlj }
Administrators = { admin }
CounterInterval = 5
)
system s1 (
)
system s2 (
) Cluster configuration stored in text file on disk.
group websg(
SystemList = { s1 = 0, s2 = 1 }
AutoStartList = { s1, s2 }
)
Mount webmnt (
MountPoint = "/webdata"
BlockDevice = "/dev/vx/dsk/webdatadg/webdatavol"
FSType = vxfs
FsckOpt = "-y"
)
23
Configuring VCS means conveying to VCS the definitions of the cluster, service groups,
resources, and resource dependencies. VCS uses two configuration files in a default
configuration:
• The main.cf file defines the entire cluster, including the cluster name, systems in the
cluster, and definitions of service groups and resources, in addition to service group and
resource dependencies.
• The types.cf file defines the resource types.
Additional files similar to types.cf may be present if agents have been added. For
example, if the Oracle enterprise agent is added, a resource types file, such as
OracleTypes.cf, is also present. The cluster configuration is saved on disk in the
/etc/VRTSvcs/conf/config directory, so the memory configuration can be re-

created after systems are restarted.

02-23
Topic: Multi-version cluster
After completing this topic, you will be able to explain the InfoScale support
for multi-version clusters.
24
This is the Multi-version cluster topic.


02-24
Support for multi-version clusters
N-nodes Cluster N-nodes Cluster
v.7.4.1 v.7.4 .2
Prior to the v.7.4.1 v.7.4 .3

InfoScale 7.4.1 From the
Downtime &
release v.7.4.1 No Downtime v.7.5.x InfoScale 7.4.2
(Previous Complexity release
releases)
v.7.4.1 v.7.5 .1
v.7.4.1 v.7.x.y
• All the nodes need to have the same HAD version. • Nodes in a cluster can have different HAD versions.
• Upgrading can be done only via a Rolling or a Full • Nodes can be upgraded in an individual manner.
upgrade. • Nodes can be added with newer HAD versions.
• If nodes are to be added to a cluster, they should have • Downgrading of nodes is allowed.
the same HAD version.
25
In earlier InfoScale versions, you needed to keep the same HAD version across all the nodes in
a cluster. To upgrade all the cluster nodes, you needed to perform either a rolling upgrade or a
full upgrade. This increases downtime and complexity in the functioning of the cluster. If a
new node is to be added to the cluster, you needed to have the same HAD version as the
other nodes in the cluster.
In the InfoScale 7.4.2 version, you can have nodes with different HAD versions. To upgrade
cluster nodes, you can upgrade nodes individually. You can also add nodes with newer HAD
versions. This ensures that there is not downtime in the functioning of the cluster. In addition,
downgrading which is the replacement of existing nodes with systems with older HAD
versions is also possible.

02-25
Requirements for the Multi-version cluster support
 The multi-version cluster feature is not backward compatible. Support for the multi-
version cluster feature is applicable only from InfoScale 7.4.2.
 All the nodes in a cluster need to have the same Cluster Protocol Number (CPN).
 In future releases, new features or agents will be enabled only if the CPN matches with
the minimum CPN required for those features or agents.
26
Let us discuss some requirements and limitations of the multi-version cluster feature. The
multi-version cluster feature is not backward compatible. It is applicable from InfoScale 7.4.2.
All the nodes in a cluster need to have the same Cluster Protocol Number (CPN). In future
releases, new features or agents will be enabled only if the CPN matches with the minimum
CPN required for those features or agents.

02-26
Benefits of the Multi-version cluster feature
Scalability Upgrade
• Nodes can be upgraded one at the time.

• Nodes can be upgraded by adding a new
While adding new nodes, you can use the latest
node with a newer version and removing the
release, independently from the version of the
older node with the previous version.
existing cluster.
• Nodes can be upgraded with zero downtime
and with no impact on the level of availability.
27
The multi-version cluster feature allows for a scalable solution. While adding new nodes, you
can use the latest release, independently from the version of the existing cluster.
In addition, upgradation is another benefit of this feature. In the latest InfoScale release,
nodes can be upgraded in an individual manner. Nodes can be upgraded by adding a new
node with a newer version and then removing an old node with the previous version. If the
upgradation process is repeated for all nodes in a cluster, it can be completed with zero
downtime and without any impact of the level of availability.

02-27
Lesson summary
• Key points
– In this lesson, you learned about VCS terminologies.
– You also learned about different VCS cluster communication mechanisms.
– In addition, you learned about the implementation of high availability in the VCS architecture.
– Finally, you learned about the InfoScale support for multi-version clusters.
– Veritas Cluster Server Bundled Agents Reference Guide
– Cluster Server Administrator’s Guide
– Advanced VCS course
28

02-28
Lab 03: VCS Building Blocks
• Exercise A: Working with the VIOM UI Dashboard and VCS Inventory information
• Exercise B: Exploring the VIOM UI settings option
• Exercise C: Working with the VIOM UI Licensing and Reports option
29

02-29
What did you learn?
lesson.
30
current lesson.

02-30
Question 1: VCS terminology
A persistent resource:
A. Can be taken offline, but not brought online.

B. Cannot be a parent to another resource.
C. Always depends on another resource.
D. Is used for monitoring hardware devices.
31

02-31
Answer 1: VCS terminology
A persistent resource:
A. Can be taken offline, but not brought online.

B. Cannot be a parent to another resource.
C. Always depends on another resource.
D. Is used for monitoring hardware devices.
The correct answer is B. Parent resource needs to go offline before the child and in case of persistent, it can’t be
taken offline and hence service group offline will be affected.
32

02-32
Question 2: VCS terminology
A resource type defines:
A. How HAD manages all resources of that type

B. How an agent manages all resources of that type
C. Attribute values of a resource
D. Attributes of all resources
33

02-33
Answer 2: VCS terminology
A resource type defines:
A. How HAD manages all resources of that type

B. How an agent manages all resources of that type
C. Attribute values of a resource
D. Attributes of all resources
The correct answer is B. There is a corresponding agent for each resource type that manages all the resources of
specific type. Resource type defines how that particular agent will manage corresponding resources.
34

02-34
Question 3: Cluster communication
Cluster systems communicate over TCP/IP on a redundant private Ethernet network.
A. True
B. False
35

02-35
Answer 3: Cluster communication
Cluster systems communicate over TCP/IP on a redundant private Ethernet network.
A. True
B. False
The correct answer is False. Cluster systems communicate over LLT on a redundant private Ethernet network.
36

02-36
Agents on each system communicate with the corresponding agents on all other
systems.
A. True
B. False
37

02-37
Agents on each system communicate with the corresponding agents on all other
systems.
A. True
B. False
The correct answer is False. Agent communicates with HAD on local systems, it is HAD that communicates over
GAB with the HAD on another cluster nodes.
38

02-38
What role does the hashadow daemon perform in a VCS cluster? The hashadow
daemon:
A. Replicates the main.cf file

B. Communicated with GAB and LLT
C. Restarts HAD if HAD fails
D. Replicates the in-memory configuration
39

02-39
What role does the hashadow daemon perform in a VCS cluster? The hashadow
daemon:
A. Replicates the main.cf file

B. Communicated with GAB and LLT
C. Restarts HAD if HAD fails
D. Replicates the in-memory configuration
The correct answer is C. Role of hashadow daemon is to keep track of local HAD on each system. In case HAD is
down, it restarts it.
40

02-40
End of presentation
02-41
Lesson 03: VCS Operations
This is the VCS Operations lesson in the Veritas InfoScale 7.4.2 Fundamentals for

03-1
Administration
Administration


03-2
Lesson objectives
Topic Objective
Common VCS tools and operations Perform common cluster administrative operations.
Service group operations Manage applications under control of VCS service groups.
Resource operations Manage resources within VCS service groups.
Explain custom scripts and new environment variables and VCS

Core VCS enhancements in 7.4.2
service support.

03-3
Topic: Common VCS tools and operations
After completing this topic, you will be able to perform common cluster
administrative operations.
This is the Common VCS tools and operations topic.


03-4
VCS management tools
VIOM CLI Java GUI

• Veritas InfoScale Operations • Command-line interface • Deprecated.
Manager • Suited for local cluster • Ongoing support only for
• Suited for large numbers of management. Windows.
clusters . • Needed for using Simulator.
• Management for all Storage
Foundation products.
You can use different VCS interfaces to manage the cluster environment, provided that you
have the proper VCS authorization. VCS user accounts are described in detail in the ‘VCS
Configuration Methods’ lesson.
• Veritas InfoScale Operations Manager (VIOM) is a Web-based interface for administering
managed hosts in local and remote clusters. Installation and configuration of the VOM
environment is described in detail in the Veritas InfoScale Operations Manager Installation
Guide.
• The VCS command-line interface is installed by default and is best suited for configuration
and management of a local cluster.
• The Java GUI runs on Windows, and although deprecated for UNIX and Linux platforms, it
can still be used for some limited configuration and management purposes.

03-5
Displaying logs
• VCS log file location: /var/VRTSvcs/log
• HAD (engine) log: engine_A.log
• VOM GUI log files:
– Is useful for learning the CLI.
– Displays logs for individual perspectives.
Viewing log files

– UNIX utilities:
tail, pg, more, view
– VCS hamsg utility:
hamsg -help
The VCS engine log is located in /var/VRTSvcs/log/engine_A.log. You can view this file with
standard UNIX text file utilities such as tail, more, or view. VCS provides the hamsg utility
that enables you to filter and sort the data in log files.
In addition, you can display the engine log in VIOM to see a variety of views of detailed status
information about activity in the cluster, using selected perspectives like Storage, Availability,
and Server level.

03-6
Displaying object information
Display resource information: Display service group information:

hares –display resource hagrp –display group
hares –list condition hagrp –list condition
hares –value resource attr hagrp –value group attribute
hares –state resource hagrp –state group
Getting help
• Command-line syntax:
– ha_command –help
– man ha_command
• Cluster Server Administrator’s Guide
The following examples show how to display resource attributes and status.
• Display values of attributes to ensure they are set properly.
hares -display webip
#Resource Attribute System Value
. . .
webip AutoStart global 1
webip Critical global 1
• Determine which resources are non-critical.
hares -list Critical=0

webapache s1
webapache s2

03-7
• Determine the virtual IP address for the websg service group.
hares -value webip Address
Webip Address global 10.10.27.93
• Determine the state of a resource on each cluster system.
hares -state webip
webip State s1 OFFLINE
webip State s2 ONLINE
Displaying service group information using the CLI
The following examples show some common uses of the hagrp command for displaying service
group information and status.
• Display values of all attributes to ensure they are set properly.
hagrp -display websg
#Group Attribute System Value
. . .
websg AutoFailOver global 1
websg AutoRestart global 1
. . .
• Determine which service groups are frozen, and are therefore not able to be stopped, started, or
failed over.
hagrp -list Frozen=1
websg s1
websg s2
• Determine whether a service group is set to automatically start.
hagrp -value websg AutoStart
1
• List the state of a service group on each system.
hagrp -state websg
#Group Attribute System Value
websg State s1 |Online|
websg State s2 |Offline|

03-8
Topic: Service group operations
After completing this topic, you will be able to manage applications under
control of VCS service groups.
This is the Service group operations topic.


03-9
Bringing service groups online
• Service groups are brought online:
– Manually, after maintenance. websg
webapache
– Automatically, when vcs is started.
• Resources are brought online in order from bottom to top.
webip webmnt
• Persistent resources do not affect service group state.
webnic webvol
webdg
Online
hagrp -online
10
When a service group is brought online, resources are brought online starting with the child
resources and progressing up the dependency tree to the parent resources. In order to bring a
failover service group online, VCS must verify that all nonpersistent resources in the service
group are offline everywhere in the cluster. If any nonpersistent resource is online on another
system, the service group is not brought online.
A service group is considered online if all of its nonpersistent and autostart resources are
online. An autostart resource is a resource whose AutoStart attribute is set to 1. The state of
persistent resources is not considered when determining the online or offline state of a
service group because persistent resources cannot be taken offline. However, a service group
is faulted if a persistent resource faults.
To bring a service group online, use either form of the hagrp command:
• hagrp -online group -sys system
or
• hagrp -online group –any
The -any option brings the service group online based on the group’s failover policy.

03-10
Taking service groups offline
• Service groups are taken offline:
– Manually, for maintenance.
– Automatically, when vcs is stopped or websg
during failover. webapache
Offline
• Resources are taken offline in order
from top to bottom. webip webmnt
• A service group is offline when all
nonpersistent resources are offline.
webnic webvol
webdg
hagrp -offline
11
When a service group is taken offline, resources are taken offline starting with the highest
(parent) resources in each branch of the resource dependency tree and progressing down the
resource dependency tree to the lowest (child) resources.
Persistent resources cannot be taken offline. Therefore, the service group is considered offline
when all nonpersistent resources are offline.
Taking a service group offline using the CLI
To take a service group offline, use either form of the hagrp command:
hagrp -offline group -sys system
Provide the service group name and the name of a system where the service group is online.
hagrp -offline group –any

Provide the service group name. The -any switch takes a failover service group offline on the
system where it is online. All instances of a parallel service group are taken offline when the -
any switch is used.

03-11
Switching service groups
You can switch a service group between systems to:
• Test failover.
• Migrate services for maintenance.
s2
VCS: s1
1. Takes resources offline on system s1.
2. Brings resources online on system s2.
Only resources online on system s1 are brought online
on system s2.
hagrp -switch
12
In order to ensure that failover can occur as expected in the event of a fault, test the failover
process by switching the service group between systems within the cluster.
Switching a service group does not have the same effect as taking a service group offline on
one system and bring the service group online on another system. When you switch a service
group, VCS replicates the state of each resource on the target system. If a resource has been
manually taken offline on a system before the switch command is run, that resource is not
brought online on the target system.
To switch a service group, type the following command:
hagrp -switch group -to system
You need to provide the service group name and the name of the system where the service
group is to be brought online.

03-12
Freezing a service group
Freeze a service group to prevent offline, online, or failover actions even if a resource faults causing the
group to fault.
Temporary Persistent Example

• Only in effect until • Remains in effect Administrator can
VCS restarts through VCS start and stop an
• TFrozen=1 restarts application outside of
• Frozen=1 VCS control
When frozen, VCS does not take action on the service group even if you
! cause a concurrency violation by bringing the service online on another

system outside of VCS.
hagrp -freeze
13
When you freeze a service group, VCS continues to monitor the resources, but it does not
allow the service group (or its resources) to be taken offline or brought online. Failover is also
disabled, even if a resource faults. You can also specify that the freeze is in effect even if VCS is
stopped and restarted throughout the cluster.
Warning: Freezing a service group effectively overrides VCS protection against a concurrency
violation—which occurs when the same application is started on more than one system
simultaneously. You can cause possible data corruption if you bring an application online
outside of VCS while the associated service group is frozen.

03-13
Freezing and unfreezing a service group using the CLI
• To freeze and unfreeze a service group temporarily, type:
hagrp -freeze group
hagrp -unfreeze group
• To freeze a service group persistently, you must first open the configuration:
haconf –makerw
hagrp -freeze group –persistent
hagrp -unfreeze group –persistent
To determine if a service group is frozen, display the Frozen (for persistent) and TFrozen (for
temporary) service group attributes for a service group.
hagrp -value group Frozen

03-14
Topic: Resource operations
After completing this topic, you will be able to manage resources within
VCS service groups
15
This is the Resource operations topic.


03-15
Bringing resources online
• Resources are brought online:
– Automatically, when a service group is brought online.
– Manually, after maintenance. websg
– In dependency order from bottom.
webapache
• Agent runs the online entry point
• Online entry point runs specific startup operations. webip webmnt
webnic webvol
webdg
Online
Disk group online example:

vxdg –t import webdatadg
hares -online
16
In normal day-to-day operations, you perform most management operations at the service
group level. However, you may need to perform maintenance tasks that require one or more
resources to be offline while others are online. Also, if you make errors during resource
configuration, you can cause a resource to fail to be brought online.
Bringing resources online using the CLI
To bring a resource online, type hares -online resource -sys system
You need to provide the resource name and the name of a system that is configured to run
the service group.
Note: The service group shown in the slide is partially online after the webdg resource is
brought online. This is depicted by the textured coloring of the service group circle.

03-16
Taking resources offline
• Resources are taken offline:
– Automatically, when a service group is taken offline. Online
– Manually, for maintenance. dbsg
– In dependency order from top.
dboracle
• Agent runs the offline entry point
• Offline entry point runs specific shutdown operations dbip dbmnt
dbnic dbvol
dbdg
Database offline example:

sqlplus "/as sysdba"
shutdown immediate
hares -offline
17
Taking resources offline should not be a normal occurrence. Taking resources offline causes
the service group to become partially online, and availability of the application service is
affected.
If a resource needs to be taken offline, for example, for maintenance of underlying hardware,
then consider switching the service group to another system.
If multiple resources need to be taken offline manually, then they must be taken offline in
resource dependency tree order, that is, from top to bottom.
Taking a resource offline and immediately bringing it online may be necessary if, for example,
the resource must reread a configuration file due to a change. Or you may need to take a
database resource offline in order to perform an update that modifies the database files.
Taking resources offline using the CLI

To take a resource offline using the CLI, type hares -offline resource -sys
system
You need to provide the resource name and the name of a system.

03-17
Topic: Core VCS enhancements in 7.4.2
After completing this topic, you will be able to learn about core VCS
enhancements in 7.4.2.
18
This is the Core VCS enhancements in 7.4.2 topic.


03-18
Overview of core VCS enhancements in 7.4.2
• HAD startup can be customized by editing two new VCS custom scripts.
• New environments variables introduced to control the evacuation of the ServiceGroups

and the enabling/disabling of the CmdServer.
• In Solaris and AIX, hastart makes internal service callbacks to svcadm enable
vcs and to /etc/init.d/vcs.rc start.
19
HAD startup can be customized by editing two new VCS custom scripts. It gives better control
on HAD startup, avoiding the customers using personal scripts. New environment variables
are introduced to control the evacuation of the ServiceGroups and the enabling/disabling of
the CmdServer. It gives control on the evacuation of service groups and the start of the
CmdServer during a reboot/shutdown or start/stop of the VCS service.
In Solaris and AIX, hastart makes internal service callbacks to svcadm enable vcs
and to /etc/init.d/vcs.rc start. Consistent VCS service state across reboot and
manual startup

03-19
Use-case
• Multiple customizations around VCS service startup/shutdown

– Different kinds of customization needs by customers during VCS service start-up.
– Custom scripts to provide more control to the user.
• Seamless upgrade experience

– No need for custom patching of VCS start-up files or command scripts.
– No need for customers to create and maintain multiple patch files for different versions of the product.
20
Multiple customizations around VCS service startup/shutdown

• Different kinds of customization needs by customers during the VCS service start-up.
• Custom scripts to provide control to the user.
For a seamless upgrade experience
• No need for custom patching of VCS start-up files or command scripts.
• No need for customers to create and maintain multiple patch files for different versions of
the product.

03-20
New custom scripts
pre_hastart custom_had_start
It is called by hastart script to allow

It is called by the VCS service startup
a custom code to start the HAD
script before hastart.
binary.
• The sample scripts are part of VRTSvcs package in “$VCS_HOME/bin/sample_scripts/VRTSvcs/”

folder and contain sample code and return value translations.
• Once customized, the scripts should be copied to $VCS_HOME/bin folder for execution.
• Only users having the right permissions for $VCS_HOME/bin folder can execute the scripts.
21
Two custom scripts pre-hastart and custom_had_start are called as part of the
VCS startup.
• pre-hastart is called by the VCS service startup script before the hastart
command is executed.
• custom_had_start is called by the hastart script to allow a custom code to start
the HAD binary. For example, HAD start-up done from the command shell that has a new
process authentication group)
Sample scripts are part of the VRTSvcs package and are available in the folder specified on the
slide. This folder contains sample code and return value transactions. After customizing the
scripts, it should be copied to the $VCS_HOME/bin folder for execution. Note that only
users with the right permission can execute the scripts.

Two scenarios exist regarding the execution of the custom_had_start command:
• If the command cannot be executed, hastart falls back to traditional way of HAD
startup.
• If the command fails, VCS service goes into a ‘failed’ state

03-21
New environment variables
NOEVACUATE STARTCMDSERVER
It controls the evacuation of a Service

It controls CmdServer start when HAD
Groups during service stop (default =
starts (default = 1).
0).
• These variables are in:

— /etc/sysconfig/vcs file for Linux
— /etc/default/vcs file for Solaris and AIX
• When the variable NOEVACUATE is set to 1, the ServiceGroups are not evacuated and hastop –local –
noautodisable is executed.
• When the variable STARTCMDSERVER is set to 0, CmdServer is not started on High Availability Daemon startup.
22
Two new environment variables are introduced in VCS:

• NOEVACUATE: it controls the evacuation of a Service Groups during service stop (default =
0)
• STARTCMDSERVER: it controls CmdServer start when HAD starts (default = 1)
These variables are available at the following locations:
— /etc/sysconfig/vcs file for Linux
— /etc/default/vcs file for Solaris and AIX
When the variable NOEVACUATE is set to 1, the ServiceGroups are not evacuated and
hastop –local –noautodisable is executed.

When the variable STARTCMDSERVER is set to 0, CmdServer is not started on High Availability
Daemon startup.

03-22
Service Callbacks during hastart
For Solaris For AIX
hastart executes svcadm /etc/init.d/vcs.rc start is

restart vcs or svcadm called.
enable vcs depending on the
status of service.
Service Callbacks:
• Ensure consistent behaviour across reboot and manual start-up.
• Ensure common behaviour across platforms.
• Provides $VCS_HOME/bin/pre_hastart execution as part of the hastart call.
• Reflects the correct status of VCS at service level.
23
For Solaris: hastart executes svcadm restart vcs’ or ‘svcadm enable vcs
depending on the status of service.
For AIX : /etc/init.d/vcs.rc start is called.
Service Callbacks ensure consistent behaviour across reboot and manual start-up. It also
ensures common behaviour across platforms. It provides
$VCS_HOME/bin/pre_hastart execution as part of the ‘hastart’ call and reflects
the correct status of VCS at the service level.

03-23
Service callback with Custom Script support
Execute
hastart
pre_hastart
Custom HAD
startup
Yes
pre_hastart No
script? Called using
Traditional
service
HAD startup
No script?
Yes
VCS service
startup
custom_had Yes
start?
No
24
The flowchart on the slide displays Service callback with Custom Script support.
There are limitations to the Custom Script including the failure to determine whether the
custom_had_start script succeeded to start the HAD daemon. A failure is treated as a
user error. When a command hangs in a custom script, the VCS service transitions to the failed
state once it reaches the start timeout. The hung process continues to run along with the
caller scripts.

03-24
Lesson summary
• Key points
– In this lesson, you learned to use VCS tools to manage applications under VCS control.
– In addition, you learned about the use of VOM to practice managing resources and service groups.
– Finally, you learned about the two new custom scripts and two new environment variables and other
enhancements in core VCS service support in 7.4.2.
– SORT downloads
– Veritas Cluster Server Release Notes
25

03-25
Lab 04:VCS Operations
• Exercise A: Displaying cluster information

• Exercise B: Displaying status and attributes
• Exercise C: Performing service group operations
• Exercise D: Manipulating resources
26

03-26
What did you learn?
lesson.
27
current lesson.

03-27
Question 1: Service group operations
After you place a service under VCS control, you can continue to start and stop the
application normally without affecting anything in the cluster.
A. True
B. False
28

03-28
Answer 1: Service group operations
After you place a service under VCS control, you can continue to start and stop the
application normally without affecting anything in the cluster.
A. True
B. False
The correct answer is False. Once an application is under VCS control, it should be managed through VCS only not
manually outside VCS.
29

03-29
To manage VCS service groups in a running cluster, use the:
A. Java GUI, which runs only on a Windows system

B. Java GUI, which runs only on a UNIX system
C. VCS Simulator on any type of system
D. VCS Java GUI, VOM, or CLI
30

03-30
To manage VCS service groups in a running cluster, use the:
A. Java GUI, which runs only on a Windows system

B. Java GUI, which runs only on a UNIX system
C. VCS Simulator on any type of system
D. VCS Java GUI, VOM, or CLI
The correct answer is D. VCS service groups can be managed through any of the following: VCS Java GUI, VOM or CLI.
31

03-31
A service group is considered offline when:
A. All persistent resources in that group are offline on a system .

B. All nonpersistent resources in that group are offline on a system and persistent
resources are probed.
C. All nonpersistent resources in that group are offline on a system.
D. All resources in that group are offline, cleared, and probed on a system.
32

03-32
A service group is considered offline when:
A. All persistent resources in that group are offline on a system .

B. All nonpersistent resources in that group are offline on a system and persistent
resources are probed.
C. All nonpersistent resources in that group are offline on a system.
D. All resources in that group are offline, cleared, and probed on a system.
The correct answer is C. A service group state will be reflected as offline only when all its nonpersistent resources are
offline on that particular system.
33

03-33
What is the effect of freezing a service group?
A. The service group cannot be failed over.

B. The service group cannot fault.
C. The resources within the service group cannot fault.
D. The service group cannot be modified.
34

03-34
What is the effect of freezing a service group?
A. The service group cannot be failed over.

B. The service group cannot fault.
C. The resources within the service group cannot fault.
D. The service group cannot be modified.
The correct answer is A. The main purpose of freezing a service group is to limit its failover during maintenance activity
on that service group.
35

03-35
Question 5: Resource operations
The VCS Simulator:
A. Enables you to simulate operations using the VCS CLI on a running cluster
B. Enables you to simulate operations using the VCS Java GUI on a running cluster
C. Requires dedicated main.cf and types.cf configuration files
D. Can be run using any valid main.cf and types.cf files
36

03-36
Answer 5: Resource operations
The VCS Simulator:
A. Enables you to simulate operations using the VCS CLI on a running cluster
B. Enables you to simulate operations using the VCS Java GUI on a running cluster
C. Requires dedicated main.cf and types.cf configuration files
D. Can be run using any valid main.cf and types.cf files
The correct answer is D. VCS simulator works pretty good with just a valid main.cf and types.cf file.
37

03-37
03-38
Lesson 04: VCS Configuration Methods
This is the VCS Configuration Methods lesson in the Veritas InfoScale 7.4.2 Fundamentals for

04-1
Administration
Administration


04-2
Lesson objectives
Topic Objective
Start and stop VCS and describe the effects of cluster attributes
Starting and stopping VCS
on cluster shutdown.
Overview of configuration
Compare and contrast VCS configuration methods.
methods
Online configuration Explain the online configuration method.
Controlling access to VCS Set user account privileges to control access to VCS.
Summarize VCS user and agent account passwords encryption
VCS Password Encryption
standards.

04-3
Topic: Starting and stopping VCS
After completing this topic, you will be able to start and stop VCS and
describe the effects of cluster attributes on cluster shutdown.
This is the Starting and stopping VCS topic.


04-4
VCS startup behavior (1/2)
s1 s2
Local build Current

No config in
Cluster discover
memory
conf wait
41
21 main.cf main.cf
had had
hashadow hashadow
1
31
hastart
The default VCS startup process is demonstrated using a cluster with two systems connected
by the cluster interconnect. To illustrate the process, assume that neither system have an
active cluster configuration.
1. The hastart command is run on s1 and starts the had and hashadow processes.
2. HAD checks for a valid configuration file (hacf -verify config_dir).
3. HAD checks for an active cluster configuration on the cluster interconnect.
4. There is no active cluster configuration, hence HAD on s1 reads the local main.cf file and
loads the cluster configuration into local memory. The s1 system is now in the VCS local
build state, which means that the VCS is building a cluster configuration in the memory on
the local system.

04-5
VCS startup behavior (2/2)
s1 s2
Local build or Remote

running Cluster Cluster build
conf conf
11
1
main.cf main.cf 61
had had
hashadow hashadow
71
10
1 51
91 hastart
81
5. The hastart command is then run on s2 and starts had and hashadow on s2. The s2
system is now in the VCS current discover wait state which means that the VCS is in a wait
state while it is discovering the current state of the cluster.
6. HAD on s2 checks for a valid configuration file on the disk.
7. HAD on s2 checks for an active cluster configuration by sending a broadcast message out
on the cluster interconnect, even if the main.cf file on s2 is valid.
8. HAD on s1 receives the request from s2 and responds.
9. HAD on s1 sends a copy of the cluster configuration over the cluster interconnect to s2.
The s1 system is now in the VCS running state, which implies that the VCS determines that
there is a running configuration in memory on system s1. The s2 system is now in the VCS
remote build state, which means that the VCS is building the cluster configuration in the
memory on the s2 system from the cluster configuration that is in a running state on s1.
10. HAD on s2 performs a remote build operation to place the cluster configuration in the
memory.

04-6
11. When the remote build process completes, HAD on s2 copies the cluster configuration into the
local main.cf file.
If s2 has valid local configuration files (main.cf and types.cf), these are saved to new files
with a name, including a date and time stamp, before the active configuration is written to the
main.cf file on the disk. Note that if the checksum of the configuration in the memory
matches the main.cf on the disk, no write to disk occurs.
The startup process is repeated on each system until all members have identical copies of the cluster
configuration in the memory and matching main.cf files on the local disks. Synchronization is
maintained by data transfer through LLT and GAB.

04-7
Stopping VCS
s2 s2
s1 s1
had had
had had
$VCS_STOP_TIMEOUT = Non zero

$VCS_STOP_TIMEOUT = 0
value
• Stop operation does not timeout. • Stop operation timed-out after the above
• HAD continues to be in the LEAVING state. mentioned value (seconds).
• Administrative intervention might be • VCS stops itself forcefully. (hastop-
required. local-force)
There are several methods of stopping the VCS engine (had and hashadow daemons) on a
cluster system. The options you specify to hastop determine where VCS is stopped, and how
resources under VCS control are affected. However, before 7.3.1 there were different
behaviors of VCS shutdown across different platforms.
In previous releases, the VCS shutdown mechanism:
Application > HAD > VXFEN > GAB > LLT
If the application hangs while going offline, it hangs init/rc forever resulting in a system
shutdown or restart is halted forever.
In release 7.3.1, a new variable $VCS_STOP_TIMEOUT is introduced.
• User can decide the VCS behavior through the $VCS_STOP_TIMEOUT variable
— Default value of the variable is 0.
— Non-zero value is considered as timeout activated.
• Control over the VCS shutdown behavior.
• Provides common behavior across supported platforms.

04-8
$VCS_STOP_TIMEOUT variable
• If $VCS_STOP_TIMEOUT is set to 0, VCS service stop will not timeout.
• If $VCS_STOP_TIMEOUT is set to a non zero value, VCS service stop will timeout after the given value.
Possible values of $VCS_STOP_TIMEOUT
Possible values VCS service Behavior

0 Do not timeout VCS service stop
Non zero value Timeout VCS service stop after the given value ( value is
in seconds)
$VCS_STOP_TIMEOUT variable location
Platform Variable Path

For Linux /etc/sysconfig/vcs
For Solaris /etc/default/vcs
For AIX /etc/default/vcs
$VCS_STOP_TIMEOUT variable
• If $VCS_STOP_TIMEOUT is set to 0, the VCS service stop will not timeout.
• If $VCS_STOP_TIMEOUT is set to a non-zero value, the VCS service stop will timeout
after the given value.
Possible values of $VCS_STOP_TIMEOUT
• 0 = Do not timeout VCS service stop
• Non-zero value = Timeout VCS service stop after the given value (value is in
seconds)
$VCS_STOP_TIMEOUT variable location
• For Linux = /etc/sysconfig/vcs
• For Solaris = /etc/default/vcs
• For AIX = /etc/default/vcs

04-9
Modifying VCS stop behavior with $VCS_STOP_TIMEOUT
$VCS_STOP_TIMEOUT
Default value
10
$VCS_STOP_TIMEOUT variable to define the VCS behavior while executing the hastop
command.

04-10
Topic: Overview of configuration methods
After completing this topic, you will be able to compare and contrast VCS
configuration methods.
11
This is the Overview of configuration methods topic.


04-11
Configuration methods
Online: HAD is running

• VIOM graphical user interfaces.
• VCS CLI and shell scripts.
# vi main.cf
~
include "types.cf"
cluster webvcs (
Offline: HAD must restart …
)
• Manual modification of configuration System s1 (
files. )
• Modification of a main.cf file using the System s2 (
)
VCS Simulator. group websg (
…
12
VCS provides several tools and methods for configuring service groups and resources,
generally categorized as:
• Online configuration
You can modify the cluster configuration while VCS is running using either the graphical
user interfaces or the command-line interface. These online methods change the cluster
configuration in memory. When finished, you write the in-memory configuration to the
main.cf file on disk to preserve the configuration.
• Offline configuration
In some circumstances, you can simplify cluster implementation and configuration using
an offline method, including:

– Editing configuration files manually
– Using the Simulator to create, modify, model, and test configurations
This method requires you to stop and restart VCS in order to build the new configuration
in memory.

04-12
Topic: Online configuration
After completing this topic, you will be able to explain the online
configuration method.
13
This is the Online configuration topic.


04-13
How VCS changes the online cluster configuration
Open the configuration. haconf -makerw
hagrp –add websg
Add a service group.
In-memory
configuration
In-memory
configuration
HAD
vxfen
HAD
GAB
vxfen
LLT
GAB
LLT
14
When you use the Cluster Manager to modify the configuration, the GUI communicates with
the HAD on the specified cluster system to which the Cluster Manager is connected.
Note: The Cluster Manager configuration requests are shown conceptually as ha commands
in the diagram, but they are implemented as system calls.
The had daemon communicates the configuration change to had on all other nodes in the
cluster, and each had daemon changes the in-memory configuration.
When the command to save the configuration is received from the Cluster Manager, had
communicates this command to all cluster systems, and each system’s had daemon writes the
in-memory configuration to the main.cf file on its local disk.
The VCS command-line interface is an alternate online configuration tool. When you run ha
commands, had responds similarly.
Note: When two administrators are changing the cluster configurations simultaneously, each
administrator can see the changes as they are being made.

04-14
Opening the cluster configuration
ReadOnly=0
1
haconf -makerw Shared cluster configuration
2 in memory.
hares –modify ...
In-memory
main.cf configuration main.cf
not equal to
main.cf
15
You must open the cluster configuration to add service groups and resources, make
modifications, and perform certain operations.
The state of the configuration is maintained in an internal attribute, i.e. ReadOnly. If you try to
stop VCS with the configuration open, a warning is displayed. This helps to ensure that you
remember to save the configuration to disk and avoid losing any changes made while the
configuration was open. You can override this protection, as described later in this lesson.

04-15
Saving the cluster configuration
ReadOnly=0
haconf -dump
Shared cluster configuration
in memory.
In-memory
matches
main.cf
16
When you save the cluster configuration, VCS copies the configuration in memory to the
main.cf file in the /etc/VRTSvcs/conf/config directory on all running cluster
systems. At this point, the configuration is still open. You have only written the in-memory
configuration to disk and have not closed the configuration.
If you save the cluster configuration after each change, you can view the main.cf file to see
how the in-memory modifications are reflected in the main.cf file.

04-16
Closing the cluster configuration
ReadOnly=1
1
haconf -dump -makero Shared cluster configuration
in memory.
2
In-memory
matches
main.cf
17
When the administrator saves and closes the configuration, VCS changes the state of the
configuration to closed where the ReadOnly attribute is equal to 1. In addition, VCS writes the
configuration in memory to the main.cf file

04-17
How VCS protects the cluster configuration
When the configuration is open:
• Configuration in memory may not match main.cf file.
• ReadOnly cluster attribute is set to 0:
haclus –value ReadOnly
• VCS warns you to close the configuration if you:
– Stop VCS on all systems.
– Close the GUI.
Use of hastop –all –force

! bypasses warning; you can lose
configuration changes
18
When the cluster configuration is open, you cannot stop VCS without overriding the warning
that the configuration is open.
• If you ignore the warning and stop VCS while the configuration is open, you may lose
configuration changes.
• If you forget to save the configuration and shut down VCS, the configuration in the
main.cf file on disk may not be the same as the configuration that was in memory
before VCS was stopped.
You can configure VCS to automatically back up the in-memory configuration to the disk to
minimize the risk of losing modifications made to a running cluster.

04-18
Automatic configuration backups
Configuring automatic main.cf backups:
• BackupInterval is 0 by default.
• When set > 0, VCS saves the configuration to main.cf.autobackup
haclus -modify BackupInterval 3
• BackupInterval minimum is 3 (minutes).
# ls –l /etc/VRTSvcs/conf/config/main*
-rw------ 2 root other 5992 Oct 10 12:07 main.cf
-rw------ 1 root root 5039 Oct 8 8:01 main.cf.08Oct2015...
-rw------ 2 root other 5051 Oct 9 17:58 main.cf.09Oct2015...
-rw------ 2 root other 5992 Oct 10 12:07 main.cf.10Oct2015...
-rw------ 1 root other 6859 Oct 11 7:43 main.cf.autobackup
-rw------ 2 root other 5051 Oct 9 17:58 main.cf.previous
19
You can set the BackupInterval cluster attribute to automatically save the in-memory
configuration to disk periodically. When set to a value greater than or equal to three minutes,
VCS automatically saves the configuration in memory to the main.cf.autobackup file.
Note: If no changes are made to the cluster configuration during the time period set in the
BackupInterval attribute, no backup copy is created. If necessary, you can copy the
main.cf.autobackup file to main.cf and restart VCS to build the configuration in
memory at the point in time of the last backup. Ensure that you understand the VCS startup
sequence described in the “Starting and Stopping VCS” section before you attempt this type
of recovery.

04-19
Topic: Controlling access to VCS
After completing this topic, you will be able to set user account privileges to
control access to VCS.
20
This is the Controlling access to VCS topic.


04-20
Relating VCS and UNIX user accounts
In non-secure mode:
• No UNIX to VCS user account mapping, except root.
• Cluster Administrator privileges given to root.
• Nonroot users are prompted for a VCS account name and password when using the CLI.
Mode Authentication Authorization

• System-level
Nonsecure VCS
• VCS
Secure—default System-level only VCS
Stronger security to VCS users with 2048 bit key and SHA256 signature, in secure mode.
21
If you have not configured security in the cluster, VCS has a completely separate list of user
accounts and passwords to control access to VCS.
When using the Cluster Manager to perform administration, you are prompted for a VCS
account name and password. Depending on the privilege level of that VCS user account, VCS
displays the Cluster Manager GUI with an appropriate set of options. If you do not have a valid
VCS account, you cannot run the Cluster Manager.
When using the command-line interface for VCS, you are also prompted to enter a VCS user
account and password after which it is determined whether that user account has proper
privileges to run the command. One exception is the UNIX root user. By default, only the UNIX
root account is able to use VCS ha commands to administer VCS from the command line.
VCS access in secure mode

• When running in secure mode, VCS uses operating system-based authentication, which
enables VCS to provide a single sign-on mechanism. All VCS users are system and domain
users and configured using fully qualified user names, for example,
[email protected].
• When running in secure mode, you can add system or domain users to VCS and assign
them VCS privileges. However, you cannot assign or change passwords using a VCS
interface.
Note: InfoScale Availability uses 2048 bit key and SHA256 signature certificates.
The vcsauthserver generates certificates with 2048 bit key and SHA256 signature and provides
stronger security to VCS users.

04-21
Simplifying VCS administrative access
To simplify administration from the CLI:
11
Set the VCS_Host environment variable to the node name to
administer global groups (in remote clusters).
21
Log in to VCS using halogin vcs_user_name.
31
Type the password.
22
The halogin command is provided to save authentication information so that users do not
have to enter credentials every time a VCS command is run.
The command stores authentication information in the user’s home directory. You must set
the VCS_HOST environment variable to the name of the node from which you are running VCS
commands to use halogin.
Note: The effect of halogin only applies for that shell session.

04-22
VCS user account privileges
Cluster-level authorizations Group-level authorizations
• Cluster Administrator: Full privileges. • Group Administrator: All operations

• Cluster Operator: All cluster, service for specified service group, except
group, and resource-level operations. deletion.
• Cluster Guest: Read-only access; new • Group Operator: Service group and
users given Cluster Guest resource online and offline;
authorization by default. temporarily freeze or unfreeze service
groups.
! An admin user must open the cluster configuration to enable

password changes for operator and guest accounts.
23
You can ensure that different types of administrators in your environment have a VCS
authority level to affect only those aspects of the cluster configuration that are appropriate to
their level of responsibility.
For example, if you have a DBA account that is authorized to take a database service group
offline or switch it to another system, you can make a VCS Group Operator account for the
service group with the same account name. The DBA can then perform operator tasks for that
service group, but cannot affect the cluster configuration or other service groups. If you set
AllowNativeCliUsers to 1, then the DBA logged on with that account can also use the VCS
command line to manage the corresponding service group.
Setting VCS privileges is described in the next section.

04-23
Configuring cluster user accounts
VIOM GUI configuration
CLI configuration
1. Open the cluster configuration.
2. Add a user:
hauser –add user
3. Type a password.
4. Add privileges:
Cluster
hauser –addpriv user priv
Group
hauser -addpriv user priv
–group group
5. Save and close the configuration.
24
VCS users are not the same as UNIX users except when running VCS in secure mode. If you
have not configured security in the cluster, VCS maintains a set of user accounts separate from
UNIX accounts. In this case, even if the same user exists in both VCS and UNIX, this user
account can be given a range of rights in VCS that does not necessarily correspond to the
user’s UNIX system privileges.
The slide shows how to use the hauser command to create users and set privileges. You can
also add privileges with the -addpriv and –deletepriv options to hauser.
In non-secure mode, VCS passwords are stored in the main.cf file in encrypted format. If
you use a GUI or CLI to set up a VCS user account, passwords are encrypted automatically. If
you edit the main.cf file, you must encrypt the password using the vcsencrypt
command.

04-24
Note: In non-secure mode, if you change a UNIX account, this change is not reflected in the VCS
configuration automatically. You must manually modify accounts in both places to synchronize them.
Modifying user accounts
Use the hauser command to make changes to a VCS user account:
• Change the password for an account.
hauser -update user_name
• Delete a user account.
hauser -delete user_name

04-25
Topic: VCS Password Encryption
After completing this topic, you will be able to summarize VCS user and
agent account passwords encryption standards.
26
This is the VCS Password Encryption topic.


04-26
AES 128 encryption
• AES 256 bit encryption provides more security to the users as compare to AES 128. The solution
proposed is cross-platform.
• Different behaviors of VCS across platforms often cause confusion to the users. VCS being cross-
platform product, ensures common behavior across OS platforms.
• AES 256 is more secure than AES 128. A 256-bit key means 2^256 possible keys to bruteforce, as
proposed to 2^128 (AES 128).
27
In versions prior to 7.3.1, AES 128 bit encryption was used. The vcsencrypt utility allowed
users to encrypt the agent passwords using a security key. The security key supports AES or
Advanced Encryption Standard encryption which creates a secure password for the agent.
Before 7.3.1, AES 128 bit encryption was used in agents.
The VCS 7.3.1 recommendations was to move to AES 256-bit encryption for enhanced security
and use the randomized initialization vector in the AES-CBC implementation.
• AES-256 bit encryption provides better security to the users as compared to the AES-128
encryption. The solution proposed is cross-platform.
• Users can get confused between different behaviors of VCS across platforms. VCS being a
cross-platform product ensures common behavior across OS platforms.
• AES-256 encryption is more secure than AES-128 encryption. It has a 256-bit key which
means that 2^256 possible keys to bruteforce, as proposed to 2^128 in AES 128-bit
encryption.

04-27
AES 256 architecture
A 256 bit encryption key is generated using the

openssl APIs, and stored in the main.cf.
Then a randomized initialization vector is generated,

which also gets stored in the main.cf.
The vscencrypt utility encrypts the password

using the secure key and the initialization vector.
Backward compatibility of the agent password is

supported.
28
A 256 bit encryption key is generated using the openssl APIs, and stored in the main.cf.
Then a randomized initialization vector is generated, which also gets stored in the
main.cf. The vscencrypt utility encrypts the password using the secure key and the
initialization vector. Backward compatibility of the agent password is supported.
Example:
• A user already uses AES 128 bit encryption and upgrades to VCS 7.3.1.
• The agent will be able to decrypt the passwords.
• But once the user decides to generate a new 256 bit secure key, they have to encrypt the
agent with the 256 bit key.

04-28
AES 256 encryption
1 Generate the 256 bit symmetric key which would be used for encryption and decryption of passwords.
2 This key is generated by accepting a passphrase from the customer and appending it with fixed value
buffer.
3 This value is now hashed, obfuscated, converted to hexadecimal and is dumped in the
configuration file.
4 When a user tries to login, they are prompted for a password.
5 An ipm connection is opened with engine and the encrypted password is sent to HAD.
6 HAD decrypts this password and the password corresponding to the user in the configuration file.
7 If these passwords match, the user can access the Cluster.
29
In versions prior to 7.4.2, VCS user passwords used home-grown algorithms which were weak
and susceptible to dictionary attacks. In the InfoScale version 7.4.2, the home-grown
algorithm is dismissed. Both User and Agent Passwords are encrypted using standard AES 256
(Advanced Encryption Standard) algorithm. As there were no prerequisites to AES 256
encryption implementation, systems and licenses requirements for InfoScale Availability were
same.
1. Generate the 256 bit symmetric key which would be used for encryption and decryption
of passwords.
2. This key is generated by accepting a passphrase from the customer and appending it with
fixed value buffer.
3. This value is now hashed, obfuscated, converted to hexadecimal, and is dumped in the
configuration file.
4. When a user tries to login, they prompted for a password.
5. An ipm connection is opened with the engine and the encrypted password is sent to HAD.
6. HAD decrypts this password and the password corresponding to the user in the
configuration file.
7. If these passwords match, the user can access the Cluster.

04-29
Generating the 256-bit secure key
• The 256-bit secure key used for encryption must be generated manually:
vcsencrypt -gensecinfo
Please enter a passphrase of minimum 8 characters.
Passphrase:
SecInfo generated successfully. Trying to update its value in config file
• User and Agent passwords can also be encrypted using the “vcsencrypt” utility and
manually added to the configuration:
User Agent
Password vcsencrypt -vcs vcsencrypt -agent
30
• The 256-bit secure key used for encryption must be generated manually by using the
following command:
vcsencrypt -gensecinfo
Please enter a passphrase of minimum 8 characters.
Passphrase:
SecInfo generated successfully.
Trying to update its value in config file.
• User and Agent passwords can also be encrypted using the “vcsencrypt” utility and
manually added to the configuration:

— User password:
vcsencrypt -vcs
— Agent password:
vcsencrypt -agent

04-30
Lesson summary
• Key points
– In this lesson, you learned to change the default VCS shutdown behavior using Cluster attributes and
back up the cluster configuration.
– In addition, you learned about online configuration that enables you to keep VCS running while making
configuration changes.
– Finally, you learned VCS user and agent account passwords encryption using AES 256.
– Veritas Cluster Server Command Line Quick Reference
31

04-31
Lab 05: VCS Configuration Methods
• Exercise A: VCS configuration state and stopping VCS
• Exercise B: Configuring automatic backup of the VCS configuration
• Exercise C: Setting non default VCS stop options
32

04-32
What did you learn?
lesson.
33
current lesson.

04-33
Question 1: Online configuration
The main.cf file is the configuration file used by _______ at startup.
A. GAB
B. LLT
C. hashadow
D. had
34

04-34
Answer 1: Online configuration
The main.cf file is the configuration file used by _______ at startup.
A. GAB
B. LLT
C. hashadow
D. had
The correct answer is D. HAD daemon reads cluster configuration from the main.cf file at startup.
35

04-35
Question 2: Controlling access to VCS
You have decided that each application manager is to have full access to only their
service groups. Which VCS user category would you assign to each application manager?
A. Cluster Operator
B. Group Administrator
C. Group Operator
D. Service Administrator
36

04-36
Answer 2: Controlling access to VCS
You have decided that each application manager is to have full access to only their
service groups. Which VCS user category would you assign to each application manager?
A. Cluster Operator
B. Group Administrator
C. Group Operator
D. Service Administrator
The correct answer is B. Group administration user will limit the full access to the specific service group.
37

04-37
Question 3: Controlling access to VCS
In nonsecure mode, you would like to provide read-only access using the VCS Java GUI to
users who are in the UNIX group named vcsreaders. How would you do this?
A. Change the UNIX group ownership for hagui to vcsreaders.

B. VCS GUIs cannot be restricted to read-only access.
C. Add each user to VCS in the Cluster Guest category.
D. These users must use Web-based access to VCS.
38

04-38
Answer 3: Controlling access to VCS
In nonsecure mode, you would like to provide read-only access using the VCS Java GUI to
users who are in the UNIX group named vcsreaders. How would you do this?
A. Change the UNIX group ownership for hagui to vcsreaders.

B. VCS GUIs cannot be restricted to read-only access.
C. Add each user to VCS in the Cluster Guest category.
D. These users must use Web-based access to VCS.
The correct answer is C. Assigning each user to Cluster Guest VCS user access.
39

04-39
Question 4: Starting and stopping VCS
You want to stop VCS on one system in a two-node cluster, but leave applications
running. Which command line do you use?
A. hastop -force
B. hastop -migrate
C. hastop -local -evacuate
D. hastop –local -force
40

04-40
Answer 4: Starting and stopping VCS
You want to stop VCS on one system in a two-node cluster, but leave applications
running. Which command line do you use?
A. hastop -force
B. hastop -migrate
C. hastop -local -evacuate
D. hastop –local -force
The correct answer is D. -force option enables the continuity of the applications even though HAD is stopped locally
on that node.
41

04-41
Question 5: Online configuration
If you want to be warned that the configuration is open if you try to stop VCS:
A. Set EngineShutdown to Disable.

B. Do not use the –force option to hastop.
C. Set ReadOnly to No.
D. Use the –warn option to hastop.
42

04-42
Answer 5: Online configuration
If you want to be warned that the configuration is open if you try to stop VCS:
A. Set EngineShutdown to Disable.

B. Do not use the –force option to hastop.
C. Set ReadOnly to No.
D. Use the –warn option to hastop.
The correct answer is B. You need to make sure that the configuration is in read-only mode, if you try to perform
hastop without –force option, it will throw an error message asking you to change the configuration to read-only.
But with –force option it doesn’t perform such check.
43

04-43
Question 6: Starting and stopping VCS
Default value of the $VCS_STOP_TIMEOUT variable is 0.
A. True
B. False
44

04-44
Answer 6: Starting and stopping VCS
Default value of the $VCS_STOP_TIMEOUT variable is 0.
A. True
B. False
The correct answer is A. Default value of the $VCS_STOP_TIMEOUT variable is 0.
45

04-45
04-46
Lesson 05: Preparing Services for VCS
This is the Preparing Services for VCS lesson in the Veritas InfoScale 7.4.2 Fundamentals
for UNIX/Linux: Administration course.

06-1
Administration
Administration


06-2
Lesson objectives
Topic Objective
Preparing applications for VCS Prepare applications for the VCS environment.
Performing one-time configuration

Perform one-time configuration tasks.
tasks
Testing the application service Test the application services before placing them under VCS control.
Stopping and migrating a service Stop resources and manually migrate a service.
Collecting configuration information Collect configuration information.

06-3
Topic: Preparing applications for VCS
After completing this topic, you will be able to prepare applications for the
VCS environment.
This is the Preparing applications for VCS topic.


06-4
Application service overview
Application
Process
Storage
File system
Volume
Network Disk group
s1
IP address 10.10.21.198
NIC
An application service is the service that the end-user perceives when accessing a particular
network address. An application service typically consists of multiple components, some
hardware- and some software-based, all cooperating together to produce a service.
For example, a service can include application software (processes), a file system containing
data files, a physical disk on which the file system resides, one or more IP addresses, and a
NIC for network access.
If this application service needs to be migrated to another system for recovery purposes, all of
the components that compose the service must migrate together to re-create the service on
another system.

06-5
Identifying components
Application
Storage
Process
Network
IP Mount
10.10.21.198
NIC Volume
DiskGroup
The first step in preparing services to be managed by VCS is to identify the components
required to support the services. These components should be itemized in your design
worksheet and may include the following, depending on the requirements of your application
services:
Shared storage resources:
• Disks or components of a logical volume manager, such as Volume Manager disk groups
and volumes
• File systems to be mounted
• Directory mount points
Network-related resources:
• IP addresses
• Network interfaces
Application-related resources:
• Identical installation and configuration procedures completed on each cluster node
• Procedures to manage and monitor the application
• The location of application binary and data files
The following sections describe the aspects of these components that are critical to
understanding how VCS manages resources.

06-6
Topic: Performing one-time configuration tasks
After completing this topic, you will be able to perform one-time

configuration tasks.
This is the Performing one-time configuration tasks topic.


06-7
Configuration and migration procedure
Before configuring VCS in production environments:
• Fully test each service on each startup or failover target system.
• Configure services on a test cluster, if possible.
Perform one-time
configuration tasks.
Start, verify, and

More Ready for
stop services on
systems? VCS
one system at a time.
Use the procedure shown in the diagram to prepare and test application services on each
system before placing the service under VCS control. Consider using a design worksheet to
obtain and record information about the service group and each resource. This is the
information you need to configure VCS to control these resources.
Details are provided in the following section.

06-8
Documenting attributes
As you configure components:
• Use a design worksheet to document details needed to configure VCS resources.
• Note differences among systems, such as network interface device names.
Resource definition Value

Service Group Name appsg
appip
Resource Name appip appproc
Resource Type IP
Required attributes appip appmnt
Device e1000g0
eth0:1
en1
lan2
Address 10.10.21.198 appnic appvol
Optional attributes
appdg
NetMask 255.0.0.0
In order to configure the operating system resources you have identified as requirements for
an application, you need the detailed configuration information used when initially
configuring and testing services.
You can use a design diagram and worksheet while performing one-time configuration tasks
and testing to:
• Show the relationships between the resources, which determine the order in which you
configure, start, and stop resources.
• Document the values needed to configure VCS resources after testing is complete.
Note: If your systems are not configured identically, you must note those differences in the
design worksheet. The “Online Configuration” lesson shows how you can configure a resource
with different attribute values for different systems.

06-9
Checking resource attributes
While documenting the configuration, verify that:
• Recorded attribute values match configured values
• All required attributes have values

appd
Service Group Name appsg
Resource Name appmnt
10… /appdata
Resource Type Mount
appvol Required attributes

lan2
MountPoint /appmnt
appdg BlockDevice /dev/vx/dsk. . .
FSType vxfs
FsckOpt -y
10
Verify that the resources specified in your design worksheet are appropriate and complete for
your platform. Refer to the Veritas Cluster Server Bundled Agents Reference Guide before you
begin configuring resources.
The examples displayed in the slides in this lesson show values for various operating system
platforms, indicated by the icons. In the case of the appsg service group shown in the slide,
the lan2 value of the Device attribute for the NIC resource is specific to HP-UX. Solaris, Linux,
and AIX have other operating system-specific values, as shown in the respective Bundled
Agents Reference Guides.

06-10
Configuring shared storage
Initialize disks. vxdisksetup –i emc0_dd5

From one system
Create a disk group. vxdg init appdatadg appdatadg01=emc0_dd5
Create a volume. vxassist -g appdatadg make appdatavol 1g
-V
Make a file system. mkfs -t
–F vxfs /dev/vx/rdsk/appdatadg/appdatavol
Make mount point. mkdir /appdata Each system
11
The diagram shows the procedure for configuring shared storage on the initial system. In this
example, Volume Manager is used to manage shared storage.
Note: Although examples used throughout this course are based on Veritas Volume Manager,
VCS also supports other volume managers. VxVM is shown for simplicity—objects and
commands are essentially the same on all platforms. The agents for other volume managers
are described in the Veritas Cluster Server Bundled Agents Reference Guide.
Preparing shared storage, such as creating disk groups, volumes, and file systems, is
performed once, from one system. Then you must create mount point directories on each
system.
The options to mkfs differ depending on platform type, as displayed in the following
examples.
AIX
mkfs -V vxfs /dev/vx/rdsk/appdatadg/appdatavol
Linux
mkfs -t vxfs /dev/vx/rdsk/appdatadg/appdatavol
Solaris/HP-UX
mkfs -F vxfs /dev/vx/rdsk/appdatadg/appdatavol

06-11
Configuring the application
Install and configure applications identically on each
target system.
• Determine file locations:
Service group name appsg
– Shared or local storage
Resource name appproc
– Binaries, data, configuration
Resource type Process
• Follow application installation and
configuration guidelines. Required attributes
• Identify startup, monitor, and PathName /appdata/appd
shutdown procedures. Optional attributes
Arguments start

Some applications have VCS-specific installation and configuration
requirements, described in the VCS agent guides for those
applications, such as Oracle.
12
You must ensure that the application is installed and configured identically on each system
that is a startup or failover target and manually test the application after all dependent
resources are configured and running.
Some VCS agents have application-specific installation instructions to ensure the application is
installed and configured properly for a cluster environment. Check the Veritas Services and
Operations Readiness Tools (SORT) Web site for application-specific guides, such as the Veritas
Cluster Server Agent for Oracle Installation and Configuration Guide.
Depending on the application requirements, you may need to
• Create user accounts.
• Configure environment variables.

• Apply licenses.
• Set up configuration files.
This ensures that you have correctly identified the information used by the VCS agent scripts
to control the application.
Note: The shutdown procedure should be a graceful stop, which performs any cleanup
operations.

06-12
Topic: Testing the application service
After completing this topic, you will be able to test the application services
before placing them under VCS control.
13
This is the Testing the application service topic.


06-13
Testing the application service
s1
Bring up resources.
Ready for
VCS
Start up all resources More
in dependency order. systems?
Shared storage
s2 . . . sn
Virtual IP address
Stop resources.
Application software
Test the application.

Test the application.
Stop resources. Bring up resources.
14
Before configuring a service group in VCS to manage an application, test the application
components on each system that can be a startup or failover target for the service group.
Following this best practice recommendation ensures that VCS can successfully manage the
application service after you configure a service group to manage the application.
The testing procedure emulates how VCS manages application services and must include:
• Startup: Online
• Shutdown: Offline
• Verification: Monitor
The actual commands used may differ from those used in this lesson. However, conceptually,
the same type of action is performed by VCS. Example operations are described for each
component throughout this section.

06-14
Bringing up resources: Online
To bring shared storage resources online:

1
Import the disk group:
vxdg –t import appdatadg
21 Start the volume, if autorecovery has been disabled (and for pre-6.0 VCS).
vxvol –g appdatadg start appdatavol
31 Mount the file system:

mount -V–F vxfs /dev/vx/dsk/appdatadg/appdatavol /appdata
-t
• Do not automount file systems controlled by VCS.
! • Example: Verify that the /etc/fstab (Linux) file has no

application related entries.
15
Verify that shared storage resources are configured properly and accessible. The examples
shown in the slide are based on using Volume Manager.
1. Import the disk group.
2. Start the volume.
3. Mount the file system.
Mount the file system manually for the purposes of testing the application service. Do not
configure the operating system to automatically mount any file system that will be controlled
by VCS.
Examples of mount commands are provided for each platform.
AIX
mount -V vxfs /dev/vx/dsk/appdatadg/appdatavol /appdata
Linux
mount -t vxfs /dev/vx/dsk/appdatadg/appdatavol /appdata
Solaris/HP-UX
mount -F vxfs /dev/vx/dsk/appdatadg/appdatavol /appdata

06-15
Virtual IP addresses
http://eweb.com
DNS
eweb.com = 10.10.21.198 Admin IP on s2 e1000g0

configured at boot
10.10.21.9
ifconfig e1000g0 addif 10.10.21.198 up
Virtual IP configured by IP resource e1000g0:1

s2
Admin IP on s1 e1000g0
configured at boot s1
10.10.21.8
16
The example in the slide demonstrates how users access services through a virtual IP address
that is specific to an application. In this scenario, VCS is managing a Web server that is
accessible to network clients over a public network.
1. A network client requests access to http://eweb.com.
2. The DNS server translates the host name to the virtual IP address of the Web server.
3. The virtual IP address is managed and monitored by a VCS IP resource in the Web service
group. The virtual IP address is associated with the next virtual network interface for
e1000g0, which is e1000g0:1 in this example of Solaris network interfaces.
4. The system which has the service group online accepts the incoming request on the
virtual IP address.
Note: The administrative IP address is associated with a physical network interface on a
specific system and is configured by the operating system during system startup. These are
also referred to as base or test IP addresses.

06-16
Virtual IP address migration
http://eweb.com
DNS
Virtual IP configured by IP resource e1000g0:1
eweb.com = 10.10.21.198 Admin IP on s2 e1000g0

configured at boot
10.10.21.9
s2
! Do not place admin IP addresses

under VCS control.
s1
17
The diagram in the slide shows what happens if the system running the Web service group
(s1) fails.
1. The IP address is no longer available on the network. Network clients may receive errors
that web pages are not accessible.
2. VCS on the running system (s2) detects the failure and starts the service group.
3. The IP resource is brought online, which configures the same virtual IP address on the
next available virtual network interface alias, e1000g:1 in this example. This virtual IP
address floats, or migrates, with the service. It is not tied to a system.
4. The network client Web request is now accepted by the s2 system.
Note: The admin IP address on s2 is also configured during system startup. This address is
unique and associated with only this system, unlike the virtual IP address.
CAUTION: The administrative IP address cannot be placed under VCS control. This address
must be configured by the operating system. Ensure that you do not configure an IP resource
with the value of the administrative IP address.

06-17
Bringing up application (virtual) IP addresses
An application IP address is:
• Added as virtual IP address to a virtual public network device.
• Associated with an application service.
• Resolved by a naming service.
• Controlled by the high availability software.
• Migrated to other systems by VCS.
• Also called service group or floating IP addresses.
18
Configure the application IP addresses associated with specific application services to ensure
that clients can access the application service using the specified address.
Application IP addresses are configured as virtual IP addresses. On most platforms, the
devices used for virtual IP addresses are defined as interface:number.
Note: These virtual IP addresses are only configured temporarily for testing purposes. You
must not configure the operating system to manage the virtual IP addresses.
The following examples show the platform-specific commands used to configure a virtual IP
address for testing purposes.
• AIX
Create an alias for the virtual interface and bring up the IP on the next available logical
interface.
ifconfig en1 inet 10.10.21.198 netmask 255.0.0.0 alias
• HP-UX
1. Configure IP address using the ifconfig command.
ifconfig lan2:1 inet 10.10.21.198
2. Use ifconfig to manually configure the IP address to test the configuration without
rebooting.
ifconfig lan2:1 up

06-18
Linux
• Configure IP address using the ip addr command.
• ip addr add 10.10.21.198/24 broadcast 10.255.255.255 \
dev eth0 label eth0:1
• ip addr show eth0
Solaris
• Plumb the virtual interface and bring up the IP address on the next available logical
interface.
Note: In each case, you can edit /etc/hosts to assign a virtual host name (application name) to
the virtual IP address.
10.10.21.198 eweb.com

06-19
Starting the application
• Manually start the application for testing purposes.
Manual startup for an example process daemon:

/appdata/appd start
• Ensure the operating system does not control application startup
Example autostart methods:

• AIX: /etc/rc.d/rc2.d files
• HP-UX: /sbin/rc2.d files
• Linux: chkconfig
• Solaris 10: svcadm
20
When all dependent resources are available, you can start the application software. Ensure
that the application is not configured to start automatically during system boot. VCS must be
able to start and stop the application using the same methods you use to control the
application manually.
Examples of operating system control of applications:
On AIX and HP-UX, rc files may be present if the application is under operating system
control.
On Linux, you can use the chkconfig command to determine if an application is under
operating system control.
On Solaris 10 platforms, you must disable the Service Management Facility (SMF) using the
svcadm command for some services, such as Apache, to ensure that SMF is not trying to
control the service.
Follow the guidelines for your platform to remove an application from operating system
control in preparation for configuring VCS to control the application.

06-20
Verifying resources: Monitor
Verify the disk group. vxdg list appdatadg
Verify the volume. vxinfo –g appdatadg –p
Verify the file system. mount | grep /appdata
Verify the NIC. ping 10.10.21.220; ping 10.10.21.230
Verify the virtual IP. netstat –in

ifconfig –a
Verify the application. ps -ef | grep "appd"
21
You can perform some simple steps, such as those shown in the slide, to verify that each
component needed for the application to function is operating at a basic level.
Note: To test the network resources, access one or more well-known addresses outside of the
cluster, such as local routers, or primary and secondary DNS servers.
This helps you identify any potential configuration problems before you test the service as a
whole, as described in the “Testing the Integrated Components” slide.

06-21
Testing the integrated components
Test the application real-world scenarios.
• Involve key users of the application.
• Test network connectivity from client systems on different subnets.
10.10.21.198
s2
s1
22
When all components of the service are running, test the service in situations that simulate
real-world use of the service.
For example, if you have an application with a backend database, you can:
1. Start the database (and listener process).
2. Start the application.
3. Connect to the application from the public network using the client software to verify
name resolution to the virtual IP address.
4. Perform user tasks, as applicable; perform queries, make updates, and run reports.
Another example that illustrates how you can test your service uses Network File System
(NFS). If you are preparing to configure a service group to manage an exported file system,
verify that you can mount the exported file system from a client on the network.

06-22
Topic: Stopping and migrating a service
After completing this topic, you will be able to stop resources and manually
migrate a service.
23
This is the Stopping and migrating a service topic.


06-23
Stopping application components: Offline
Stop resources in
order from top down.
Stop the application.
kill opts PID
Take down the virtual IP. Unmount the file system.
ifconfig e1000g0 removeif 10.10... umount /appdata
Stop the volume.
vxvol –g appdatadg stop appdatavol
Deport the disk group.
! Ensure application is not

automatically shut down by OS. vxdg deport appdatadg
24
Stop resources in the order of the dependency tree from the top down after you have finished
testing the service. You must have all resources offline in order to migrate the application
service to another system for testing. The procedure also illustrates how VCS stops resources.
The ifconfig options are platform-specific, as shown in the following examples.
AIX
ifconfig en1 10.10.21.198 delete
HP-UX
ifconfig lan2:1 0.0.0.0
Linux
ifdown eth0:1
Solaris
ifconfig e1000g0 removeif 10.10.21.198

06-24
Manually migrating the application service
Application
Process
Storage
File system Network

Volume
10.10.21.198
Disk group IP address
s2
NIC
s1
25
After you have verified that the application service works properly on one system, manually
migrate the service between all intended target systems. Performing these operations enables
you to:
• Ensure that your operating system and application resources are properly configured on all
potential target cluster systems.
• Validate or complete your design worksheet to document the information required to
configure VCS to manage the services.
Perform the same type of testing used to validate the resources on the initial system,
including real-world scenarios, such client access from the network.

06-25
Topic: Collecting configuration information
After completing this topic, you will be able to stop resources and manually
migrate a service.
26
This is the Collecting configuration information topic.


06-26
Documenting resource dependencies
• Determine parent and child relationships according to service group diagrams.
• Verify that the dependencies match the online and offline tests you performed.
appsg appproc
Resource dependency definition
appip appmnt Service group appsg
Parent resource Requires Child resource
appnic appvol appvol appdg
appmnt appvol
appdg appip appnic
appproc appmnt
appproc appip
27
Ensure that the steps you perform to bring resources online and take them offline while
testing the service are accurately reflected in a design worksheet. Compare the worksheet
with service group diagrams you have created or that have been provided to you.
The slide shows the resource dependency definition for the application used as an example in
this lesson.

06-27
Validating service group attributes
Service group definition Value

Failover
appsg Group appsg
system
Required Attributes
SystemList s1=0, s2=1
s2 Optional Attributes
s1 AutoStartList s1, s2
Parallel 0
Startup
system
28
Check the service group attributes in your design worksheet to ensure that the appropriate
startup and failover systems are listed. Other service group attributes may be included in your
design worksheet, according to the requirements of each service.
Service group definitions consist of the attributes of a particular service group. These
attributes are described in more detail later in the course.

06-28
Lesson summary
• Key points
– In this lesson, you learned to prepare each component of a service and document attributes.
– Finally, you learned to test services in preparation for configuring VCS service groups.
29

06-29
Lab 06: Preparing Services for VCS
• Exercise A: Configuring and examining storage for the service
• Exercise B: Examining the application
• Exercise C: Manually starting and stopping the application
30

06-30
What did you learn?
lesson.
31
current lesson.

06-31
Question 1: Preparing applications for VCS
A recommended task in preparing application services for VCS is:
A. Starting and stopping the service under VCS control before creating the SystemList
B. Making sure all resources in the service group can be probed
C. Manually testing each service on each system that is a startup or failover target
before placing the service under VCS control
D. Testing all services on the same system before placing under VCS control
32

06-32
Answer 1: Preparing applications for VCS
A recommended task in preparing application services for VCS is:
A. Starting and stopping the service under VCS control before creating the SystemList
B. Making sure all resources in the service group can be probed
C. Manually testing each service on each system that is a startup or failover target
before placing the service under VCS control
D. Testing all services on the same system before placing under VCS control
The correct answer is C. As a best practice, you should manually test the application startup and stop procedures on all
the cluster nodes, before you put it under VCS.
33

06-33
Question 2: Testing the application service
When testing resource components, start the components:
A. In any order; the order for starting resources is irrelevant

B. Based on whether they are storage or network resources
C. According to the inherent component dependencies
D. From the top of the resource dependency tree
34

06-34
Answer 2: Testing the application service
When testing resource components, start the components:
A. In any order; the order for starting resources is irrelevant

B. Based on whether they are storage or network resources
C. According to the inherent component dependencies
D. From the top of the resource dependency tree
The correct answer is C. Follow the dependency order of application components while starting or stopping the related
resources components.
35

06-35
Question 3 Testing the application service
In VCS, application IP addresses are added as virtual IP addresses to the network
interface, and are:
A. Configured to come up outside of VCS control when the application comes up

B. Migrated to other systems if the current system or application service fails
C. The same as the Administrative IP address for each system
D. Configured to come up when the operating system is started
36

06-36
In VCS, application IP addresses are added as virtual IP addresses to the network
interface, and are:
A. Configured to come up outside of VCS control when the application comes up

B. Migrated to other systems if the current system or application service fails
C. The same as the Administrative IP address for each system
D. Configured to come up when the operating system is started
The correct answer is B. Application IP resource is a required key components for the accessibility of the application by
the outside world so it needs to be migrated to the failover system.
37

06-37
Why is an administrative IP address required for some operating system platforms if HA
services require network access?
A. The application (virtual) IP address cannot be configured without an administrative

IP address.
B. The administrative IP address migrates between systems during failover.
C. The Java GUI uses the administrative IP address to initiate failover.
D. VCS updates the DNS server with the administrative IP address during failover.
38

06-38
Why is an administrative IP address required for some operating system platforms if HA
services require network access?
A. The application (virtual) IP address cannot be configured without an administrative

IP address.
B. The administrative IP address migrates between systems during failover.
C. The Java GUI uses the administrative IP address to initiate failover.
D. VCS updates the DNS server with the administrative IP address during failover.
The correct answer is A. As the application IP is a virtual IP hence it needs an already existing IP address i.e.
administrative IP address on the NIC as the base IP.
39

06-39
Question 5: Performing one-time configuration tasks
If IMF is configured online monitoring, what does the agent do after receiving
notification of unexpected offline from the AMF driver?
A. Creating the file system.

B. Creating a volume.
C. Making a disk group.
D. Making a mount point.
40

06-40
Answer 5: Performing one-time configuration tasks
A. Creating the file system.

B. Creating a volume.
C. Making a disk group.
D. Making a mount point.
The correct answer is D. A mount point needs to be manually created on each system.
41

06-41
Why would you avoid specifying the administrative IP address in the IP resource
definition?
A. Failover of the resource would remove the admin IP addresses.

B. IP resources can only be used with 10.x.x.x type addresses.
C. IP resources can only be used with non-routable IP addresses.
D. Administrative IP addresses must be IPv6 format.
42

06-42
Why would you avoid specifying the administrative IP address in the IP resource
definition?
A. Failover of the resource would remove the admin IP addresses.

B. IP resources can only be used with 10.x.x.x type addresses.
C. IP resources can only be used with non-routable IP addresses.
D. Administrative IP addresses must be IPv6 format.
The correct answer is A. Administrative IP address is configured at OS level and configuring the same under VCS will
remove it during the failover of that service group to another system.
43

06-43
Appendix
presentation now.
44
This is an Appendix topic.


06-44
Configuring an application IP address on AIX
To set up manually for testing purposes:
1. Create an alias for the virtual interface and bring up the IP on the next available logical interface.
ifconfig en1 inet 10.10.21.198 netmask 255.0.0.0 alias
2. Edit /etc/hosts to assign a virtual hostname (application service name) to the IP address.
10.10.21.198 eweb.com
45

06-45
Configuring an application IP address on HP-UX
1. Configure IP address using the ifconfig command.
ifconfig lan2:1 inet 10.10.21.198
2. Use ifconfig to manually configure the IP address to test the configuration without rebooting.
ifconfig lan2:1 up
3. Edit /etc/hosts and assign a virtual host name (application service name) to the virtual IP address.
10.10.21.198 eweb.com
46

06-46
Configuring an application IP address on Linux
1. Configure the IP address using the ip addr command.
ip addr add 10.10.21.198/24 broadcast \

10.255.255.255 dev eth0 label eth0:1
ip addr show eth0
2. Edit /etc/hosts to assign a virtual host name (application service name) to the IP address.
10.10.21.198 eweb.com
47

06-47
Configuring an application IP address on Solaris
1. Plumb the virtual interface and bring up the IP on the next available logical interface.
2. Edit /etc/hosts to assign a virtual hostname (application service name) to the IP address.
10.10.21.198 eweb.com
48

06-48
Configuring an administrative IP address
For public network access to high availability services, you must configure an administrative IP address associated with
the physical network interface.
• Each system needs a unique administrative IP address for each interface.
• Configure the operating system to bring up the administrative IP address during system boot.
• The IP addresses are used by VCS to monitor network interfaces.
• These addresses are also sometimes referred to as base, maintenance, or test IP addresses.
The administrative IP address may already be configured and only needs to be verified.
Procedure AIX HP-UX Linux Solaris
49

06-49
Configuring an administrative IP address on AIX
1. Use SMIT or mktcpip to configure the IP address to come up during system boot.
2. Edit /etc/hosts and assign an IP address to the interface name.
10.10.21.8 s1_en1
3. Use ifconfig to manually configure the IP address or it will be configured during the next reboot.
ifconfig en1 inet 10.10.21.8 netmask 255.0.0.0 +
ifconfig en1 up
50

06-50
Configuring an administrative IP address on HP-UX
1. Add an entry in /etc/rc.config.d/netconf to include the configuration information for the interface.
INTERFACE_NAME[0]=lan2
IP_ADDRESS[0]=10.10.21.8
SUBNET_MASK[0]=“255.0.0.0”
BROADCAST_ADDRESS[0]=“”
DHCP_ENABLE[0]=“0”
10.10.21.8 s1_lan2
3. Use ifconfig to manually configure the IP address to test the configuration without rebooting:
ifconfig lan2 inet 10.10.21.8 netmask 255.0.0.0
ifconfig lan2 up
51

06-51
Configuring an administrative IP address on Linux
1. Add an entry in the appropriate file in /etc/sysconfig/network-scripts/ to include the interface.
# cd /etc/sysconfig/network-scripts
# ls
ifcfg-eth0 ifcfg-eth1 ifcfg-eth2 ifcfg-eth3
# more ifcfg-eth0
DEVICE=eth0
BOOTPROTO=static
BROADCAST=10.255.255.255
IPADDR=10.10.21.8
NETMASK=255.0.0.0
NETWORK=10.10.21.0
. . .
10.10.21.8 s1_lan2
ifconfig eth2 10.10.21.8 netmask 255.0.0.0
52

06-52
Configuring an administrative IP address on Solaris
1. Create /etc/hostname.interface with the desired interface name so the IP address is configured during
system boot:
s1_e1000g0
10.10.21.8 s1_e1000g0
ifconfig e1000g0 inet 10.10.21.8 netmask 255.0.0.0 +
ifconfig e1000g0 up
53

06-53
End of presentation
06-54
06-55
Lesson 06: Online Configuration
This is the Online Configuration lesson in the Veritas InfoScale 7.4.2 Fundamentals for

06-1
Administration
Administration


06-2
Lesson objectives
Topic Objective
Online service group configuration Configure an online configuration procedure.
Adding resources Create resources using online configuration tools.
Solving common configuration

Resolve common errors made during online configuration.
errors
Testing the service group Test the service group to ensure proper configuration.

06-3
Topic: Online service group configuration
After completing this topic, you will be able to configure an online

configuration procedure.
This is the Online service group configuration topic.


06-4
Online configuration procedure
Open the cluster configuration Add service group
Modify the cluster configuration Set SystemList

using the GUI or CLI
Set opt attributes
Save and close the configuration
Add/test resources
Resource flow chart
This procedure assumes:

• You have prepared and tested the
application service on each system More? Test
• Resources are offline everywhere
The diagram illustrates a high-level procedure that can be used as a standard methodology to
create service groups and resources while the VCS is running. There are multiple ways to use
this configuration procedure, but following a recommended practice simplifies and
streamlines the initial configuration and facilitates troubleshooting if you encounter problems
during configuration.

06-5
Naming convention suggestions
• Choose a prefix based on purpose of application
• Match prefixes used for resources in a service group
• Take care when using delimiters
Dash (-) and underscore (_) are problematic on some keyboards
Conventions Examples
• <unique_id>_<VCS_type>_<instance> • sap_nic_lan1, sap_nic_lan2
• <unique_id><VCS_type><instance> • paydbmountredo, paydbmountarch
hares –state | grep paydb

. . .
paydbmountredo State s1 ONLINE
paydbmountarch State s1 ONLINE
Using a consistent pattern for selecting names for VCS objects simplifies initial configuration
of high availability. Perhaps more importantly, applying a naming convention helps avoid
administrator errors and significantly reduces troubleshooting efforts when errors or faults
occur.
As displayed on the slide, Veritas recommends the use of the pattern based on the function of
the service group, and match some portion of the name among all resources and the service
group in which the resources are contained.
When deciding upon a naming convention, consider delimiters, such as dash (-) and
underscore (_), with care. Differences in keyboards may prevent use of some characters,
especially in the case where clusters span geographic locations.

06-6
Adding a service group using the GUI
main.cf
haconf –makerw group appsg (

hagrp –add appsg SystemList = {s1 = 0, s2 = 1}
hagrp –modify appsg SystemList s1 0 s2 1
AutoStartList = {s1, s2}
hagrp –modify appsg AutoStartList s1 s2
)
haconf –dump -makero
The minimum required information to create a service group is:

• A unique name: Using a consistent naming scheme helps identify the purpose of the
service group and all associated resources.
• The list of systems on which the service group can run
The SystemList attribute for the service group defines where the service group can run, as
displayed in the excerpt from the sample main.cf file. A priority number is associated
with each system to determine the order systems are selected for failover. The lower-
numbered system is selected first.
• The list of systems where the service group can be started
The Startup box specifies that the service group starts automatically when VCS starts on
the system, if the service group is not already online elsewhere in the cluster. This is
defined by the AutoStartList attribute of the service group. In the example displayed in
the slide, the s1 system is selected as the system on which appsg is started when VCS
starts up.
• The type of service group
The Service Group Type selection is Failover by default.
If you save the configuration after creating the service group, you can view the main.cf file
to see the effect of HAD modifying the configuration and writing the changes to the local disk.
Note: You can click the Show Command button to see the commands that are run when you
click OK.

06-7
Adding a service group using the CLI
You can also use the VCS command-line interface to modify a running cluster configuration.
The next example shows how to use hagrp commands to add the appsg service group and
modify its attributes.
haconf –makerw
hagrp –add appsg
hagrp –modify appsg SystemList s1 0 s2 1
hagrp –modify appsg AutoStartList s1 s2
haconf –dump –makero
The corresponding main.cf excerpt for appsg is shown in the slide.
Notice that the main.cf definition for the appsg service group does not include the Parallel
attribute. When a default value is specified for a resource, the attribute is not written to the
main.cf file. To display all values for all attributes:
• In the GUI, select the object (resource, service group, system, or cluster), click the
Properties tag, and click Show all attributes.
• From the command line, use the -display option to the corresponding ha command.
For example:
hagrp -display appsg
See the command-line reference card provided with this course for a list of commonly used
ha commands.

06-8
Topic: Adding resources
After completing this topic, you will be able to create resources using online
configuration tools.
This is the Adding resources topic.


06-9
Online resource configuration procedure
Add resource
Considerations:
Set non-critical • Add resources in order of dependency, starting at the bottom.
• Set each resource to non-critical until testing has completed.
Modify attributes • Configure all required attributes.
• Enable the resource.
• Bring each resource online before adding the next resource.
Enable resource
Bring online
Online? Troubleshoot resources
Done
10
Add resources to a service group in the order of resource dependencies starting from the
child resource (bottom up). This enables each resource to be tested as it is added to the
service group.
While adding a resource, you need to specify:
• The service group name
• The unique resource name
If you prefix the resource name with the service group name, you can more easily identify
the service group to which it belongs. When you display a list of resources from the
command line using the hares -list command, the resources are sorted
alphabetically.
• The resource type
• Attribute values
Use the procedure shown in the diagram to configure a resource.
Notes:
• It is recommended that you set each resource to be non-critical during initial
configuration. This simplifies testing and troubleshooting in the event that you have
specified incorrect configuration information. If a resource faults due to a configuration
error, the service group does not fail over if resources are non-critical.
• Enabling a resource signals the agent to start monitoring the resource.

06-10
Adding a resource using the GUI: NIC example
• NIC monitoring methods depend on

UNIX platform
• Device is only required attribute for
Solaris, Linux, AIX
HP-UX requires PingOptimize or
NetworkHosts
• Administrative IP address must be NIC appnic (
configured first outside of VCS Critical = 0 main.cf
Device = eth0
)
11
The NIC resource has only one required attribute, Device, for all platforms other than HP-UX,
which also requires NetworkHosts unless PingOptimize is set to 0.
Optional attributes for NIC vary by platform. Refer to the Veritas Cluster Server Bundled
Agents Reference Guide for a complete definition. These optional attributes are common to all
platforms.
• NetworkType: Type of network, Ethernet (ether)
• PingOptimize: Number of monitor cycles to detect if the configured interface is inactive
A value of 1 optimizes broadcast pings and requires two monitor cycles. A value of 0
performs a broadcast ping during each monitor cycle and detects the inactive interface
within the cycle. The default is 1.

Note: On the HP-UX platform, if the PingOptimize attribute is set to 1, the monitor entry
point does not send broadcast pings.
• NetworkHosts: The list of hosts on the network that are used to determine if the network
connection is alive
It is recommended that you specify the IP address of the host rather than the host name
to prevent the monitor cycle from timing out due to DNS problems.
Example device attribute values: AIX: en0; HP-UX: lan2; Linux: eth0; Solaris: e1000g0

06-11
Adding an IP resource
Virtual IP addresses:
• Are configured by the agent using the
operating system-specific command
– AIX/HP-UX/Solaris: ifconfig
IP appip(
– Linux: ip addr Critical = 0 main.cf
• Must be different from the administrative Device = eth0
Address = "10.10.21.198“
IP address
NetMask: 255.255.255.0
)
12
The slide displays the attribute values for an IP resource (on Solaris) in the appsg service
group, reflected in the main.cf snippet.
The IP resource on Solaris has two required attributes: Device and Address, which specify the
network interface and virtual IP address, respectively. The required attributes vary depending
on the platform.
Optional attributes
• NetMask: Netmask associated with the application IP address
− The value may be specified in decimal (base 10) or hexadecimal (base 16). The default
is the netmask corresponding to the IP address class.
− It is a required attribute on AIX.

• Options: Options to be used with the ifconfig command
• ArpDelay: Number of seconds to sleep between configuring an interface and sending out a
broadcast to inform routers about this IP address; the default is 1 second
• IfconfigTwice: If set to 1, this attribute causes an IP address to be configured twice, using
an ifconfig up-down-up sequence. This behavior increases the probability of gratuitous
ARPs (caused by ifconfig up) reaching clients; the default is 0.

06-12
Adding a resource using commands: DiskGroup example
hares command: Add a resource and • Imports and deports a disk group
modify resource attributes. • Monitors the disk group using vxdg
haconf –makerw
hares –add appdg DiskGroup appsg
hares –modify appdg Critical 0
hares –modify appdg DiskGroup
appdatadg
hares –modify appdg Enabled 1 main.cf
haconf –dump –makero
DiskGroup appdg(
Critical = 0
DiskGroup = appdatadg
)
13
You can use the hares command to add a resource and configure the required attributes.
This example shows how to add a DiskGroup resource.
The DiskGroup resource has only one required attribute, DiskGroup.
Note: As of version 4.1, when a disk group is brought under VCS control, VCS sets the vxdg
autoimport flag to no, to ensure the disgroup is not autoimported during the system boot
process..
Example optional attributes:
• StartVolumes: Starts all volumes after importing the disk group
This also starts layered volumes by running vxrecover -s. The default is 1, enabled,
on all UNIX platforms except Linux.

• StopVolumes: Stops all volumes before deporting the disk group with vxvol
The default is 1, enabled, on all UNIX platforms except Linux.

06-13
The Volume resource
• Is used in case:
Service group name appsg
– VCS version is pre-6.0
Resource name appvol
– Autostart is disabled
Resource type Volume • Starts a volume using vxrecover and stops a
Required attributes volume using vxvol
Volume appdatavol • Reads a block from the raw device interface
DiskGroup appdatadg using dd to determine volume status
main.cf
Volume appvol(
Volume resources: Critical = 0
• Are not required; Volumes are started DiskGroup = appdatadg
automatically during disk group import Volume = appdatavol
• Provide additional monitoring )
14
The Volume resource can be used to manage a VxVM volume. Although the Volume resource
is not strictly required, it provides additional monitoring. You can use a DiskGroup resource to
start volumes when the DiskGroup resource is brought online. This has the effect of starting
volumes more quickly, but only the disk group is monitored.
If you have a large number of volumes on a single disk group, the DiskGroup resource can
time out when trying to start or stop all the volumes simultaneously. In this case, you can set
the StartVolume and StopVolume attributes of the DiskGroup to 0, and create Volume
resources to start the volumes individually.
Also, if you are using volumes as raw devices with no file systems, and, therefore, no Mount
resources, consider using Volume resources for the additional level of monitoring. The
Volume resource has no optional attributes.

06-14
The Mount resource
Service group name appsg • Mounts and unmounts a block device on
the directory
Resource name appmnt
• Runs fsck before remounting, if mount
Resource type Mount fails
Required attributes
MountPoint /appdata
BlockDevice /dev/vx/dsk/appdatadg/appdatavol
FSType vxfs
FsckOpt -y
main.cf
Mount appmnt(
CLI escape character Critical = 0
MountPoint = /appdata
hares –modify appmnt FsckOpt %-y BlockDevice = /dev/vx/dsk/appdatadg/appdatavol
FSType = vxfs
FsckOpt = -y
)
15
The Mount resource has the required attributes displayed in the main.cf file excerpt in the
slide.
Example optional attributes:
• MountOpt: Specifies options for the mount command
When setting attributes with arguments starting with a dash (-), use the percent (%)
character to escape the arguments. Examples:
hares -modify appmnt FsckOpt %-y
The percent character is an escape character for the VCS CLI which prevents VCS from
interpreting the string as an argument to hares.
• SnapUmount: Determines whether VxFS snapshots are unmounted when the file system is
taken offline (unmounted)
The default is 0, meaning that snapshots are not automatically unmounted when the file
system is unmounted.
Note: If SnapUmount is set to 0 and a VxFS snapshot of the file system is mounted, the
unmount operation fails when the resource is taken offline, and the service group is not
able to fail over.
This is desired behavior in some situations, such as when a backup is being performed
from the snapshot.

06-15
File system locking
• Prevents inadvertently unmounting file systems
• Enables a key lock to be placed on a file system
• Is the default behavior for the Mount resource
Operating system view

umount /appdata
umount -f /appdata
UX:vxfs umount: ERROR: V-3-26360: file system /appdata has been mount locked
umount -o mntunlock="VCS" /appdata
VCS view
hares -display appmnt -attribute VxFSMountLock
appmnt VxFSMountLock global 1
16
Storage Foundation enables a file system to be mounted with a key which must be used to
unmount the file system. The Mount resource has a VxFSMountLock attribute to manage the
file system mount key.
This attribute is set to the ‘VCS’ string by default when a Mount resource is added. The Mount
agent uses this key for online and offline operations to ensure the file system cannot be
inadvertently unmounted outside of VCS control.
You can unlock a file system without unmounting by using the fsadm command:
/opt/VRTS/bin/fsadm -o mntunlock="key" mount_point_name
Note: The example operating system commands for unmounting a locked file system are
specific to Solaris. Other operating systems may use different methods for unmounting file
systems.

06-16
The Process resource
Service group name appsg • Starts and stops a daemon-type process
Resource name appproc • Monitors the process by scanning the process table
Resource type Process
Required attributes
PathName /bin/sh
Optional attributes
Arguments “/appdata/appd
start”
main.cf
Process appproc(
Critical = 0
PathName = "/bin/sh"
Arguments = "/appdata/appd start"
)
17
The Process resource controls the application and is added last because it requires all other
resources to be online in order to start. The Process resource is used to start, stop, and
monitor the status of a process.
• Online: Starts the process specified in the PathName attribute, with options, if specified,
in the Arguments attribute
• Offline: Sends SIGTERM to the process; SIGKILL is sent if process does not exit within one
second
• Monitor: Determines if the process is running by scanning the process table
The optional Arguments attribute specifies any command-line options to use when starting
the process.

06-17
Process attribute specification
Executed command: /appdata/appd start
Attribute Binary executable Shell script
Fully-qualified path to binary file Fully-qualified path to shell

PathName
/appdata/appd /bin/sh
Parameters passed to binary file Path to shell script and parameters

Arguments
(optional)
start “/appdata/appd start”
18
If the executable is a shell script, you must specify the script name followed by arguments.
You must also specify the full path for the shell in the PathName attribute.
The monitor script calls ps and matches the process name. The process name field is limited
to 80 characters in the ps output. If you specify a path name to a process that is longer than
80 characters, the monitor entry point fails.
If the executable is a binary, you specify the full path to the executable in the PathName
attribute, and any options to be passed are specified in the Arguments attribute.

06-18
Topic: Solving common configuration errors
After completing this topic, you will be able to resolve common errors made
during online configuration.
19
This is the Solving common configuration errors topic.


06-19
Troubleshooting resources
Bring online
Y
Done Online?
Modify attributes
Primary focus
N
Incorrect
Values? Flush group
Correct
Check logs
Fix problems
N
Faulted?
Y
Clear resource
20
Verify that each resource is online on the local system before continuing the service group
configuration procedure. If the online operation hangs with the resource stuck in WAITING TO
GO ONLINE state, you can flush the service group using the hagrp –flush command instead of
waiting until the online operation times out and the resource faults. Flushing the service
group changes the WAITING TO GO ONLINE resource state back to OFFLINE.
Note: The flush operation does not halt the resource online operation if it is still running. If a
running operation succeeds after a flush command was fired, the resource state might still
change.”If you are unable to bring a resource online, use the procedure in the diagram to find
and fix the problem. You can view the logs through Cluster Manager or in the
/var/VRTSvcs/logs directory if you need to determine the cause of errors. VCS log
entries are written to engine_A.log and agent entries are written to resource_A.log
files.
Note: Some resources must be disabled and re-enabled. Only resources whose agents have
open and close entry points, such as MultiNICB, require you to disable and enable again after
fixing the problem. By contrast, a Mount resource does not need to be disabled if, for
example, you incorrectly specify the MountPoint attribute.
However, it is generally a good practice to disable and enable regardless because it is difficult
to remember when it is required and when it is not. In addition, a resource is immediately
monitored upon enabling, which would indicate potential problems with attribute
specification. More detail on performing tasks necessary for solving resource configuration
problems is provided in the following sections.

06-20
Clearing resource faults
• Agents monitor both online and offline resources.
• A resource fault occurs when:
– The underlying component is not available
– The next monitor entry point returns an unexpected offline status
• Fix the underlying problem.
• Clear the fault in VCS:
– Clear nonpersistent resources and bring them back online manually.
– Probe persistent resources; otherwise resources do not show online until the
next OfflineMonitorInterval.
hares –clear resource –sys sysname

hares –probe resource –sys sysname
21
A fault indicates that the monitor entry point is reporting an unexpected offline state for a
previously online resource. This indicates a problem with the underlying component being
managed by the resource.
Before clearing a fault, you must resolve the problem that caused the fault. Use the VCS logs
to help you determine which resource has faulted and why. It is important to clear faults for
critical resources after fixing underlying problems so that the system where the fault originally
occurred can be a failover target for the service group. In a two-node cluster, a faulted critical
resource would prevent the service group from failing back if another fault occurred. You can
clear a faulted resource on a particular system, or on all systems where the service group can
run.
Note: Persistent resource faults should be probed to force the agent to monitor the resource
immediately. Otherwise, the resource status is not online until the next
OfflineMonitorInterval, which may be up to five minutes (default is 300 seconds).
Clearing and probing resources using the CLI
To clear a faulted resource, type:
hares -clear resource [-sys system]
If the system name is not specified then the resource is cleared on all systems.
To probe a resource, type:
hares -probe resource -sys system

06-21
Topic: Testing the service group
After completing this topic, you will be able to explain the implementation
of high availability in the VCS architecture.
22
This is the Testing the service group topic.


06-22
Test procedure
Start
Link resources
After all resources are
online locally:
Fix as indicated Run virtual firedrill
1. Link resources.
2. Run virtual firedrill.
Check logs/fix Test switching 3. Switch the service group to each
system.
4. Set resources to critical, as needed.
Set critical
5. Test failover.
Check logs/fix Test failover
Done
23
After you have successfully brought each resource online, link the resources and switch the
service group to each system on which the service group can run.

06-23
Linking resources
Right click the parent resource, and select Link.
main.cf
hares –link appip appnic
hares –dep
hares –unlink appip appnic appip requires appnic
24
When you link a parent resource to a child resource, the dependency becomes a component
of the service group configuration. When you save the cluster configuration, each dependency
is listed at the end of the service group definition in the main.cf file, after the resource
specifications. In addition, VCS creates a dependency tree in the main.cf file at the end of
the service group definition to provide a more visual view of resource dependencies. This is
not part of the cluster configuration, as denoted by the // comment markers.
// resource dependency tree
//
//group appsg
//{
//IP appip
// {
// NIC appnic
. . .
Note: You cannot use the // characters as general comment delimiters. VCS strips out all lines
with // upon startup and re-creates these lines based on the requires statements in
main.cf.

06-24
Running a virtual fire drill

Service group must be
fully online on one node.
havfd appsg –sys sym2 -v
25
You can run a virtual fire drill for a service group to check that the underlying infrastructure is
properly configured to enable failover to other systems. The service group must be fully online
on one system, and can then be checked on all other systems where it is offline.
You can select which type of infrastructure components to check, or run all checks. In some
cases, you can use the virtual fire drill to correct problems, such as making a mount point
directory if it does not exist. However, not all resources have defined actions for virtual fire
drills, in which case a message is displayed indicating that no checks were performed. You can
also run fire drills using the havfd command, as shown in the slide.

06-25
Setting the critical attribute
• Attribute is removed from main.cf
Critical=1 is default for all resources
• Entire service group is faulted if resource faults
main.cf
DiskGroup appdg(
hares –modify appdg Critical 1 DiskGroup = appdatadg
)
26
The Critical attribute is set to 1, or true, by default. When you initially configure a resource,
you set the Critical attribute to 0, or false. This enables you to test the resources as you add
them without the resource faulting and causing the service group to fail over as a result of
configuration errors you make.
Some resources may always be set to non-critical. For example, a resource monitoring an
Oracle reporting database may not be critical to the overall service being provided to users. In
this case, you can set the resource to non-critical to prevent downtime due to failover in the
event that it was the only resource that faulted.
Note: When you set an attribute to a default value, the attribute is removed from main.cf.
For example, after you set Critical to 1 for a resource, the Critical = 0 line is removed from the
resource configuration because it is now set to the default value for the resource type.
To see the values of all attributes for a resource, use the hares command. For example:
hares -display appdg

06-26
Lesson summary
• Key points
– In this lesson, you learned about the standard procedure to create and test service groups.
– You also learned to recognize common configuration problems.
– Finally, you learned to apply a methodology for finding solutions.
– Veritas Cluster Server Bundled Agent Reference Guide
27

06-27
Lab 07: Online Configuration
• Exercise A: Creating a service group for the loopy application
• Exercise B: Configuring resources for the loopy application
• Exercise C: Performing a virtual fire drill on the service group
• Exercise D: Testing the service group
• Exercise E: Setting resources to critical
• Exercise F: (Optional) Examining Veritas File System locking by VCS
28

06-28
What did you learn?
lesson.
29
current lesson.

06-29
Question 1: Adding resources
When you add resources using the CLI:
A. VCS automatically writes the resource definition to the main.cf file on disk.
B. Resources are automatically enabled as they are added.
C. Resources are automatically brought online when its required attributes are set.
D. VCS sets resources to Critical by default.
30

06-30
Answer 1: Adding resources
When you add resources using the CLI:
A. VCS automatically writes the resource definition to the main.cf file on disk.
B. Resources are automatically enabled as they are added.
C. Resources are automatically brought online when its required attributes are set.
D. VCS sets resources to Critical by default.
The correct answer is D. By default when a resource is added through CLI, it is set to critical.
31

06-31
What is the effect of disabling a resource?
A. The agent stops monitoring that resource.

B. The agent takes the resource offline on the local system.
C. The resource attributes are checked for syntax errors.
D. The resource is considered faulted.
32

06-32
What is the effect of disabling a resource?
A. The agent stops monitoring that resource.

B. The agent takes the resource offline on the local system.
C. The resource attributes are checked for syntax errors.
D. The resource is considered faulted.
The correct answer is A. Corresponding resource agent will stop monitoring that resource if it is disabled.
33

06-33
File system locking:
A. Must be configured after adding a Mount resource

B. Prevents a file system from being brought online outside of VCS control
C. Prevents inadvertently unmounting file systems
D. Works in conjunction with I/O fencing
34

06-34
File system locking:
A. Must be configured after adding a Mount resource

B. Prevents a file system from being brought online outside of VCS control
C. Prevents inadvertently unmounting file systems
D. Works in conjunction with I/O fencing
The correct answer is C. File system locking feature provided by VCS prevents accidental/manual unmounting of
VCS filesystem.
35

06-35
Question 4: Online service group configuration
When you create a service group:
A. It is automatically set to run on all cluster systems.

B. It is automatically set to start and run on all cluster systems.
C. You must explicitly specify which systems can run the service group.
D. You must add all resources before the service group can be brought online.
36

06-36
Answer 4: Online service group configuration
When you create a service group:
A. It is automatically set to run on all cluster systems.

B. It is automatically set to start and run on all cluster systems.
C. You must explicitly specify which systems can run the service group.
D. You must add all resources before the service group can be brought online.
The correct answer is C. You need to explicitly specify the SystemList attribute in order to
specify the list of systems where that service group can come online.
37

06-37
Question 5: Online service group configuration
When you create a resource:
A. All child resources must first be online.

B. All child resources must first be offline.
C. All parent resources must be disabled.
D. All required attributes must be set before you can bring the resource online.
38

06-38
Answer 5: Online service group configuration
When you create a resource:
A. All child resources must first be online.

B. All child resources must first be offline.
C. All parent resources must be disabled.
D. All required attributes must be set before you can bring the resource online.
The correct answer is D. Resource will be probed when you try to bring it online so it is a must to set all
required attributes to be configured before the resource may come online.
39

06-39
End of presentation
06-40
Lesson 07: Offline Configuration
This is the Offline Configuration lesson in the Veritas InfoScale Availability 7.4.2 for

07-1
Administration
Administration


07-2
Lesson objectives
Topic Objective
Offline configuration examples Identify scenarios where offline configuration is applicable.
Offline configuration procedures Outline offline configuration procedures.
Solving offline configuration

Resolve common errors made during offline configuration.
problems
Testing the service group Test the service group to ensure proper configuration.

07-3
Topic: Offline configuration examples
After completing this topic, you will be able to identify scenarios where
offline configuration is applicable.
This is the Offline configuration examples topic.


07-4
Example 1: Reusing a cluster configuration
hr2011db
main.cf mkt2011db

db2011clus
group hr2011db (
SystemList = {s1=0,s2=1} s2
AutoStartList = {s1, s2} s1
)
hr2012db
main.cf
mkt2012db

db2012clus
group hr2012db (
SystemList = {s3=0,s4=1}
s4
AutoStartList = {s3, s4} s3
)
One example where offline configuration is appropriate is when your high availability
environment is expanding and you are adding clusters with similar configurations.
In the example displayed on the slide, the original cluster consists of two systems, with each
system running a database instance. Another cluster with essentially the same configuration is
being added, but it is managing different databases. You can copy configuration files from the
original cluster, make the necessary changes, and then restart VCS as described later in this
lesson. This method may be more efficient than creating each service group and resource
using a graphical user interface or the VCS command-line interface.

07-5
Example 2: Reusing a service group definition
Service group definition Service group definition Value
Value
Group extwebsg Group intwebsg
Required attributes Required attributes
FailOverPolicy Priority FailOverPolicy Priority
SystemList s1=0, s2=1 SystemList s2=0, s1=1
Optional attributes Optional attributes
AutoStartList s1, s2 AutoStartList s2, s1
extwebapache intwebapache
extwebip extwebmnt intwebip intwebmnt
extwebnic extwebvol intwebnic intwebvol
extwebdg intwebdg
Another example of using offline configuration is when you want to add a service group with a
similar set of resources as another service group in the same cluster.
In the example displayed on the slide, the portion of the main.cf file that defines the
extwebsg service group is copied and edited as necessary to define a new intwebsg service
group.

07-6
Example 3: Modeling a configuration
# vi main.cf
~
include "types.cf"
cluster modelclus(
…
)
system s1 (
)
system s2 (
)
group websg (
…
main.cf
modelclus

s2
s1
You can use the VCS Simulator to create and test a cluster configuration on the Windows
operating system and then copy the finalized configuration files into a real cluster
environment. The Simulator enables you to create configurations for all supported UNIX,
Linux, and Windows platforms.
This only applies to the cluster configuration. You must perform all the preparation tasks to
create and test the underlying resources, such as virtual IP addresses, shared storage objects,
and applications. After the cluster configuration is copied to the real cluster and VCS is
restarted, you must test all the objects, as shown later in this lesson.

07-7
Topic: Offline configuration procedures
After completing this topic, you will be able to outline offline configuration
procedures.
This is the Offline configuration procedures topic.


07-8
Offline configuration procedure: New cluster
Save and close the configuration. haconf –dump -makero
Change to configuration directory. cd /etc/VRTSvcs/conf/config
Stop VCS on all systems. hastop -all
Edit the configuration file. vi main.cf
Verify configuration file syntax. hacf –verify .
Errors? Primary system
Start VCS on this system. hastart
Verify that VCS is running. hastatus -sum
Start VCS on all other systems. hastart All other systems
The diagram illustrates a process for modifying the cluster configuration when you are
configuring your first service group and do not have services already running in the cluster.
Select one system to be your primary node for configuration. Work from this system for all
steps up to the final point of restarting VCS.
1. Save and close the configuration. Always save and close the configuration before making
any modifications. This ensures that the configuration in the main.cf file on disk is the
most recent in-memory configuration.
2. Change to the configuration directory. The examples used in this procedure assume you
are working in the /etc/VRTSvcs/conf/config directory.
3. Stop VCS. Stop VCS on all cluster systems. This ensures that there is no possibility of
another administrator changing the cluster configuration while you are modifying the
main.cf file.
4. Edit the configuration files. You must choose any system to modify the main.cf file.
However, you must then start VCS first on that system.

07-9
5. Verify the configuration file syntax.
Run the hacf command in the /etc/VRTSvcs/conf/config directory to verify the
syntax of the main.cf and types.cf files after you have modified them. VCS cannot
start if the configuration files have syntax errors. Run the command in the config
directory using the dot (.) to indicate the current working directory, or specify the full
path.
Note: The hacf command only identifies the syntax errors, not the configuration errors.
6. Start VCS on the system with the modified configuration file. Start VCS first on the primary
system with the modified main.cf file.
7. Verify that VCS is running. Verify that VCS is running on the primary configuration system
before starting VCS on other systems.
8. Start other systems. After VCS is in a running state on the first system, start VCS on all
other systems. If you cannot bring VCS to a running state on all systems, see the ‘Solving
Offline Configuration Problems’ topic.

07-10
Existing cluster procedure part 1
Save and close the configuration. haconf –dump -makero
Change to configuration directory. cd /etc/VRTSvcs/conf/config
Backup up the main.cf file. cp -p main.cf main.cf.orig
Create a staging directory. mkdir stage
Copy main.cf and *types.cf. cp –p *.cf stage

First system
Change to the staging directory. cd stage
Edit the configuration file. vi main.cf
Set Frozen attribute. Set Frozen=1 for modified groups.
11
The diagram illustrates a process for modifying the cluster configuration when you want to
minimize the time that VCS is not running to protect existing services. This procedure includes
several built-in protections from common configuration errors and maximizes high availability.
First system
Designate one system as the primary change management node. This makes troubleshooting
easier if you encounter problems with the configuration.
1. Save and close the configuration. Save and close the cluster configuration before you start
making changes. This ensures that the working copy has the latest in-memory
configuration.
2. Back up the main.cf file. Make a copy of the main.cf file with a different name. This
ensures that you have a backup of the configuration that was in memory when you saved
the configuration to disk.
Note: If any *types.cf files are being modified, also back up these files.
3. Make a staging directory. Make a subdirectory of /etc/VRTSvcs/conf/config in
which you can edit a copy of the main.cf file. This helps ensure that your edits are not
overwritten if another administrator changes the configuration simultaneously.

07-11
4. Copy the configuration files.
Copy the *.cf files /etc/VRTSvcs/conf/config to the staging directory.
5. Modify the configuration files. Modify the main.cf file in the staging directory on one
system. The diagram on the slide refers to this as the first system.
6. Freeze the service groups. If you are modifying existing service groups, freeze those
service groups persistently by setting the Frozen attribute to 1. This simplifies fixing
resource configuration problems after VCS is started because the service groups will not
fail over between systems if faults occur.
. . .
group extwebsg (
SystemList = { s1 = 1, s2 = 0}
Operators = { extwebsgoper }
Frozen = 1
)

07-12
Existing cluster procedure part 2
Verify configuration file syntax. hacf –verify .
Stop VCS; leave services running. hastop -all -force
Copy the main.cf file back. cp -p main.cf ..
Start VCS on this system. hastart

Primary system
Verify that HAD is running. hastatus -sum
Start VCS on all other systems. hastart All other systems
After you verify that all resources come online properly:

! • Unfreeze service groups (hagrp –unfreeze group –persistent).
• Test switching the service groups.
13
7. Verify the configuration file syntax. Run the hacf command in the staging directory to
verify the syntax of the main.cf and types.cf files after you have modified them.
Note: The dot (.) argument indicates that the current working directory is used as the
path to the configuration files. You can run hacf -verify from any directory by
specifying the path to the configuration directory:
hacf -verify /etc/VRTSvcs/conf/config
8. Stop VCS. Stop VCS on all cluster systems after making configuration changes. To leave
applications running, use the -force option, as shown in the diagram.
9. Copy the new configuration file. Copy the modified main.cf file and all *types.cf
files from the staging directory back into the configuration directory.
10. Start VCS. Start VCS first on the system with the modified main.cf file.
11. Verify that VCS is in a local build or running state on the primary system.
12. Start other systems. After VCS is in a running state on the first system, start VCS on all
other systems. You must wait until the first system has built a cluster configuration in
memory and is in a running state to ensure the other systems perform a remote build
from the first system’s configuration in memory.

07-13
VCS startup using a specific main.cf file
5
Display VCS status
Local build root@s1
Cluster
conf
4
2 main.cf
had
hashadow
1
3
hastart
s1
System with modified main.cf file.
14
The diagram illustrates how to start VCS to ensure that the cluster configuration in memory is
built from a specific main.cf file.
Starting VCS using a modified main.cf file
Ensure that VCS builds the new configuration in memory on the system where the changes
were made to the main.cf file. All other systems must wait for the build to successfully
complete and the system to transition to the running state before VCS is started elsewhere.
1. Run hastart on s1 to start the had and hashadow processes.
2. HAD checks for a valid main.cf file.
3. HAD checks for an active cluster configuration on the cluster interconnect.
4. Since there is no active cluster configuration, HAD on s1 reads the local main.cf file and
loads the cluster configuration into local memory on s1.
5. Verify that VCS is in a local build or running state on s1 using hastatus -sum.

07-14
Building the configuration
s1 s2

conf conf
11
main.cf main.cf 7
had had
hashadow hashadow
10 8 6
9 hastart
15
6. When VCS is in a running state on s1, run hastart on s2 to start the had and
hashadow processes.
7. HAD on s2 checks for a valid main.cf file.
8. HAD on s2 checks for an active cluster configuration on the cluster interconnect.
9. The s1 system sends a copy of the cluster configuration over the cluster interconnect to
s2.
10. The s2 system performs a remote build to load the new cluster configuration in memory.
11. HAD on s2 backs up the existing main.cf and types.cf files and saves the current
in-memory configuration to disk.

07-15
Resource dependencies
Resource dependency definition • Ensure resource dependencies are defined
Service group intwebsg • Review these rules:
– No persistent parent resources
Parent resource Requires Child resource – No links between resources in different groups
intwebvol intwebdg – No limits to the number of parent and child resources
– No cyclical dependencies
intwebmnt intwebvol
intwebip intwebnic
intwebapache intwebmnt
intwebapache intwebip
intwebvol requires intwebdg

intwebmnt requires intwebvol
intwebip requires intwebnic
intwebapache requires intwebmnt
intwebapache requires intwebip
16
Ensure that you create the resource dependency definitions at the end of the service group
definition. Add the links using the syntax shown in the slide.

07-16
A completed configuration file
Original group
group extwebsg (
AutoStartList = { s1, s2 } Copied group
Operators = { extwebsgoper }
group intwebsg (
)
DiskGroup extwebdg ( AutoStartList = { s2, s1 }
DiskGroup = extwebdatadg Operators = { intwebsgoper }
) Frozen = 1
)
IP extwebip (
Device = e1000g0 DiskGroup intwebdg (
Address = "10.10.21.108" DiskGroup = extwebdatadg
) )
. . .
IP extwebip (
Device = e1000g0
Address = "10.10.21.109"
)
Check these common problems. . . .
17
A portion of the completed main.cf file with a new service group definition for intwebsg is
displayed in the slide. This service group was created by copying the extwebsg service group
definition and changing the attribute names and values.
Two errors are intentionally shown in the example on the slide.
• The extwebip resource name was not changed in the intwebsg service group. This causes a
syntax error when the main.cf file is checked using hacf -verify because you cannot
have duplicate resource names within the cluster.
• The intwebdg resource has the value of extwebdatadg for the DiskGroup attribute. This
does not cause a syntax error, but is not a correct attribute value for this resource. The
extwebdatadg disk group is being used by the extwebsg service group and cannot be
imported by another failover service group.

Note: You cannot include comment lines in the main.cf file. The lines you see starting with
// are generated by VCS to show resource dependencies. Any lines starting with // are
stripped out during VCS startup.

07-17
Topic: Solving offline configuration errors
After completing this topic, you will be able to resolve common errors made
during offline configuration.
18
This is the Solving offline configuration errors topic.


07-18
Starting from an old configuration
If you start the cluster from the wrong system, VCS starts but builds the in-memory configuration from an old
configuration. To recover from an older configuration:
1 Close the configuration, if open.
Stop VCS on all systems and keep applications

2 running.
Copy main.cf.previous to main.cf on the good

3 system.
4 Verify the syntax.
Start VCS on this system using the hastart

5 command.
6 Verify that VCS is running using hastatus.
Start VCS on all other systems after the first system

7
is running.
19
If you are running an old cluster configuration because you started VCS on the wrong system
first, you can recover the main.cf file on the system, where you originally made the
modifications using the main.cf.previous backup file created automatically by VCS.
To recover from an old configuration, use the offline configuration procedure to restart VCS
using the recovered main.cf file. Note: You must ensure that VCS is in the local build or
running state on the system with the recovered main.cf file before starting VCS on other
systems.

07-19
Forcing VCS to start from a wait state (1/2)
1
Local build Attempt to start VCS on s1
Cluster root@s1
root@s1
conf
5
6
main.cf
2
had
hashadow
s1 4 hasys -force s1
3
hacf -verify /etc/VRTSvcs/conf/config
20
This scenario results in all cluster systems entering a wait state:

• Your new main.cf file has a syntax problem.
• You forget to check the file with hacf -verify.
• You start VCS on the first system with hastart.
• The first system cannot build a configuration and goes into a wait state, such as
ADMIN_WAIT.
To force VCS to start from a wait state, follow the given steps :
1. An attempt to start VCS on a s1 with a bad main.cf results in the cluster in a wait state.
2. Visually inspect the main.cf file and modify or replace the file as necessary to ensure it
contains the correct configuration content.
3. Verify the configuration with hacf -verify /opt/VRTSvcs/conf/config.
4. Run hasys -force s1 on s1. This starts the local build process. You must have a valid
main.cf file to force VCS to a running state. If the main.cf file has a syntax error, VCS
enters the ADMIN_WAIT state.
5. HAD checks for a valid main.cf file.
6. HAD on s1 reads the local main.cf file, and if it has no syntax errors, HAD loads the
cluster configuration into local memory on s1.

07-20
Forcing VCS to start from a wait state (2/2)
s1 s2

conf conf
12
main.cf main.cf 9
had had
hashadow hashadow
10
8
11
7 hastart
21
7. When HAD is in a running state on s1, this state change is broadcast on the cluster
interconnect by GAB.
8. Next, run hastart on s2 to start HAD.
9. HAD on s2 checks for a valid main.cf file. This system has an old version of the
main.cf.
10. HAD on s2 then checks for another node in a local build or running state.
11. Since s1 is in a local build or running state, HAD on s2 performs a remote build from the
configuration on s1.
12. HAD on s2 copies the cluster configuration into the local main.cf and types.cf files
after moving the original files to backup copies with timestamps.

07-21
Configuration file backups
ls –l /etc/VRTSvcs/conf/config/main*
-rw------ 2 . . . 5992 Jan 10 12:07 main.cf
-rw------ 1 . . . 5039 Jan 8 8:01 main.cf.08Jan2017.13.13.36
-rw------ 2 . . . 5051 Jan 9 17:58 main.cf.09Jan2017.13.41.26
-rw------ 2 . . . 5992 Jan 10 12:07 main.cf.10Jan2017.13.14.20
-rw------ 2 . . . 5051 Jan 9 17:58 main.cf.previous
main.cf linked to last timestamp file

main.cf.previous linked to main.cf.09Jan2017.13.41.26
22
Each time you save the cluster configuration, VCS maintains backup copies of the main.cf
and types.cf files.
This occurs as follows:
1. New main.cf.datetime and *types.cf.datetime files are created.
2. The hard links for main.cf, main.cf.previous, types.cf and
types.cf.previous (as well as any others) are changed to point to the correct
versions
Although it is always recommended that you copy configuration files before modifying them,
you can revert to an earlier version of these files if they are damaged or lost.

07-22
Topic: Testing the service group
After completing this topic, you will be able to test the service group to
ensure proper configuration.
23
This is the Testing the service group topic.


07-23
Service group test procedure
Start Online? Troubleshoot resources
Fix as indicated Run virtual firedrill

After all resources are online locally:
1. Link resources.
2. Run virtual firedrill.
Check logs/fix Test switching 3. Switch the service group to each system.
4. Set resources to critical, as needed.
5. Test failover.
Set critical
Check logs/fix Test failover
Done
24
After you restart VCS throughout the cluster, use the procedure shown in the slide to verify
that your configuration additions or changes are correct.
Notes:
• This process is slightly different from online configuration, which tests each resource
before creating the next and before creating dependencies.
• Resources should come online after you restart VCS if you have specified the appropriate
attributes to automatically start the service group.
In case of any configuration problems, use the procedures shown in the “Online
Configuration” lesson. If you need to make additional modifications, you can use one of the
online tools or modify the configuration files using the offline procedure.

07-24
Lesson summary
• Key points
– In this lesson, you learned to use a text editor or the VCS Simulator to modify VCS configuration files.
– You also learned to apply a methodology for modifying VCS configuration.
– Finally, you learned to test a VC configuration.
25

07-25
Lab 08: Offline Configuration
• Exercise A: Editing a copy of the main.cf file using a system editor
• Exercise B: Stopping VCS
• Exercise C: Restarting VCS using the edited main.cf file
26

07-26
What did you learn?
lesson.
27
current lesson.

07-27
Question 1: Offline configuration procedures
After manually editing the main.cf file, which command should you run to ensure that
the file is free of syntax errors?
A. haconfig -verify config_dir

B. hacf -check
C. hacf -verify config_dir
D. haconf -verify main.cf
28

07-28
Answer 1: Offline configuration procedures
After manually editing the main.cf file, which command should you run to ensure that
the file is free of syntax errors?
A. haconfig -verify config_dir

B. hacf -check
C. hacf -verify config_dir
D. haconf -verify main.cf
The correct answer is C. hacf –verify is the command that needs to be executed on the directory
holding the configuration files.
29

07-29
A recommended practice for offline configuration is to:
A. Copy the main.cf and *types.cf into a staging directory before editing.
B. Change the original main.cf while the cluster is up, then bring the cluster down.
C. Make changes to the service groups before changing cluster attributes.
D. Change resources, then service groups, then cluster attributes.
30

07-30
A recommended practice for offline configuration is to:
A. Copy the main.cf and *types.cf into a staging directory before editing.
B. Change the original main.cf while the cluster is up, then bring the cluster down.
C. Make changes to the service groups before changing cluster attributes.
D. Change resources, then service groups, then cluster attributes.
The correct answer is A. As a best practice you should always copy the configuration files to a staging directory
where you will apply changes.
31

07-31
Question 3
If you inadvertently start VCS with an old configuration file, you can:
A. Stop VCS and start it from another system.

B. Stop VCS, copy the main.cf.previous to main.cf, and restart VCS.
C. Restore main.cf from a backup if you have lost any changes.
D. Stop VCS without closing the configuration, and restart on another system.
32

07-32
If you inadvertently start VCS with an old configuration file, you can:
A. Stop VCS and start it from another system.

B. Stop VCS, copy the main.cf.previous to main.cf, and restart VCS.
C. Restore main.cf from a backup if you have lost any changes.
D. Stop VCS without closing the configuration, and restart on another system.
The correct answer is B. You need to stop HAD on all the nodes and copy the main.cf.previous file to
main.cf and restart VCS on that node.
33

07-33
Why do you wait for VCS to be running on the system with the modified main.cf
before starting VCS on other systems?
A. To ensure that all systems build from their local main.cf files
B. To ensure that the other systems wait and perform a remote build
C. To ensure all other systems remain in ADMIN_WAIT until you force a local build
D. To force VCS to perform a local build on each system
34

07-34
Why do you wait for VCS to be running on the system with the modified main.cf
before starting VCS on other systems?
A. To ensure that all systems build from their local main.cf files
B. To ensure that the other systems wait and perform a remote build
C. To ensure all other systems remain in ADMIN_WAIT until you force a local build
D. To force VCS to perform a local build on each system
The correct answer is B. You wait for local build to get completed on the first node of the cluster
and once the modified configuration is in memory you can start VCS on rest of the nodes for a
remote build.
35

07-35
Why do you save and close the cluster configuration before making changes to
main.cf?
A. To ensure that the latest in-memory configuration is written to disk

B. To check the syntax of the configuration in memory before making changes
C. To reset locks on the configuration files so you can make changes
D. To prevent another administrator from modifying the main.cf on another system
36

07-36
Why do you save and close the cluster configuration before making changes to
main.cf?
A. To ensure that the latest in-memory configuration is written to disk

B. To check the syntax of the configuration in memory before making changes
C. To reset locks on the configuration files so you can make changes
D. To prevent another administrator from modifying the main.cf on another system
The correct answer is A. So that the latest in-memory configuration is saved to main.cf file.
37

07-37
End of presentation
07-38
Lesson 08: Configuring Notification
This is the Configuring Notification lesson in the Veritas InfoScale 7.4.2 Fundamentals for

08-1
Administration
Administration


08-2
Lesson objectives
Topic Objective
• Explain VCS notification service.

Notification overview
• Summarize the functions of ClusterService group.
Configuring notification Configure notifications using the NotifierMngr resource.
Overview of triggers Configure triggers to customize VCS behavior in response to events.

08-3
Topic: Notification overview

• Explain VCS notification service.
• Summarize the functions of ClusterService group.
This is the Notification overview topic.


08-4
Notification overview
1. HAD sends a message to the notifier daemon when an event occurs.
2. The notifier daemon:
a. Formats the event message.
b. Sends an SNMP trap or e-mail message (or both) to designated recipients.
HAD
SMTP SNMP


notifier HAD 
 Replicated message queue

 To view: haclus -notes

When VCS detects certain events, you can configure the notifier to:
• Generate an SNMP (V2) trap to specified SNMP consoles.
• Send an e-mail message to designated recipients.
VCS ensures that no event messages are lost while the VCS engine is running, even if the
notifier daemon stops or is not started. The HAD daemons throughout the cluster
communicate to maintain a replicated message queue. If the service group with the notifier
configured as a resource fails on one of the nodes, the notifier fails over to another node in
the cluster. Since the message queue is guaranteed to be consistent and replicated across
nodes, the notifier can resume message delivery from where it left off after it fails over to the
new node.
Messages are stored in the queue until one of these conditions is met:
• The notifier daemon sends an acknowledgement to HAD that at least one recipient has
received the message.
Or
• The queue is full. The queue is circular—the last (oldest) message is deleted in order to
write the current (newest) message.

08-5
Messages in the queue for one hour are deleted if the notifier is unable to deliver to the recipient.
Note: Before the notifier daemon connects to HAD, messages are stored permanently in the queue
until one of the last two conditions is met.
You can view the entries in the message cue using the haclus -notes command. You can also
delete all queued messages on each cluster node using haclus -delnotes, but the notifier
must be stopped first.

08-6
High availability for notification
• The notifier daemon is managed by a NotifierMngr type resource.
• The ClusterService group:
– contains the notifier and csgnic resources.
– is configured using the CPI, VOM, or CLI.
– is a failover group, online on one cluster node only.
– contains other resources, depending on the cluster options configured.
ClusterService notifier
HAD
csgnic


HAD 
notifier



The notification service is managed by a NotifierMngr type resource contained in the

ClusterService group. ClusterService is created automatically during cluster configuration, if
certain configuration options are selected.
You can also configure ClusterService later using the VCS command-line interface or the
Veritas Operations Manager. After configuring the notification, ClusterService contains a
NotifierMngr type resource to manage the notifier daemon, and a csgnic resource to monitor
the network interface used by the notifier for sending messages.
ClusterService is a special-purpose service group:
• It is the first group to come online on the first node in a running state.
• It can failover despite being frozen

• It cannot be autodisabled.
• It switches to another node upon hastop -local on the online system.
• It attempts to start on all miniclusters if a network partition occurs
• It also manages the wide-area connector process in a global cluster environment.
CAUTION: Do not add resources to the ClusterService for managing non-VCS
applications.

08-7
Message severity levels
Resource state
unknown
Service group is online Resource has faulted
Warning Error
Information
Concurrency violation
SevereError
HAD
SMTP SNMP


notifier HAD 

 See the VCS Administrator's Guide
 for a complete list of events.
Event messages are assigned one of four severity levels by the notifier:
• Information: Normal cluster activity is occurring, such as resources being brought online.
• Warning: Cluster or resource states are changing unexpectedly, such as a resource in an
unknown state.
• Error: Services are interrupted, such as a service group faulting that cannot be failed over.
• SevereError: Potential data corruption is occurring, such as a concurrency violation.
The administrator can configure the notifier to specify which recipients are sent messages
based on the severity level.
A complete list of events and corresponding severity levels is provided in the Veritas Cluster
Server Administrator’s Guide.

08-8
Notifier and log events
2017/01/06 15:19:37 VCS ERROR V-16-1-10205
engine_A.log Group websg is faulted on system s1
From [email protected] Thu Jan 6 15:19:53 2017

Date: Thu, 06 Jan 2015 15:19:37 -0700
From: Notifier
Subject: VCS Error, Service group has faulted
Event Time: Thu Jan 6 15:19:37 2015 Notifier

e-mail Log file
Entity Name: websg Information INFO
Entity Type: Service Group NOTICE
Entity Subtype: Failover
Warning WARNING
Entity State: Service group has faulted
Traps Origin: Veritas_Cluster_Server Error ERROR
System Name: s1 SevereError CRITICAL
Entities Container Name: webvcs
Entities Container Type: VCS
9
The table on this slide displays how the notifier levels shown in e-mail messages compare to
the log file codes for corresponding events.
This example shows a log entry for a VCS ERROR and the corresponding e-mail message.
Notice that the notifier SevereError events correlate with CRITICAL entries in the engine log.

08-9
Viewing logs
• Engine log location: /var/VRTSvcs/log/engine_A.log
• View logs using the GUI or the hamsg command:
hamsg engine_A
• Agent logs kept in /var/VRTSvcs/log
Unique Message
Identifier (UMI)
2017/05/20 16:00:09 VCS NOTICE V-16-1-11022
VCS engine (had) started
2017/05/20 16:01:27 VCS INFO V-16-1-10196
Cluster logger started
2017/05/20 16:01:31 VCS ERROR V-16-1-11309
Configuration must be ReadWrite
Most Recent
! Support Web site

10
In addition to the primary VCS engine log file, VCS logs information for had, hashadow, and all
agent programs.
Starting with the 4.0 version of VCS, messages in VCS logs have a unique message identifier.
Each entry includes a text code indicating the severity, from CRITICAL entries to INFO entries
with the status information.
• CRITICAL entries indicate problems requiring immediate attention—contact Support
immediately.
• ERROR entries indicate exceptions that need to be investigated.

08-10
Topic: Configuring notification
After completing this topic, you will be able to configure notification using
the NotifierMngr resource.
11
This is the Configuring notification topic.


08-11
Configuration methods
CLI VIOM Installer

• Use hares • Veritas InfoScale Respond to
command. Operations prompts during
Manager. cluster
• Add resource of
NotifierMngr • Add NotifierMngr configuration.
type. resource to
ClusterService.
• Set attributes.
12
Although, you can start and stop the notifier daemon manually outside of VCS, you should
make the notifier component highly available by placing the daemon under VCS control.
You can configure VCS to manage the notifier manually using the command-line interface or
the Veritas Operations Manager, or set up notifications during initial cluster configuration.

08-12
Notification configuration
Add a NotifierMngr resource with a link to csgnic.
Modify the SmtpServer and SmtpRecipients attributes. If SMTP

notification
Optionally, modify ResourceOwner and GroupOwner. is required.
Modify SnmpConsoles, if using SNMP notification. If SNMP

notification
Configure the SNMP console to receive VCS traps. is required.
Modify any other optional attributes, as appropriate.
! Add a NotifierMngr resource to only

one service group, ClusterService.
13
High-level tasks are required to manually configure highly available notifications.

Add a NotifierMngr type of resource to the ClusterService group and link the resource to the
csgnic resource that is present
If SMTP notification is required:
• Modify the SmtpServer and SmtpRecipients attributes of the NotifierMngr resource.
• Optionally, modify the ResourceOwner attribute of individual resources.
• Optionally, specify a GroupOwner e-mail address for each service group.
If SNMP notification is required:
• Modify the SnmpConsoles attribute of the NotifierMngr resource.

• Verify that the SNMPTrapPort attribute value matches the port configured for the
SNMP console. The default is port 162.
• Configure the SNMP console to receive VCS traps (described later in the lesson).
Modify any other optional attributes of the NotifierMngr resource. See the manual pages for
notifier and hanotify for a complete description of configuration options.

08-13
The NotifierMngr resource type
Resource definition Value • Required attributes:

Service group name ClusterService – Either SnmpConsoles or SmtpXxx
attributes must be specified.
Resource name notifier
– Both can be specified.
Resource type NotifierMngr
• Restart resource if you change attributes.
Required attributes
SmtpServer mailserver.company.com
SmtpRecipients [email protected] Error
main.cf
NotifierMngr notifier (
SmtpServer = "mailserver.company.com"
SmtpRecipients = { "[email protected]" = Error }
)
. . .
notifier requires csgnic
. . .
14
The notifier daemon runs on only one system in the cluster, where it processes messages
from the local had daemon. If the notifier daemon fails on that system, the NotifierMngr
agent detects the failure and migrates the service group containing the NotifierMngr resource
to another system.
Because the message queue is replicated throughout the cluster, any system that is a target
for the service group has an identical queue. When the NotifierMngr resource is brought
online, had sends the queued messages to the notifier daemon.
The example in the slide shows the configuration of a notifier resource for e-mail notification.
See the Veritas Cluster Server Bundled Agents Reference Guide for detailed information about
the NotifierMngr agent.
Note: Before modifying resource attributes, ensure that you take the resource offline and
disable it. The notifier daemon must be stopped and restarted with new parameters in order
for changes to take effect.

08-14
The ResourceOwner attribute
• Provides e-mail notification for individual resources.
• Writes an entry in the log file.
• Requires notifier to be configured to enable e-mail to be sent.
2017/01/03 11:23:48 VCS INFO V-16-1-10304

engine_A.log Resource dboracle([email protected],
Group=dbsg) is offline on s1
ResourceStateUnknown ResourceRestartingByAgent
Notification events
ResourceMonitorTimeout ResourceWentOnlineByItself
ResourceNotGoingOffline ResourceFaulted
CLI hares –modify dboracle ResourceOwner [email protected]
15
You can set the ResourceOwner attribute to define an owner for a resource. After the
attribute is set to a valid e-mail address and notification is configured, an e-mail message is
sent to the defined recipient when one of the resource-related events occurs, shown in the
table in the slide. VCS also creates an entry in the log file in addition to sending an e-mail
message.
ResourceOwner can be specified as an e-mail ID ([email protected]) or a user account
(gene). If a user account is specified, the e-mail address is constructed as login@smtp_system,
where smtp_system is the system that was specified in the SmtpServer attribute of the
NotifierMngr resource.

08-15
The GroupOwner attribute
• Provides e-mail notification for individual service groups.
• Requires notifier to be configured for e-mail to be sent.
From: Notifier
Subject: VCS Information, Service group is
E-mail message
online
Event Time: Wed Feb 23 18:23:09 2015
. . .
Entities Owner: [email protected]
Concurrency violation Restarting
Notification events Online Faulted and cannot failover

Offline Autodisabled
Switching
CLI hagrp –modify dbsg GroupOwner [email protected]
16
You can set the GroupOwner attribute to define an owner for a service group. After the
attribute is set to a valid e-mail address and notification is configured, an e-mail message is
sent to the defined recipient when one of the group-related events occurs, as shown in the
table in the slide.
GroupOwner can be specified as an e-mail ID ([email protected]) or a user account (gene).
If a user account is specified, the e-mail address is constructed as login@smtp_system, where
smtp_system is the system that was specified in the SmtpServer attribute of the NotifierMngr
resource.

08-16
Additional recipients for notifications
Attributes: Values:
• ResourceRecipients • Fully-qualified
• TypeRecipients e-mail addresses
• GroupRecipients • Severity:
• SystemRecipients – SevereError, Error, Warning and Information.
• ClusterRecipients
main.cf snippet
cluster vcs_cluster (
UserNames = { admin = ************ }
Administrators = { admin }
ClusterRecipients= { " [email protected]" Error }
)
17
Additional attributes are added to enable broader specification of users to be notified of

resource and service group events. These attributes are configured at the corresponding
object level. For example, the GroupRecipients attribute is configured within a service group
definition.
These attributes are specified by a list of e-mail ids of users along with the severity level. The
registered users would get only those events which have severity equal to or greater than the
severity requested. For example, if janedoe is configured in the ClusterRecipients attribute
with a severity level Warning, she would get events of severity Warning, Error and SevereError
but would not get events with severity Information. A cluster event, such as a cluster fault,
which is Error level, would be sent to janedoe.

08-17
Configuring the SNMPv3 console (1/2)
Prerequisites for configuring SNMPv3
• OS RHEL6, RHEL7 and Windows 2012
• Python 3.5 or later and PySNMP
To install PySNMP through pip run the following command:

pip install pysnmp
Configuration Authentication
No authentication and no privacy Username
(noAuthNoPriv)
Authentication and no privacy (authNoPriv) Username, authentication protocol,
authentication password
Authentication and privacy (authPriv) Username, authentication protocol,
authentication password, privacy protocol,
privacy password
18
Using the Management Server console in VIOM, you can configure SNMPv3 settings.
Note: To perform this task, your user group must be assigned the Admin role on the
Management Server perspective.
Prerequisites for configuring SNMPv3 settings are as follows:
• Install Python 3.5.
• Install PySNMP.
To install PySNMP through pip run the following command:

pip install pysnmp
To install PySNMP off-line, the following packages must be downloaded and installed for
PySNMP to be operational:
PyASN1 - https://pypi.python.org/pypi/pyasn1
PySNMP - https://pypi.python.org/pypi/pysnmp/
PyCryptodomex - https://pypi.python.org/pypi/pycryptodomex/

08-18
Configuring the SNMPv3 console (2/2)
19
To configure SNMPv3 settings for SNMP traps

1. In the Home page on the Management Server console, click Settings.
2. In the Settings tab, click Management Server.
3. In the Server settings tab, under SNMPv3 Settings check the SNMPv3 checkbox.
4. Enter the User Name.
5. For authentication, check the Authentication checkbox and perform the following steps:
a) Select the Authentication Protocol from the drop-down.
b) Enter the Authentication Password.
6. For encryption, check the Privacy checkbox and perform the following steps:
a) Select the Privacy Protocol from the drop-down.
b) Enter the Privacy Password.
7. Enter the path for the python executable.
8. Click Save Settings.
9. To delete the settings, click Delete Settings.

08-19
VIOM 2FA
Requirements and pre-requisites for 2FA:
• It does not require special pre-requisites.
• It supports RHEL6, RHEL7 and Windows 2012.
• OpenSSL must be installed on the VIOM Management Server.
• In Linux, the OpenSSL is installed by default.
External CA
Login request
Request certificate
Client VIOM
20
2FA is a two factor authentication, it can also be a multi-factor authentication. It adds an

enhanced layer of security with the user authentication. There are different types of 2FAs
available like SMS 2FA, TOTP, pushed based 2FA and certificate-based Authentication. 2FA
authentication is required by the public sector companies. It is mandatory for their
compliance. It adds an extra layer of security to the product.
VIOM uses certificate-based authentication.
• Import third party signed SSL certificate to VIOM 7.4.
• Configure VIOM web server to authenticate client certificates.
Requirements and pre-requisites for 2FA:

• It does not require special pre-requisites.
• It supports RHEL6, RHEL7 and Windows 2012.
• OpenSSL must be installed on the VIOM Management Server.
• In Linux, the OpenSSL is installed by default.

08-20
2FA Architecture
21
A public key certificate that identifies a root certificate authority (CA). There can be one or
more intermediate certificates. Server, intermediate and root certificates are in PEM format.

08-21
Configure VIOM web server to authenticate Linux/Unix
Modify the Tomcat

Stop the web server. Start the web server.
server.xml file.
Client uses this certificate

Get the client certification
Import the certificate in for authentication while
from the CA in pkcs12
the browser. accessing the VIOM
format.
console.
22
Steps to configure VIOM web server to authenticate Linux/Unix.

Note: OpenSSl should be installed on the Veritas InfoScale Operations Manager CMs.
1. Stop the web server
/opt/VRTSsfmcs/bin/vomsc --stop web
2. Modify /opt/VRTSsfmcs/webgui/tomcat/conf/server.xml as follows,
a) Set clientauth=“true” in the connector section.
Add
truststorefile=“${vom.webgui.install.dir}/tomcat/cert.keystore”
truststorePass=“changeit” after clientAuth=“true”.

Note: The password must be the same as Keystore password.
b) Add<Opensslpath path =“/usr/bin/openssl”>
</Opensslpath> after the </Host> tag.
3. Start the web server.
/opt/VRTSsfmcs/bin/vomsc -- start web
4. Get the client certification from the CA in the pkcs12 format.
5. Import the certificate in the browser.
6. Use this certificate for authentication while accessing the VIOM console.

08-22
2FA: example (1/3)
Certificate signing request created.
23
The screenshot on the slide displays commands for creating certificate signing request for CA.

08-23
2FA: example (2/3)
24
The commands to create client CSR and certificates are displayed on the slide.

08-24
2FA: example (3/3)
Importing root CA to VOM server
Importing intermediate CA to VOM server
Importing the server certificate
Stopping the web server.
25
Location of root CA and Intermediate CA.

/2FA/new_certs/cacerts/certs/ca.cert.pem
/2FA/new_certs/intermediatecerts/certs/intermediate.cert.pem
The slide displays commands to import root and intermediate CA to the VIOM server.

08-25
VIOM APIs (1/3)
Perform Operation VIOM Web services APIs
To move managed host in the Data Center. https://veritasdomain.example.com:14161/vom/api/op/server/
{host_id}/moveto/datacenter/
To move managed host from Organization OE1 https://veritasdomain.example.com:14161/vom/api/
to child-Organization OE3 of Organization OE2. op/server/{host_id}/moveto_oe/OE2,OE3/
To move cluster in the Data Center. https://veritasdomain.example.com:14161/vom/api/
op/hadr/{cluster_id}/moveto/datacenter/
To move cluster from an organization OE1 to https://veritasdomain.example.com:14161/vom/api/
child-Organization OE3 of Organization OE2. op/hadr/{cluster_id}/moveto_oe/OE2,OE3/
To freeze service group with the parameter https://veritasdomain.example.com:14161/vom/api/

as persistent. op/hadr/servicegroup/{service_group_id}/{persistent}/freeze/
To unfreeze service group. https://veritasdomain.example.com:14161/vom/api/

op/hadr/servicegroup/{service_group_id}/unfreeze/
To clear fault on service group. https://veritasdomain.example.com:14161/vom/api/
op/hadr/servicegroup/{service_group_id}/host/{hostname}/clear/
To clear the clearadminwait state for service https://veritasdomain.example.com:14161/vom/api/
group. op/hadr/servicegroup/{service_group_id}/host/{hostname}/{fault}/clearadminwait/
26
The table on the slide displays a list of performing operations using Veritas InfoScale
Operations Manager Web services APIs for version 7.4.2.
APIs were introduced in the InfoScale 6.X version. Additional APIS were introduced in later
versions. Users can add these APIs to their scripts and perform the operations with out
logging to the VIOM GUI. During the maintenance activity, the APIS are useful when the user
is handling hundreds of clusters.
• VIOM APIs are accessible using the HTTPS protocol or using standard HTTPS client such as
cURL. REST
• APIs are used to query the VIOM server to discover and manage data like setting the
objects and listing their properties. APIs are used to manage user-defined attributes and to
perform some operations on objects like service group freeze and service group unfreeze
and so on.
Using the Veritas InfoScale Operations Manager Web services API, you can also perform the
following operations:
• Start and stop Virtual Business Services. This operation can be performed in
the Server and Availability perspective.
• Start and stop VVR replication.
• Run a recovery plan.
• Provision storage using a storage template. To perform this operation, you need to install
the Storage Provisioning and Enclosure Migration Add-on version 6.1.

08-26
VIOM APIs (2/3)
To flush service group. https://veritasdomain.example.com:14161/vom/api/
op/hadr/servicegroup/{service_group_id}/host/{hostname}/flush/
To get metadata for a host https://veritasdomain.example.com:14161/vom/api/meta/server/host
To define an extended attribute https://veritasdomain.example.com:14161/vom/api/meta/hadr/cluster/add?name=department

named department on object type cluster
To modify an extended attribute https://veritasdomain.example.com:14161/vom/api/meta/hadr/cluster/modify?name=departme
named department to Dept_new on object nt&new_name=Dept_new
type cluster
To delete an extended attribute https://veritasdomain.example.com:14161/vom/api/meta/hadr/cluster/delete?name=dept_new
named Dept_new on object type cluster
To update the extended attribute value for a https://veritasdomain.example.com:14161/vom/api/update/server/host/myhost_id?location=1st
host where the extended attribute is location floor
To filter disk group objects whose display https://veritasdomain.example.com:14161/vom/api/query/server/host/myhost_id/diskgroup?dis

type is private play_type=private
To run a recovery plan https://veritasdomain.example.com:14161/vom/api/op/hadr/recoveryplan/rplan_id/execute/
To Switch service group. https://veritasdomain.example.com:14161/vom/api/op/hadr/servicegroup/service_group_id/hos

t/{hostname}/switch/
27

08-27
VIOM APIs (3/3)
To provision storage using a storage https://veritasdomain.example.com:14161/vom/api/op/server/

template, enter the following URL to view template/storage/storage_template_id/provision
the payload information.
To online service group with the parameters https://veritasdomain.example.com:14161/vom/api/op/hadr/servicegroup/

as force or propogate. service_group_id/host/{hostname}/{force}/{propogate}/online
To offline service group with the parameters https://veritasdomain.example.com:14161/vom/api/op/hadr/servicegroup/

as force, probe, or propogate service_group_id/host/{hostname}/{force}/{probe}/{propogate}/offline
28

08-28
Topic: Overview of triggers
After completing this topic, you will be able to configure triggers to

customize VCS behavior in response to events.
29
This is the Overview of triggers topic.


08-29
Using triggers
Resource keys
• Triggers are:
– programs run by VCS when defined events occur. RESADMINWAIT
– an alternative method of notification. RESNOTOFF
– useful for customizing VCS behavior in response to events. RESSTATECHANGE
• The TriggersEnabled attribute: RESRESTART
– used to enable triggers. RESFAULT
– set to one or more keys, which

correspond to VCS events. Service group keys
– applies to resources and service groups.
PREONLINE
– when set at the service group, applies.
to all resources in a service group. POSTONLINE
– can be localized per cluster node. POSTOFFLINE
VIOLATION
NOFAILOVER
+ 3 resource keys
30
VCS provides an additional method for notifying users of important events. When VCS detects
certain events, you can configure a trigger to notify an administrator or perform other actions.
You can use event triggers in place of, or in conjunction with, notification.
Triggers are executable programs, batch files, shell or Perl scripts associated with the
predefined event types supported by VCS that are shown in the slide.
Triggers are configured by specifying one or more keys in the TriggersEnabled attribute. Some
keys are specific to service groups or resources.
The RESSTATECHANGE, RESRESTART, and RESFAULT keys apply to both resources and service
groups. When one of these keys is specified in TriggerPath at the service group level, the
trigger applies to each resource in the service group.
Examples of some trigger keys include:

• POSTOFFLINE: The service group went offline from a PARTIAL or ONLINE state.
• POSTONLINE: The service group went online from OFFLINE state.
• RESFAULT: A resource faulted.
• RESRESTART: A resource was restarted after a fault.
For a complete description of triggers, see the Veritas Cluster Server Administrator’s Guide.

08-30
Sample triggers
Sample triggers:
• Provided for each type of trigger.
• Can be copied and modified.
• Are located in:
/opt/VRTSvcs/bin/sample_triggers/VRTSvcs
more /opt/VRTSvcs/bin/sample_triggers/resfault
. . .
# Usage:
# resfault <system> <resource> <oldstate>
#
# <system>: is the name of the system where resource faulted.
# <resource>: is the name of the resource that faulted.
# <oldstate>: is the previous state of the resource that
# faulted.
#
# Possible values for oldstate are ONLINE and OFFLINE.
. . .
31
A set of sample trigger scripts is provided in

/opt/VRTSvcs/bin/sample_triggers/VRTSvcs. These scripts can be copied to
/opt/VRTSvcs/bin/triggers and modified to your specifications.
The sample scripts include comments that explain how the trigger is invoked and provide
guidance about modifying the sample trigger.
Note: In some cases, sample trigger scripts have not yet been updated and may still refer to
older implementations of triggers. For example, the preonline script still refers to setting
PreOnline attribute to 1 rather than using the newer TriggersEnabled attribute to configure
the PreOnline trigger.

08-31
Location of triggers
Triggers:
• are placed in /opt/vrtsvcs/bin/triggers by default.
• can be relocated by specifying triggerpath attribute at service group or resource level.
• enables customization of trigger scripts for different resources or service groups.
– Example: NoFailover trigger for websg
• TriggerPath = "bin/triggers/websg"
• Path to trigger: /opt/VRTSvcs/bin/triggers/websg
/opt/VRTSvcs/bin/triggers/websg/nofailover
group websg (
SystemList = { s1 = 0, s2 =1 } main.cf
TriggersEnabled = { NOFAILOVER }
TriggerPath = "bin/triggers/websg"
)
32
Trigger executable programs, batch files, shell or Perl scripts reside in

/opt/VRTSvcs/bin/triggers by default.
You can change the location of triggers by specifying the TriggerPath attribute at the service
group or resource level. This attribute enables you to set up different trigger programs for
resources or service groups. In previous versions of VCS, the same triggers applied to all
resources or service groups in the cluster.
The value of the TriggerPath attribute is appended to /opt/VRTSvcs (also referred to as
VCS_HOME) to form a directory containing the trigger programs. In the example shown in the
slide, TriggerPath is set to bin/triggers/websg. Therefore, the files executed when the
NOFAILOVER key is specified for the websg service group must be located in
/opt/VRTSvcs/bin/trigers/websg.
The example portion of the main.cf file shows the NOFAILOVER trigger enabled for websg
(globally), and the trigger path customized to map to
/opt/VRTSvcs/bin/triggers/websg.

08-32
Example configuration
E-mail notification of any resource fault in the websg service group:
1. Copy the resfault script from the sample_triggers/VRTSvcs directory to the triggers directory.
2. Edit the copied resfault script.
2. Verify executable permission is set for root.
3. Copy the trigger script to all other cluster nodes.
4. Set TriggersEnabled to RESFAULT for websg for all systems.
hagrp –modify websg TriggersEnabled RESFAULT
33
This slide displays the basic procedure for creating a trigger using a sample script provided
with VCS.
In this case, the resfault script is copied from the sample_triggers directory and then modified
to use the Linux /bin/mail program to send e-mail to the modified recipients list.
The only changes required to make use of the sample resfault trigger in this example are the
following two lines:
@recipients=("student\@mgt.example.com");
. . .
"/bin/mail -s resfault $recipient < $msgfile";
After a trigger is modified, you must ensure the file is executable by root, and then copy the
script or program to each system in the cluster that can run the trigger. Finally, modify the
TriggersEnabled attribute to specify the key for each system that can run the trigger.

08-33
Using multiple scripts for a trigger
• Follows the UNIX startup script model:
rcN.d, Snumscript or Knumscript
• Scripts are executed in ascending order based on num.
• Example:
– TriggerPath set to bin/triggers/websg
– Multiple preonline scripts located in /TriggerPath/trigger_name
– Scripts are named Tnumscript.
Examples: T01backup, T02setenv, and so on
ls /opt/VRTSvcs/bin/triggers/websg/preonline/
T01backup
T02setenv
T03online
34
VCS supports the use of multiple scripts for a single trigger. This enables you to break the logic
of a trigger into components rather than having all trigger logic in one monolithic script.
The number contained in the file name determines the order in which the scripts are run,
similar to legacy UNIX startup and kill scripts in rcN.d directories. To use multiple files for a
single trigger, you must specify a custom path using the TriggerPath attribute.

08-34
Lesson summary
• Key points
– In this lesson, you learned about VCS notification service and the functions of ClusterService group.
– In addition, you learned to configure notifications using the NotifierMngr resource.
– Finally, you learned to configure triggers to customize the notification facilities to meet your specific
requirements.
35

08-35
Lab 09: Configuring Notification
• Exercise A: Configuring and testing the notifier using VIOM

• Exercise B: Configuring trigger scripts
36

08-36
What did you learn?
lesson.
37
current lesson.

08-37
Question 1: Notification overview
Which process generates the appropriate SNMP trap?
A. hasnmp
B. hanotify
C. notifier
D. notifiermngr
38

08-38
Answer 1: Notification overview
Which process generates the appropriate SNMP trap?
A. hasnmp
B. hanotify
C. notifier
D. notifiermngr
The correct answer is C. The notifier process configures how messages are received from VCS and how they are
delivered to SNMP consoles and SMTP servers.
39

08-39
Question 2: Configuration notification
Which attribute can be set to send an e-mail message to the person designated as
responsible for a particular service group?
A. GroupOwner
B. ResourceOwner
C. GroupAdmin
D. GroupOperator
40

08-40
Answer 2: Configuration notification
Which attribute can be set to send an e-mail message to the person designated as
responsible for a particular service group?
A. GroupOwner
B. ResourceOwner
C. GroupAdmin
D. GroupOperator
The correct answer is A. GroupOwner attribute when set properly for specific service group enables all service group
events to be notified through email to specified user.
41

08-41
Question 3: Configuring notification
How do you make a notification highly available?
A. Configure a Process resource to monitor the notification daemon.

B. You cannot make notification highly available.
C. Add a NotifierMngr resource to the ClusterService group.
D. Configure a Phantom resource for the notification daemon.
42

08-42
Answer 3: Configuring notification
How do you make a notification highly available?
A. Configure a Process resource to monitor the notification daemon.

B. You cannot make notification highly available.
C. Add a NotifierMngr resource to the ClusterService group.
D. Configure a Phantom resource for the notification daemon.
43

08-43
Based on the notifier configuration example, who will receive a notification when a
service group goes online?
A. admin
B. operator
C. sysadmin
D. operator, admin, and sysadmin
SmtpServer = "smtp.acme.com"
SmtpRecipients = { "[email protected]" = Information
"[email protected]" = Warning "[email protected]" = Error }
PathName = "/opt/VRTSvcs/bin/notifier"
)
44

08-44
Based on the notifier configuration example, who will receive a notification when a
service group goes online?
A. admin
B. operator
C. sysadmin
D. operator, admin, and sysadmin
SmtpServer = "smtp.acme.com"
SmtpRecipients = { "[email protected]" = Information
"[email protected]" = Warning "[email protected]" = Error }
PathName = "/opt/VRTSvcs/bin/notifier"
)
The correct answer is B. Since the notification type is Information, it will be received by the operator as per the
configuration.
45

08-45
Question 5: Overview of triggers
When a resfault trigger script is present on all cluster systems, the trigger is run
when:
A. A resource faults in a service group with TriggersEnabled set to RESFAULT.

B. A critical resource faults on any system.
C. A resource faults and the service group has no failover target.
D. A resource faults in a service group with TriggerResStateChange=1.
46

08-46
Answer 5: Overview of triggers
When a resfault trigger script is present on all cluster systems, the trigger is run
when:
A. A resource faults in a service group with TriggersEnabled set to RESFAULT.

B. A critical resource faults on any system.
C. A resource faults and the service group has no failover target.
D. A resource faults in a service group with TriggerResStateChange=1.
The correct answer is A. The trigger will be executed when a resource faults in a service group with the attribute
TriggerEnabled set to RESFAULT.
47

08-47
OpenSSl should be installed on the Veritas InfoScale Operations Manager CMs.
A. True
B. False
48

08-48
OpenSSl should be installed on the Veritas InfoScale Operations Manager CMs.
A. True
B. False
49

08-49
08-50
Lesson 09: Handling Resource Faults
This is the Handling Resource Faults lesson in the Veritas InfoScale 7.4.2 Fundamentals for

09-1
Administration
Administration


09-2
Lesson objectives
Topic Objective
VCS response to resource faults Explain how VCS responds to resources faults.
Determine failover duration for a service group with traditional and

Determining failover duration
IMF monitoring.
Controlling fault behavior Control fault behavior using resource type attributes.
Recovering from resource faults Recover from resource faults.
Fault notification and event handling Configure fault notification and triggers.

09-3
Topic: VCS response to resource faults
After completing this topic, you will be able to explain how VCS responds to
resource faults.
This is thea VCS response to resource faults topic.


09-4
Failover decisions and critical resources
• A service group must have at least one critical resource to enable automatic failover.
• Other attributes modify this behavior, as described throughout this lesson.
Default VCS behavior for a failover service group:

• If a critical resource faults, the service group fails over.
• If any critical resource is taken offline as a result of a fault, the service group fails over.
Critical resources define the basis for failover decisions made by the VCS. When the monitor
entry point for a resource returns with an unexpected offline status, the action taken by the
VCS engine depends on whether the resource is critical.
By default, if a critical resource in a failover service group faults or is taken offline as a result
of another resource fault, VCS determines that the service group is faulted. VCS then fails the
service group over to another cluster system, as defined by a set of service group attributes.
The default failover behavior for a service group can be modified using one or more optional
service group attributes. Failover determination and behavior are described throughout this
lesson.

09-5
How VCS responds to resource faults
Resource goes Execute clean
offline unexpectedly entry point
Default
Fault the resource behavior
Take all resources

in path offline
Y Critical N Keep group

Fault group online resource
in path? partially online
Take entire
group offline
Failover N Bring group

target Keep group offline
available? online elsewhere
Y
VCS responds in a specific and predictable manner to faults. When VCS detects a resource
failure, it performs the following actions:
1. Instructs the agent to execute the clean entry point for the failed resource to ensure that
the resource is completely offline. The resource transitions to a FAULTED state.
2. Takes all resources in the path of the fault offline starting from the faulted resource up to
the top of the dependency tree.
3. If an online critical resource is part of the path that was faulted or taken offline, faults the
service group and takes the group offline to prepare for failover. If no online critical
resources are affected, no more action occurs.
4. Attempts to start the service group on another system in the SystemList attribute
according to the FailOverPolicy defined for that service group and the relationships
between multiple service group. Note: The state of the group on the new system prior to
failover must be offline (not faulted).
5. If no other systems are available, the service group remains offline. VCS also executes
certain triggers and carries out notification while it performs each task in response to
resource faults. The role of notification and event triggers in resource faults is explained in
detail later in this lesson.

09-6
The impact of service group attributes (1/2)
Resource goes NONE Place resource in

ManageFaults
offline unexpectedly ADMIN_WAIT state
ALL
Y Fault resource and

Frozen?
group
Path for default settings
N
Execute clean entry point
Fault the resource and the service group
Take all resources

in path offline
Several service group attributes can be used to change the default behavior of VCS while
responding to resource faults.
ManageFaults
The ManageFaults attribute can be used to prevent VCS from taking any automatic actions
whenever a resource failure is detected. Essentially, ManageFaults determines whether VCS or
an administrator handles faults for a service group.
If ManageFaults is set to the default value of ALL, VCS manages faults by executing the clean
entry point for that resource to ensure that the resource is completely offline, as shown
previously. This is the default value (ALL).
If this attribute is set to NONE, VCS places the resource in an ADMIN_WAIT state and waits for
administrative intervention. This is often used for service groups that manage database
instances. You may need to leave the database in its FAULTED state in order to perform
problem analysis and recovery operations.
Note: This attribute is set at the service group level. This means that any resource fault within
that service group requires administrative intervention if the ManageFaults attribute for the
service group is set to NONE.

09-7
These service group attributes are used to indicate that the service group is frozen due to an
administrative command. When a service group is frozen, all agent online and offline actions
are disabled.
• If the service group is temporarily frozen using the hagrp –freeze group
command, the TFrozen attribute is set to 1.
• If the service group is persistently frozen using the hagrp -freeze group
-persistent command, the Frozen attribute is set to 1.
• When the service group is unfrozen using the hagrp -unfreeze group
[-persistent] command, the corresponding attribute is set back to the default value
of 0.

09-8
The impact of service group attributes (2/2)
Critical N
1 Keep group
online resource
in path? partially online
Y
Take entire 0
AutoFailOver?
group offline
1
Choose a failover target from
SystemList based on FailOverPolicy
Y Failover N
Bring group Keep
target
online elsewhere available? group offline
This attribute determines whether automatic failover takes place when a resource or system
faults. The default value of 1 indicates that the service group should be failed over to other
available systems if at all possible. However, if the attribute is set to 0, no automatic failover is
attempted for the service group, and the service group is left in an OFFLINE | FAULTED state.

09-9
Topic: Determining failover duration
After completing this topic, you will be able to determine the duration of
service group failover with traditional and IMF monitoring.
10
This is the Determining failover duration topic.


09-10
Failover duration when a resource faults
• Service group failover time is the sum of the duration of each failover task.
• You can affect failover time behavior by setting resource type attributes.
Traditional monitoring
+ Detect the resource failure (< MonitorInterval).

+ Fault the resource.
+ Take the entire service group offline.
+ Select a failover target.
+ Bring the service group online on another system in the cluster.
= Failover duration
11
When a resource failure occurs, application services may be disrupted until either the
resource is restarted on the same system or the application services migrate to another
system in the cluster. The time required to address the failure is a combination of the time
required to:
• Detect the failure
When traditional monitoring is configured, a resource failure is only detected when the
monitor entry point of that resource returns an offline status unexpectedly. The resource
type attributes used to tune the frequency of monitoring a resource are MonitorInterval
(default of 60 seconds) and OfflineMonitorInterval (default of 300 seconds).
• Fault the resource
This is related to two factors:

• How much tolerance you want VCS to have for false failure detections
For example, in an overloaded network environment, the NIC resource can return an
occasional failure even though there is nothing wrong with the physical connection.
You may want VCS to verify the failure a couple of times before faulting the resource.
• Whether or not you want to attempt a restart before failing over
For example, it may be much faster to restart a failed process on the same system
rather than to migrate the entire service group to another system.

09-11
• Take the entire service group offline. In general, the time required for a resource to be
taken offline is dependent on the type of resource and what the offline procedure
includes. However, VCS enables you to define the maximum time allowed for a normal
offline procedure before attempting to force the resource to be taken offline. The resource
type attributes related to this factor are OfflineTimeout and CleanTimeout.
• Select a failover target. The time required for the VCS policy module to determine the
target system is negligible, less than one second in all cases, in comparison to the other
factors.
• Bring the service group online on another system in the cluster. In most cases, in order to
start an application service after a failure, you need to carry out some recovery
procedures. For example, a file system’s metadata needs to be checked if it is not
unmounted properly, or a database needs to carry out recovery procedures, such as
applying the redo logs to recover from sudden failures.
• Take these considerations into account when you determine the amount of time you want
VCS to allow for an online process. The resource type attributes related to bringing a
service group online are OnlineTimeout, OnlineWaitLimit, and OnlineRetryLimit.
For more information on attributes that affect failover, refer to the Veritas Cluster Server
Bundled Agents Reference Guide.

09-12
Adjusting monitoring
MonitorInterval
• Defines frequency of online monitoring
• Is set to default value of 60 seconds for most resource types
• Can be reduced for testing (for example, to 10 or 20 seconds)
Use caution when changing this value; lower values increase load on cluster systems.
OfflineMonitorInterval
• Defines frequency of offline monitoring
• Is set to default value of 300 seconds for most resource types
• Can be reduced for testing (for example, to 60 seconds)
! If you change a resource type attribute, you affect all resources of that type.
13
You can change some resource type attributes to facilitate failover testing. For example, you
can change the monitor interval to see the results of faults more quickly. You can also adjust
these attributes to affect how quickly an application fails over when a fault occurs.
MonitorInterval
This is the duration (in seconds) between two consecutive monitor calls for an online or
transitioning resource. The default is 60 seconds for most resource types.
OfflineMonitorInterval
This is the duration (in seconds) between two consecutive monitor calls for an offline
resource. If set to 0, offline resources are not monitored. The default is 300 seconds for most
resource types.
Refer to the Veritas Cluster Server Bundled Agents Reference Guide for the applicable monitor
interval defaults for specific resource types.

09-13
Adjusting timeouts
Timeout interval values define the maximum time within which the entry points must finish or be terminated.
OnlineTimeout and OfflineTimeout:

• Is set to 300 seconds by default
• Can be increased if all resources of a type require more time to be brought online or taken offline in your environment
• Can result in false resource faults when lowered if resources cannot respond in the interval specified.
MonitorTimeout: Set to 60 seconds by default for most resource types
Before modifying defaults:

• Measure the online and offline times outside of VCS.
• Measure the monitor time:
1. Fault the resource.
2. Issue a probe.
14
The attributes MonitorTimeout, OnlineTimeout, and OfflineTimeout indicate the maximum

time (in seconds) within which the monitor, online, and offline entry points must finish or be
terminated. The defaults for the OnlineTimeout and OfflineTimeout attributes are 300
seconds.
For best results, measure the length of time required to bring a resource online, take it offline,
and monitor it before modifying the defaults. Simply issue an online or offline command to
measure the time required for each action. The default for the MonitorTimeout attribute is 60
seconds. To measure the time taken to monitor a resource, fault the resource, and then issue
a probe, or bring the resource online outside of VCS control and issue a probe.

09-14
Topic: Controlling fault behavior
After completing this topic, you will be able to control fault behavior using
resource type attributes.
15
This is the Controlling fault behavior topic.


09-15
Type attributes related to resource faults
RestartLimit
• Controls the number of times a resource is restarted on the same system
before it is marked as FAULTED
• Is set to 0 by default
ConfInterval
Determines the amount of time that must elapse before restart and
tolerance counters are reset to zero Is set to 600 by default.
ToleranceLimit
• Enables the monitor entry point to return OFFLINE several times
before the resource is declared FAULTED
• Is set to 0 by default
16
Although the failover capability of VCS helps to minimize the disruption of application services
when resources fail, the process of migrating a service to another system can be time-
consuming. In some cases, you may want to attempt to restart a resource on the same system
before failing it over to another system.
Whether a resource can be restarted depends on the application service:
• The resource must be successfully cleared (taken offline) after failure.
• The resource must not be a child resource with dependent parent resources that must be
restarted.
If you have determined that a resource can be restarted without impacting the integrity of the
application, you can potentially avoid service group failover by configuring the RestartLimit,
ConfInterval, and ToleranceLimit resource type attributes.
For example, you can set the ToleranceLimit to a value greater than 0 to allow the monitor
entry point to run several times before a resource is determined to be faulted. This is useful
when the system is very busy and a service, such as a database, is slow to respond.

09-16
Restart example
The resource is restarted one time within the

RestartLimit = 1 ConfInterval timeframe.
ConfInterval = 180 The resource can be restarted once within a
three-minute interval.
MonitorInterval = 60 The resource is monitored every 60 seconds.
On On On Off On Off
Monitor
Faulted
Confidence
Restart 0 0 0 1 1 1
Counter
17
This example illustrates how the RestartLimit and ConfInterval attributes can be configured for
modifying the behavior of VCS when a resource is faulted. Setting RestartLimit = 1 and
ConfInterval = 180 has this effect when a resource faults:
1. The resource stops after running for 10 minutes.
2. The next monitor returns offline.
3. The ConfInterval counter is set to 0.
4. The agent checks the value of RestartLimit.
5. The resource is restarted because RestartLimit is set to 1, which allows one restart within
the ConfInterval counter
6. The next monitor returns online.

7. The ConfInterval counter is now 60; one monitor cycle has completed.
8. The resource stops again.
9. The next monitor returns offline.
10. The ConfInterval counter is now 120; two monitor cycles have completed.
11. The resource is not restarted because the RestartLimit counter is now 1 and the
ConfInterval counter is 120 (seconds). Because the resource has not been online for the
ConfInterval time of 180 seconds, it is not restarted.
12. VCS faults the resource. If the resource had remained online for 180 seconds, the internal
RestartLimit counter would have been reset to 0.

09-17
Modifying resource type attributes
• Can be used to optimize agents

• Is applied to all resources of the specified type
hatype –modify NIC ToleranceLimit 2
types.cf
type NIC (
. . .
static int ToleranceLimit = 2
static str ArgList[] = { Device, …
. . .
)
18
You can modify the resource type attributes to affect how an agent monitors all resources of a
given type. For example, agents usually check their online resources every 60 seconds. You
can modify that period so that the resource type is checked more often. This is good for either
testing situations or time-critical resources.
You can also change the period so that the resource type is checked less often. This reduces
the load on VCS overall, as well as on the individual systems, but increases the time it takes to
detect resource failures.
For example, to change the ToleranceLimit attribute for all NIC resources so that the agent
ignores occasional network problems, type hatype -modify NIC ToleranceLimit
2

09-18
Overriding resource type attributes
Resource type attributes apply to all resources of that type and can be overridden to change its value for a
specific resource.
hares
–override webnic MonitorInterval Override MonitorInterval

–modify webnic MonitorInterval 10 Modify overridden attribute
–display –ovalues webnic Display overridden values
–undo_override webnic MonitorInterval Restore default settings
main.cf
NIC webnic(
Device = eth0
. . .
MonitorInterval=10
. . .
)
19
Resource type attributes apply to all resources of that type. You can override a resource type
attribute to change its value for a specific resource. Use the options to hares shown on the
slide to override resource type attributes.
Note: The configuration must be in read-write mode in order to modify and override resource
type attributes. The changes are reflected in the main.cf file only after you save the
configuration using the haconf -dump command.
Some predefined static resource type attributes (those resource type attributes that do not
appear in types.cf unless their value is changed, such as MonitorInterval) and all static
attributes that are not predefined (static attributes that are defined in the type definition file)
can be overridden. For a detailed list of predefined static attributes that can be overridden,
refer to the Veritas Cluster Server Administrator’s Guide.

09-19
Topic: Recovering from resource faults
After completing this topic, you will be able to recover from resource faults.
20
This is the Recovering from resource faults topic.


09-20
Recovering a resource from a faulted state
To clear a non-persistent resource fault:

1. Verify that:
– The fault is fixed outside of VCS
– The resource is completely offline
2. Run hares –clear resource to clear the FAULTED status
To clear a persistent resource fault:

1. Ensure that the fault is fixed outside of VCS
2. Either wait for the monitor cycle to run, or probe the resource manually:
hares –probe resource –sys system
21
When a resource failure is detected, the resource is put into a FAULTED or an ADMIN_WAIT
state depending on the cluster configuration. In either case, administrative intervention is
required to bring the resource status back to normal.
Recovering a resource from a faulted state
A critical resource in FAULTED state cannot be brought online on a system. When a critical
resource is FAULTED on a system, the service group status also changes to FAULTED on that
system, and that system can no longer be considered as an available target during a service
group failover.
You have to clear the FAULTED status of a nonpersistent resource manually. Before clearing
the FAULTED status, ensure that the resource is completely offline and that the fault is fixed
outside of VCS.
Note: You can also run hagrp -clear group [-sys system] to clear all FAULTED
resources in a service group. However, you have to ensure that all of the FAULTED resources
are completely offline and the faults are fixed on all the corresponding systems before running
this command.
The FAULTED status of a resource is cleared when the monitor returns an online status for
that resource. Note that offline resources are monitored according to the value of
OfflineMonitorInterval, which is 300 seconds (five minutes) by default. To avoid waiting for
the periodic monitoring, you can initiate the monitoring of the resource manually by probing
the resource.

09-21
Topic: Fault notification and event handling
After completing this topic, you will be able to configure fault notification
and triggers.
22
This is the Fault notification and event handling topic.


09-22
Fault notification
A resource becomes Send notification (Error);
offline unexpectedly e-mail ResourceOwner, ResourceRecipients
Send notification (Warning);

A resource state is unknown e-mail ResourceOwner, ResourceRecipients
Service group Send notification (SevereError);

concurrency fault e-mail GroupOwner, GroupRecipients
The service group is Send notification (Information);

brought online or taken e-mail GroupOwner, GroupRecipients
offline successfully
Send notification (Error)

The failover target
e-mail GroupOwner, GroupRecipients
does not exist
23
As a response to a resource fault, VCS carries out tasks to take resources or service groups
offline and to bring them back online elsewhere in the cluster. While carrying out these tasks,
VCS generates certain messages with a variety of severity levels and the VCS engine passes
these messages to the notifier daemon. Whether these messages are used for SNMP
traps or SMTP notification depends on how the notification component of VCS is configured.
The following events are examples that result in a notification message being generated:
• A resource becomes offline unexpectedly; that is, a resource is faulted.
• VCS cannot determine the state of a resource.
• A failover service group is online on more than one system.
• The service group is brought online or taken offline successfully.

• The service group has faulted on all nodes where the group could be brought online, and
there are no nodes to which the group can fail over.

09-23
Event handling
Resource becomes
Call resfault, resstatechange
offline unexpectedly
Resource cannot
Call resnotoff
be taken offline
Resource is placed
Call resadminwait
in ADMIN_WAIT
Successful resource
Call resstatechange
online/offline
Resource restarted Call resrestart
No failover target
Call nofailover
exists
24
You can use triggers to customize how VCS responds to events that occur in the cluster. For
example, you could use the ResAdminWait trigger to automate the task of taking diagnostics
of the application as part of the failover and recovery process. If you set ManageFaults to
NONE for a service group, VCS places faulted resources into the ADMIN_WAIT state. If the
resadminwait trigger is configured, VCS runs the script when a resource enters ADMIN_WAIT.
Within the trigger script, you can run a diagnostic tool and log information about the
resource, and then take a desired action, such as clearing the state and faulting the resource:
hagrp -clearadminwait -fault group -sys system
Lets look at the role of triggers in resource faults. As a response to a resource fault, VCS
carries out tasks to take resources or service groups offline and to bring them back online
elsewhere in the cluster. While these tasks are being carried out, certain events take place. If
corresponding event triggers are configured, VCS executes the triggers, as shown in the slide.
Triggers are placed in the /opt/VRTSvcs/bin/triggers directory by default. Sample
trigger scripts are provided in /opt/VRTSvcs/bin/ sample_triggers. Trigger
configuration is described in the “Configuring Notification” lesson and the VERITAS Cluster
Server Administrator’s Guide.

09-24
Lesson summary
• Key points
– In this lesson, you learned how to customize VCS responses to faults by configuring attributes.
– You also learned to adjust the failover duration to meet specific requirements.
25

09-25
Lab 10: Handling Resource Faults
• Exercise A: Observing non-critical resource faults
• Exercise B: Observing critical resource faults
• Exercise C: (Optional) Observing faults in frozen service groups
• Exercise D: (Optional) Observing ManageFaults behavior
• Exercise E: (Optional) Observing restart limit behavior
26

09-26
What did you learn?
lesson.
27
current lesson.

09-27
Question 1: VCS response to resource faults
If a critical resource in a service group faults, the default behavior of VCS is:
A. Take offline only the resources in the service group that are dependent on the
faulted resource.
B. Take offline all resources in the service group on the current node and attempt to fail
the service group over to another node.
C. Report the faulted resource, but continue running the service group on the current
node.
D. Attempt to fail the faulted resource over to another node.
28

09-28
Answer 1: VCS response to resource faults
If a critical resource in a service group faults, the default behavior of VCS is:
A. Take offline only the resources in the service group that are dependent on the
faulted resource.
B. Take offline all resources in the service group on the current node and attempt to fail
the service group over to another node.
C. Report the faulted resource, but continue running the service group on the current
node.
D. Attempt to fail the faulted resource over to another node.
The correct answer is B. Failure of a critical resource will take all the resources offline that belongs to
corresponding service group and attempt failover of the service group to another node.
29

09-29
If a non-critical resource in a service group faults, and only one online resource is a
parent of the faulted resource, the default VCS behavior is:
A. Take no action when noncritical resources fail.

B. Fail the service group to another node.
C. Fail the service group to another node only if the parent resource is critical.
D. Fail the service group to another node only if the parent resource is offline.
30

09-30
If a non-critical resource in a service group faults, and only one online resource is a
parent of the faulted resource, the default VCS behavior is:
A. Take no action when noncritical resources fail.

B. Fail the service group to another node.
C. Fail the service group to another node only if the parent resource is critical.
D. Fail the service group to another node only if the parent resource is offline.
The correct answer is C. If the parent resource is critical resource then in that case only it will failover to another
node.
31

09-31
If a resource is faulted on all nodes, clearing the fault on one node:
A. Allows the resource to be brought online on any node

B. Allows the resource to be brought online only on that particular node
C. Clears the fault on all nodes automatically
D. Fixes the underlying operating system or hardware problem
32

09-32
If a resource is faulted on all nodes, clearing the fault on one node:
A. Allows the resource to be brought online on any node

B. Allows the resource to be brought online only on that particular node
C. Clears the fault on all nodes automatically
D. Fixes the underlying operating system or hardware problem
The correct answer is B. A faulted resource can be attempted online on the node where it failed, you need to clear
the fault first.
33

09-33
Question 4: Recovering from resource faults
What does probing a resource do?
A. It forces the agent to run the clean entry point to clear internal states.
B. It forces the agent to monitor the resource and report the resource status.
C. It checks the configuration of the resource for syntax errors.
D. It clears transitional states of the resource.
34

09-34
Answer 4: Recovering from resource faults
What does probing a resource do?
A. It forces the agent to run the clean entry point to clear internal states.
B. It forces the agent to monitor the resource and report the resource status.
C. It checks the configuration of the resource for syntax errors.
D. It clears transitional states of the resource.
The correct answer is B. Probing a resource forces the corresponding agent to run a monitor
cycle on that resource and report it status.
35

09-35
Setting the ____________ service group attribute to NONE causes VCS to place faulted
resources in an ADMIN_WAIT state.
A. ResFaults
B. AdminWait
C. ResWait
D. ManageFaults
36

09-36
Setting the ____________ service group attribute to NONE causes VCS to place faulted
resources in an ADMIN_WAIT state.
A. ResFaults
B. AdminWait
C. ResWait
D. ManageFaults
The correct answer is D. ManageFaults attribute control the failover behavior of a service group. If its value is
selected to NONE then service group enters ADMIN_WAIT state and no failover happens in case of service
group fault.
37

09-37
End of presentation
09-38
Lesson 10: Intelligent Monitoring Framework
This is the Intelligent Monitoring Framework lesson in the Veritas InfoScale 7.4.2

10-1
Administration
Administration


10-2
Lesson objectives
Topic Objective
Describe how the Intelligent Monitoring Framework (IMF) improves

IMF overview
fault detection.
IMF configuration Configure IMF for supported agents.
Faults and failover with intelligent

Determine how VCS behaves when IMF monitoring detects faults.
monitoring

10-3
Topic: IMF overview
After completing this topic, you will be able to describe how the Intelligent
Monitoring Framework (IMF) improves fault detection.
This is the IMF overview topic.


10-4
Drawbacks of traditional monitoring
• Agents poll resources periodically Traditional monitoring
• Polling interval:
– Defined by MonitorInterval OfflineMonitorInteval attributes
– Default values are 60 and 300 seconds, respectively
HAD
• Fault detection bound by MonitorInterval
• Adding resources => Increased system load
Agent framework
Faulted
Resources
Faulting…
Agents tightly coupled with resources
The Intelligent Monitoring Framework was created to meet customer demands for supporting
increasing numbers of highly available services. Some environments are supporting large
numbers of resources (hundreds of mount points, for example) running on already loaded
systems.
With traditional monitoring, VCS agents poll each resource every 60 seconds, by default. This
can add a substantial system load in large-scale environments. The periodic nature of
traditional monitoring, coupled with the requirement to run the monitor process for each
resource, results in the state of the resource being unknown between monitor cycles, and
requires additional system resources.

10-5
IMF event-driven notification
• Extremely fast failure detection and remediation: Intelligent Monitoring Framework (IMF)
– Agent registers resource with the AMF kernel driver and
provides resource specific information
– Agent blocks and awaits notification on resource status change
HAD
– Resource status change instantaneously relayed to the blocking
agent for handling
Agent framework
• Significant reduction in system resource utilization thus
devoting key resources for other processing AMF driver
Faulted
Resources
Registering…
Agents, loosely coupled with resources, wait for

notification from AMF driver
The intelligent monitoring framework is a notification-based mechanism that minimizes load

on the system and provides immediate notification of faults. IMF is implemented through the
asynchronous monitoring framework (AMF) kernel module.
When IMF monitoring is configured for a resource, the agent registers the resource with the
AMF module. The AMF module receives event notifications from the operating system when a
registered resource changes states. The AMF module passes the notification to the agent for
handling, as described later in the lesson.

10-6
Agents with IMF support
OS specific agents Common Bundled agents

• Zone and LDom (Solaris) • Process
• WPAR (AIX) • Application
• Mount
Enterprise agents • DiskGroup
• DB2 : Db2udb • Apache HTTP server
• Sybase : Sybase and SybaseBk
• Oracle : Oracle and Netlsnr
IMF was introduced in VCS 5.1 SP1, with every major release agents were further added to
IMF supported list. The agents listed on the right side of the slide are common to all
supported OS and supports IMF monitoring.
In addition, you can create custom agents that use IMF monitoring by linking the AMF plug-ins
with the script agent and creating an XML file to enable registration with the AMF module. For
more information about using IMF monitoring for custom agents, see the VCS Agent
Developer’s Guide.
Note: There are other VCS agents available through the agent pack on the SORT Web site
(https://sort.veritas.com/agents).
Some of these application agents, such as WebSphereMQ also support IMF. Refer to the
specific application agent documentation to check if the agent supports IMF.

10-7
Topic: IMF configuration
After completing this topic, you will be able to describe how to configure
IMF.
This is the IMF configuration topic.


10-8
IMF modes
Actions configured using IMF attribute Mode key: Default setting is 3.
0 — No intelligent monitoring
1 — Intelligent monitoring for offline resources; poll-based for
online resources
2 — Intelligent monitoring for online resources; poll-based
monitoring for offline resources
3 — Intelligent monitoring for both online and offline resources
The Mode key of the IMF attribute determines whether IMF or traditional monitoring is
configured for a resource. Accepted values are:
0—Does not perform intelligent resource monitoring
1—Performs intelligent resource monitoring for offline resources and performs poll-based
monitoring for online resources
2—Performs intelligent resource monitoring for online resources and performs poll-based
monitoring for offline resources
3—Performs intelligent resource monitoring for both online and for offline resources

10-9
MonitorFreq and RegisterRetryLimit
Default setting is 5.
MonitorFreq
• Frequency at which agent invokes the monitor agent function.

• Monitor function is called by agent as follows:
− For online components: (MonitorFreq X MonitorInterval) number of seconds.
− For offline components: (MonitorFreq X OfflineMonitorInterval) number of seconds.
RegisterRetryLimit
• Number of times the agent must retry registration for a component.
Default setting is 3. • If unable to register, intelligent monitoring is disabled for that component.
10
MonitorFreq is a key value that specifies the frequency at which the agent invokes the
monitor agent function. The value of this key is an integer.
After the component is registered for IMF-based monitoring, the agent calls the monitor
agent function as follows:
• For online components: (MonitorFreq X MonitorInterval) number of seconds.
• For offline components: (MonitorFreq X OfflineMonitorInterval) number of seconds.
For agents that support IMF, the default value is 5. You can set this attribute to a non-zero
value in cases where the agent requires to perform poll-based monitoring in addition to the
intelligent monitoring.
RegisterRetryLimit is a key value that determines the number of times the agent must retry
registration for a component. If the agent cannot register the component within the limit that
is specified, then intelligent monitoring is disabled until the component state changes or the
value of the Mode key changes. The default value is 3.

10-10
Combining IMF and poll-based monitoring
• Set IMF Mode key to 1, 2, or 3
• Set MonitorFreq > 0
Example for Process resource
Process sendmail(
Pathname = "/usr/bin/sendmail"
Arguments = "-db –q3om"
PidFile = "/var/run/sendmail.pid
. . .
IMF Mode=2, MonitorFreq=5
. . .
)
When sendmail is online, monitor entry point runs every 5 x 60 seconds
11
The configuration snippet in the slide shows a Process resource with IMF enabled for
monitoring online resources and traditional poll-based monitoring for offline resources. For
Process resources that are offline, the agent runs monitor entry point periodically as specified
by the OfflineMonitorInterval attribute.
With MonitorFreq set to 5, the agent runs the monitor entry point periodically, calculated by
multiplying the value of MonitorFreq (5) by MonitorInterval (60 seconds), which results in a
poll-based monitor occurring every five minutes for an online resource. Poll-based monitoring
is performed by checking the process table for the process IDs listed in the PidFile.

10-11
VCS Cloned agent support for IMF/AMF
• The Application agent is used to make applications highly available when ISV agents are not available.
• VCS operator permission specific to the application’s expert is assigned to the service group that corresponds to an
application.
• In case of multiple such applications, provision to specify different VCS operation permission is required for each
application.
• In case of failure of an application, logs should be separate for each application.
• Clone the Application agent (binaries and type definition) and configure their resources in a separate service groups for
each application.
• Assign appropriate operator permissions to each service group managing an application.
• IMF environment seamlessly identifies each cloned Application agents, and makes them IMF aware.
12
InfoScale lets you clone the Application agent so that you can configure a different service
group for each application. You must then assign the appropriate operator permissions for
each service group for it to function as expected.
The Application agent is IMF-aware and uses asynchronous monitoring framework (AMF)
kernel driver for IMF notification. For more information about IMF and intelligent resource
monitoring, refer to the Cluster Server Administrator's Guide.

10-12
IMF support for NFS Mount
Intelligent Monitoring Framework (IMF) uses AMF kernel driver for immediate notifications, enabling VCS to detect
state changes instantly.
In InfoScale 7.4.1, Mount agent is IMF-aware, and can process notification for VxFS, EXT4 and XFS.
Benefits of IMF support for NFS file system are:
• Detecting NFS Mount instantly
• Detecting NFS Unmount instantly InfoScale 7.4.2 adds IMF

support to NFS file systems.
• Fast fault detection
• Fast failover
• CPU conservation
13
Intelligent Monitoring Framework (IMF) uses AMF kernel driver for immediate notifications,
enabling VCS to detect state changes instantly.
In InfoScale 7.4.1, Mount agent is IMF-aware, and can process notification for VxFS, EXT4 and
XFS. The slide lists benefits of IMF support for NFS file system. For the requirements and
supportability, refer to the Cluster Server Administrator's Guide.

10-13
Topic: Faults and failover with intelligent monitoring
After completing this topic, you will be able to determine how VCS behaves
when IMF monitoring detects faults.
14
This is the Faults and failover with intelligent monitoring topic.


10-14
Failover duration when a resource faults
• Service group failover time is the sum of the duration of each failover task.
• You can affect failover time by modifying the IMF mode setting
IMF monitoring
+ Detect the resource failure:
• AMF driver notifies agent of state change
• Agent runs monitor entry point to determine state
+ Fault the resource.
+ Take the entire service group offline.
+ Select a failover target.
= Failover duration
15
Failover duration for service groups when an IMF-monitored resource is determined in similar
fashion to the process described in the “Handling Resource Faults” lesson.
The key difference is the time required to detect a resource fault. Depending on when a
resource faults in the traditional poll-based model, the detection of the fault can take up to 60
seconds. For IMF-monitored resources, a fault is detected and the agent probes the resource
immediately to determine the resource state.
When a process dies or hangs, the operating system generates an alert. The agent is
registered to receive such alerts from the operating system, through the AMF kernel module.
The agent then probes the resource to determine the state and notifies HAD if the resources
is faulted. HAD can then take action within seconds of a resource fault, rather than minutes,
as with poll-based monitoring.

10-15
Lesson summary
• Key points
– In this lesson, you learned about the Intelligent monitoring framework.
– You also learned about combining traditional poll-based monitoring and intelligent monitoring.
– Finally, you also learned to adjust resource attributes to meet specific application requirements.
– Veritas Cluster Server Administrator's Guide
16

10-16
Lab 11: Intelligent Monitoring Framework
• Exercise A: Examining IMF monitoring on a resource
• Exercise B: (Optional) Examining the IMF default configuration
17

10-17
What did you learn?
lesson.
18
current lesson.

10-18
Question 1: IMF configuration
How is resource monitoring performed if the IMF mode is set to 2?
A. IMF for offline resources, poll-based for online resources

B. IMF for online resources, poll-based for offline resources
C. Traditional poll-based monitoring for both offline and online resources
D. IMF monitoring for both offline and online resources
19

10-19
Answer 1: IMF configuration
How is resource monitoring performed if the IMF mode is set to 2?
A. IMF for offline resources, poll-based for online resources

B. IMF for online resources, poll-based for offline resources
C. Traditional poll-based monitoring for both offline and online resources
D. IMF monitoring for both offline and online resources
The correct answer is B. When IMF mode is set to 2 then Intelligent monitoring for online resources and poll-based
monitoring for offline resources is conducted.
20

10-20
Question 2: IMF configuration
If IMF Mode is set to 2, MonitorFreq is 4, and MonitorInterval is 60, how often is poll-
based monitoring performed for online resources?
A. Every minute
B. Never
C. Every four minutes
D. Every two minutes
21

10-21
If IMF Mode is set to 2, MonitorFreq is 4, and MonitorInterval is 60, how often is poll-
based monitoring performed for online resources?
A. Every minute
B. Never
C. Every four minutes
D. Every two minutes
The correct answer is C. After 4 monitor cycles(MonitorFreq=4) of 60 seconds (MonitorInterval=60) that is 4 minutes.
22

10-22
Question 3 IMF configuration
If IMF mode is set to 1, LevelTwoMonitorFreq is 5, and MonitorFreq is 0, how often does
second level monitoring occur for online resources?
A. Every five minutes

B. Never
C. Every minute
D. Every ten minutes
23

10-23
If IMF mode is set to 1, LevelTwoMonitorFreq is 5, and MonitorFreq is 0, how often does
second level monitoring occur for online resources?
A. Every five minutes

B. Never
C. Every minute
D. Every ten minutes
24

10-24
Question 4: Faults and failover with intelligent monitoring
What configuration is necessary to enable only IMF monitoring for all online and offline
resources of type Process?
A. haimfconfig –enable
hatype –modify Process IMF –update Mode 3
B. hatype –modify Process IMF –update Mode 3
C. hatype –modify Process IMF –update Mode 1
D. haimfconfig –agent -Process
25

10-25
Answer 4: Faults and failover with intelligent monitoring
What configuration is necessary to enable only IMF monitoring for all online and offline
resources of type Process?
A. haimfconfig –enable
B. hatype –modify Process IMF –update Mode 3
C. hatype –modify Process IMF –update Mode 1
D. haimfconfig –agent -Process
The correct answer is B. IMF mode 3 enables intelligent monitoring for both online and offline resources.
26

10-26
Question 5: Faults and failover with intelligent monitoring
A. Restarts the resource

B. Fails over the service group
C. Marks the resource as faulted
D. Runs the monitor entry point
27

10-27
Answer 5: Faults and failover with intelligent monitoring
A. Restarts the resource

B. Fails over the service group
C. Marks the resource as faulted
D. Runs the monitor entry point
The correct answer is D. As the offline event is reported, monitoring entry point is initiated.
28

10-28
End of presentation
10-29
Lesson 11: Cluster Communications
This is the Cluster Communications lesson in the Veritas InfoScale 7.4.2 Fundamentals for

11-1
Administration
Administration


11-2
Lesson objectives
Topic Objective
VCS communications review Describe how components communicate in a VCS environment.
Describe the files that specify the cluster interconnect
Cluster interconnect configuration
configuration.
Describe how systems join the cluster membership and services
Cluster startup
come online.
System and cluster interconnect
Describe how VCS responds to common failures.
failure
Changing the interconnect
Change the cluster interconnect configuration.
configuration

11-3
Topic: VCS communications review
After completing this topic, you will be able to describe how components
communicate in a VCS environment.
This is the VCS communications review topic.


11-4
On-node and off-node communication
Agents Agents Agents
HAD HAD HAD

vxfen vxfen vxfen
GAB GAB GAB
LLT LLT LLT
LLT sends a heartbeat Each LLT module LLT forwards the

on each interface tracks the status of heartbeat status of
every ½ second. heartbeats from each node to GAB.
each peer on each
interface.
5
VCS maintains the cluster state by tracking the status of all resources and service groups in the
cluster. The state is communicated between HAD processes on each cluster system by way of
the atomic broadcast capability of Group Membership Services/Atomic Broadcast (GAB). HAD
is a replicated state machine, which uses the GAB atomic broadcast mechanism to ensure that
all systems within the cluster are immediately notified of changes in resource status, cluster
membership, and configuration.
Atomic means that all systems receive updates, or all systems are rolled back to the previous
state, much like a database atomic commit. If a failure occurs while transmitting status
changes, GAB’s atomicity ensures that, upon recovery, all systems have the same information
regarding the status of any monitored resource in the cluster.

11-5
VCS uses agents to manage resources within the cluster. Agents perform resource specific
tasks on behalf of HAD, such as online, offline, and monitoring actions. These actions can be
initiated by an administrator issuing directives using the VCS graphical or command-line
interfaces, or by other events that require HAD to take some action. Agents also report
resource status back to HAD. Agents do not communicate with one another, but only with
HAD.
The HAD processes on each cluster system communicate cluster status information over the
cluster interconnect.
In order to replicate the state of the cluster to all cluster systems, VCS must determine which
systems are participating in the cluster membership. This is accomplished by the group
membership services mechanism of GAB.
Cluster membership refers to all systems configured with the same cluster ID and
interconnected by a pair of redundant Ethernet LLT links. Under normal operation, all systems
configured as part of the cluster during VCS installation actively participate in cluster
communications.
Systems join a cluster by issuing a cluster join message during GAB startup. Cluster
membership is maintained by heartbeats. Heartbeats are signals sent periodically from one
system to another to determine system state. Heartbeats are transmitted by the LLT protocol.
VCS communications stack summary
The hierarchy of VCS mechanisms that participate in maintaining and communicating cluster
membership and status information is shown in the slide diagram.
• Agents communicate with HAD.
• The HAD processes on each system communicate status information by way of GAB.
• GAB determines cluster membership by monitoring heartbeats transmitted from each
system over LLT.

11-6
Cluster interconnect specifications
Link limit High priority Low priority
Up to eight links per node. • Heartbeats every half- • Heartbeats every second.
second. • No cluster status sent.
• Cluster status information • Automatically promoted to
carried over links. high priority if there are no
• Usually configured for high-priority links
dedicated cluster network functioning.
links. • Can be configured on public
network interfaces.
LLT can be configured to designate links as high-priority or low-priority links. High-priority links
are used for cluster communications (GAB) as well as heartbeats. Low-priority links carry only
heartbeats unless there is a failure of all configured high-priority links. At this time, LLT
switches cluster communications to the first available low-priority link. Traffic reverts to high-
priority links as soon as they are available.

11-7
GAB membership notation
20s placeholder
Cluster with four nodes: 0, 1, 21, 22 Nodes 0 and 1
# gabconfig -a
GAB Port Memberships
===============================================
Port a gen f7c001 membership 01 ; ;12
Port b gen f7c004 membership 01 ; ;12
Port h gen f7c002 membership 01 ; ;12
HAD is communicating 10s placeholder

Fencing is communicating (0 displayed if node 10
GAB is communicating is a member)
Cluster with 22 nodes Nodes 21 and 22
Port a gen a4e095 membership 0123456789012345678901
Port b gen a4e098 membership 0123456789012345678901
Port h gen a4e096 membership 0123456789012345678901
To display the cluster membership status, type gabconfig on each system. For example:
gabconfig -a
The first example in the slide shows:
• Port a, GAB membership, has four nodes: 0, 1, 21, and 22
• Port b, fencing membership, has four nodes: 0, 1, 21, and 22
• Port h, VCS membership, has four nodes: 0, 1, 21, and 22
Note: The port a, port b, and port h generation numbers change each time the membership
changes.
The gabconfig output uses a positional notation to indicate which systems are members of
the cluster. Only the last digit of the node number is displayed relative to semicolons that
indicate the 10s digit. The second example shows gabconfig output for a cluster with 22
nodes.

11-8
Topic: Cluster interconnect configuration
After completing this topic, you will be able to describe the files that specify
the cluster interconnect configuration.
This is the Cluster interconnect configuration topic.


11-9
The llttab file
• Assigns node numbers to systems.
• Sets the cluster ID number to identify a system with a cluster.
• Specifies the network devices used for the cluster interconnect.
• Modifies default LLT behavior, such as heartbeat frequency.
/etc/llttab
set-cluster 10
set-node s1
link nxge0 /dev/nxge:0 - ether - -
10
The VCS installation utility sets up all cluster interconnect configuration files and starts LLT and
GAB. You may never need to modify communication configuration files. Understanding how
these files work together to define the cluster communication mechanism helps you to
understand VCS behavior.
LLT configuration files
The LLT configuration files are located in the /etc directory.
The llttab file
The llttab file is the primary LLT configuration file and is used to:
• Set the cluster ID number.

• Set system ID numbers.
• Specify the network device names used for the cluster interconnect.
• Modify LLT behavior, such as heartbeat frequency.
Note: Ensure that there is only one set-node line in the llttab file. This is the minimum
recommended set of directives required to configure LLT. The basic format of the file is an LLT
configuration directive followed by a value. These directives and their values are described in
more detail in the next sections.
For a complete list of directives, see the sample_llttab file in the /opt/VRTS/llt
directory and the llttab manual page.

11-10
The llthosts file
• Associates a system name with a VCS cluster node ID.
• Has the same entries on all systems.
• Maps unique node numbers to system names.
• Matches system names with llttab and main.cf
• Matches system names with sysname
/etc/llthosts
0 s1
1 s2
The system (node) name does not need to be the UNIX host name found using the hostname command.
11
The llthosts file associates a system name with a VCS cluster node ID number. This file
must be present in the /etc directory on every system in the cluster. It must contain a line
with the unique name and node ID for each system in the cluster. The format is:
node_number name
The critical requirements for llthosts entries are:
• Node numbers must be unique. If duplicate node IDs are detected on the Ethernet LLT
cluster interconnect, LLT in VCS 4.0 is stopped on the joining node. In VCS versions before
4.0, the joining node panics.
• The system name must match the name in llttab if a name is configured for the set-
node directive (rather than a number).
• System names must match those in main.cf, or VCS cannot start.

Note: The system (node) name does not need to be the UNIX host name found using the
hostname command. However, Veritas recommends that you keep the names the same to
simplify administration, as described in the next section. See the llthosts manual page
for a complete description of the file.

11-11
How node and cluster numbers are specified
/etc/llttab
set-node s1
0 to 64k -1 set-cluster 10
link qfe0 /dev/qfe:0 - ether - -
link qfe4 /dev/qfe:4 - ether - -
/etc/llthosts
0 s1
0 to 63 1 s2
12
A unique number must be assigned to each system in a cluster using the set-node
directive. Each system in the cluster must have a unique llttab file, which has a unique
value for set-node, which can be one of the following:
• An integer in the range of 0 through 63 (64 systems per cluster maximum)
• A system name matching an entry in /etc/llthosts
The set-cluster directive
LLT uses the set-cluster directive to assign a unique number to each cluster. A cluster ID
is set during installation and can be validated as a unique ID among all clusters sharing a
network for the cluster interconnect.
Note: You can use the same cluster interconnect network infrastructure for multiple clusters.
The llttab file must specify the appropriate cluster ID to ensure that there are no
conflicting node IDs.
If you bypass the installer mechanisms for ensuring the cluster ID is unique and LLT detects
multiple systems with the same node ID and cluster ID on a private network, the LLT interface
is disabled on the node that is starting up. This prevents a possible split-brain condition,
where a service group might be brought online on the two systems with the same node ID.

11-12
The sysname file
• Contains a short-form system name used by VCS.
• Removes VCS dependency on UNIX uname.
• Is optional, but created by default during cluster configuration using CPI.
• Can be specified in llttab so the same file can be copied to multiple nodes.
/etc/VRTSvcs/conf/sysname
s1 s1 set-node /etc/VRTSvcs/conf/sysname
s1 s1
s1 s1 set-cluster 10
s1 s1
s1 s1
s1 s1 s1 s12 link dev1 /dev/dev:1 - ether - -
s1 s1 s12
s1 s1 link dev2 /dev/dev:2 - ether - -
s1 s1
s1 s1
s1 s12
13
The sysname file is an optional LLT configuration file that is configured automatically during
VCS installation. This file is used to store the short-form of the system (node) name. The
purpose of the sysname file is to enable specification of a VCS node name other than the
UNIX host name. This may be desirable, for example, when the UNIX host names are long and
you want VCS to use shorter names.
Note: If the sysname file contains a different name from the llttab/
llthosts/main.cf files, this “phantom” system is added to the cluster upon cluster
startup.
The sysname file can be specified for the set-node directive in the llttab file. In this
case, the llttab file can be identical on every node, which may simplify reconfiguring the
cluster interconnect in some situations. Refer the sysname manual page for a complete
description of the file.

11-13
The GAB configuration file
The /etc/gabtab file:
• Contains the command to start GAB:
/sbin/gabconfig -c -n number_of_systems
• Specifies the number of systems that must be communicating to allow VCS to start.
/etc/gabtab
/sbin/gabconfig –c –n 4
• The -n option must always be set to the total number of systems in the cluster.
14
GAB is configured with the /etc/gabtab file. This file contains one line that is used to start
GAB. For example: /sbin/gabconfig -c -n 4
This example starts GAB and specifies that four systems are required to be running GAB to
start within the cluster. The -n option must always be set to the total number of systems in
the cluster. A sample gabtab file is included in /opt/VRTSgab.
Note: Other gabconfig options are discussed later in this lesson. Refer the gabconfig
manual page for a complete description of the file.

11-14
Topic: Cluster startup
After completing this topic, you will be able to describe how systems join
the cluster membership and services come online.
15
This is the Cluster startup topic.


11-15
Seeding during startup
HAD 31 HAD HAD
I am vxfen 21 I am I am
vxfen vxfen
alive alive alive
GAB GAB GAB
1 LLT LLT LLT

s1 s2 s3
1 LLT starts on each system.

GAB starts on each system with a seed value equal to the number of systems in the
21 cluster: gabconfig –c –n 3
When GAB sees three members, the cluster is seeded.
31 HAD starts only after GAB is communicating on all systems.
16
GAB and LLT are started automatically when a system starts up. HAD can only start after GAB
membership has been established among all cluster systems. The mechanism that ensures
that all cluster systems are visible on the cluster interconnect is GAB seeding.
Seeding during startup
Seeding is a mechanism to ensure that systems in a cluster are able to communicate before
VCS can start. Only systems that have been seeded can participate in a cluster. Seeding is also
used to define how many systems must be online and communicating before a cluster is
formed.
By default, a system is not seeded when it boots. This prevents VCS from starting, which
prevents applications (service groups) from starting. If the system cannot communicate with
the cluster, it cannot be seeded.
Seeding is a function of GAB and is performed automatically or manually, depending on how
GAB is configured. GAB seeds a system automatically in one of two ways:
• When an unseeded system communicates with a seeded system
• When all systems in the cluster are unseeded and able to communicate with each other
The number of systems that must be seeded before VCS is started on any system is also
determined by the GAB configuration.

11-16
When the cluster is seeded, each node is listed in the port a membership displayed by
gabconfig -a. In the following example, all four systems (nodes 0, 1, 2, and 3) are seeded, as
shown by port a membership:
# gabconfig -a
GAB Port Memberships
=======================================================
LLT, GAB, and VCS startup files
These startup files are placed on the system when VCS is installed.
AIX
/etc/rc.d/rc2.d/S70llt Checks for /etc/llttab and runs
/sbin/lltconfig -c to start LLT
/etc/rc.d/rc2.d/S92gab Calls /etc/gabtab
/etc/rc.d/rc2.d/S99vcs Runs /opt/VRTSvcs/bin/hastart
Earlier Linux distributions

/etc/rc[2345].d/llt Checks for /etc/llttab and runs
/etc/rcX.d/gab Calls /etc/gabtab
/etc/rcX.d/vcs Runs /opt/VRTSvcs/bin/hastart
RHEL 7, SLES 12 Linux:

systemctl stop | start vxfen Runs /opt/VRTSvcs/vxfen/bin/vxfen
Solaris
/lib/svc/method/llt Checks for /etc/llttab and runs
/lib/svc/method/gab Calls /etc/gabtab
/lib/svc/method/vcs Runs /opt/VRTSvcs/bin/hastart

11-17
Probing resources during normal startup
1 A, B autodisabled for s1, s2
A B
21
Monitor
31 31
HAD HAD
s1 s2
1 During startup, HAD autodisables service groups.
HAD directs agents to probe (monitor) all resources on all systems in the SystemList
21
to determine their status.
If agents successfully probe resources, HAD brings service groups online according to
31
AutoStart and AutoStartList attributes (or where resources were running before
hastop –all –force).
18
During initial startup, VCS autodisables a service group until all its resources are probed on all
systems in the SystemList. When a service group is autodisabled, VCS sets the AutoDisabled
attribute to 1 (true), which prevents the service group from starting on any system. This
protects against a situation where enough systems are running LLT and GAB to seed the
cluster, but not all systems have HAD running.
In this case, port a membership is complete, but port h is not. VCS cannot detect whether a
service is running on a system where HAD is not running. Rather than allowing a potential
concurrency violation to occur, VCS prevents the service group from starting anywhere until all
resources are probed on all systems.
After all resources are probed on all systems, a service group can come online by bringing
offline resources online. If the resources are already online, as in the case where HAD has
been stopped with the hastop -all -force option, the resources are marked as online.

11-18
Topic: System and cluster interconnect failures
After completing this topic, you will be able to describe how VCS responds
to common failures.
19
This is the System and cluster interconnect failures topic.


11-19
VCS response to system failure
A C B C
s1 s2 s3
s3 faults; C started on s1 or s2 No membership: s3

Regular membership: s1, s2
20
The example cluster used throughout most of this section contains three systems, s1, s2, and
s3, each of which can run any of the three service groups, A, B, and C.
The abbreviated system and service group names are used to simplify the diagrams.
In this example, there are two Ethernet LLT links for the cluster interconnect. Prior to any
failures, systems s1, s2, and s3 are part of the regular membership of cluster number 1. When
the s3 system fails, it is no longer part of the cluster membership. Service group C fails over
and starts up on either s1 or s2, according to the SystemList and FailOverPolicy values.

11-20
Failover duration on a system failure
+ Detect the system failure—21 seconds for heartbeat timeouts.

+ Select a failover target—less than one second.
= Failover duration
21
When a system faults, application services that were running on that system are disrupted
until the services are started up on another system in the cluster. The time required to
address a system fault is a combination of the time required to:
• Detect the system failure.
A system is determined to be faulted according to these default timeout periods:
– LLT timeout: If LLT on a running system does not receive a heartbeat from a system for
16 seconds, LLT notifies GAB of a heartbeat failure.
– GAB stable timeout: GAB determines that a membership change is occurring, and after
five seconds, GAB delivers the membership change to HAD.
• Select a failover target. The time required for the VCS policy module to determine the
target system is negligible, less than one second in all cases, in comparison to the other
factors.
• Bring the service group online on another system in the cluster. As described in an earlier
lesson, the time required for the application service to start up is a key factor in
determining the total failover time.

11-21
Manual seeding
51
Seeded HAD Seeds HAD

vxfen vxfen 1
41
31 GAB GAB
21
LLT LLT
s1 s2 s3
1 s3 is down for maintenance. s1 and s2 are rebooted.
21 LLT starts on s1 and s2. GAB starts but cannot seed with s3 down.
31 Manually force GAB to seed on s1: gabconfig -x.
41 GAB on s2 now seeds because it can detect another seeded system (s1).
51 Start HAD on s1 and s2.

! Warning: Manual seeding can cause
split-brain condition.
22
You can override the seed values in the gabtab file and manually force GAB to seed a system
using the gabconfig command. This is useful when one of the systems in the cluster is out
of service and you want to start VCS on the remaining systems.
To seed the cluster if GAB is already running, use gabconfig with the –x option to override
the -n value set in the gabtab file. For example, type:
gabconfig -x
If GAB is not already started, you can start and force GAB to seed using -c and -x options to
gabconfig:
gabconfig -c -x
CAUTION: Only manually seed the cluster when you are sure that no other systems have GAB
seeded. In clusters that do not use I/O fencing, you can potentially create a split brain
condition by using gabconfig improperly.
After you have started GAB on one system, start GAB on other systems using gabconfig
with only the -c option. You do not need to force GAB to start with the -x option on other
systems. When GAB starts on the other systems, it determines that GAB is already seeded and
starts up.

11-22
Single LLT link failure
Regular membership: s1, s2, s3
A B C
s1 s2 s3
gabconfig -a
Port a gen a6e003 jeopardy ; 2
Port b gen a6e006 membership 012 Jeopardy membership: s3
Port b gen a6e006 jeopardy ; 2
Port h gen a6e004 membership 012
Port h gen a6e004 jeopardy ; 2
23
In the case where a node has only one functional LLT link, the node is a member of the regular
membership and the jeopardy membership. Being in a regular membership and jeopardy
membership at the same time changes only the failover behavior on system fault. All other
cluster functions remain. This means that failover due to a resource fault or switchover of
service groups at operator request is unaffected.
The only change is that other systems prevented from starting service groups on system fault.
VCS continues to operate as a single cluster when at least one network channel exists
between the systems.
In the example shown in the diagram where one LLT link fails:
• A jeopardy membership is formed that includes just system s3.
• System s3 is also a member of the regular cluster membership with systems s1 and s2.
• Service groups A, B, and C continue to run and all other cluster functions remain
unaffected.
• Failover due to a resource fault or an operator switch a service group is unaffected.
• If system s3 now faults or its last LLT link is lost, service group C is not started on systems
s1 or s2.

11-23
Interconnect failure and potential split brain condition
A C B A B C
21
s1 s2 s3
s1 and s2 determine that s3 is faulted. No jeopardy

1
occurs; no groups are autodisabled.
s3 determines that s1 and s2
1
If all systems are in all groups' SystemList, VCS are faulted.
21
tries to bring them online.
24
When both LLT links fail simultaneously:

• The cluster partitions into two separate clusters. No jeopardy membership is formed and
no service groups are autodisabled.
• Each cluster determines that the other systems are down and tries to start the service
groups.
If an application starts on multiple systems and can gain control of what are normally
exclusive resources, such as disks in a shared storage device, split brain condition results and
data can be corrupted.

11-24
Interconnect failures with a low-priority public link
A B C
21
1
s1 s2 s3
1 No change in membership. Jeopardy membership: s3

21 Public network is now used for
21 Regular membership: s1, s2, s3. heartbeat and status.
25
LLT can be configured to use a low-priority network link as a backup to normal heartbeat
channels. Low-priority links are typically configured on the public network or administrative
network.
In normal operation, the low-priority link carries only heartbeat traffic for cluster membership
and link state maintenance. The frequency of heartbeats is reduced by half to minimize
network overhead. When the low-priority link is the only remaining LLT link, LLT switches all
cluster status traffic over the link. Upon repair of any configured link, LLT switches cluster
status traffic back to the high-priority link.
Notes:
• Nodes must be on the same public network segment in order to configure low-priority
links. LLT is a non-routable protocol.

• You can have up to eight LLT links total, which can be a combination of low and high-
priority links. For example, if you have three high-priority links, you have the same
progression to jeopardy membership. The difference is that all three links are used for
regular heartbeats and cluster status information.

11-25
Topic: Changing the interconnect configuration
After completing this topic, you will be able to change the cluster
interconnect configuration.
26
This is the Changing the interconnect configuration topic.


11-26
Example reconfiguration scenarios s4
s3
• Manual configuration: s2
– Merging clusters. s1
– Changing parameters persistently across reboots, such
as the peer inactive timeout.
• Automatic changes by CPI:
Adding or removing cluster nodes.
set-node s1
• The lltconfig command:
set-cluster 10
– Changes parameters temporarily.
– Requires running on each node to affect cluster-wide.
. . .
• Use lltstat –c to display parameters. set-timer peerinact:1200
s7
s6
s5
27
You may never need to perform any manual configuration of the cluster interconnect because
the VCS installation utility sets up the interconnect based on the information you provide
about the cluster.
However, certain configuration tasks require you to modify VCS communication configuration
files, as shown in the slide.

11-27
Manually modifying the interconnect
/etc/llttab /etc/gabtab
set-node s1
set-cluster 10 /etc/llthosts path/gabconfig –c –n #

0 s1
1 s2 One system
Back up and
edit files All systems
haconf –dump -makero

Stop VCS Start VCS hastart
hastop –all -force
Stop fencing vxfenconfig -U Start fencing vxfen_startup_file
Stop GAB gabconfig -U Start GAB sh /etc/gabtab
Stop LLT lltcnfig -U Start LLT lltconfig -c
28
The procedure shown in the diagram can be used for any type of change to the VCS
communications configuration. The first task includes saving and closing the cluster
configuration before backing up and editing files.
Although some types of modifications do not require you to stop both GAB and LLT, using this
procedure ensures that any type of change you make takes effect.
For example, if you added a system to a running cluster, you can change the value of -n in the
gabtab file without having to restart GAB. However, if you added the -j option to change
the recovery behavior, you must either restart GAB or execute the gabtab command
manually for the change to take effect.
Similarly, if you add a host entry to llthosts, you do not need to restart LLT. However, if
you change llttab, or you change a host name in llthosts, you must stop and restart
LLT, and, therefore, GAB.
Following this procedure ensures that any type of changes take effect. You can also use the
scripts in the rc*.d directories to stop and start services. Use the UNIX or Linux-specific
method for starting and stopping services for fencing, GAB, and LLT.
• AIX: /etc/init.d/vxfen.rc stop|start
• RHEL 7, SLES 12 Linux: systemctl stop|start vxfen
• Earlier Linux distributions: /etc/init.d/vxfen stop|start
• Solaris: svcadm disable -t vxfen

11-28
Example LLT link specification
Range (all) SAP

/etc/llttab
set-node s1
set-cluster 10
# Solaris example
link-lowpri e1000g0 /dev/e1000g:0 - ether - -
Tag name Device:Unit Link type MTU
29
You can add links to the LLT configuration as additional layers of redundancy for the cluster
interconnect. You may want an additional interconnect link for:
• VCS for heartbeat redundancy
• Storage Foundation for Oracle RAC for additional bandwidth
To add an Ethernet link to the cluster interconnect:
1. Cable the link on all systems.
2. Use the process on the previous page to modify the llttab file on each system to add
the new link directive.
To add a low-priority public network link, add a link-lowpri directive using the same
syntax as the link directive, as shown in the llttab file example in the slide.
VCS uses the low-priority link only for heartbeats (at half the normal rate), unless it is the only
remaining link in the cluster interconnect.

11-29
Lesson summary
• Key points
– In this lesson, you learned about the cluster interconnect.
– You also learned to configure the cluster interconnect.
– Veritas Cluster Server Installation Guide
– Veritas Cluster Server Administrator's Guide
30

11-30
Lab 12: Cluster Communications
• Exercise A: Reconfiguring LLT

• Exercise B: Observing jeopardy membership
31

11-31
What did you learn?
lesson.
32
current lesson.

11-32
Question 1: Cluster interconnect configuration
Which files are required for LLT configuration?
A. llthosts, llttab
B. lltnodes, llttab
C. llttab, lltconfig
D. llttab, sysname
33

11-33
Answer 1: Cluster interconnect configuration
Which files are required for LLT configuration?
A. llthosts, llttab
B. lltnodes, llttab
C. llttab, lltconfig
D. llttab, sysname
The correct answer is A. /etc/llthosts and /etc/llttab files are responsible for LLT configuration.
34

11-34
Question 2: GAB configuration
Which command line can be used to start GAB in a two-node cluster?
A. gabstart -a 2
B. gabconfig -all
C. gabconfig -c -n 2
D. gabtab -c -n 2
35

11-35
Answer 2: GAB configuration
Which command line can be used to start GAB in a two-node cluster?
A. gabstart -a 2
B. gabconfig -all
C. gabconfig -c -n 2
D. gabtab -c -n 2
The correct answer is C. Option –n specifies the number of nodes.
36

11-36
Question 3 Cluster interconnect configuration
Which file is used to start GAB at boot time?
A. gabconf
B. gabstart
C. gabtab
D. gabconfig
37

11-37
Answer 3: Cluster interconnect configuration
Which file is used to start GAB at boot time?
A. gabconf
B. gabstart
C. gabtab
D. gabconfig
The correct answer is C. /etc/gabtab file is responsible for GAB configuration.
38

11-38
Question 4: Cluster startup
What is the purpose of seeding the cluster?
A. To ensure that the state of cluster systems is known before HAD is started.
B. To force GAB to communicate with HAD on each system in the cluster.
C. To determine which systems have transitioned from running to faulted.
D. To force HAD on each system to start sending heartbeats to each other system.
39

11-39
Answer 4: Cluster startup
What is the purpose of seeding the cluster?
A. To ensure that the state of cluster systems is known before HAD is started.
B. To force GAB to communicate with HAD on each system in the cluster.
C. To determine which systems have transitioned from running to faulted.
D. To force HAD on each system to start sending heartbeats to each other system.
The correct answer is A. HAD starts after the initial seeding is completed, it waits until all the systems seeds and join the
GAB. Once all the systems join GAB, state of all systems is known across the cluster.
40

11-40
Question 5: System and cluster interconnect failures
Why are service groups autodisabled during HAD startup?
A. To ensure that all service groups are started manually when VCS first starts up.
B. To determine where resources are online before starting any service groups.
C. To ensure that you manually reset the AutoDisabled attribute each time HAD starts
up.
D. Service groups are only autodisabled during HAD startup if they faulted before HAD
was shut down.
41

11-41
Answer 5: System and cluster interconnect failures
Why are service groups autodisabled during HAD startup?
A. To ensure that all service groups are started manually when VCS first starts up.
B. To determine where resources are online before starting any service groups.
C. To ensure that you manually reset the AutoDisabled attribute each time HAD starts
up.
D. Service groups are only autodisabled during HAD startup if they faulted before HAD
was shut down.
The correct answer is B. Service groups remain autodisabled until its resource state has been probed and determined on
all the cluster nodes.
42

11-42
End of presentation
11-43

Info Scale

Uploaded by

Copyright:

Available Formats

Info Scale

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Info Scale

Uploaded by

Copyright:

Available Formats

Veritas InfoScale 7.4.

Not for Distribution.

Course Developer Lead Subject Matter Experts Technical Contributors and

Aditya Konde Pratik Boharapi Gayatri Deshpande

InfoScale Storage Basics

Lesson 01: Installing and Licensing InfoScale

Lesson 02: Virtual Objects

Lesson 03: Creating a Volume and File System

Lesson 04: Working with Volumes with Different Layouts

Lesson 05: Making Configuration Changes

Lesson 06: Administering File System

Table of Contents iii

Not for Distribution.

PART 2: Veritas InfoScale Availability 7.4.2 for UNIX/Linux: Administration

InfoScale Availability Basics

Lesson 01: High Availability Concepts

Lesson 02: VCS Building Blocks

Lesson 03: VCS Operations

Lesson 04: VCS Configuration Methods

VCS Password encryption .........................................................................................................4-26

Lesson 05: Preparing Services for VCS

Lesson 06: Online Configuration

iv Veritas InfoScale 7.4.2 Fundamentals for UNIX/Linux: Administration (Lessons)

Not for Distribution.

Lesson 08: Configuring Notification

InfoScale Availability Additions

Lesson 09: Handling Resource Faults

Lesson 10: Intelligent Monitoring Framework

Lesson 11: Cluster Communications

Changing the interconnect configuration...............................................................................11-26

Not for Distribution.

Not for Distribution.

This topic describes the contents of this course.

This topic describes the contents of this course.

Not for Distribution.

Not for Distribution.

Not for Distribution.

Knowledge of and hands-on experience with UNIX/Linux systems administration is required.

Not for Distribution.

Not for Distribution.

Not for Distribution.

• The latest technology articles from industry

Best of all, it is free. Sign up to become a member: https://vox.veritas.com/

Not for Distribution.

Not for Distribution.

• Certification Paths: Backup & Recovery,

• APTARE Training: Learn About APTARE training

• View FAQs about Education Services

• Manage your training transcript and print certificates of

Not for Distribution.

Lesson 01: Installing and Licensing InfoScale

Not for Distribution.

The lessons in this course are displayed on the slide.

Not for Distribution.

Not for Distribution.

After completing this topic, you will be able to:

This is the Introducing the Veritas InfoScale product suite topic.

Not for Distribution.

InfoScale family of products draws on Veritas' long heritage of

Not for Distribution.