V5.4.0.

cover

򔻐򗗠򙳰 Front cover

AIX Problem Determination II:

LVM Issues

(Course code QV012)

Student Exercises
ERC 1.6

UNIX Software Service Enablement

Student Exercises

Trademarks
IBM® and the IBM logo are registered trademarks of International Business Machines
Corporation.
The following are trademarks of International Business Machines Corporation, registered in
many jurisdictions worldwide:
DB2® HACMP™ System i™
System p™ System x™ System z™
Windows is a trademark of Microsoft Corporation in the United States, other countries, or
both.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.

August 2012 edition

The information contained in this document has not been submitted to any formal IBM test and is distributed on an “as is” basis without
any warranty either express or implied. The use of this information or the implementation of any of these techniques is a customer
responsibility and depends on the customer’s ability to evaluate and integrate them into the customer’s operational environment. While
each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will
result elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk.

© Copyright International Business Machines Corporation 2012.

This document may not be reproduced in whole or in part without the prior written permission of IBM.
Note to U.S. Government Users — Documentation related to restricted rights — Use, duplication or disclosure is subject to restrictions
set forth in GSA ADP Schedule Contract with IBM Corp.
V7.0
Student Exercises

TOC Contents
Exercise 1. LVM components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1

Exercise 2. Mirroring logical volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1

Exercise 3. Working with LVM metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1

Exercise 4. Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1

© Copyright IBM Corp. 2012 Contents iii

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

iv AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

pref Text highlighting

The following text highlighting conventions are used throughout this book:
Bold Identifies file names, file paths, directories, user names,
principals, menu paths and menu selections. Also identifies
graphical objects such as buttons, labels and icons that the
user selects.
Italics Identifies links to web sites, publication titles, is used where the
word or phrase is meant to stand out from the surrounding text,
and identifies parameters whose actual names or values are to
be supplied by the user.
Monospace Identifies attributes, variables, file listings, SMIT menus, code
examples and command output that you would see displayed
on a terminal, and messages from the system.
Monospace bold Identifies commands, subroutines, daemons, and text the user
would type.

© Copyright IBM Corp. 2012 v

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

vi AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty Exercise 1. LVM components

Introduction
This lab covers creating and manipulating the components of the
Logical Volume Manager.

Exercise Objectives
At the end of the lab, you should be able to:
• Create volume groups and logical volumes
• Adjust the t factor to determine physical partition sizes

© Copyright IBM Corp. 2012 Exercise 1. LVM components 1-1

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

Exercise instructions
Preface
This exercise includes information for you to read, and exercise steps for you to
perform. The following examples illustrate the numbered checklist format used to
identify exercise steps:
__ 1. (This is example step one.) Login to ...
__ 2. (This is example step two.) Execute the following ...
Two versions of these instructions are available: one with hints and one without. You
can use either version to complete this exercise (or move back and forth between the
two versions). Also, please don’t hesitate to ask the instructor if you have questions.
Part 1 of this exercise shows the command line and the SMIT menus in the Exercises
with Hints Guide.The remaining exercises only show the command line in the Exercises
with Hints Guide.

Part 1 - LVM Concepts

__ 1. List the volume groups on your system.

__ 2. List the physical volumes on your system.

__ 3. Create a new volume group with the following specifications:

• Type of volume group: Original
• Volume group name: labvg
• Physical volume name: hdisk1 (Or, another disk that is available. The output of
lspv will show you where rootvg resides. Only use a disk where it shows None
for the volume group.)
Note: The rest of this lab assumes the disk is named hdisk1, but use whatever
disk is available on your system.

1-2 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 4. List the contents of the labvg volume group.

__ 5. Create a logical volume with the following specifications:

• Volume group name: labvg
• Logical volume name: lab_lv
• Logical volume type: jfs2
• Logical partitions: 3
• Defaults for all other fields (make note of the defaults)

__ 6. Add another disk into the labvg volume group.

Note: The rest of this lab assumes the additional disk is named hdisk2, but use
whatever disk is available on your system.

__ 7. Try to remove a physical volume that belongs to labvg. (Do not force it.)
Could you do it? Why or why not?

__ 8. Remove the labvg volume group.

© Copyright IBM Corp. 2012 Exercise 1. LVM components 1-3

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

Part 2 - Physical partition size

This exercise section deals with the physical partition size and its relation to the number
of physical volumes and the number of physical partitions in a volume group.
__ 9. List the disk drives on your system, then use the command bootinfo -s <disk> to
find out the size of each.

__ 10. Create a volume group named ppsizevg (original type) with the smallest disk
available.

__ 11. Display the information for the volume group ppsizevg to answer the following:
a. Physical partition size: ____________

b. Number of physical partitions: ___________

c. Maximum physical partitions per physical volume: ____________

d. Maximum physical volumes: ___________

__ 12. Try to add a larger disk to the ppsizevg volume group. What happened?

__ 13. Modify the ppsizevg volume group to allow the larger disk to be added.

1-4 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 14. Display the information for the volume group ppsizevg to answer the following.
a. Physical partition size: ____________

b. Number of physical partitions: ___________

c. Maximum physical partitions per physical volume: ____________

d. Maximum physical volumes: ___________

Explain the differences in the values between now and before the second disk was
added.

__ 15. Remove the ppsizevg volume group.

__ 16. Create a volume group named ppsize2vg (original type) with the largest disk
available.

__ 17. Display the information for the volume group ppsize2vg to answer the following:
a. Physical partition size: ____________

b. Number of physical partitions: ___________

c. Maximum physical partitions per physical volume: ____________

d. Maximum physical volumes: ___________

© Copyright IBM Corp. 2012 Exercise 1. LVM components 1-5

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 18. Try to add a smaller disk to the ppsize2vg volume group. What happened?

__ 19. Display the information for the volume group ppsize2vg to answer the following.
a. Physical partition size: ____________

b. Number of physical partitions: ___________

c. Maximum physical partitions per physical volume: ____________

d. Maximum physical volumes: ___________

Explain the differences in the values between now and before the second disk was
added.

__ 20. Remove the ppsize2vg volume group.

__ 21. Create a volume group named ppsizesvg (scalable type) with the smallest disk
available.

__ 22. Display the information for the volume group ppsizesvg to answer the following:
a. Physical partition size: ____________

1-6 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty
b. Number of physical partitions: ___________

c. Maximum physical partitions per physical volume: ____________

d. Maximum physical volumes: ___________

__ 23. Try to add a larger disk to the ppsizesvg volume group. How does the result
compare to what happened when a larger disk was added to an original type volume
group?

__ 24. Display the information for the volume group ppsizesvg to answer the following.
a. Physical partition size: ____________

b. Number of physical partitions: ___________

c. Maximum physical partitions per physical volume: ____________

d. Maximum physical volumes: ___________

__ 25. Remove the ppsizesvg volume group.

End of exercise

© Copyright IBM Corp. 2012 Exercise 1. LVM components 1-7

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

1-8 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty Exercise 2. Mirroring logical volumes

Introduction
This lab covers creating mirrored logical volumes and mirror pools.

Exercise Objectives
At the end of the lab, you should be able to:
• Create mirrored logical volumes
• Create and use mirror pools

© Copyright IBM Corp. 2012 Exercise 2. Mirroring logical volumes 2-1

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

Exercise instructions
Preface
This exercise includes information for you to read, and exercise steps for you to
perform. The following examples illustrate the numbered checklist format used to
identify exercise steps:
__ 1. (This is example step one.) Login to ...
__ 2. (This is example step two.) Execute the following ...
Two versions of these instructions are available: one with hints and one without. You
can use either version to complete this exercise (or move back and forth between the
two versions). Also, please don’t hesitate to ask the instructor if you have questions.
Part 1 of this exercise shows the command line and the SMIT menus in the Exercises
with Hints Guide.The remaining exercises only show the command line in the Exercises
with Hints Guide.

Part 1 - Mirroring
__ 1. List the volume groups on your system. If there are any volume groups on the
system (with the exception of rootvg) delete them.

__ 2. List the physical volumes on your system.

__ 3. Create a new volume group with the following specifications:

• Type of volume group: Original
• Volume group name: labvg
• Physical volume name: hdisk1 (Or, another disk that is available. The output of
lspv will show you where rootvg resides. Only use a disk where it shows None
for the volume group.)
Note: The rest of this lab assumes the disk is named hdisk1, but use whatever
disk is available on your system.

2-2 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 4. Try to create a logical volume with the following specifications:

• Volume group name: labvg
• Logical volume name: lab_lv
• Logical volume type: jfs2
• Number of copies of each logical partition: 2
• Logical partitions: 3
• Defaults for all other fields (make note of the defaults)
Did it work? Why not?

__ 5. Create the logical volume with just one copy of each logical partition.

__ 6. Add another disk into the labvg volume group.

Note: The rest of this lab assumes the additional disk is named hdisk2, but use
whatever disk is available on your system.

__ 7. Create a mirror for the logical volume lab_lv with the following specifications:
• Logical partition copies: 2
• Physical volume name: hdisk2
• Defaults for all other fields (make note of the defaults)

© Copyright IBM Corp. 2012 Exercise 2. Mirroring logical volumes 2-3

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 8. Display the information for the logical volume lab_lv to answer the following:
a. Are the logical volume copies synchronized? _________

b. Are there any stale partitions? __________

c. Number of copies: ________

d. Number of logical partitions: ________

e. Number of physical partitions: ________

__ 9. If lab_lv is not synchronized, synchronize it. Verify that it is synchronized and that
there are no stale partitions.

__ 10. Remove the labvg volume group.

2-4 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty Part 2 - Mirror pools

__ 11. Create a scalable volume group named myvg with hdisk1 and hdisk2 in a mirror
pool named MirrorP1.

__ 12. Verify that hdisk1 and hdisk2 are in the mirror pool, MirrorP1 in the myvg volume
group.

__ 13. Add hdisk3 into the myvg volume group and include it in the MirrorP1 mirror pool.

__ 14. Verify that hdisk3 is in the mirror pool, MirrorP1 in the myvg volume group.

__ 15. Try to add hdisk4 into mirror pool, MirrorP1. What happened?

__ 16. Do what you need to do, to get hdisk4 into the MirrorP1 mirror pool.

__ 17. Remove hdisk3 and hdisk4 from the MirrorP1 mirror pool.

__ 18. Create a new mirror pool named MirrorP2 with hdisk3.

© Copyright IBM Corp. 2012 Exercise 2. Mirroring logical volumes 2-5

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 19. Verify that hdisk1 and hdisk2 are in MirrorP1, hdisk3 is in MirrorP2 and hdisk4 is
not in a mirror pool.

__ 20. Create a logical volume in myvg with the following specifications:

• Name: mylv1
• Logical partitions: 2
• Mirror pool: MirrorP1

__ 21. Verify the logical volume, mylv1, is using MirrorP1.

__ 22. Use mirrorvg to create a second copy of mylv1 using MirrorP2.

__ 23. Verify the logical volume, mylv1, is using MirrorP1 for the first copy and MirrorP2
for the second copy.

__ 24. Try to create a third copy of mylv1 on hdisk4. hdisk4 is not in a mirror pool. Did it
work?

2-6 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty __ 25. Turn on mirror pool strictness. Did it give you any messages regarding the logical
volume mylv1?

__ 26. Try to create a logical volume named mylv2 with 4 logical partitions in myvg. Did it
work?
» No. The strict policy is enforced for new logical volumes created.

__ 27. Remove the third copy of mylv1 (which is on hdisk4).

__ 28. Confirm that the first copy of mylv1 is using MirrorP1 and the second copy is using
MirrorP2.

__ 29. Try to change mylv1 to use MirrorP2 for the first copy and MirrorP1 for the second
copy. Did it work?

__ 30. Bring mylv1 into compliance with the correct mirror pools. Look at the mapping of
mylv1 to confirm it’s in compliance.

__ 31. Remove the myvg volume group and its mirror pools, MirrorP1 and MirrorP2.

End of exercise

© Copyright IBM Corp. 2012 Exercise 2. Mirroring logical volumes 2-7

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

2-8 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty Exercise 3. Working with LVM metadata

Introduction
This lab covers the use of informational high and intermediate level
commands to view LVM components and metadata.

Exercise Objectives
At the end of the lab, you should be able to:
• Identify LVM metadata information
• Repair a damaged file system
• Fix an ODM corruption
• Understand the quorum mechanism

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-1
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

Exercise instructions
Preface
This exercise includes information for you to read, and exercise steps for you to
perform. The following examples illustrate the numbered checklist format used to
identify exercise steps:
__ 1. (This is example step one.) Login to ...
__ 2. (This is example step two.) Execute the following ...
Two versions of these instructions are available: one with hints and one without. You
can use either version to complete this exercise (or move back and forth between the
two versions). Also, please don’t hesitate to ask the instructor if you have questions.

Part 1 - Intermediate level informational commands

Normally, it’s easier to use the high level commands to get information about LVM.
However, if there is a problem with data mismatched between the ODM and the
on-disk metadata, high level commands may error out. In this case, using the
intermediate level commands may allow you to get the information you need to
correct a problem.
You will experiment with several of these intermediate level informational
commands.
Notes:
• IMPORTANT: Do NOT use the intermediate level commands which make
changes to LVM metadata unless you have a detailed understanding of the
internal LVM structure.
• Use care when using the intermediate level informational commands.
Remember that these commands are not supported, documented commands.
Syntax and format of the output may change between AIX releases, and since
they are undocumented, you may need to use your understanding of LVM
structure to interpret the results.

__ 1. Create a regular volume group named testvg using one unused disk. The lspv
command can be used to see which disks are currently used. (We will assume that
this disk is hdisk1 in the rest of this exercise.)

3-2 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty __ 2. Create a logical volume named lv1 in testvg that uses 5 partitions.
Note: You will be adding a file system to this logical volume later in these exercises.

__ 3. Create a logical volume named lv2 in testvg that uses 3 partitions.

__ 4. Use the lqueryvg -p hdisk1 -At command to view volume group information
from the VGDA on hdisk1.
This syntax says:
- Read the VGDA on hdisk1 (-p hdisk1)
- Display all attributes for the volume group (static attributes, logical volume
details and physical volume details) (-A)
- With descriptions or tags for each attribute (-t)
Execute the command and answer the following questions.
(Substitute your disk for hdisk1.)

__ a. What are the LVIDs for each logical volume?

__ b. What is the PVID for the disk? ______________________________

__ c. How many VGDAs are in this volume group? __________

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-3
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 5. Compare the output of lqueryvg with lsvg testvg.

__ 6. Use the command lquerypv -p <hdisk1_PVID> -N hdisk1 -at to look at

physical volume information for hdisk1 in the VGDA on hdisk1. Use lspv to get the
PVID for hdisk1.
This syntax says:
- Read the VGDA from hdisk1 (-N hdisk1)
- Display all static attributes (-a)
- With descriptions (-t)
- For hdisk1 (-p <hdisk1_PVID>)
Normally, most of this information could be more easily obtained using 
lspv <PVname>. The only piece of information not returned by lspv is the total active
VGDAs on this disk. hdisk1 is the only disk in testvg, so we already know it will
have two active VGDAs. In the case of a two disk volume group, you can use
lquerypv to determine which disk has two copies of the VGDA.

__ 7. Use lsvg testvg to get the VGID for testvg. Then, use the command 
lquerypv -p <hdisk1_PVID> -N hdisk1 -At | grep <testvg_VGID> 
to look at the partition map for hdisk1. The -A flag says to display the static
attributes and the partition map.
The partition map is large, one line for every physical partition on the disk. In this
example, we grep for the VGID to limit the output to just those partitions which have
been allocated. You could also send the output to more and page through it until you
get to the partitions which have been allocated.

What are the physical partition numbers being used for lv1 and lv2?

3-4 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty __ 8. Compare the output of lquerypv with lspv -M hdisk1.

__ 9. Use the command getlvcb -AT LVname for each logical volume in testvg. This
verifies that the Logical Volume Control Block (LVCB) for each logical volume is
readable and contains data.
Note: The getlvcb command has many flags which can be used to extract specific
pieces of information from the LVCB. For more information, see:
SG24-5433 AIX Logical Volume Manager from A to Z:
Troubleshooting and Commands

__ 10. Use the command lquerylv -L <LVID> -p hdisk1 -At for each logical volume in
testvg. This displays the logical volume information stored in the VGDA. Compare
LVMAP lines with the PVMAP output from lquerypv. If this were a mirrored logical
volume, the 2nd and 3rd copies would also be listed.

__ 11. Compare the output for physical partitions of lquerylv -L <LVID> -p hdisk1 -At
with lslv lv1 and lslv -m lv1.

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-5
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

Part 2 - File systems tests

__ 12. Create a JFS file system on lv1 with the mount point /lv1fs.
# crfs -v jfs -d lv1 -m /lv1fs

__ 13. Mount the file system.

__ 14. Copy some files to the file system and verify that they are there.

__ 15. Unmount /lv1fs in preparation to simulate corruption of the superblock.

__ 16. Use the dd command to write zeros through the superblock.

# dd if=/dev/zero of=/dev/lv1 bs=4096 seek=1 count=1
This dd command copies from the input file (if=/dev/zero) to the output file
(of=/dev/lv1). /dev/zero is a pseudo device which can be used as a source of
zeros. The bs=4096 flag sets the block size to 4096 bytes. seek=1 skips to the
second block. The superblock is the second 4 KB block on the logical volume. The
first block contains the LVCB. count=1 indicates that we want to read one block from
the input file and write it to the output file. If the count flag is omitted, dd will continue
until it reaches the end of either the input file or the output file.

3-6 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty __ 17. Try to mount the /lv1fs file system. What happened?

__ 18. Try to use the fsck command without any flags to repair the superblock.
# fsck /dev/lv1
What happened?

__ 19. Use fsck -p to repair the damage.

With the -p flag (preen mode), fsck analyzes the file system and performs simple
repairs without prompting. These simple repairs include repairing the superblock, if
corrupted, by copying the secondary superblock.

__ 20. Try to mount the /lv1fs file system again. What happened?

Verify that your files are still there.

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-7
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

Part 3 - ODM tests

__ 21. You are going to delete some objects from ODM to simulate ODM corruption. As a
precaution, make a copy of the files in /etc/objrepos, just in case.

__ 22. In this step, you will simulate a corruption of the ODM by deleting the CuAt
information for the logical volume lv2.
__ a. First, take a look at the entries in CuAt for lv2. You will use the query from this
step for the query in step b.

__ b. Issue the following command to delete lv2 objects from CuAt:

# odmdelete -q name=lv2 -o CuAt
The following message is displayed:
0518-307 odmdelete: 2 objects deleted.

__ 23. Try to create a file system on lv2 with the mount point /lv2fs. Did it work? If not, why
not?

__ 24. Fix the corruption. (Hint: What command removes volume group information from
the ODM? What command reads the VGDA and adds volume group information to
the ODM?)

3-8 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty __ 25. Create a file system on lv2 with the mount point /lv2fs.

__ 26. Mount the file system.

__ 27. Simulate the corruption again by issuing the following command:

# odmdelete -q name=lv2 -o CuAt
You should get the following message:
0518-307 odmdelete: 3 objects deleted.

Note: When you created a file system on lv2, another ODM object was created in
CuAt.

__ 28. Is the file system mounted on /lv2fs still mounted? Why or why not?

__ 29. Try to copy the files /bin/a* to the file system /lv2fs. Did it work?

__ 30. Unmount the file system /lv2fs.

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-9
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 31. Try to increase the size of the file system /lv2fs by 1 GB. Could you increase it? If
not, why not?

__ 32. Fix the corruption.

__ 33. Remove the volume group testvg.

3-10 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty Part 4 - Working with quorum

__ 34. Create a volume group named quorumvg using one unused disk. (We will assume
that this disk is hdisk1 in the rest of this exercise.)

__ 35. Use the lsvg command to find the volume group information for the quorumvg
volume group.
a. Quorum: ________

b. Quorum enabled or disabled? _____________

c. Number of VGDAs: ____________

d. Active physical volumes: ___________

__ 36. Use the lqueryvg -p hdisk1 -At command to find the volume group information
from the disk’s perspective.
a. Total VGDAs: ______________

b. Number of VGDAs on this disk: ____________

c. Quorum enabled or disabled? _________________

__ 37. Add a second disk to the quorumvg volume group. (We will assume that this disk is
hdisk2 in the rest of this exercise.)

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-11
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 38. Use the lsvg command to find the volume group information for the quorumvg
volume group.
a. Quorum: ____________________

b. Quorum enabled or disabled? ________________

c. Number of VGDAs: ____________

d. Active physical volumes: ___________

__ 39. Use the lqueryvg -p hdisk1 -At and lqueryvg -p hdisk2 -At commands to
find the volume group information from each disk’s perspective.
a. Total VGDAs: ______________

b. Number of VGDAs on hdisk1: ____________

c. Number of VGDAs on hdisk2: ____________

Was the information the same or different from the two disk’s perspective?

__ 40. Use the lspv command to get the PVIDs of the two disks in the quorumvg volume
group.
- PVID for hdisk1: ________________________________________

- PVID for hdisk2: ________________________________________

3-12 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty __ 41. Use the command lquerypv -p <PVID> -N <disk> -at for each disk to query the
attributes of each physical volume.
Is there a difference in the values on the two physical volumes?

__ 42. Make the second disk unavailable using the following steps.
a. Varyoff the quorumvg volume group.

b. Make the second disk unavailable (hdisk2 in our example) using rmdev. Do
not delete it from CuDv.

__ 43. Try to varyon the quorumvg volume group. Did it varyon?

__ 44. Look in /usr/include/lvm.h to find the description for the status of the physical
volumes.

__ 45. Look in the error log file to see if any errors were logged.

__ 46. Use the command lquerypv -p <PVID> -N <disk> -at for each disk to query the
attributes of each physical volume. What happened?

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-13
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 47. Use lqueryvg -p hdisk1 -At to look at the VGDA information on hdisk1 (the first
disk). Has it changed?

__ 48. Use the lsvg command to find the volume group information for the quorumvg
volume group. Has any of the following information changed?
a. Quorum: ________

b. Quorum enabled or disabled? _____________

c. Number of VGDAs: ____________

d. Active physical volumes: ___________

__ 49. Bring hdisk2 back online.

__ 50. Use the command lquerypv -p <PVID> -N hdisk2 -at for hdisk2 disk to query
its attributes. Was it successful?

__ 51. Use the lsvg command to find the volume group information for the quorumvg
volume group. Has any of the following information changed?
a. Quorum: ________

3-14 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty b. Quorum enabled or disabled? _____________

c. Number of VGDAs: ____________

d. Active physical volumes: ___________

__ 52. Try to create a logical volume with 1 physical partition on hdisk2. Did it work?

__ 53. Try running the command syncvg -v quorumvg to have hdisk2 recognized as
active by the lsvg command. Did it work?

__ 54. What do you think will bring hdisk2 into an active state in the quorumvg volume
group? Try it. (Look at the Hints if you do not know.)
Verify it worked by running the lsvg quorumvg command.

__ 55. In the previous steps, you removed the second disk that was added to a two disk
volume group. In the following steps, you will remove the first disk.
Make the first disk (hdisk1) unavailable using the following steps.
a. Varyoff the quorumvg volume group.

b. Make the first disk unavailable (hdisk1) using rmdev. Do not delete it from
CuDv. (Be sure you have the PVID for hdisk1 written down before you do
this step!)

__ 56. Try to varyon the quorumvg volume group. Did it varyon? If not, why?

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-15
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 57. Look in the error log file to see if any errors were logged.

__ 58. Use the command lquerypv -p <PVID> -N <disk> -at for each disk to query the
attributes of each physical volume. What happened?

__ 59. Use lqueryvg -p hdisk2 -At to look at the VGDA information on hdisk2 (the
second disk). Has it changed?

__ 60. Try to run the lsvg command on quorumvg. Why didn’t it work?

__ 61. Varyon the quorumvg volume group using the force (-f) flag.

__ 62. Look in /usr/include/lvm.h to find the description for the status of the physical
volume for the disk that was made unavailable.

__ 63. Look in the error log file to see if any errors were logged.

3-16 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty
__ 64. Use the lsvg command to find the volume group information for the quorumvg
volume group. Has any of the following information changed from when the
quorumvg volume group had both disks available?
a. Quorum: ________

b. Quorum enabled or disabled? _____________

c. Number of VGDAs: ____________

d. Active physical volumes: ___________

__ 65. The output of the lsvg quorumvg command showed that there were 2 VGDAs. In
our previous steps we saw that hdisk2 only had 1 VGDA (because it was the
second disk added to a two disk volume group). Since hdisk2 is the only disk active,
look at the VGDA information on hdisk2 to see what happened. 
(Hint: Use the lqueryvg -p hdisk2 -At command.)

__ 66. Bring hdisk1 back online.

__ 67. Use the command lquerypv -p <PVID> -N <disk> -at for each disk to query the
attributes of each physical volume.
How do the values of Total VGDAs compare to what they were previously?

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-17
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 68. Compare the VGDA information on each disk. Normally, the VGDA information
should be the same on both disks. Why aren’t they in this case?
(Hint: Use lqueryvg -p <disk> -At)

__ 69. Use the lsvg command to find the volume group information for the quorumvg
volume group. Has any of the following information changed from when the
quorumvg volume group had both disks available? Is it different from when hdisk2
was unavailable and then became available?
a. Quorum: ________

b. Quorum enabled or disabled? _____________

c. Number of VGDAs: ____________

d. Active physical volumes: ___________

__ 70. In the previous test when we took the second disk (hdisk2) offline then brought it
back online, the varyonvg command rebuilt the VGDA information and brought the
volume group back to the original state.
Try the varyonvg command now, then the lsvg quorumvg command. Did it fix the
volume group VGDA information?

__ 71. Varyoff the quorumvg volume group and then vary it back on. Did it varyon? Are
both disks available?

3-18 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty
__ 72. Use the reducevg command to officially remove hdisk1 from the quorumvg volume
group.

__ 73. Use lqueryvg -p hdisk1 -At to look at the VGDA information on hdisk1. What
does it show?

__ 74. Use lqueryvg -p hdisk2 -At to look at the VGDA information on hdisk2. What
does it show?

__ 75. Now, add hdisk1 back into the quorumvg volume group. What did you have to do
to get it to work? Why?

__ 76. Use the command lquerypv -p <PVID> -N <disk> -at for each disk to query the
attributes of each physical volume.
Is there a difference in the values on the two physical volumes?

How does it differ from when the quorumvg volume group was originally created
with both disks (hdisk1 first and hdisk2 second)?

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-19
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 77. Use the lqueryvg -p hdisk1 -At and lqueryvg -p hdisk2 -At commands to
find the volume group information from each disk’s perspective. Do they look
correct?

__ 78. Now, you will go through some steps to see what happens when quorum is turned
off. Turn off quorum on the quorumvg volume group.

__ 79. Use the lsvg command to find the volume group information for the quorumvg
volume group.
a. Quorum: ________

b. Quorum enabled or disabled? _____________

c. Number of VGDAs: ____________

d. Active physical volumes: ___________

__ 80. Use the lqueryvg -p hdisk1 -At and lqueryvg -p hdisk2 -At commands to
find the volume group information from each disk’s perspective.
a. Total VGDAs: ______________

b. Number of VGDAs on hdisk1: ____________

c. Number of VGDAs on hdisk2: ____________

d. Quorum enabled or disabled? _______________

3-20 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 81. Varyoff the quorumvg volume group.

__ 82. Make hdisk1 unavailable using rmdev. Do not delete it from CuDv.

__ 83. Try to varyon the quorumvg volume group. Did it varyon? Why or why not?

__ 84. Look at the VGDA information from hdisk2 (because that is the only disk that is
available in the quorumvg volume group). What does it show regarding quorum?

__ 85. Force a varyon of the quorumvg volume group.

__ 86. Is quorum still disabled, or did forcing the varyon enable it?

__ 87. Bring hdisk1 back online.

__ 88. Varyon the quorumvg volume group so hdisk1 is recognized.

© Copyright IBM Corp. 2012 Exercise 3. Working with LVM metadata 3-21
Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 89. Run lsvg quorumvg to confirm that there are now 2 active physical volumes.

__ 90. Remove the quorumvg volume group.

End of exercise

3-22 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty Exercise 4. Problem determination

Introduction
This lab covers some problems with ODM corruption.

Exercise Objectives
At the end of the lab, you should be able to:
• Identify LVM problems when the /tmp file system is full
• Identify and fix ODM corruption with redefinevg and synclvodm
• Identify and fix a corrupted PVID

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-1

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

Exercise instructions
Preface
This exercise includes information for you to read, and exercise steps for you to
perform. The following examples illustrate the numbered checklist format used to
identify exercise steps:
__ 1. (This is example step one.) Login to ...
__ 2. (This is example step two.) Execute the following ...
Two versions of these instructions are available: one with hints and one without. You
can use either version to complete this exercise (or move back and forth between the
two versions). Also, please don’t hesitate to ask the instructor if you have questions.

Part 1 - Full file system

__ 1. List the physical volumes on your system and check to see if hdisk1, hdisk2 and
hdisk3 are available. If not, substitute other disks that are available for this part of
the exercise. The rest of this exercise assumes hdisk1, hdisk2 and hdisk3 are the
disks being used.

__ 2. Create a volume group named b4vg that includes hdisk1.

__ 3. Create a JFS file system in b4vg with the following specifications:

• Size of one physical partition
• Mount point /b4fs

__ 4. Mount the /b4fs file system.

4-2 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty __ 5. Create a file in the /b4fs file system. This can be done by issuing the following
command:
echo "Contents of file before full file system" > /b4fs/myfile

__ 6. Fill up the /tmp file system by executing the following command:

# dd if=/dev/zero of=/tmp/bigfile
It will give you the following message when the file system is filled:
dd: write error: No space left on device

__ 7. Try to list the contents of the /b4fs/myfile file. Did it work?

__ 8. Try to create another file in /b4fs. Did it work?

__ 9. Try to create a JFS file system in b4vg with the following specifications.
• Size of one physical partition
• Mount point /afterfs
Did it work?

__ 10. Try to create another volume group named aftervg using hdisk2. Did it work?

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-3

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 11. Try to list the information about the aftervg volume group. Did it work?

__ 12. Try to create a logical volume in aftervg. Did it work?

__ 13. Extend the b4vg volume group by adding hdisk3.

__ 14. Try to list the volume groups on the system, then all the active volume groups. Did it
work?

__ 15. Try to unmount the /b4fs file system.

__ 16. Try to remove the b4vg and aftervg volume groups. Did it work?

__ 17. Delete the /tmp/bigfile that was created to fill up the /tmp file system.

__ 18. Remove the b4vg and aftervg volume groups.

4-4 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty Part 2 - ODM corruption - redefinevg and synclvodm

__ 19. List the physical volumes on your system and check to see if hdisk1 is available. If
not, choose another disk that is available. The rest of this exercise assumes hdisk1
is the disk being used.

__ 20. Create a volume group named myvg that includes hdisk1.

__ 21. Find and write down the following information about the mkvg volume group:
a. VGID: ________________________________

b. Physical partition size: ____________________

__ 22. Create a JFS file system in myvg with the following specifications:
• Size of one physical partition
• Two log partitions
• Mount point /myfs

__ 23. Display the logical volumes in myvg. Write down the logical volume names.
____________________________________

____________________________________

__ 24. Try to use the rendev command to rename the log logical volume, loglv00, to
mylog. Did it work? If not, why not?

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-5

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 25. Change the name of the log logical volume to mylog using the command
chlv -n mylog loglv00 (assuming loglv00 is the name of the log device).

__ 26. Change the myfs file system to use the log device name mylog using the
command:
chfs -a log=/dev/mylog /myfs

__ 27. Change the name of the logical volume to mylv using the command
chlv -n mylv lv00 (assuming the original name is lv00).

__ 28. Verify the logical volume name changes in myvg.

__ 29. Verify the changes are in the /etc/filesystems file.

__ 30. Mount the myfs file system.

__ 31. Create a file in the /myfs file system. This can be done by issuing the following
command: echo "Data for myfs" > /myfs/data

4-6 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 32. You are going to delete some objects from ODM to simulate ODM corruption. As a
precaution, make a copy of the files in /etc/objrepos, just in case:

__ 33. Issue the following commands (carefully!) to replicate a failure where all the
information about the myvg volume group is gone.
Note: It is always a good idea to test your query with the odmget command before
you do the odmdelete command.
# odmget -q "name LIKE my*" CuAt
# odmdelete -o CuAt -q "name LIKE my*"

# odmget -q "value3 LIKE my*" CuDvDr

# odmdelete -o CuDvDr -q "value3 LIKE my*"

# odmget -q "value1=myvg" CuDvDr

# odmdelete -o CuDvDr -q "value1=myvg"

# odmget -q "name LIKE my*" CuDv

# odmdelete -o CuDv -q "name LIKE my*"

# odmget -q "name=myvg" CuDep

# odmdelete -o CuDep -q "name=myvg"

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-7

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 34. List all the volume groups on the system. There should be two, rootvg and myvg.
What did you see?

__ 35. List all the active volume groups (use the lsvg -o flag). There should be two,
rootvg and myvg. What did you see?

__ 36. Look at the information for the physical volume, hdisk1. What did you see?

__ 37. Look at the information for the logical volume, mylv. What did you see?

__ 38. You’ve seen that the ODM is missing information, but is the file system data really
gone? Issue the ls command for the /myfs file system. What happened?

__ 39. Try unmounting and mounting the /myfs file system. What happened?

__ 40. Issue the ls command for the /myfs file system. Are the files still there?

__ 41. Do you think any of the following actions will fix the problem? Give them each a try,
then explain what happened.

4-8 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty • exportvg

• varyonvg

• synclvodm

__ 42. There is a script on your system in /home/qv012. It’s called find-odm-objects.

This script searches the ODM object classes PdDv CuDv PdAt CuAt CuDep, and
CuDvDr for a given string. The contents of this script is:
#!/usr/bin/ksh
for odmclass in PdDv CuDv PdAt CuAt CuDep CuDvDr
do
odmget $odmclass | grep -ip $1
done
Run this script to look for the string my using the command:
# /home/qv012/find-odm-objects my
Was there anything in the ODM that referenced the myvg volume group or its
dependent logical volume?

__ 43. The first thing to do is to regain basic ODM information regarding the myvg volume
group. This can be done with the redefinevg command. When inconsistencies
occur between the ODM and the LVM, the redefinevg command determines which
physical volumes belong to the specified volume group and re-enters this
information in the ODM. The redefinevg command checks for inconsistencies by
reading the reserved areas of all the configured physical volumes attached to the
system.The redefinevg command needs the volume group name and either a
VGID or a physical volume name.

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-9

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

The physical volume that was in myvg was hdisk1. To confirm the VGDA
information on hdisk1 is still correct, run the lqueryvg command:
# lqueryvg -p hdisk1 -At

__ 44. Now, run the redefinevg command for myvg.

__ 45. Check the state of the volume groups using lsvg and lsvg -o commands. Does it
show the myvg volume group?

__ 46. Use the lsvg -l myvg command to check the state of the logical volumes in the
myvg volume group. Has everything been fixed?

__ 47. Use the /home/qv012/find-odm-objects script to see what is defined in the ODM
regarding the myvg volume group and its logical volumes.
(Hint: search for the string my).

__ 48. There is no information for the logical volumes in the myvg volume group (only the
volume group information has been stored). The logical volume information needs to
be restored from the LVCBs.
The synclvodm command will synchronize or rebuild the LVCB, the ODM, and the
VGDAs on the physical volumes. To complete the repair, run the synclvodm
command.

4-10 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty
__ 49. Verify everything has been repaired for the myvg volume group and its logical
volumes.

__ 50. Unmount the /myfs file system.

__ 51. Remove the myvg volume group.

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-11

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

Part 3 - Corrupted disk recovery

Introduction
In this exercise, you will simulate disk corruption by writing zeros over the PVID of one
of your disks. Since the actual data in the logical volumes was not affected by this
corruption, it may be possible to recover the logical volumes on the corrupted disk
without having to restore or recreate the data. There are several important things to
note about this scenario:
• If you are not careful, your recovery actions could cause the partition map for the
affected logical volumes to be lost. Once the partition map is lost, the data is
inaccessible and is effectively lost. In this case, you would need to restore or
recreate the data.
• This underscores the importance of backups. Before you take any corrective
actions, make a backup of all logical volumes involved, if possible. Depending on
the problem, it may not be possible, but your first step should be to try.
• In the real world, you may never see this particular type of corruption. This
exercise will not give you a cookbook procedure that you can follow in every
case, but it should help you to understand some of the tools and structures so
that you can create your own recipes.
In this exercise, there are many references to the PVID, VGID, and LVID’s of the system
to be used for the exercise. The values used in the hints are examples. Your numbers
will be different.
Look at the solutions for explanations of what is happening at various steps.

Exercise
__ 52. List the physical volumes on your system and check to see if hdisk1 and hdisk2 are
available. If not, choose two other disks that are available. The rest of this exercise
assumes hdisk1 and hdisk2 are the disks being used.
Write down the PVIDs of the two disks you are using:
_________________________________________

_________________________________________

__ 53. Assign hdisk1 and hdisk2 to a volume group called testvg.

4-12 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 54. Create a logical volume called first on hdisk2 with 5 logical partitions.

__ 55. You’ve created a logical volume called first that is 5 logical partitions and have told
mklv to try to put those 5 partitions on hdisk2. You know this request was fulfilled,
because the logical volume was just created, but do you know the location and order
of the physical partitions that were used on hdisk2? Does it matter? To satisfy our
curiosity, let’s take a look and see what the created logical partitions look like.
Verify the logical partitions for the logical volume first.

__ 56. Create a file called map which looks like this:

hdisk1:54
hdisk1:50
hdisk1:53
hdisk1:51
hdisk1:52

__ 57. Create a logical volume called second with the map file (map) you just created. It
will have 5 logical partitions.
You use the map file to create a complex logical volume for our example. If and
when you successfully recreate a logical volume and it’s file system, you want it very
apparent that it wasn’t just luck.

__ 58. Verify that the logical volume was created in the manner you specified.

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-13

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 59. Create and mount two JFS file systems with the following characteristics:
• File system 1: named firstfs, use the logical volume first
• File system 2: named secondfs, use the logical volume second

__ 60. Now, you will populate the file systems with data using the following shell script.
Create a shell script called vgtester with the information shown, set the
permissions to 755, and execute:
for copy in 1 2 3 4 5 6 7 8 9 10
do
cp /etc/inittab /firstfs/first.$copy
cp /etc/inittab /secondfs/second.$copy
done

__ 61. Create checksums for the file systems. You will use them later to provide validation
that you have successfully recovered the data. Redirect the checksums to a file
called checkfirst for /firstfs and checksecond for /secondfs:
# sum /firstfs/* > /tmp/checkfirst
# sum /secondfs/* > /tmp/checksecond
Note: We are using checksums to validate that the data in the file systems have
been successfully recovered. In point of fact, if the logical volume is not recovered
successfully, the file system won’t mount. The checksums just provide extra
validation.

__ 62. Unmount the /firstfs and /secondfs file systems, and varyoff the volume group
testvg.

4-14 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 63. Corrupt the PVID on hdisk1 by executing the following command:

# dd if=/dev/zero of=/dev/hdisk1 count=1
If you do not specify a block size, dd uses a block size of 512 bytes. The PVID is part
of the boot record, the first sector (512 bytes) on the disk. The PVID is 32 bytes
starting at address 0x80.

__ 64. Use lspv to examine what disks and volume groups are still on the system. What is
wrong with the output?

__ 65. Verify that hdisk1 really does not have a PVID with the following command:
# lquerypv -h /dev/hdisk1

lquerypv -h displays 256 bytes from the specified disk. It will start with address 0,
unless you specify an offset, for example: lquerypv -h /dev/hdisk1 0x100 would
display 256 bytes starting at address 100 hex. The PVID is 32 bytes starting at
address 0x80.
To see what a PVID should look like, compare this output with that for hdisk2.
# lquerypv -h /dev/hdisk2

__ 66. Use odmget to verify what the ODM knows about hdisk1.

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-15

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 67. List the volume group information about testvg. What happened?

__ 68. Try to varyon the testvg volume group. Did it work? Why not?

__ 69. Now, what can you do to fix this? You can’t start the repair of a volume group unless
you can talk to the volume group to see how much damage was done with the
dd.command. And it looks like you can’t start the volume group until you repair the
damage.
Fortunately, there is a way around this.
In this case, you will need to force the varyon of the volume group (-f flag).
Note: You might think that you could change the quorum checking attribute.
However, turning off quorum checking only stops quorum checking once the volume
group is varied on. Turning off quorum checking actually makes it more difficult to
varyon a volume group. When quorum checking is turned off, all VGDAs must be
available to varyon the volume group.
Force the volume group to varyon. What happens?

__ 70. Try to mount the file system that still resides on hdisk2 (remember, this is /firstfs). It
should still be readable since only hdisk1 was lost. What happens?

__ 71. At this point, although the PVID has been removed from hdisk1, the actual data on
the disk should still be there. LVM can’t access hdisk1 because it can’t find a disk
with hdisk1’s PVID. And you can’t mount the file system from the logical volume on
hdisk2 because it needs the JFS log on hdisk1.

4-16 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty You will need to recreate hdisk1’s PVID, but the process of doing this will remove
the partition map of any logical volumes on hdisk1 from the good VGDA. Before you
recreate the PVID, you must collect information about the partition map for any
logical volumes that reside on hdisk1 and create a map file. This will allow you to
recreate the partition map so that you should not have to restore or recreate the
actual logical volume data.
Every VGDA on a disk not only has information about the logical volumes that are
on that particular disk, but it also contains information about the logical volumes on
OTHER disks. So you can use the VGDA on hdisk2 to get the information you
need.
With the uncertain status of what hdisk1 is really called and its uncertain PVID
status, you will want to use the VGID because the volume group is obviously active
and there isn’t more than one VGID. Now, go find the VGID and write it down.
VGID: _______________________________________________________

__ 72. Use lqueryvg and the VGDA on hdisk2 to query about the logical volumes on
hdisk1 and where they are located.

__ 73. Use lquerylv to identify where the logical volume second resides on hdisk1. Is it
in the same place that it was created in step 58?

__ 74. Identify where the logical volume loglv00 resides on hdisk1.

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-17

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 75. Create a map file called fixlog that correlates to the logical volume location.
This map file will be used in a later step to recreate the loglv00 logical volume.

__ 76. At this point you have a volume group which knows partially about the old hdisk1’s
PVID, which no longer exists. You have an ODM that only knows hdisk1 by the
missing PVID. But if the PVID is really missing from the corruption, how will you
remove something from the VGDA that doesn’t exist? LVM allows you to remove a
disk by its PVID. The PVID doesn’t really have to exist on the system anymore.
Another example of this problem occurs when someone replaces a dead disk
without running reducevg first.
Use reducevg to remove the history of the old PVID from the VGDA.
# reducevg -d testvg <PVID of hdisk1>
Because you didn’t use the -f flag, LVM will ask you if it’s okay to delete the defined
logical volumes that reside on hdisk1. You must answer yes to all these queries. It
is good to not use the -f flag because it is a double-check that you gathered all the
information on all the logical volumes that exist on hdisk1. Don’t be alarmed about
the concept of deleting the logical volume. The system doesn’t go out to the disk and
literally place zeroes in those physical partitions. It just removes the partition map for
that logical volume in the VGDA. This is why it’s very important to get the partition
map for the logical volumes and create a map file before using reducevg.

__ 77. Look at the status of hdisk1 to see the results of reducevg.

__ 78. Use rmdev to remove hdisk1 from the system, which will remove the existing PVID
from hdisk1.
Note. The rmdev at this point is EXTREMELY important. In the practice exercise to
generate these notes, this command was left out and left our system in a deadlock
situation where there is no way to recover the lost logical volumes.

4-18 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 79. Reconfigure the drive, without rebooting.

__ 80. After this completes, check to see if hdisk1 has come back clean and with no PVID.

__ 81. Add a PVID to hdisk1.

__ 82. Check the status of the physical volumes. 

What is the new PVID for hdisk1? ___________________________________

__ 83. Make hdisk1 part of testvg by extending the volume group to include hdisk1. (You
may have to force it.)

Adding the disk into the volume group does wipe out any old remnants of the old
VGDA on hdisk1, but it does not zero out the data that resides in the physical
partitions that were used by the second and loglv00 logical volumes.

__ 84. Recreate the two missing logical volumes using the map files you created:
- map for second
- fixlog for loglv00

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-19

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

__ 85. When the reducevg was done in a previous step, the information for /secondfs was
removed from /etc/filesystems. This information needs to be reentered.
Edit /etc/filesystems and add the following:
/secondfs:
dev = /dev/second
vfs = jfs
log = /dev/loglv00
mount = false
account = false

__ 86. Mount the /secondfs file system.

__ 87. Mount the /firstfs file system.

__ 88. Create a checksum for the files in each file system and compare them to the
checksums initially created near the beginning of this optional exercise. Redirect the
checksums to a file called checkfirst2 for /firstfs and checksecond2 for
/secondfs.
Compare checkfirst to checkfirst2 and checksecond to checksecond2. This is to
verify that the data was correctly reconstructed.

The checksums should be the same. This confirms that everything was correctly
reconstructed.

__ 89. Unmount the /firstfs and /secondfs file systems.

4-20 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0
Student Exercises

EXempty

__ 90. Remove the testvg volume group.

End of exercise

© Copyright IBM Corp. 2012 Exercise 4. Problem determination 4-21

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
Student Exercises

4-22 AIX LVM Issues © Copyright IBM Corp. 2012

Course materials may not be reproduced in whole or in part
without the prior written permission of IBM.
V7.0

backpg
Back page