Backup Configuration PPS

Copyright © EVRY.
Backup installation: RMAN to tape v2

Installing TDP software
Prerequisites
Create a CATALOG database (if it doesn't exist)
On the CATALOG database
Install TDPO software on AIX
Installation
Order access to TSM server (TDPO node)
Configure TDPO node
Add dsm.sys
Configuring RMAN for TDP
Environment
Database configuration
Create/unlock sysbackup account, enable block change tracking and list archiving status
Configure RMAN
Add crontab entries
Post activities
Add script
Add to crontab
Changes when upgrading to new Oracle version where ORACLE_HOME has changed
Create symbolic link
Run script to change environment variables for RMAN
Check status of RMAN backup
Deletion of databases with RMAN TDPO backup
Remove backup
Unregister from CATALOG
Delete database from helper table
Order deletion of TDPO
Appendix
Finding the service name for your database
Restore with RMAN TDP from scripts
Restore with RMAN TDP manually (recommended)
Troubleshoot for RMAN restore
Installing TDP software

Prerequisites
root/administrator access.
Access to log in as the oracle OS user
Requires a CATALOG database
Create a CATALOG database (if it doesn't exist)

Click here to expand...
On the CATALOG database

See which CATALOG to use at RMAN Catalog.
How to create CATALOG

export ORACLE_SID=cdbrman_1
sqlplus "/as sysdba"
#Lager pluggable database rcatalog, med en local admin bruker, og åpner

den.
create pluggable database rcatalog admin user rman_adm identified by
xxxxxx;
Page 1
Copyright © EVRY.
alter pluggable database rcatalog open;

#Kobler til pluggable basen og lager et users tablespace
alter session set container=rcatalog;
CREATE TABLESPACE users DATAFILE SIZE 10M AUTOEXTEND ON MAXSIZE
UNLIMITED;
#Lager rman tablespace, user og granter slik man har det på eksisterende
rcatalog base
CREATE TABLESPACE rman_cat DATAFILE SIZE 5M AUTOEXTEND ON MAXSIZE
UNLIMITED;
CREATE USER rman IDENTIFIED BY <password> TEMPORARY TABLESPACE temp
DEFAULT TABLESPACE rman_cat QUOTA UNLIMITED ON rman_cat;
GRANT CONNECT,RECOVERY_CATALOG_OWNER TO rman;
# Ifra root container (container databasen) laget en trigger slik at
pluggable base starter når container basen starter
CREATE or REPLACE trigger OPEN_ALL_PLUGGABLES
after startup
on database
BEGIN
execute immediate 'alter pluggable database all open';
END open_all_pdbs;
#I pluggable basen laget en profil og assignet den lokale admin brukeren
til denne profilen
create profile EVRY_APP_PROFILE limit
failed_login_attempts unlimited --Don't lock users in test.
password_life_time unlimited -- self-explainatory
password_reuse_max 1 -- password changes required before
password can be reused
password_reuse_time 1
password_lock_time 1
password_grace_time 365
;
alter user rman_adm profile evry_app_profile;
#lagt inn i oracle@dat-rman-ora01
/u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/tnsnames.ora (må
også inn på dat-rman-ora02)
rcatalog =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = dat-rman-scan)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = rcatalog)
)
)
#kobler til RMAN og lager catalog

oracle@dat-rman-ora01
/u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin > rman
Page 2
Copyright © EVRY.
connect catalog rman/<password>@rcatalog

create catalog;
quit;
Requires the helper_partition_dbid in the rman schema in the CATALOG database (this will exists unless the CATALOG server is renewed)
sqlplus rman/<password>@dat-rman-scan:1521/rcatalog
CREATE TABLE helper_partition_dbid
( dbid number(10) NOT NULL,
partition varchar2(40) NOT NULL,
sid varchar2(10)
)
Install TDPO software on AIX
Installation
Check for old version

lslpp -l "tivoli.*"
Uninstall if it is 32 bit client or version 5 (not common anymore)
Uninstall old client

installp -u "tivoli.tsm.client.*"
Install the new client (It will only install if there's newer software in the repo)
Page 3
Copyright © EVRY.
Install from repo (as root)

#Mount repo
mount /depot
#uninstall old clients
installp -u "tivoli.tsm.client.*"
#install tsm
installp -acY -d/depot/sw/tsm/v7160 \
GSKit8.gskcrypt64.ppc.rte GSKit8.gskssl64.ppc.rte \
tivoli.tsm.client.api.64bit tivoli.tsm.client.ba.64bit
tivoli.tsm.client.jbb.64bit tivoli.tsm.filepath_aix
#install tdpo
installp -acY -d/depot/sw/tsm/v7110/TSMORA_AIX GSKit8
tivoli.tsm.client.api.64bit tivoli.tsm.client.oracle.aix.64bit \
tivoli.tsm.client.oracle.tools.aix.64bit
tivoli.tsm.loc.client.oracle.aix.64bit.ela xlsmp.aix53.rte xlsmp.rte
#unmount repo
umount /depot
#list software versions
lslpp -l "tivoli.*"
You must link libraries to all $ORACLE_HOMES
As oracle user: For each $ORACLE_HOME on all nodes add softlinks

if [ ùname`="Linux" ]; then
ln -sf /opt/tivoli/tsm/client/oracle/bin64/libobk.so
$ORACLE_HOME/lib/libobk.so
ln -sf /opt/tivoli/tsm/client/oracle/bin64/libobk.so
$ORACLE_HOME/lib/libobk64.so
else
ln -sf /usr/tivoli/tsm/client/oracle/bin64/libobk64.a
$ORACLE_HOME/lib/libobk.a
ln -sf /usr/tivoli/tsm/client/oracle/bin64/libobk64.a
$ORACLE_HOME/lib/libobk64.a
fi
Order access to TSM server (TDPO node)

You need to have a node for each database registered on the TSM server. You order TDPO nodes in EVRY Order > Universal . The order shall
be placed on GI-Backup-Services-GeneralBackupOperations. Several nodes, and nodes on different partitions can be placed in the same order.
TDPO Nodes for RAC one

For clusters use the cluster name instead of hostname. example:

The dat-sa cluster has 3 nodes: dat-sa-ora01, dat-sa-ora02 and dat-sa-ora03. The cluster name is "dat-sa". When ordering TDPO
nodes we use:
t096car_dat_sa-oracle
d096car_dat_sa-oracle
Page 4
Copyright © EVRY.
and so on...
Information to include in the order

The order shall include the following information:
Field Value
User info: <fill inn your etident>
Service area: Storage
Order description - in English: Create TDPO node(s): (list of nodes)
<dbname>_<hostname/clustername>-oracle
Delete TDPO node(s): (list of nodes)
<dbname>_<hostname/clustername>-oracle
Size of database: larger than/or less than 500GB?
Server: <server (or list of servers if cluster)>
Customer name: EVRY
Project name: Internal backup of Oracle databases.
Invoice the customer?: No
Agreement/Contract number:
Cost center/project code:
Project Activity- code for work hours: P2348.TM.BACKUP
Assignment group: GI-Backup-Services-GeneralBackupOperations
Configure TDPO node
Add dsm.sys
Page 5
Copyright © EVRY.
Add entries to dsm.sys

TIVOLIDIR=/usr/tivoli
#[ ùname`="Linux" ] && TIVOLIDIR=/opt/tivoli
echo "SErvername HOMER

TCPServeraddress 153.110.140.61
TCPPort 3010
Passwordaccess PROMPT
ERRORLOGname /u01/app/oracle/admin/backup/tdp_err.log
DISKBUFFSIZE 32
SErvername MARGE
TCPPort 3010
DISKBUFFSIZE 32
SErvername BART
TCPPort 3010
DISKBUFFSIZE 32" > $TIVOLIDIR/tsm/client/api/bin64/dsm.sys
chmod 644 $TIVOLIDIR/tsm/client/api/bin64/dsm.sys
If you change the ERRORLOGname you must remember to script below accordingly
Create dsm.opt files

#[ ùname`="Linux" ] && TIVOLIDIR=/opt/tivoli
echo "SErvername MARGE

SUBDIR Yes
TAPEPrompt No
DATEformat 4" > $TIVOLIDIR/tsm/client/oracle/bin64/dsm.opt_MARGE
echo "SErvername HOMER
SUBDIR Yes
TAPEPrompt No
DATEformat 4" > $TIVOLIDIR/tsm/client/oracle/bin64/dsm.opt_HOMER
echo "SErvername BART
SUBDIR Yes
TAPEPrompt No
DATEformat 4" > $TIVOLIDIR/tsm/client/oracle/bin64/dsm.opt_BART
chmod 644 $TIVOLIDIR/tsm/client/oracle/bin64/dsm.opt*
Modify the script below to fit your environment and run it
Page 6
Copyright © EVRY.
Script to automatically configure TDPO

#Change as needed
#AIX
#Linux
#TIVOLIDIR=/opt/tivoli
export TDPO_BASEDIR=$TIVOLIDIR/tsm/client/oracle/bin64
export ORACLE_BASE=/u01/app/oracle
export DSMOPTFILE=dsm.opt_HOMER
export CLUSTERNAME=DAT-SA
#export CLUSTERNAME=`hostname`
export NODEEXT=oracle
#Change DBLIST to a space separated list of database names to be backed up

export DBLIST="p001pbi pwhtest t004rbs t011pbi t111pbi t411pbi u011pbi"
#export DBLIST=`ps -fu oracle | grep smon | egrep -v 'grep|ASM' | cut -d_
-f3 | tr "\\n" " "`
#Do not change anything below

#=======================================================================
#Create directories
mkdir -p $ORACLE_BASE/admin/backup
chown oracle:dba $ORACLE_BASE/admin/backup
touch $ORACLE_BASE/admin/backup/tdp_err.log
chown oracle:dba $ORACLE_BASE/admin/backup/tdp_err.log
touch $TDPO_BASEDIR/tdpoerror.log
chmod 777 $TDPO_BASEDIR/tdpoerror.log
for ORACLE_UNQNAME in $DBLIST; do
echo "Working on $ORACLE_UNQNAME"
#Create log directories
mkdir -p $ORACLE_BASE/admin/$ORACLE_UNQNAME/backup
touch $ORACLE_BASE/admin/$ORACLE_UNQNAME/backup/dsmi.log
chown oracle:dba $ORACLE_BASE/admin/$ORACLE_UNQNAME
chown oracle:dba $ORACLE_BASE/admin/$ORACLE_UNQNAME/backup
chown oracle:dba $ORACLE_BASE/admin/$ORACLE_UNQNAME/backup/dsmi.log
#Create tdpo directory and config file
mkdir -p $TDPO_BASEDIR/$ORACLE_UNQNAME
chmod 750 $TDPO_BASEDIR/$ORACLE_UNQNAME
chown root:dba $TDPO_BASEDIR/$ORACLE_UNQNAME
echo "DSMI_ORC_CONFIG $TIVOLIDIR/tsm/client/oracle/bin64/$DSMOPTFILE
DSMI_LOG $ORACLE_BASE/admin/$ORACLE_UNQNAME/backup/dsmi.log
TDPO_FS adsmorc
TDPO_NODE ${ORACLE_UNQNAME}_${CLUSTERNAME}-${NODEEXT}
TDPO_OWNER oracle
TDPO_PSWDPATH $TDPO_BASEDIR/$ORACLE_UNQNAME" >
$TDPO_BASEDIR/$ORACLE_UNQNAME/tdpo.opt
chmod 640 $TDPO_BASEDIR/$ORACLE_UNQNAME/*
chown root:dba $TDPO_BASEDIR/$ORACLE_UNQNAME/*
done
Page 7
Copyright © EVRY.
Set passwords (requires environment variables from script above)

cd $TDPO_BASEDIR
for ORACLE_UNQNAME in $DBLIST; do
echo setter passord for
echo $ORACLE_UNQNAME
sleep 2
./tdpoconf PASSWord -TDPO_OPTfile=$ORACLE_UNQNAME/tdpo.opt
done
Verify that all entries work as expected (run as oracle user)
Show configuration (requires environment variables from script above)

cd $TDPO_BASEDIR
for b in $DBLIST; do
./tdpoconf SHOWENVironment -TDPO_OPTfile=$b/tdpo.opt
echo ============================
echo "$b"
read hepp
done
Sample files for t096car@dat-sa

/usr/tivoli/tsm/client/oracle/bin64/t096car/tdpo.opt
DSMI_ORC_CONFIG /usr/tivoli/tsm/client/oracle/bin64/dsm.opt_MARGE
DSMI_LOG /u01/app/oracle/admin/t096car/backup/dsmi.log
TDPO_FS adsmorc
TDPO_NODE T096CAR_DAT-SA-ORACLE
TDPO_OWNER oracle
TDPO_PSWDPATH /usr/tivoli/tsm/client/oracle/bin64/t096car
Content of /usr/tivoli/tsm/client/oracle/bin64/t096car
-rw-r----- 1 root dba 291 okt. 26 11:29 tdpo.opt
-rw-r--r-- 1 root system 66 okt. 26 11:29
TDPO.T096CAR_DAT-SA-ORACLE
For a cluster the dsm.sys and files under /usr/tivoli/tsm/client/oracle/bin64 must be copied to all nodes:
Page 8
Copyright © EVRY.
copytsm.sh (Sample script to copy configuration to other nodes in a cluster)

#!/bin/sh
TDPODIR=/usr/tivoli/tsm/client/oracle/bin64
DSMSYSDIR=/usr/tivoli/tsm/client/api/bin64
MACHINEPREFIX=dat-efs-r-c05ora0
cd $TDPODIR
for b in 2 3; do
for c in s1011cas; do
scp -rp $c $MACHINEPREFIX$b:$TDPODIR
ssh $MACHINEPREFIX$b "chown -R root:dba $TDPODIR/$c"
done
scp -p dsm.opt* $MACHINEPREFIX$b:$TDPODIR
ssh $MACHINEPREFIX$b "chown root:system $TDPODIR/dsm.opt*"
ssh $MACHINEPREFIX$b "chmod 644 $TDPODIR/dsm.opt*"
scp -p $DSMSYSDIR/dsm.sys $MACHINEPREFIX$b:$DSMSYSDIR
ssh $MACHINEPREFIX$b "chown root:system $DSMSYSDIR/dsm.sys"
ssh $MACHINEPREFIX$b "chmod 644 $DSMSYSDIR/dsm.sys"
done
This script requires ssh passwordless login to be enabled for the root user among the nodes in the cluster (similar to the oracle user).
Schedule it in cron as root in order to propagate changes automatically
Configuring RMAN for TDP
For a cluster this should be done only on one of the nodes which you designates to run the backup scripts.
Environment
Make sure ORACLE_HOME and TNS_ADMIN are pointing to the correct directories for the database you are configuring.
Database configuration
For PCI hardened databases or databases where the sys account is disabled the script must be run on the actual node where the database is
running (set ORACLE_SID and run sqlplus /nolog). For other databases the script can be executed from a remote node if you know the sys
password. In order to enable archiving you must shut down all but one node (RAC only) of the database and run the script from the last node from
sqlplus after setting correct ORACLE_SID.
Create/unlock sysbackup account, enable block change tracking and list archiving status
Page 9
Copyright © EVRY.
Oracle 11
conn / as sysdba
create profile EVRY_SYSBACKUP_PROFILE limit
failed_login_attempts unlimited;
alter profile EVRY_SYSBACKUP_PROFILE limit
failed_login_attempts unlimited
password_life_time unlimited
password_reuse_max unlimited
password_reuse_time unlimited
password_lock_time 1
password_grace_time 365
password_verify_function NULL
;
create user sysbackup identified by XYXYXYXYXYXY;
alter user sysbackup profile EVRY_SYSBACKUP_PROFILE;
grant connect,sysdba to sysbackup;
alter user sysbackup identified by XYXYXYXYXYXY;
alter user sysbackup account unlock;
ALTER DATABASE ENABLE BLOCK CHANGE TRACKING;
archive log list;
Oracle 12
connect / as sysdba
alter user sysbackup identified by XYXYXYXYXYXY;
alter user sysbackup account unlock;
grant sysbackup to sysbackup;
ALTER DATABASE ENABLE BLOCK CHANGE TRACKING;
archive log list;
If the "archive log list" shows that the database isn't running in archivelog mode immediately execute the script below to enable archiving:
Enable archiving
shutdown immediate;
startup mount;
alter database archivelog;
alter database open;
Configure RMAN
1. Add rcatalog entry to the correct tnsnames.ora
2. If it is not already present add an entry for the database you are going to back up in the tnsnames.ora (must use same name as the
service).
3. Download the script package from the PPS repo and untar it.
4. Execute create_backup_script_pps.sh
Page 10
4.
Copyright © EVRY.
Sample run
oracle@dat-efs-c04ora01 /home/oracle/q > ./create_backup_script_pps.sh
New ORACLE_SERVICE name: crcdb01
SYSBACKUP user password: XYXYXYXYXYXY
Oracle version 11g or older (Y/N): Y
ORACLE_BASE (/u01/app/oracle):
ORACLE_HOME (/u01/app/oracle/product/12.1.0/dbhome_1):
TNS_ADMIN (/u01/app/12.1.0.2/grid/network/admin):
RMAN catalog user (rman):
RMAN catalog user password (XYXYXYXYXYXY):
RMAN catalog database (rcatalog):
RMAN catalog is rcatalog
xspaceASxspaceSYSDBA
rman_env_mal
OBS !!
Check if environment file .rman_env is ok
/u01/app/oracle/admin/crcdb01/backup/rman/.rman_env
archivelog_backup.rcv
configure_rman.rcv
delete_obsolete.rcv
getdbid.sql
incr0_backup.rcv
incr1_backup.rcv
incr2_backup.rcv
insertdbid_rcatalog.sh
list.rcv
full_restore_point_in_time.rcv
full_restore.rcv
restore_datafile.rcv
crcdb01_archivelog_backup.rcv
crcdb01_configure_rman.rcv
crcdb01_crontab_list
crcdb01_delete_obsolete.rcv
crcdb01_getdbid.sql
crcdb01_incr0_backup.rcv
crcdb01_insertdbid_rcatalog.sh
crcdb01_list.rcv
crcdb01.dbid
crcdb01_full_restore_point_in_time.rcv
crcdb01_full_restore.rcv
crcdb01_restore_datafile.rcv
crcdb01_archivelog_backup.rcv
crcdb01_configure_rman.rcv
crcdb01_crontab_list
crcdb01_delete_obsolete.rcv
crcdb01_getdbid.sql
crcdb01_insertdbid_rcatalog.sh
Page 11
Copyright © EVRY.
crcdb01_list.rcv
crcdb01.dbid
crcdb01_full_restore_point_in_time.rcv
crcdb01_full_restore.rcv
crcdb01_restore_datafile.rcv
crontab_list_mal
OBS !!
Check and modify backup schedule
/u01/app/oracle/admin/crcdb01/backup/rman/crcdb01_crontab_list
********************************************************************
* You have now created all scripts for backup and restore file
* Open the file
/u01/app/oracle/admin/crcdb01/backup/rman/crcdb01_crontab_list
* and add its content to crontab. Remember to uncomment the lines
Page 12
Copyright © EVRY.
********************************************************************
* To configure rman settings execute
* /u01/app/oracle/admin/crcdb01/backup/rman/crcdb01_configure_rman.rcv
and execute the <service>_configure_rman.rcv script in order to configure corect settings for backup, and register the database with
the catalog server.
Add crontab entries

1. Open the file $ORACLE_HOME/admin/<service>/backup/rman/<service>_crontab_list, and add its content to crontab.
2. Verify that the scripts are working by executing the "archive archivelogs2 script.
3. Uncomment the rest of the backup jobs after you have verified that the previos step finishes without errors.
4. If there are many databases change the schedule so not everything is backed up at once.
Try to set time of incremental backup (0-2) between 05-22, as 2230 until 0500 is a peak time for the TSM server. This to avoid
overloading the TSM server, which when this happens in return will result in that PPS backup is stopped.
Post activities
If you earlier had TSM backup of /rman/backup directory, and now have moved all databases to TDP, remember to remove this directory,
and free the diskspace. Also remember to remove TSM backup of this filesystem
When RMAN backup is done for all databases on the server, and this backup is verified, remember to do the following:
disable backup of /fs/oracle_home<x> and /oradata filesystems on the server.
remove all old backups on /rman/backup. If the file system then is free, remove to free disk space. Use some of this space and
add it to the archive directory.
If this frees a lot of disk space, remember to remove this in the correct manner from the partition to reduce disk cost.
Remove stop and start of database from the TSM configuration.
Monitoring (overvåking) of the RMAN backup status
Monitoring of RMAN backup for PPS Oracle databases is available at http://alt-aot-app01.man.cosng.net/rman/. Note! Oracle 9i databases
are not included
Add rotation scripts of the RMAN logs
Add script
Can be downloaded
Download rotate_rman_logs.sh
sudo su - oracle
mkdir -p $ORACLE_BASE/admin/scripts
cd $ORACLE_BASE/admin/scripts
wget http://153.110.72.43/software/Oracle/RMAN_TDPO/rotate_rman_logs.sh
chmod 755 rotate_rman_logs.sh
Or create the script rotate_rman_logs.sh with the following content
rotate_rman_logs.sh
#!/usr/bin/ksh
# Rotation of old rman log files, and deletion of old files
# Rotate logs, and delete files older than x DAYS
# Run at 0600 Friday
# Tom Fladsrud 2014
# Crontab entry for oracle user
# RMAN TDPO log rotation
# 0 06 * * 5 $ORACLE_BASE/admin/scripts/rotate_rman_logs.sh
Page 13
Copyright © EVRY.
$ORACLE_BASE/admin 2>&1
#i.e
# 0 06 * * 5 /fs/oracle_home/admin/scripts/rotate_rman_logs.sh
/fs/oracle_home/admin 2>&1
#
DAYS=30
#FILE_TYPE=_backup.log
#DATO=`/usr/bin/date +'%y%m%d'`
Usage()
{
echo "Usage: $0 <directory>"
echo "<directory>: the directory to start searching for logfiles"
echo "Example: sh $0 $ORACLE_HOME/admin"
}
if [ $# -lt 1 ]
then
Usage
exit 1
fi
CUR_DIR=`pwd`
START_DIR=$1
cd "$START_DIR"
function rotate
{
date_daily=`date +"%d-%m-%Y"`
#echo "$date_daily"
#rotate the logs
for FILE1 in `ls | grep _backup.log | grep -v old`
do
cp $FILE1 $FILE1-old-$date_daily
> $FILE1
done
# clean up old logs
CUR_D=`pwd`
cd ..
#echo "find $CUR_D -type f -name '*-old-*' -mtime +$DAYS -print"
for FILE2 in `find $CUR_D -type f -name '*-old-*' -mtime +$DAYS
-print`
do
#echo "File $FILE2 to old. Will be deleted"
rm -f $FILE2
done
cd -
}
#ROTATE LOG FILES
for FILE in `ls`
do
if [ -d $FILE ]
then
#echo "$FILE is a subdirectory..."
if [ -d "$FILE/backup/rman/log" ]
then
#echo "$FILE/backup/rman/log exists"
Page 14
Copyright © EVRY.
cd "$FILE/backup/rman/log"
rotate
cd $START_DIR
Page 15
Copyright © EVRY.
fi
fi
done
Add to crontab
Add the following to crontab. Remember to uncomment
Entry in crontab
# RMAN TDPO log rotation
0 6 * * 5 /u01/app/oracle/admin/scripts/rotate_rman_logs.sh
/u01/app/oracle/admin 2>&1
Changes when upgrading to new Oracle version where ORACLE_HOME

has changed
cd $ORACLE_HOME/lib. If $ORACLE_HOME is not defined for root user, change directory manually. NB! This should be the new ORACLE_HO
ME.
ln -s /usr/tivoli/tsm/client/oracle/bin64/libobk64.a libobk.a
ln -s /usr/tivoli/tsm/client/oracle/bin64/libobk64.a libobk64.a

Copy tnsnames.ora entry for rcatalog to the new $ORACLE_HOME/network/admin. This shall as a minimum include the RMAN Catalog.
Run script to change environment variables for RMAN

Run the new Oracle profile, and run the following for the databases moved to a new $ORACLE_HOME:
cd /u0/app/oracle/admin/rman_scripts_temp/
sh create_backup_script_pps.sh
Check status of RMAN backup

Make sure the RMAN backup runs as it should after these changes.
Deletion of databases with RMAN TDPO backup

Remove backup
When removing a database using TDPO backup, backup must be removed before deleting the database.
For connecting to Oracle 11 or lower, remove AS SYSBACKUP from the command. See HowtoconnecttoRMANandCATALOG for details.
Procedure for removing TDPO backup
rman
Page 16
Copyright © EVRY.
connect target SYSBACKUP/<PASSWORD>@DATABASE AS SYSBACKUP;

connect CATALOG rman/<password>@rcatalog;
DELETE FORCE NOPROMPT BACKUP;
If the database is not to be reused, notify "EDB Drift TURNUS storage open" (z999sdms) that the tdpo node for the database can be removed,
and that all backup for this database shall be removed.
Unregister from CATALOG

If the database is registered in CATALOG, it must be unregistered before you delete the database.
Unregister database from CATALOG - Oracle 10g and higher
Make sure the DBID is set when logged in, before running the unregister command
rman
UNREGISTER DATABASE NOPROMPT;
Unregister database from CATALOG - Oracle 9i

The unregister function was first introduced in Oracle 10g. For Oracle 9i one must remove the database from the CATALOG database. First
you must find the dbid of your database.
To be executed on the database you want to unregister

SELECT DBID FROM V$DATABASE;
Then you must find the correct row of the DB table in the CATALOG database. Using the DB_ID you fond in he previous SQL
To be executed on the RMAN CATALOG database (rcatalog)

SELECT DB_KEY, DB_ID FROM DB WHERE DB_ID = <DBID>;
Then you can remove the database from the CATALOG database with the following SQL
To be executed on the RMAN CATALOG database

EXECUTE dbms_rcvcat.unregisterdatabase(<db_key>, <db_id>)
Delete database from helper table
sqlplus rman/<rman password>@rcatalog

DELETE FROM helper_partition_dbid where dbid=<dbid>;
commit;
quit;
Page 17
Copyright © EVRY.
Order deletion of TDPO

If you are not going to use the TDPO node anymore. Place an order of deletion of the TDPO node in EVRY Order > Universal . The order shall be
placed on GI-Backup-Services-GeneralBackupOperations. Removal of several nodes, and nodes on different partitions can be placed in the same
order.
Information included in the order
The order shall include the following information:
Field Value
Brukerinformasjon <fill inn your etident>
Tjenesteområde Datalagring
Oppdragsbeskrivelse Order of DELETION of TDPO node:
Node name: <service>_<hostname>-oracle (i.e. ppscdb02_dat-efs-c04-oracle)

Partition of database:
Hostname: <hostname> (i.e. dat-efs-c04ora01 (dat-efs-c04ora01.man.cosng.net))
SLAN IP Address: <SLAN-IP> (i.e. 10.150.128.181)
ProdLAN IP Address: <ProdLAN-IP> (i.e 192.168.179.188)
TSM Server used for filesystem backups on the database server (found in
/usr/tivoli/tsm/client/ba/bin/dsm.sys):
SErvername: <name of TSM server> (i.e. brundtland)
TCPserveraddress: <IP of TSM server> (i.e. 153.110.140.52)
TCPPort: <TSM port> (i.e. 3010)
Database centre location: <data hall where the database is located> (i.e. Datahall 3 (S3)
Shall the data be stored or deleted at once? If stored, for how long?
Kunde/prosjekt Internal backup of Oracle databases. P2348.TM.BACKUP
Skal kunden faktureres Nei
IFS – kode for timeføring P2348.TM.BACKUP
Hvis kjent - legg inn utførende GI-Backup-Services-GeneralBackupOperations

enhet
Appendix
Finding the service name for your database
How to find the service name of your database

lsnrctl status
Restore with RMAN TDP from scripts

Restore scripts for RMAN can be found at $ORACLE_HOME/admin/ppscdb02/backup/rman/restore
This are scripts for
restore of datafile
full restore
full restore at a specific restore point
Before running the script, you must of course alter the script to suit your need. What datafile you want to restore, what time and so on. Examples
is given in the files. The actual work is commented out to avoid someone running the scripts by accident, causing problems. Suggest they are
commented out again after usage.
Not customized for Oracle 12c or higher
Page 18
Copyright © EVRY.
Be aware that these scripts are not customized for Oracle 12c. For Oracle 12c or higher, use manual routines.
Restore with RMAN TDP manually (recommended)

Before running any of these recovery and restore function, always comment out the backup scripts for that database in crontab. When database is
restored, create a fresh backup of the database and uncomment (activate) the cronjobs.
Corrupt or missing control file (i.e. ORA-00210)
rman
shutdown abort;
startup nomount;
restore controlfile;
You might get a error message saing "RMAN-06004: ORACLE error from recovery catalog database: RMAN-20005: target
database name is ambiguous":
Solution:
Find the dbid from one of the rman backup logfiles. Then set the dbid:
set dbid <id>
Then run "restore controlfile;" once more
alter database mount;

recover database;
alter database open resetlogs;
Then when someone has verified the database, run a full backup (incr0) uncomment the cronjobs for this database.
All datafiles and control files is deleted or corrupt

sqlplus "/as sysdba"

shutdown abort;
startup nomount;
rman
set dbid <id>
Find the dbid from one of the rman backup logfiles.
i.e set dbid 1769537117
Page 19
Copyright © EVRY.
restore controlfile;
RESTORE DATABASE; # Restore datafiles from full backup.
RECOVER DATABASE; # Append archive files to database.
If you get a message during recover database like this:
arkivloggtrden 1 sekvensen 5 er allerede p disk som filen /u01/app/oracle/fra/fd11/1_5.dbf arkivlogg,

filnavn=/u01/app/oracle/fra/fd11/1_5.dbf trd=1 sevens=5 finner ikke arkivloggen arkivlogg trd=1 sekvens=6
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 02/19/2014 21:17:52
RMAN-06054: media recovery requesting unknown log: thread 1 scn 241709
Solution
Oracle does not stop the recovery at the last available archive log in the database backupsets as per the restored control file.
Instead it keeps on requesting for the "next available archive log sequence" number. But in reality, as per the above example,
there is no log sequence 3. So the database is already consistent after applying log sequence 2 (final archive log sequence as per
the restored control file) and hence the recovery is in fact successful. Just try to open the database with the resetlogs option. It
should work
alter database open resetlogs
You should get the "Database altered" message which indicates that the database was opened successfully.
Then when someone has verified the database, run a full backup (incr0) uncomment the cronjobs for this database.
One or more datafiles are corrupt or deleted - Restore and Recover entire database at one spesific point in time
This is not necessarily discovered during backup, but first discovered when using the database. It could the be desirable to restore
to a point in time where you know this worked.
Example of such error:
select * from Persons; select * from Persons

* FEIL p linje 1: ORA-00376: filen 11 kan ikke leses n
ORA-01110: datafil 11:
/fs/oracle_home3/oradata/fd11/fd11_stab_space_01.dbf
Page 20
Copyright © EVRY.
Recovery with RMAN when controlfile is ok

set NLS_DATE_FORMAT='Mon DD YYYY HH24:MI:SS'
rman
shutdown abort;
startup mount;
RUN {
set until time "TO_DATE('Feb 24 2014 11:20:00', 'MON DD YYYY
HH24:MI:SS')";
# SET UNTIL SCN 1000; # alternatively, you can specify SCN
# SET UNTIL SEQUENCE 9923; # alternatively, you can specify log
sequence number
RESTORE DATABASE;
RECOVER DATABASE;
}
ALTER DATABASE OPEN RESETLOGS;
Recovery with RMAN when controlfile also must be restored at a specific time
set NLS_DATE_FORMAT='Mon DD YYYY HH24:MI:SS'
rman
shutdown abort;
startup force nomount;
run
{
set until time "to_date('2015-05-20:16:00:00',
'yyyy-mm-dd:hh24:mi:ss')";
RESTORE CONTROLFILE;
}
run
{
set until time "to_date('2015-05-20:16:00:00',
'yyyy-mm-dd:hh24:mi:ss')";
restore database;
recover database;
}
alter database open resetlogs;
Recommended to create a fresh backup

It is recommended that you immediately back up the database, preferably with the database
mounted (to avoid possible data loss in an open database).
Because the database is a new incarnation, the backups made before the RESETLOGS are not easily
usable. For example, run the following to back up the database:
Page 21
Copyright © EVRY.
SHUTDOWN IMMEDIATE
STARTUP MOUNT
BACKUP DATABASE;
ALTER DATABASE OPEN;
Several datafiles are deleted - Restore and Recovery of each datafile

This is not necessarily discovered during backup, but first discovered when using the database. I.e. during insert.
ERROR: ORA-00604: det oppstod en feil p rekursivt SQL-niv 1

ORA-01116: feil ved pning av databasefilen 1
ORA-01110: datafil 1: /fs/oracle_home4/fd11/system01.dbf ORA-27041: kan
ikke pne filen
IBM AIX RISC System/6000 Error: 2: No such file or directory
Error when loging into RMAN - ORA-01110

You might get an error during login to RMAN like shown below.
rman
Gjenopprettingsstyrer : Release 11.1.0.7.0 - Production on On Mar 5
09:38:10 2014
Copyright (c) 1982, 2007, Oracle. All rights reserved.
RMAN-00571:
===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS
===============
RMAN-00571:
===========================================================
RMAN-00554: feil ved initialisering av intern
gjenopprettingsstyrerpakke
RMAN-04005: feil fra mldatabasen:
ORA-00604: det oppstod en feil p rekursivt SQL-niv 1
ORA-01116: feil ved pning av databasefilen 1
ORA-01110: datafil 1: /fs/oracle_home4/fd11/system01.dbf
ORA-27041: kan ikke pne filen IBM AIX RISC System/6000 Error: 2: No
such file or directory Additional information: 3
If so, run ps -ef | grep smon_<oracle sid> and kill the process of the database you are working on.
Page 22
Copyright © EVRY.
rman
set dbid 1770049508
startup mount
RESTORE DATAFILE '/fs/oracle_home4/fd11/system01.dbf';
RECOVER DATAFILE '/fs/oracle_home4/fd11/system01.dbf';
Get ORA-01110 during startup

If there are other datafiles missing, you will get an error similar to the one below during alter database open command
RMAN-00571:
===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS
===============
RMAN-00571:
===========================================================
RMAN-03002: kommandoen alter db mislyktes ved 03/05/2014 10:41:35
ORA-01157: kan ikke identifisere/lse datafilen 3 - se sporingsfilen
for DBWR
ORA-01110: datafil 3: /fs/oracle_home4/fd11/undotbs01.dbf
If so run
RESTORE DATAFILE '/fs/oracle_home4/fd11/undotbs01.dbf';

RECOVER DATAFILE '/fs/oracle_home4/fd11/undotbs01.dbf';
Do this for all datafiles missing.
Troubleshoot for RMAN restore

RMAN-06004: ORACLE error from recovery catalog database: RMAN-20207: UNTIL TIME or RECOVERY WINDOW is before RESETLOGS
time
If you have tried a restore and run ALTER DATABASE OPEN RESETLOGS, and this backup did not work, so you must go back in time, you
might get an error "RMAN-06004: ORACLE error from recovery catalog database: RMAN-20207: UNTIL TIME or RECOVERY WINDOW is
before RESETLOGS time"
If so, try the following, where you set the incarnation to the Inc Key of the PARENT. In this case the parent Inc Key was19669935
Page 23
Copyright © EVRY.
RMAN> ALTER DATABASE MOUNT;

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of alter db command at 05/21/2015 08:12:28
ORA-01100: databasen er allerede montert
RMAN> list incarnation of database UNB01;
List of Database Incarnations

DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time
------- ------- -------- ---------------- --- ---------- ----------
19669934 19669935 UNB01 2427545002 PARENT 1
26.10.2010
19669934 67414091 UNB01 2427545002 CURRENT 567754925
21.05.2015
RMAN> RESET DATABASE TO INCARNATION 19669935;
database reset to incarnation 19669935
Then retry the operation you tried when getting the RMAN-06004 message.
Page 24

Backup Configuration PPS

Uploaded by

Copyright:

Available Formats

Backup Configuration PPS

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Backup Configuration PPS

Uploaded by

Copyright:

Available Formats

Copyright © EVRY.

Backup installation: RMAN to tape v2

Installing TDP software

Create a CATALOG database (if it doesn't exist)

On the CATALOG database

How to create CATALOG

#Lager pluggable database rcatalog, med en local admin bruker, og åpner

alter pluggable database rcatalog open;

#kobler til RMAN og lager catalog

connect catalog rman/<password>@rcatalog

Install TDPO software on AIX

Check for old version

Uninstall if it is 32 bit client or version 5 (not common anymore)

Uninstall old client

Install from repo (as root)

You must link libraries to all $ORACLE_HOMES

As oracle user: For each $ORACLE_HOME on all nodes add softlinks

Order access to TSM server (TDPO node)

TDPO Nodes for RAC one

Click here to expand...

Information to include in the order

User info: <fill inn your etident>

Service area: Storage

Order description - in English: Create TDPO node(s): (list of nodes)

Delete TDPO node(s): (list of nodes)

Size of database: larger than/or less than 500GB?

Server: <server (or list of servers if cluster)>

Customer name: EVRY

Project name: Internal backup of Oracle databases.

Invoice the customer?: No

Cost center/project code:

Project Activity- code for work hours: P2348.TM.BACKUP

Assignment group: GI-Backup-Services-GeneralBackupOperations

Configure TDPO node

Add entries to dsm.sys

echo "SErvername HOMER

Create dsm.opt files

echo "SErvername MARGE

Modify the script below to fit your environment and run it

Script to automatically configure TDPO

#Change DBLIST to a space separated list of database names to be backed up

#Do not change anything below

Set passwords (requires environment variables from script above)

Verify that all entries work as expected (run as oracle user)

Show configuration (requires environment variables from script above)

Sample files for t096car@dat-sa

copytsm.sh (Sample script to copy configuration to other nodes in a cluster)

Configuring RMAN for TDP

Add crontab entries

Add rotation scripts of the RMAN logs

chmod 755 rotate_rman_logs.sh

Or create the script rotate_rman_logs.sh with the following content

Changes when upgrading to new Oracle version where ORACLE_HOME

Create symbolic link

Run script to change environment variables for RMAN

Check status of RMAN backup

Deletion of databases with RMAN TDPO backup

connect target SYSBACKUP/<PASSWORD>@DATABASE AS SYSBACKUP;

DELETE FORCE NOPROMPT BACKUP;

Unregister from CATALOG