ORA-600 [kfcNullConvert20] error on ASM instance:
I have experienced ORA-600 error on ASM instance (10.2.0.2), which caused ASM instance restart on that particular node. Due to which all databases instances on that node are restarted, because they lost connectivity with ASM.
This issue is a known Bug 4682861 in Oracle and the effected version are 10.1.0.4 ,10.1.0.5 ,10.2.0.1 ,10.2.0.2. This BUG is fixed in 10.2.0.3 Server patch set and 11.1.0.6 Base release.
Check for Possible Error messages in ASM/Database alert.log and trace files.
If you find below SYMPTOMS then you are hitting a BUG 4682861.
ASM alert. log errors:
ORA-00600: internal error code, arguments: [kfcNullConvert20], [], [], [], [], [], [], []
DBW0: terminating instance due to error 471
Trace file (ASM DBWR trace file contains Stack Trace: kfcNullConvert and errors):-
error 600 detected in background process
ORA-00600: internal error code, arguments: [kfcNullConvert20], [], [], [], [], [], [], []
ksuitm: waiting for [5] seconds before killing DIAG
Database alert log errors:
ORA-15064: communication failure with ASM instance
ORA-03113: end-of-file on communication channel
ASMB: terminating instance due to error 15064
Cause:
Due to Bug 4682861, ASM instance restarted.
Solution: ***************
Option 1:- Upgrade to 10.2.0.3 or higher release
Option 2:-Download the patch from MetaLink:
i) Click on Patches & Updates Link.
ii) Enter patch number: 4682861 and Select your O/S
iii) Click Go.
iv) Download the patch for the Oracle release that you experienced this issue.
Please note that ANY ORA-600 error indicates Oracle has detected an internal inconsistency or a problem which it doesn’t know how best to address. These are *NOT* necessarily bugs and can occur for reasons such as running out of some resource, Operating System IO problems.. etc.
************************************************
Create, Drop and Alter ASM disk groups
Crete Disk Group:
Create Disk groups using the CREATE DISKGROUP statement and specify the level of redundancy.
Disk group redundancy types:-
NORMAL REDUNDANCY - Two-way mirroring, requiring two failure groups.
HIGH REDUNDANCY - Three-way mirroring, requiring three failure groups.
EXTERNAL REDUNDANCY - No mirroring for disks that are already protected using hardware RAID or mirroring.
SQL> CREATE DISKGROUP data NORMAL REDUNDANCY
FAILGROUP failure_group_1 DISK '/dev/sda1' NAME dataa1,'/dev/sda2' NAME dataa2,
FAILGROUP failure_group_2 DISK '/dev/sdb1' NAME datab1,'/dev/sdb2' NAME datab2;
Drop Disk Group:
Drop disk group using DROP DISKGROUP statement.
SQL> DROP DISKGROUP data INCLUDING CONTENTS;
Alter Disk Group:
Add or remove disks from disk groups Using ALTER DISKGROUP statement. You can also use wildcard "*" to reference disks.
Add a disk.
SQL> ALTER DISKGROUP data ADD DISK '/dev/datac1', '/dev/datac2';
Add all reference disks
SQL> ALTER DISKGROUP data ADD DISK '/dev/datad*;
Drop/remove a disk.
SQL> ALTER DISKGROUP data DROP DISK datab2;
The UNDROP command used to undo only pending drop of disks. After you drop the disks you cannot revert.
SQL> ALTER DISKGROUP data UNDROP DISKS;
Diskgroup Rebalance:
Disk groups can be rebalanced manually Using REBALANCE clause and you can modify the POWER clause default value.
SQL> ALTER DISKGROUP disk_group_1 REBALANCE POWER 5;
MOUNT and DISMOUNT DiskGroups:
Normally Disk groups are mounted at ASM instance startup and dismounted at shutdown. Using MOUNT and DISMOUNT options you can make one or more Disk Groups available or unavailable.
SQL> ALTER DISKGROUP data MOUNT;
SQL> ALTER DISKGROUP data DISMOUNT;
SQL> ALTER DISKGROUP ALL MOUNT;
SQL> ALTER DISKGROUP ALL DISMOUNT;
DiskGroup Check:
Use CHECK ALL to verify the internal consistency of disk group metadata and repair in case of any error.
SQL> ALTER DISKGROUP data CHECK ALL;
DiskGroup resize:
Resize the one or all disks in the Diskgroup.
Resize all disks in a failure group.
SQL> ALTER DISKGROUP data RESIZE DISKS IN FAILGROUP failure_group_1 SIZE 1024G;
Resize a specific disk.
SQL> ALTER DISKGROUP data RESIZE DISK dataa1 SIZE 1024G;
Resize all disks in a disk group.
SQL> ALTER DISKGROUP data RESIZE ALL SIZE 1024
***************************************************
Migrate Database to ASM Using RMAN: *******
We are Using RMAN to relocate non-ASM files to ASM files. The ASM files cannot be accessed through normal OS interfaces.
Step1: Query V$CONTROLFILE and V$LOGFILE to get the file names.
SQL> select * from V$CONTROLFILE;
SQL> select * from V$LOGFILE;
Step 2: Shutdown the database.
SQL> SHUTDOWN IMMEDIATE;
Step3: Modify the target database parameter file:
(i) Remove the CONTROL_FILES parameter from the spfile, so the control files will be created automatically in ASM Disk group mentioned in DB_CREATE_FILE_DEST destination
Using a pfile then set CONTROL_FILES parameter to the appropriate ASM files or aliases.
(ii) Set the DB_CREATE_FILE_DEST and DB_CREATE_ONLINE_LOG_DEST_n parameters to the relevant ASM disk groups.
Step 4: Startup nomount mode and Restore control file
RMAN> STARTUP NOMOUNT;
Restore the control file into the new location.
RMAN> RESTORE CONTROLFILE FROM 'old_controlfile_name';
Step 5: Startup mount mode and backup the Database.
RMAN> ALTER DATABASE MOUNT;
Backup database into ASM disk group.
RMAN> BACKUP AS COPY DATABASE FORMAT '+diskgroup1';
Step 6: Switch database and create or rename Redo log members
Switch all data files to the new ASM Diskgroup location.
RMAN> SWITCH DATABASE TO COPY;
RMAN> SQL “ALTER DATABASE RENAME ‘old_redolog_member’ to ‘+diskgroup2’;
or
Create new redo logs in ASM Disk group and delete the old redo log files.
Step 7: Open Database and create temporary tablespace.
Open database using resetlogs
SQL> ALTER DATABASE OPEN RESETLOGS;
Create temporary tablespace in ASM disk group.
SQL> CREATE TABLESPACE temp1 ADD TEMPFILE ‘+diskgroup1’;
Step 8: Drop old database files.
1. SQL> DROP TABLESPACE ‘old_temporary_tablespace’ including contents and datafiles;
2. Remove all remaining Non-ASM database files using OS commands
***************************************
Createdisk, Deletedisk and Querydisk in ASM :
The /etc/init.d/oracleasm script is used to create, delete and query ASM disks and make disks available.
Create and Delete ASM disk:
Run below command to Create ASM disks.
# /etc/init.d/oracleasm createdisk DATA1 /dev/sdc
Creating Oracle ASM disk "DATA1" [ OK ]
Run below command to Delete ASM disks.
# /etc/init.d/oracleasm deletedisk DATA1
Deleting Oracle ASM disk "DATA1" [ OK ]
Query/List/Scan ASM disk:
Run the below querydisk command to see if the Disk/Device is used by ASM:
# /etc/init.d/oracleasm querydisk /dev/sdc
Checking if device "/dev/sdc" is an Oracle ASM disk [ OK ]
# /etc/init.d/oracleasm querydisk DATA1
Checking for ASM disk "DATA1" [ OK ]
Run below command to list Existing disks
# /etc/init.d/oracleasm listdisks
DATA1
Run the below command to scan the disks created on another node.
# /etc/init.d/oracleasm scandisks
Scanning system for ASM disks [ OK ]
********************************************
Delete Archivelog files without using RMAN :
Here I am explaining two methods to delete archive logs and other database files with out using RMAN from ASM Disk Group in Oracle 10g/11g.
Method 1: asmcmd - ASM command-line utility
ASMCMD> rm file_name
For ex:-ASMCMD> rm ‘+dgroup2/testdb/archivelogs/thread_1_seq_363.510.1’
Or
ASMCMD> rm ‘+dgroup2/testdb/datafile/USERS.250.5334166963’
If you use a wildcard, rm deletes all matches except non-empty directories (unless the -r flag is used). The rm command can delete the file or alias only if the file is not currently in use by a client database.
Method 2: SQLPLUS utility
SQL> ALTER DISKGROUP DROP file
For ex:- SQL> ALTER DISKGROUP FLASH DROP FILE ‘+FLASH/testdb/archivelog/2009_08_11/thread_1_seq_363.510.1';
The “asmcmd” and “sqlplus” commands will not update the database views (V$ARCHIVED_LOG, V$FLASH_RECOVERY_AREA_USAGE), controlfile, Recovery Catalog that the files have been removed.
To update the Database views, control file or RMAN Catalog about deleted files you need to run the below command from RMAN.
RMAN> CROSSCHECK ARCHIVELOG ALL;
RMAN> DELETE EXPIRED ARCHIVELOG ALL;
*****************************************
Install and Configure ASMLib in 10g (Automatic Storage Management):
The Oracle ASM feature was introduced in Oracle 10g Release 1.
There are two methods to configure ASM on Linux.
1. Configure ASM with ASMLib I/O: This method creates all Oracle database files on raw block devices, which are managed by ASM using ASMLib calls. ASMLib works with block devices and raw devices are not required with this method.
2. Configure ASM with Standard Linux I/O: This method creates Oracle database files on raw character devices, which are managed by ASM using standard Linux I/O system calls. It requires creating RAW devices for all disk partitions used by the ASM.
Here we will “Configure ASM with ASMLib I/O” method.
Step 1: Download and Install ASMLib
Download Oracle “ASMLib” software from below link and follow the link for your platform.
http://www.oracle.com/technology/tech/linux/asmlib/index.html
You must install all three packages for the kernel you are running. Use “uname –r “command to determine the version of your kernel.
oracleasm-support-version.cpu_type.rpm
oracleasm-kernel-version.cpu_type.rpm
oracleasmlib-version.cpu_type.rpm
See the below example to install the packages and run the command as root.
# rpm -ivh oracleasm-support-2.0.3-1.x86_64.rpm \
> oracleasm-2.6.9-67.ELsmp-2.0.3-1.x86_64.rpm \
> oracleasmlib-2.0.2-1.x86_64.rpm
Step 2: Configure and Enable Oracle ASM
Run the below command to configure the Oracle ASM and it will ask for the user and group that default to owing the ASM drivers access point.
# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Fix permissions of Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration [ OK ]
Creating /dev/oracleasm mount point [ OK ]
Loading module "oracleasm" [ OK ]
Mounting ASMlib driver filesystem [ OK ]
Scanning system for ASM disks [ OK ]
This command will load the ASM driver and mount the ASM driver filesystem. By selecting “y” during the configuration, the system will always load the module and mount the file system on system boot.
Run the below command to enable automatic start
#/etc/init.d/oracleasm enable
No comments:
Post a Comment