srvctl modify service -d DBFAP -s S999 -n -i DBFAP1,DBFAP2,DBFAP3
PRKP-1029 : Failed to register the service S999
=== ODM Action Plan ==========
1.- Provide the output of the following
srvctl config service -d
2.- Before modifying you need to stop the resources. Please confirm you did this
3.- Try to recreate/modify the Service invoking using dbca
************
Here is the output
srvctl config service -d DBFAP -s S999
S999 PREF: DBFAP1 DBFAP2 AVAIL:
We didnt stop the service before modifying .. what would be the impact for users using this service if we stop ?
I dont have GUI access, hence cannot use DBCA at this moment. Please provide me alternateways.
*******************
there are lot of resource on this .. which resources need to be down for service modify. Also we are able to modify other services while all are up, but only this S999 service is throwing the error.
oracle@lxdb0109c03:/u01/app/oracle/product/10.2.0/crs/bin> crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ebr_vip application ONLINE ONLINE lxdb0109c02
ora....P1.inst application ONLINE ONLINE lxdb0109c01
ora....P2.inst application ONLINE ONLINE lxdb0109c02
ora....P3.inst application ONLINE ONLINE lxdb0109c03
ora....AP1.srv application ONLINE ONLINE lxdb0109c02
ora....AP2.srv application ONLINE ONLINE lxdb0109c01
ora....AP3.srv application ONLINE ONLINE lxdb0109c03
ora....S001.cs application ONLINE ONLINE lxdb0109c02
ora....AP1.srv application ONLINE ONLINE lxdb0109c01
ora....AP2.srv application ONLINE ONLINE lxdb0109c02
ora....AP3.srv application ONLINE ONLINE lxdb0109c03
ora....S002.cs application ONLINE ONLINE lxdb0109c02
ora....AP1.srv application ONLINE ONLINE lxdb0109c02
ora....AP2.srv application ONLINE ONLINE lxdb0109c01
ora....AP3.srv application ONLINE ONLINE lxdb0109c03
ora....S003.cs application ONLINE ONLINE lxdb0109c02
ora....AP1.srv application ONLINE ONLINE lxdb0109c01
ora....AP2.srv application ONLINE ONLINE lxdb0109c02
ora....AP3.srv application ONLINE ONLINE lxdb0109c03
ora....S004.cs application ONLINE ONLINE lxdb0109c02
ora....AP1.srv application ONLINE ONLINE lxdb0109c02
ora....AP2.srv application ONLINE ONLINE lxdb0109c01
ora....AP3.srv application ONLINE ONLINE lxdb0109c03
ora....S005.cs application ONLINE ONLINE lxdb0109c02
ora....AP1.srv application ONLINE ONLINE lxdb0109c01
ora....AP2.srv application ONLINE ONLINE lxdb0109c02
ora....AP3.srv application ONLINE ONLINE lxdb0109c03
ora....S006.cs application ONLINE ONLINE lxdb0109c02
ora.DBFAP.db application ONLINE ONLINE lxdb0109c02
ora....AP1.srv application ONLINE ONLINE lxdb0109c01
ora....AP2.srv application ONLINE ONLINE lxdb0109c02
ora....s999.cs application ONLINE ONLINE lxdb0109c02
ora....01.lsnr application ONLINE ONLINE lxdb0109c01
ora....c01.gsd application ONLINE ONLINE lxdb0109c01
ora....c01.ons application ONLINE ONLINE lxdb0109c01
ora....c01.vip application ONLINE ONLINE lxdb0109c01
ora....02.lsnr application ONLINE ONLINE lxdb0109c02
ora....c02.gsd application ONLINE ONLINE lxdb0109c02
ora....c02.ons application ONLINE ONLINE lxdb0109c02
ora....c02.vip application ONLINE ONLINE lxdb0109c02
ora....03.lsnr application ONLINE ONLINE lxdb0109c03
ora....c03.gsd application ONLINE ONLINE lxdb0109c03
ora....c03.ons application ONLINE ONLINE lxdb0109c03
ora....c03.vip application ONLINE ONLINE lxdb0109c03
oracle@lxdb0109c03:/u01/app/oracle/product/10.2.0/crs/bin>
*****************************************************************
=== ODM Action Plan ===
S999 PREF: DBFAP1 DBFAP2 AVAIL:
1. When did You first add the service S999?
2. Were there any errors reported when creating the service?
3. check for available ocr backups:
ocrconfig -showbackup
4. on both nodes backup the files in $CRS_HOME/cdata/
You may copy them to another directory
5. try the following commands
script /tmp/srvctl.log
export SRVM_TRACE=true
srvctl config service -d DBFAP
srvctl status service -d DBFAP -s S999 -f -v
srvctl modify service -d DBFAP -s S999 -n -i DBFAP1,DBFAP2 -f
exit
upload output and resulting trace files
******************************************
This is a three node rac, Third node is added recently to existing 2 node setup. The instances are DBFAP1, DBFAP2. DBFAP3 , database is DBFAP. So you want me to backup for all three nodes and run the commands to modify the service from third node ?
Also please confirm whether modify command is : (includes three instances)
srvctl modify service -d DBFAP -s S999 -n -i DBFAP1,DBFAP2,DBFAP3
Hi,
As this is Three node RAC,
Please update whether the modify command should be
srvctl modify service -d DBFAP -s S999 -n -i DBFAP1,DBFAP2 -f -------------- OR
srvctl modify service -d DBFAP -s S999 -n -i DBFAP1,DBFAP2,DBFAP3 -f
*********************
Hi!
It have to use:
srvctl modify service -d DBFAP -s S999 -n -i DBFAP1,DBFAP2,DBFAP3 -f
**********************
oracle@lxdb0109c03:/u01/app/oracle/product/10.2.0/crs/bin> ocrconfig -showbackup
lxdb0109c02 2010/08/27 21:14:46 /u04/oracm/DBFPCluster1
lxdb0109c02 2010/08/27 17:14:40 /u04/oracm/DBFPCluster1
lxdb0109c02 2010/08/27 13:14:34 /u04/oracm/DBFPCluster1
lxdb0109c02 2010/08/26 21:14:07 /u04/oracm/DBFPCluster1
lxdb0109c02 2010/08/20 21:10:22 /u04/oracm/DBFPCluster1
oracle@lxdb0109c03:/u01/app/oracle/product/10.2.0/crs/bin>
**********************************
FileComment (trace file output)
----------------------
[main] [1:46:48:626] [HASContext.getCRSHome:457] /u01/app/oracle/product/10.2.0/crs
[main] [1:46:48:626] [Util.getCRSHome:458] getCRSHome: ret=/u01/app/oracle/product/10.2.0/crs
[main] [1:46:48:626] [HAOperation.setCRSHome:148] CRS_HOME: /u01/app/oracle/product/10.2.0/crs
[main] [1:46:48:627] [HAOperation.checkCommandVerb:211] Checking an existence of CRS command
[main] [1:46:48:627] [Service.genSCProfile:915] Creating profile for Service Composite
[main] [1:46:48:627] [LocalCommand.execute:54] LocalCommand.execute: Calling run method
[main] [1:46:48:627] [HAProfileOperation.run:175] Executing the HA operation /u01/app/oracle/product/10.2.0/crs/bin/crs_profile ora.DBFAP.S999.cs
[main] [1:46:48:627] [HAOperationImpl.writeProfileOnDisk:1167] Writing the profile on disk for ora.DBFAP.S999.cs
[main] [1:46:48:628] [HAOperationImpl.writeProfileOnDisk:1168] Running /u01/app/oracle/product/10.2.0/crs/bin/crs_stat for ora.DBFAP.S999.cs
[main] [1:46:48:628] [HAOperationImpl.runCommand:1246] CRS cmd is: /u01/app/oracle/product/10.2.0/crs/bin/crs_stat -p ora.DBFAP.S999.cs
[main] [1:46:48:628] [RuntimeExec.runCommand:74] Calling Runtime.exec() with the command
[main] [1:46:48:628] [RuntimeExec.runCommand:76] /u01/app/oracle/product/10.2.0/crs/bin/crs_stat
[main] [1:46:48:628] [RuntimeExec.runCommand:76] -p
[main] [1:46:48:628] [RuntimeExec.runCommand:76] ora.DBFAP.S999.cs
[Thread-3] [1:46:48:632] [StreamReader.run:61] In StreamReader.run
[main] [1:46:48:632] [RuntimeExec.runCommand:131] runCommand: Waiting for the process
[Thread-2] [1:46:48:632] [StreamReader.run:61] In StreamReader.run
[Thread-2] [1:46:48:678] [StreamReader.run:65] OUTPUT>CRS-0210: Could not find resource 'ora.DBFAP.S999.cs'.
[Thread-2] [1:46:48:678] [StreamReader.run:65] OUTPUT>
[main] [1:46:48:680] [RuntimeExec.runCommand:133] runCommand: process returns -46
[main] [1:46:48:680] [RuntimeExec.runCommand:147] RunTimeExec: output>
[main] [1:46:48:680] [RuntimeExec.runCommand:150] CRS-0210: Could not find resource 'ora.DBFAP.S999.cs'.
[main] [1:46:48:680] [RuntimeExec.runCommand:150]
[main] [1:46:48:680] [RuntimeExec.runCommand:155] RunTimeExec: error>
[main] [1:46:48:680] [RuntimeExec.runCommand:175] Returning from RunTimeExec.runCommand
[main] [1:46:48:681] [HAOperationImpl.runCommand:1264] Unsigned return code = 210
[main] [1:46:48:681] [HAOperationImpl.runCommand:1278] set status HA_RES_NOT_EXIST_ERR
[main] [1:46:48:681] [HASContext.getInstance:197] Module init : 6
[main] [1:46:48:681] [HASContext.getInstance:221] Local Module init : 0
[main] [1:46:48:681] [HASContext.getInstance:256] HAS Context Allocated: 9 to oracle.ops.mgmt.has.Util@998b08
[main] [1:46:48:681] [Util.
[main] [1:46:48:682] [has.UtilNative.Native] prsr_trace: Native: hasHAPrivilege
[main] [1:46:48:682] [HAOperationImpl.writeProfileOnDisk:1175] Writing profile on disk at /u01/app/oracle/product/10.2.0/crs/crs/public/ora.DBFAP.S999.cs.cap
[main] [1:46:48:683] [HAOperationImpl.runCommand:1246] CRS cmd is: /u01/app/oracle/product/10.2.0/crs/bin/crs_profile -update ora.DBFAP.S999.cs -h lxdb0109c01 lxdb0109c02 lxdb0109c03
[main] [1:46:48:683] [RuntimeExec.runCommand:74] Calling Runtime.exec() with the command
[main] [1:46:48:683] [RuntimeExec.runCommand:76] /u01/app/oracle/product/10.2.0/crs/bin/crs_profile
[main] [1:46:48:683] [RuntimeExec.runCommand:76] -update
[main] [1:46:48:683] [RuntimeExec.runCommand:76] ora.DBFAP.S999.cs
[main] [1:46:48:683] [RuntimeExec.runCommand:76] -h
[main] [1:46:48:684] [RuntimeExec.runCommand:76] lxdb0109c01 lxdb0109c02 lxdb0109c03
[main] [1:46:48:687] [RuntimeExec.runCommand:131] runCommand: Waiting for the process
[Thread-4] [1:46:48:687] [StreamReader.run:61] In StreamReader.run
[Thread-5] [1:46:48:687] [StreamReader.run:61] In StreamReader.run
[Thread-4] [1:46:48:722] [StreamReader.run:65] OUTPUT>Syntax error in /u01/app/oracle/product/10.2.0/crs/crs/public/ora.DBFAP.S999.cs.cap, line number 1: "CRS-0210: Could not find resource 'ora.DBFAP.S999.cs'."
[Thread-4] [1:46:48:722] [StreamReader.run:65] OUTPUT>(File: caa_profile.cpp, line: 155
[Thread-4] [1:46:48:722] [StreamReader.run:65] OUTPUT>
[main] [1:46:48:723] [RuntimeExec.runCommand:133] runCommand: process returns 1
[main] [1:46:48:724] [RuntimeExec.runCommand:147] RunTimeExec: output>
[main] [1:46:48:724] [RuntimeExec.runCommand:150] Syntax error in /u01/app/oracle/product/10.2.0/crs/crs/public/ora.DBFAP.S999.cs.cap, line number 1: "CRS-0210: Could not find resource 'ora.DBFAP.S999.cs'."
[main] [1:46:48:724] [RuntimeExec.runCommand:150] (File: caa_profile.cpp, line: 155
[main] [1:46:48:724] [RuntimeExec.runCommand:150]
[main] [1:46:48:724] [RuntimeExec.runCommand:155] RunTimeExec: error>
[main] [1:46:48:724] [RuntimeExec.runCommand:175] Returning from RunTimeExec.runCommand
[main] [1:46:48:724] [HAOperationImpl.runCommand:1290] set status FAILED
[main] [1:46:48:724] [HAProfileOperation.run:178] Returned from executing the HA Operation
[main] [1:46:48:725] [HAProfileOperation.run:183] OUTPUT> Syntax error in /u01/app/oracle/product/10.2.0/crs/crs/public/ora.DBFAP.S999.cs.cap, line number 1: "CRS-0210: Could not find resource 'ora.DBFAP.S999.cs'."
[main] [1:46:48:725] [HAProfileOperation.run:183] OUTPUT> (File: caa_profile.cpp, line: 155
[main] [1:46:48:725] [HAProfileOperation.run:183] OUTPUT>
[main] [1:46:48:725] [HAProfileOperation.run:189] ERROR>
[main] [1:46:48:725] [LocalCommand.execute:56] LocalCommand.execute: Returned from run method
[main] [1:46:48:725] [Service.genSCProfile:920] HA Profile operation failed
[main] [1:46:48:725] [HAOperationResult.getOutputAll:114] outLine is []
[main] [1:46:48:726] [HAOperationResult.getOutputAll:115] errLine is []
[main] [1:46:48:726] [Action.releaseLock:345] Release CSS lock SRVM.DBFAP
[main] [1:46:48:726] [has.ClusterLockNative.Native] prsr_trace: Native: unlock
PRKP-1029 : Failed to register the service S999.
oracle@lxdb0109c03:/u01/app/oracle/product/10.2.0/crs/bin>
oracle@lxdb0109c03:/u01/app/oracle/product/10.2.0/crs/bin> exit
exit
************************
=== ODM Action Plan ===
1.- The command is failing because it can not find the file:
/u01/app/oracle/product/10.2.0/crs/crs/public/ora.DBFAP.S999.cs.cap
This file is created when you run the "srvctl add service" command on the node where the command is ran.
Please check if this file exist in node 1 and 2, Execute the command from the node where the file exist.
************************************
On Node 1 and 3 file exists and has the following contents. ... Node 2 doesn't have the file .
oracle@lxdb0109c03:/u01/app/oracle/product/10.2.0/crs/bin> cat /u01/app/oracle
/product/10.2.0/crs/crs/public/ora.DBFAP.S999.cs.cap
CRS-0210: Could not find resource 'ora.DBFAP.S999.cs'.
*******************************************
=== ODM Action Plan ===
1.- I talk to the customer and explained that the command is failing because it can not allocate the file /u01/app/oracle/product/10.2.0/crs/crs/public/ora.DBFAP.S999.cs.cap
This is the profile for the services and it is created on the node where the "srvctl add service" command is ran. We checked cat /u01/app/oracle/product/10.2.0/crs/crs/profile on each node but the file does not exist either.
The commun fix a this issue is:
- Stop the services
- Drop the service
- Create the service
Customer can not have downtime on this maintanence window, he is going to request for the next windows. He asked us to investigate if there is another way to recreate the file
*********************************
Generic Note
------------------------
NOTE: Please keep in mind that stopping the services will not cause any outage, it will not allow any new connection with this service name and existing connection will work fine.
****************************
Generic Note
------------------------
Hi!
I will recommend to backup the OCR and then stop/remove and services as it is explained on this note.
Note 259301.1: CRS and 10g/11.1 Real Application Clusters
This should be done from node 1, keep in mind that you might need to user the "-f" option to remove the services.
I dont know of any pre-check to create the services.
**************************************
Generic Note
------------------------
Hi!
I had reviewed all the bugs related to this issue and I could not find any workaround beside the one that we had already exposed.
I believe given above steps will work out in you environment.
Thanks for your time and patience.
No comments:
Post a Comment