I was actually hoping I had already written enough about applying the PSU for 18.104.22.168.0 but today I tried to apply the patch to my Oracle Restart home in my lab VM and guess what-it wasn’t as automatic as I thought.
I must have performed the classic copy & paste error and applied the patch to the GRID_HOME only. Please don’t do this! The README clearly states you can but if you do, you will run into this mess. At the time of writing there was no hit for the error on my favourite search engine, hence this post. If you see this on a real database and not on your personal lab VM you should of course consult support what to do. This is for educational purposes only, so to say.
I initiated the patch after upgrading OPatch on the RDBMS and GRID homes. The GI PSU (17272829) has been unzipped as the grid owner (“grid”) in /u01/app/grid/stage. I always use separation of duties in my play-VMs to test the edge cases. The path included the OPatch directory from the GRID_HOME. The command I executed was:
[root@server1 ~]$ opatchauto apply /u01/app/grid/stage/17272829 \ > -oh /u01/app/grid/product/22.214.171.124/grid -ocmrf /tmp/ocm.rsp
To my surprise this completed fine. After I realised my mistake-check the -oh flag-I wanted to patch the only RDBMS home too-should be simple. I unzipped the patch as oracle in /u01/app/oracle/stage and pointed all environment variables to the RDBMS home before I executed the patch command:
[root@server1 stage]# opatchauto apply /u01/app/oracle/stage/17272829 \ > -oh /u01/app/oracle/product/126.96.36.199/dbhome_1 -ocmrf /tmp/ocm.rsp OPatch Automation Tool Copyright (c) 2013, Oracle Corporation. All rights reserved. OPatchauto version : 188.8.131.52.2 OUI version : 184.108.40.206.0 Running from : /u01/app/oracle/product/220.127.116.11/dbhome_1 opatchauto log file: /u01/app/grid/product/18.104.22.168/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-11-18_04-01-04_deploy.log Parameter Validation: Successful System Configuration Collection failed: oracle.osysmodel.driver.crs.productdriver.ProductDriverException: Unknown host name in config:server1.example.com opatchauto failed with error code 2.
Looks like a DNS error. The log file showed this:
$ cat /u01/app/grid/product/22.214.171.124/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-11-18_03-43-14_deploy.log ===============START of log file================ Initializing command-line Information: Grid Home : /u01/app/grid/product/126.96.36.199/grid Patch Location : /u01/app/oracle/stage/17272829 OCM Response File location : /tmp/ocm.rsp Nodename : server1 Analyze mode : false No-restart : false Patching Home(s) selected : /u01/app/oracle/product/188.8.131.52/dbhome_1, Database Name specified : false Initialization Successful. Parameter Validation: Successful 2013-11-18_03-43-15 : Collect Patch Data: In Progress... 2013-11-18_03-43-16 : Collect Patch Data: Completed. 2013-11-18_03-43-16 : Collect Target Data: In Progress... 2013-11-18_03-43-16 : GIPatchingHelper::createSystemInstance() called for selected homes 2013-11-18_03-43-19 : GIPatchingHelper::createSystemInstance() Failed oracle.osysmodel.driver.crs.AbstractCrsProductDriver.findHostByName(AbstractCrsProductDriver.java:145) oracle.osysmodel.driver.crs.AbstractCrsProductDriver.findOrMakeHome(AbstractCrsProductDriver.java:415) oracle.osysmodel.driver.crs.AbstractCrsProductDriver.makeDBHomes(AbstractCrsProductDriver.java:507) oracle.osysmodel.driver.crs.CrsProductDriver.buildConfigGraph(CrsProductDriver.java:295) oracle.osysmodel.driver.crs.CrsProductDriver.buildConfigGraph(CrsProductDriver.java:143) oracle.opatch.gi.GIPatchingHelper.createSystemInstance(GIPatchingHelper.java:251) oracle.opatch.gi.GIPatchingSessionInfo.createSystemInstance(GIPatchingSessionInfo.java:310) oracle.opatch.gi.GIPatchingSessionInfo.initializeConfiguration(GIPatchingSessionInfo.java:269) oracle.opatch.gi.GIPatchingSessionInfo.initializeConfigData(GIPatchingSessionInfo.java:288) oracle.opatch.gi.GIPatching.process(GIPatching.java:584) oracle.opatch.GIPatchingSession.process(GIPatchingSession.java:89) oracle.opatch.OPatchSession.main(OPatchSession.java:2871) oracle.opatch.OPatch.main(OPatch.java:690) System Configuration Collection failed: oracle.osysmodel.driver.crs.productdriver.ProductDriverException: Unknown host name in config:server1.example.com
I double checked DNS, /etc/resolv.conf, /etc/nsswitch.conf and many other but couldn’t get to the bottom of the problem in the time I allowed myself for troubleshooting. MOS wasn’t too helpful either. The note explaining how to manually apply the GI patch was for RAC, not Oracle Restart.
Until I decided to repeat the patch from the GRID_HOME just the way I performed the first patch-but without the -oh flag-and hurray, it worked:
[root@server1 stage]# opatchauto apply /u01/app/grid/stage/17272829 -ocmrf /tmp/ocm.rsp OPatch Automation Tool Copyright (c) 2013, Oracle Corporation. All rights reserved. OPatchauto version : 184.108.40.206.2 OUI version : 220.127.116.11.0 Running from : /u01/app/grid/product/18.104.22.168/grid opatchauto log file: /u01/app/grid/product/22.214.171.124/grid/cfgtoollogs/opatchauto/17272829/opatch_gi_2013-11-18_04-48-42_deploy.log Parameter Validation: Successful Grid Infrastructure home: /u01/app/grid/product/126.96.36.199/grid RAC home(s): /u01/app/oracle/product/188.8.131.52/dbhome_1 Configuration Validation: Successful Patch Location: /u01/app/grid/stage/17272829 Grid Infrastructure Patch(es): 17027533 17077442 17303297 RAC Patch(es): 17027533 17077442 Patch Validation: Successful Stopping RAC (/u01/app/oracle/product/184.108.40.206/dbhome_1) ... Successful Applying patch(es) to "/u01/app/oracle/product/220.127.116.11/dbhome_1" ... Patch "/u01/app/grid/stage/17272829/17027533" successfully applied to "/u01/app/oracle/product/18.104.22.168/dbhome_1". Patch "/u01/app/grid/stage/17272829/17077442" successfully applied to "/u01/app/oracle/product/22.214.171.124/dbhome_1". Applying patch(es) to "/u01/app/grid/product/126.96.36.199/grid" ... Patch "/u01/app/grid/stage/17272829/17027533" is already installed on "/u01/app/grid/product/188.8.131.52/grid". Please rollback the existing identical patch first. Patch "/u01/app/grid/stage/17272829/17077442" is already installed on "/u01/app/grid/product/184.108.40.206/grid". Please rollback the existing identical patch first. Patch "/u01/app/grid/stage/17272829/17303297" is already installed on "/u01/app/grid/product/220.127.116.11/grid". Please rollback the existing identical patch first. Starting RAC (/u01/app/oracle/product/18.104.22.168/dbhome_1) ... Successful [WARNING] The local database(s) on "/u01/app/oracle/product/22.214.171.124/dbhome_1" is not running. SQL changes, if any, cannot be applied. Apply Summary: Following patch(es) are successfully installed: RAC Home: /u01/app/oracle/product/126.96.36.199/dbhome_1: 17027533, 17077442 opatchauto ran into some warnings during patch installation (Please see log file for details): GI Home: /u01/app/grid/product/188.8.131.52/grid: 17027533, 17077442, 17303297 opatchauto succeeded.
It seemed to have worked, the patches in the GRID_HOME were identical and skipped. In a real live situation you should consult Oracle Support to get their blessing.
Moral of the story: don’t apply the patch to a single Oracle home only, let it do the whole patch! Also ensure you run datapatch to apply the SQL changes to any database from your RDBMS home.