Grid control 11g... and unable to patch using deployment procedures

Recently upgraded grid control from 10.2.0.5 to 11g, and now unable to apply patches using the standard deployment procedure.

The job bombs out at the initialize step.

Here's the error

Downloading the patches from My Oracle Support.
Error while evaluating expression:[Error evaluating expression: ${doInitialization(step.guid)} : [no oraInstaller in java.library.path]]

Found note 427577.1, installed patches 9495798, 9705138, and 10388644. Still not working.

Couldn't find any additional info, so opened a ticket with Oracle support...

Resolution log:
1) Reran OPatch Update job and Refresh from Metalink, restarted job, still broke
2) Deleted the job, restarted from scratch again and noticed a warning during the "Search and select software updates" screen -- the component already exists in the software library.
So I chose not to overwrite the version in the software library, and instead of failing at the initialize stage, now the job fails a few steps down at the "Stage Patches" step.

Here's that error:

Received null input stream of binary file, null', for the item identified by, 'Components/Oracle Software Updates/p10248523_11.2.0.2.0_46(Revision: 0.1)'. See log file '$ORACLE_HOME/sysman/log/emoms.log on Oracle Management Service host for more details.

At this point my guess is there's something wrong with the software library not getting upgraded correctly from 10g to 11g. We'll have to see what Oracle support recommends...

--- Update 2/8
Oracle support eventually concurred it's likely a problem with the software library. "Trying fix described in Oracle Internal Support Portal: Document 602886.1", which looks to be a pretty straightforward method to rebuild the software library.

Making a cold backup now, will post more info later.
--- Update 2/17

Rebuilding the software library was successful, but did not resolve our issues with patching. Oracle support suggests that either we used the wrong JDK version (we used 1.6.0_24 rather than the required 1.6.0_18), or I did not use the required 64 bit flag when installing the weblogic server.

So -- we're reinstalling grid control in a new location on the same server to try and see if that resolves our patching issues, and if it does migrate all our targets to it.

--- Update 2/17
Found someone else who experienced a similar error to the one above who also had to reinstall.

Seems this is the correct way to start the weblogic installer for grid (where JAVA_HOME is set to your 1.6.0_18 JDK):

$JAVA_HOME/bin/java -d64 -jar wls1032_generic.jar

As for install guides, I think I would recommend this one from CamOra ICT.

x forwarding and sudo for oracle installs

From my linux desktop I ssh to my oracle box and forward X packets back to my desktop over ssh.

bill@wgwsvm:~$ ssh -X 192.168.56.101

When I invoke xclock this works like you would expect.

If I wanted to forward xpackets from other users that I might ssh to I have to do two additional steps. Obviously set the DISPLAY and additionally set the x auth token to forward through the connection.

bill@wgdbvm:/home/bill> xauth list
wgdbvm/unix:10  MIT-MAGIC-COOKIE-1  4696e4a4a52fac8af8b85e42195baa84
bill@wgdbvm:/home/bill> echo $DISPLAY
localhost:10.0
bill@wgdbvm:/home/bill> sudo su - oracle
[sudo] password for bill: 
oracle@wgdbvm:/home/oracle> xauth add wgdbvm/unix:10  MIT-MAGIC-COOKIE-1  4696e4a4a52fac8af8b85e42195baa84
oracle@wgdbvm:/home/oracle> export DISPLAY=localhost:10.0

One note, when disconnecting from my bill account the xauth file is cleaned up automatically for me. But oracle's xauth will still contain the token, so it needs to be deleted manually.

oracle@wgdbvm:/home/oracle> xauth list
wgdbvm/unix:10  MIT-MAGIC-COOKIE-1  4696e4a4a52fac8af8b85e42195baa84
oracle@wgdbvm:/home/oracle> xauth remove wgdbvm/unix:10

Check for view recompilation

Use the following to identify which DB's need view recompilation by using the following PL/SQL as the parameter for a SQL Script job:

WHENEVER SQLERROR EXIT FAILURE;
declare 
 xvar number;
begin
SELECT count(*) into xvar FROM sys.registry$history where ID = '6452863';
if xvar=0 then
  dbms_output.put_line('Needs View Recompliation');
  raise_application_error(-20101,'Needs View Recompilation');
else
  dbms_output.put_line('View Recompliation Already Done');
end if;
end;
/

Run that against all targets, any that fail likely need the view recompilation job run.

Error after 11g upgrade

Saw the following error in utlrp after upgrading a db to 11gR2:

Warning: XDB now invalid, could not find xdbconfig

After a bit of digging I found metalink doc 944088.1.

Check your LD_LIBRARY_PATH for DB and Listener. If not set correctly you'll need to restart both DB and Listener, and then try utlrp again.

Now with syntax highlighting goodness

Via this excellent post, we now have syntax highlighting.

code goes here

Much better looking than quotes or pre alone.

RMAN duplicate set until clause for variable dates

I've used the following for my rman duplicate scripts to set a specific timestamp for clones.

set until time "to_date('03/29/2010 08:00:00','MM/DD/YYYY HH24:MI:SS')";

However, most of the time I just want to have my data as of an hour ago no matter what time I start the clone and don't want to edit the timestamp each time (less stuff to mess up). Apparently this syntax also works.

set until time "sysdate - 1/24";

You can take this a step further in complication with anything for dates, such as cloning to the most recent archivelog backup (assuming a scheduled archive log backup takes place every hour at 35 after)

set until time "trunc((sysdate - 30/1440),'HH24')+30/1440";

Comparing files with slight differences

File compile-test.log is an output of a big compile from a database named TEST, and file is stored in TEST subdirectory.

File compile-prod.log is output of a big compile from a database named PROD, file is stored in PROD subdirectory.

diff PROD/compile-prod.log <(sed 's/TEST/PROD/g' ../TEST/compile-test.log