Null in trigger source issue for expdp

Ran into a nasty little bug while doing a full database import using datapump.

ORA-39125: Worker unexpected fatal error in KUPW$WORKER.PUT_DDLS while calling DBMS_METADATA.CONVERT [TRIGGER:"SCHEMANAME"."SOME_TRIGGER_NAME"]
ORA-06502: PL/SQL: numeric or value error
LPX-00007: unexpected end-of-file encountered
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 105
ORA-06512: at "SYS.KUPW$WORKER", line 6313

Found a number of Metalink DocID's that fit the bill, but the best suspect is 3897122.8. The issue is actually a bug in expdp where source with null lines causes expdp to basically create a corrupted export file. Fantastic.

Here's a query we used to identify the bad source code to be fixed:

select * from dba_source where text like '%'||CHR(0)||'%';

Found two triggers with null lines, probably put there by some kind of SQL dev tool. Rebuilt the triggers without null lines, and can now create an export file that can actually be imported.

Start screen after sudo su to another user

Sudo'ing to a user then running screen doesn't work out of the box.  Typically you get the following error:

Cannot open your terminal '/dev/pts/1' - please check.

Recently I found an answer to this problem here:  http://www.hjackson.org/blog/archives/2008/11/29/cannot-open-your-terminal-dev-pts-please-check

From the blog, here's the procedure

sudo su - someuser

script /dev/null

screen

Should work now.  

You never know how well your shoes fit...

... until you try someone elses out.

At some point I'll make a tabular layout of the most common unix commands and their windows counterparts... but for now lets just put a recent lil tool:

Unix:

ps -ef | grep "something"

Windows:

tasklist | findstr "something"

Good stuff!

I hear that taskkill is the kill -9 equivilent, but I need to test that to confirm it...

HangRMAN D_bug

Resolved an issue this afternoon, and in the process learned some things about debugging rman.

Issue description:  RMAN would hang on exit.  It was really slow on this server, but it would seem to hang on exit.  I say seem to, because I did leave one exit hanging while I went to lunch.  An hour or two after I got back from lunch, I noticed that the exit completed successfully.

Interesting, no?

Here's what things looked like:

$ rman target /

Recovery Manager: Release 10.2.0.3.0 - Production on Mon Mar 2 18:35:32 2009

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

connected to target database: X (DBID=1)

RMAN> exit

Recovery Manager complete.

At that point RMAN would hang for 3 hours.  My initial thought and troubleshooting involved truss and lsof and whatnot, attempting to pin the problem on our avamar backup system -- nope.

Using the most excellent DocID: 412950.1 titled "How to determine if an RMAN failure or hang lies in the Media Manager Layer or not"

I found the following very easy way to trace runs of RMAN:

rman target / trace tracefilename.trc debug

You can also use catalog= in the example.  Anyhow, at the end of this trace file I found the following query which appeared to hang:

select round(sum(MBYTES_PROCESSED)) ,round(sum(INPUT_BYTES)) ,round(sum(OUTPUT_BYTES))  from V$RMAN_STATUS  start with (RECID=:b1 and STAMP=:b1)  connect by prior RECID=parent_recid

The trace was nice enough to also provide the bind vars.  Now the fun part, figuring out what to do.

Luckily, it was all spelled out in DocID 375386.1  "Rman Backup is Very Slow selecting from V$RMAN_STATUS".

The issue was resolved by refreshing fixed object stats.  Huzzah.    

One more thing, section 22 of 11g Database Backup and Recovery User's Guide titled "Troubleshooting RMAN Operations" was valuable as well.  

SSH Encrypted TOAD

Thanks to a post over at OraFAQ, I was able to encrypt a TOAD connection using SecureCRT.

Lest I forget:

Options >> Session Options

Port Forwarding >> Add

Name: something

Port: 1521

Port: 1521

Open up toad.  Use localhost for hostname.   

Goodbye vncserver, hello xming

For the last 8 or so years, the my team has been using a combination of vncserver on a utility box and vncclient on our desktops to draw xwindow objects from various hosts.

Typically you would see this while using the Oracle Universal Installer.

The following steps describe how to use Xming, a far simpler (and some would argue safer) method to perform the same function as vncserver:

1)  Download Xming: http://sourceforge.net/projects/xming

2)  Install Xming, no special installation instructions.  (Screenshots of Xming are available:  http://www.straightrunning.com/XmingNotes/)

3)  Make sure Xming is running (You should see a swirly X in the system tray)

4)  Enable SecureCRT X11 Forwarding

    - Connect a session

    - Options -> Session Options

    - Select Remote/X11

    - Put a checkmark in "Forward X11 Packets"

    - OK

    - Logout and log back in for that session

5)  Try "xclock" from the command line, if it doesn't work...

    - Check DISPLAY environment variable.  It should be set to something like localhost:10.0

    - Is the X in the system tray?

    - Did you setup X11 forwarding?  Did you logout and log back in?

Cool stuff.  The three tier process (server -> utility host -> desktop) was difficult to explain much less troubleshoot.  

Fun with clntsh

Installing iAS 10.1.2.2 onto an Oracle oms grid control box.

Got the following error during linking, looks to be a Solaris 10 + 10R2 issue.

ld: fatal: file /u01/app/oracle/product/oms10g//lib32//libclntsh.so: unknown file type
ld: fatal: File processing errors. No output written to webcached
*** Error code 1
make: Fatal error: Command failed for target `webcached'

Here's the fix:

Edit $ORACLE_HOME/genclntsh
Change LD="ld -m -i -G -z text -L${ORACLE_HOME}/${LIB}"
to LD="ld -m -i -G -z text -L${ORACLE_HOME}/${LIB} -Bsymbolic"

Then run $ORACLE_HOME/bin/genclntsh

Then retry the linking phase.