Multipath with HP LeftHand on RHEL

HP has acquired LeftHand networks and re-branded the LeftHand storage product to the HP P4000 storage series. This is a iSCSI SAN product targeted at entry level businesses. We are using this product for various projects, where our Oracle RAC cluster is one of them. For this cluster where are setting up MPIO. Because we are using interface bonding at the moment the LUN’s are configured already and don’t have to be created.

Quick environment specs:

  • RedHat EL5.8
  • 2 NIC’s in the storage network (eth3 and eth4)

[Read more...]

Oracle ASM lifesaver

I had a very nasty experience with Oracle ASM recently. A long story short, ASM crashed on both our RAC nodes when removing a corrupted archive file on file and incarnation number (removing it on it’s name didn’t work). Starting the instance failed. When removing the failing diskgroup ASM was happy again. Luckily the failing diskgroup only housed archivelog files, except for one database it also held it’s index datafile.  So that one wouldn’t come up. If you’re in such a situation, you can use the Oracle tools amdu in combination with kfed to extract the missing datafile or archivelogs (in our situation) from a dismounted diskgroup. If I knew that from the start it saved me alot of frustrating hours.

Oracle 11g ASM disk discovery

During the installation of Oracle 11gR2 on Red Hat Enterprise Linux 5 I stumbled upon a incorrect default setting for ASM disk discovery when using ASM lib. The discovery path is set to ‘/dev/raw/sd*’, which is incorrect because ASM lib creates the devices under ‘/dev/oracleasm/disks’.

Another thing to be aware of is the ownership of the disks. You need to run ‘/etc/init.d/oracleasm configure’ in order to configure the owner. If you just hit return the ownership of the disks is set to root. Because you will start the Oracle installation as user oracle the disks will not appear in the discovery step during the installation.

[Read more...]