Installing Oracle 12.1.0.2 RAC on Oracle Linux 7-part 1

Now that 12.1.0.2 is certified on RedHat Linux 7 and spin-off environments it’s time to test the installation of RAC on such a system.

The installation of the OS is different from Oracle Linux 5 and 6-with these distributions was very straight forward how to install the operating system the method has changed significantly in release 7. I won’t cover the complete installation here, as always Tim Hall was quicker than me, but it makes me wonder who signed off the user interface for the partitioning “wizard”… I personally think that the kickstart partitioning-information is a lot easier to understand.

http://oracle-base.com/articles/linux/oracle-linux-7-installation.php

After walking through the steps in the graphical installer I thought that I’d rather kickstart the whole lot. If you have not yet used kickstart to create Oracle/Red Hat/CentOS hosts then now is the time to start admiring the technology!

My host is Oracle Linux 6.6 and there aren’t any packages that aren’t part of the distribution, with one exception: I have compiled dnsmasq from the current source since it had a bug when specifying the –bind-interfaces command line option if other dnsmasq processes were already running. libvirt uses dnsmasq extensively for its own networking (I since then upgraded the host to Oracle Linux 7 too and the problem is no longer present)

I don’t like repetitive tasks

Anything that is repetitive is not the best use of anyone’s time, especially not mine. I prefer to run a single command, do something different and come back with the OS already installed. For the installation to work I am relying on PXE booting to pass the kernel and initial RAM disk. TFTP is in charge of that. A Kickstart file is used for the silent installation. Let’s start off with the Pre Execution Boot Environment (I am putting a shameless plug in here for my 12c book which covers this subject extensively if you need more background). Here is my default configuration file in pxelinux.cfg:

default menu.c32
prompt 0

menu title PXE Boot Menu

label oracle-linux 6.6 x86-64
    menu label oracle-linux 6.6 x86-64
    kernel /tftpboot/webroot/images/ol/6/6/vmlinuz
    append initrd=/tftpboot/webroot/images/ol/6/6/initrd.img ks=http://192.168.150.1/ol66.ks ksdevice=eth0

label oracle-linux 7.0 x86-64
    menu label oracle-linux 7.0 x86-64
    kernel /tftpboot/webroot/images/ol/7/0/vmlinuz
    append initrd=/tftpboot/webroot/images/ol/7/0/initrd.img inst.ks=http://192.168.150.1/ol70.ks inst.repo=http://192.168.150.1/ol70

The next step is to ensure that you have the images you are pointing to.

[root@ol62 0]# cp -iv /media/ol70/images/pxeboot/* .
`/media/ol70/images/pxeboot/initrd.img' -> `./initrd.img'
`/media/ol70/images/pxeboot/TRANS.TBL' -> `./TRANS.TBL'
`/media/ol70/images/pxeboot/upgrade.img' -> `./upgrade.img'
`/media/ol70/images/pxeboot/vmlinuz' -> `./vmlinuz'
[root@ol62 0]# pwd
/tftpboot/webroot/images/ol/7/0

The Pre Execution Environment (PXE) will supply the KS (kickstart) directive to the boot process. The Initial RAMdisk and kernel will be pulled via tftp-don’t forget to enable xinetd and tftp if you are not using dnsmasq like me or this won’t work. You also need a DHCP server that provides the NIC of the VM with an IP and the reference to the PXE boot configuration. If you are using a lab server then dnsmasq can do this for you. I believe that admins in real life might not do it that way… The official Red Hat documentation has a nice section on how to configure your environment for pxeboot with the ISC DHCP package, too. Here is the pxe.conf file I used successfully (with that newly compiled dnsnasq-again the stock Oracle 6.x version can’t handle the –bind-interface at the time of writing.

[root@ol62 ~]# cat /etc/dnsmasq.d/pxe.conf
tftp-root=/tftpboot/webroot
enable-tftp

dhcp-boot=pxelinux.0
dhcp-option=vendor:PXEClient,6,2b
dhcp-no-override

pxe-prompt="Press F8 for boot menu", 3
pxe-service=X86PC, "Boot from network", pxelinux
pxe-service=X86PC, "Boot from local hard disk", 0

#log-queries
#log-dhcp

domain=example.com
dhcp-range=192.168.150.50,192.168.150.150,12h

This file then went into /etc/dnsmasq.d/ as you can see, and I restarted the daemon to make it read the file. Initially I found the log-queries and log-dhcp output quite useful to work out what was happening, you can see these messages in /var/log/messages. There are many … so you might want to turn that off as soon as you fixed any potential problems.

Kickstarting

My kickstart file assumes that the installation tree is available via http from 192.168.150.1/ol70. It’s just a loopback mounted ISO image so easy to implement in my host.

My kickstart file is not really special, it simply sets the partitioning schema creating 2 volume groups: rootvg for system related things such as swap, boot and root plus oraclevg for the oracle binaries. I based it primarily on the anaconda-ks.cfg file in /root that has been created during the installation. The kickstart documentationis also rather nice. Since there is no preinstall RPM available right now I resorted to a %post script to add the Oracle-specific settings:

UPDATE

Since I wrote this post Oracle Linux 7.1 has been released which features the 11.2 and 12.1 RDBMS preinstall RPMs rendering some of the steps shown here obsolete.

%post --log=/root/ks-post.log

cat >> /etc/sysctl.conf <<EOF
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
kernel.panic_on_oops = 1
EOF

sysctl -f --system

cat >> /etc/security/limits.conf <<EOF
oracle   soft   nofile    1024
oracle   hard   nofile    65536
oracle   soft   nproc     2047
oracle   hard   nproc     16384
oracle   soft   stack     10240
oracle   hard   stack     32768
oracle   soft   memlock   60397977
oracle   hard   memlock   60397977

grid   soft   nofile    1024
grid   hard   nofile    65536
grid   soft   nproc     2047
grid   hard   nproc     16384
grid   soft   stack     10240
grid   hard   stack     32768
grid   soft   memlock   60397977
grid   hard   memlock   60397977
EOF

cat > /etc/udev/rules.d/61-asm.rules <<EOF
KERNEL=="vd[cdefgh]1", OWNER="grid", GROUP="asmdba" MODE="0660"
EOF

gid=10000
for group in asmdba asmadmin kmdba dgdba backupdba dba oinstall; do
  groupadd -g $(( gid += 1 )) $group ;
done

useradd -u 10000 -g oinstall -G asmdba,kmdba,dgdba,backupdba,dba oracle
useradd -u 10001 -g oinstall -G asmdba,asmadmin,dba grid

mkdir -p /u01/app/oracle
mkdir -p /u01/app/grid

chown -R grid:oinstall /u01
chown -R oracle:oinstall /u01/app/oracle

This is of course just an example-you need to adjust this to your environment. I am assuming a separation of duties here as well which not everyone will (and should!) apply in his environment. There are also quite a few individual packages in the %package list right now that I’ll replace with the call to the preinstall RPM once it’s available. Oracle Linux 7 reflects the fact that many system daemons use a conf.d directory to separate customisations from the main configuration file. Oracle 12.1.0.2 does not check for this: when I tried moving my Oracle specific code into /etc/sysctl.d/50-oracle.conf OUI could not find and parse the file. It subsequently complained about an unknown configuration. So therefore we are back to the old days.

Now all I need to do is to call virt-install on the command line and wait for the KVM environment to be created.

References

• oracle-base.com – enough said. Had the OS install first, and the installation of the RDBMS.
• Kickstart reference for Red Hat Linux 7
• Boot options can be found here – they have changed a bit in release 7!
• I have documented the whole process in more detail in Expert Consolidation in Oracle 12c