Installing Oracle 126.96.36.199 RAC on Oracle Linux 7-part 1
Posted by Martin Bach on January 5, 2015
Now that 188.8.131.52 is certified on RedHat Linux 7 and spin-off environments it’s time to test the installation of RAC on such a system.
The installation of the OS is different from Oracle Linux 5 and 6-with these distributions was very straight forward how to install the operating system the method has changed significantly in release 7. I won’t cover the complete installation here, as always Tim Hall was quicker than me, but it makes me wonder who signed off the user interface for the partitioning “wizard”… I personally think that the kickstart partitioning-information is a lot easier to understand.
After walking through the steps in the graphical installer I thought that I’d rather kickstart the whole lot. If you have not yet used kickstart to create Oracle/Red Hat/CentOS hosts then now is the time to start admiring the technology!
My host is Oracle Linux 6.6 and there aren’t any packages that aren’t part of the distribution, with one exception: I have compiled dnsmasq from the current source since it had a bug when specifying the –bind-interfaces command line option if other dnsmasq processes were already running. libvirt uses dnsmasq extensively for its own networking (I since then upgraded the host to Oracle Linux 7 too and the problem is no longer present)
I don’t like repetitive tasks
Anything that is repetitive is not the best use of anyone’s time, especially not mine. I prefer to run a single command, do something different and come back with the OS already installed. For the installation to work I am relying on PXE booting to pass the kernel and initial RAM disk. TFTP is in charge of that. A Kickstart file is used for the silent installation. Let’s start off with the Pre Execution Boot Environment (I am putting a shameless plug in here for my 12c book which covers this subject extensively if you need more background). Here is my default configuration file in pxelinux.cfg:
default menu.c32 prompt 0 menu title PXE Boot Menu label oracle-linux 6.6 x86-64 menu label oracle-linux 6.6 x86-64 kernel /tftpboot/webroot/images/ol/6/6/vmlinuz append initrd=/tftpboot/webroot/images/ol/6/6/initrd.img ks=http://192.168.150.1/ol66.ks ksdevice=eth0 label oracle-linux 7.0 x86-64 menu label oracle-linux 7.0 x86-64 kernel /tftpboot/webroot/images/ol/7/0/vmlinuz append initrd=/tftpboot/webroot/images/ol/7/0/initrd.img inst.ks=http://192.168.150.1/ol70.ks inst.repo=http://192.168.150.1/ol70
The next step is to ensure that you have the images you are pointing to.
[root@ol62 0]# cp -iv /media/ol70/images/pxeboot/* . `/media/ol70/images/pxeboot/initrd.img' -> `./initrd.img' `/media/ol70/images/pxeboot/TRANS.TBL' -> `./TRANS.TBL' `/media/ol70/images/pxeboot/upgrade.img' -> `./upgrade.img' `/media/ol70/images/pxeboot/vmlinuz' -> `./vmlinuz' [root@ol62 0]# pwd /tftpboot/webroot/images/ol/7/0
The Pre Execution Environment (PXE) will supply the KS (kickstart) directive to the boot process. The Initial RAMdisk and kernel will be pulled via tftp-don’t forget to enable xinetd and tftp if you are not using dnsmasq like me or this won’t work. You also need a DHCP server that provides the NIC of the VM with an IP and the reference to the PXE boot configuration. If you are using a lab server then dnsmasq can do this for you. I believe that admins in real life might not do it that way… The official Red Hat documentation has a nice section on how to configure your environment for pxeboot with the ISC DHCP package, too. Here is the pxe.conf file I used successfully (with that newly compiled dnsnasq-again the stock Oracle 6.x version can’t handle the –bind-interface at the time of writing.
[root@ol62 ~]# cat /etc/dnsmasq.d/pxe.conf tftp-root=/tftpboot/webroot enable-tftp dhcp-boot=pxelinux.0 dhcp-option=vendor:PXEClient,6,2b dhcp-no-override pxe-prompt="Press F8 for boot menu", 3 pxe-service=X86PC, "Boot from network", pxelinux pxe-service=X86PC, "Boot from local hard disk", 0 #log-queries #log-dhcp domain=example.com dhcp-range=192.168.150.50,192.168.150.150,12h
This file then went into /etc/dnsmasq.d/ as you can see, and I restarted the daemon to make it read the file. Initially I found the log-queries and log-dhcp output quite useful to work out what was happening, you can see these messages in /var/log/messages. There are many … so you might want to turn that off as soon as you fixed any potential problems.
My kickstart file assumes that the installation tree is available via http from 192.168.150.1/ol70. It’s just a loopback mounted ISO image so easy to implement in my host.
My kickstart file is not really special, it simply sets the partitioning schema creating 2 volume groups: rootvg for system related things such as swap, boot and root plus oraclevg for the oracle binaries. I based it primarily on the anaconda-ks.cfg file in /root that has been created during the installation. The kickstart documentationis also rather nice. Since there is no preinstall RPM available right now I resorted to a %post script to add the Oracle-specific settings:
Since I wrote this post Oracle Linux 7.1 has been released which features the 11.2 and 12.1 RDBMS preinstall RPMs rendering some of the steps shown here obsolete.
%post --log=/root/ks-post.log cat >> /etc/sysctl.conf <<EOF fs.file-max = 6815744 kernel.sem = 250 32000 100 128 kernel.shmmni = 4096 kernel.shmall = 1073741824 kernel.shmmax = 4398046511104 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 fs.aio-max-nr = 1048576 net.ipv4.ip_local_port_range = 9000 65500 kernel.panic_on_oops = 1 EOF sysctl -f --system cat >> /etc/security/limits.conf <<EOF oracle soft nofile 1024 oracle hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft stack 10240 oracle hard stack 32768 oracle soft memlock 60397977 oracle hard memlock 60397977 grid soft nofile 1024 grid hard nofile 65536 grid soft nproc 2047 grid hard nproc 16384 grid soft stack 10240 grid hard stack 32768 grid soft memlock 60397977 grid hard memlock 60397977 EOF cat > /etc/udev/rules.d/61-asm.rules <<EOF KERNEL=="vd[cdefgh]1", OWNER="grid", GROUP="asmdba" MODE="0660" EOF gid=10000 for group in asmdba asmadmin kmdba dgdba backupdba dba oinstall; do groupadd -g $(( gid += 1 )) $group ; done useradd -u 10000 -g oinstall -G asmdba,kmdba,dgdba,backupdba,dba oracle useradd -u 10001 -g oinstall -G asmdba,asmadmin,dba grid mkdir -p /u01/app/oracle mkdir -p /u01/app/grid chown -R grid:oinstall /u01 chown -R oracle:oinstall /u01/app/oracle
This is of course just an example-you need to adjust this to your environment. I am assuming a separation of duties here as well which not everyone will (and should!) apply in his environment. There are also quite a few individual packages in the %package list right now that I’ll replace with the call to the preinstall RPM once it’s available. Oracle Linux 7 reflects the fact that many system daemons use a conf.d directory to separate customisations from the main configuration file. Oracle 184.108.40.206 does not check for this: when I tried moving my Oracle specific code into /etc/sysctl.d/50-oracle.conf OUI could not find and parse the file. It subsequently complained about an unknown configuration. So therefore we are back to the old days.
Now all I need to do is to call virt-install on the command line and wait for the KVM environment to be created.