Thinking about automation a lot, especially in the context of cloud computing, I have decided to create a small series of related posts that hopefully help someone deploying environments in a fully automated way. As my colleague @fritshoogland has said many times: the only way to deploy database servers (or any other server for that matter) in a consistent way, is to do it via software. No matter how hard you try to set up 10 identical systems manually, there will be some, maybe ever so subtle, differences between them. And with the cloud you probably have 42 systems or more to deal with! In this context, my first post could be a building block: the provisioning of the Oracle RDBMS the form of an RPM.
The idea
In a nutshell, I would like to
- Create a “golden image” of my database installation
- Make this available for deployment via NFS
- Create an RPM that allows me to deploy the “golden image” on the server I want to install and call clone.pl to complete the process
- Execute the root scripts (orainstRoot.sh, root.sh)
This is actually quite easy to do! The oracle-validated RPM, and nowadays oracle-rdbms-server-release-preinstall.x86_64 packages simplify the task greatly as they can pull in dependent packages, set sensible default parameters and generally perform a lot of the tedious setup work for you. All I have to do in my RPM is to enforce the version of the system is Oracle Linux 7 and pull the preinstall RPM in if not yet installed. Since there is no more 12c RDBMS server for i686 this entire post implicitly is about Linux x86-64.
The Implementation
I am using a development system (linuxdev in this example) to build the RPM. It’s an Oracle Linux 7.2 system with all upgrades up to 24/10/2016. The first thing to do is to install the RPM dev tools and their dependencies. As always, you use yum to do so.
[root@linuxdev ~]# yum info rpmdevtools
Loaded plugins: ulninfo
Available Packages
Name : rpmdevtools
Arch : noarch
Version : 8.3
Release : 5.0.1.el7
Size : 96 k
Repo : ol7_latest/x86_64
Summary : RPM Development Tools
URL : https://fedorahosted.org/rpmdevtools/
Licence : GPLv2+ and GPLv2
Description : This package contains scripts and (X)Emacs support files to aid in
: development of RPM packages.
: rpmdev-setuptree Create RPM build tree within user's home directory
: rpmdev-diff Diff contents of two archives
: rpmdev-newspec Creates new .spec from template
: rpmdev-rmdevelrpms Find (and optionally remove) "development" RPMs
: rpmdev-checksig Check package signatures using alternate RPM keyring
: rpminfo Print information about executables and libraries
: rpmdev-md5/sha* Display checksums of all files in an archive file
: rpmdev-vercmp RPM version comparison checker
: spectool Expand and download sources and patches in specfiles
: rpmdev-wipetree Erase all files within dirs created by rpmdev-setuptree
: rpmdev-extract Extract various archives, "tar xvf" style
: rpmdev-bumpspec Bump revision in specfile
: ...and many more.
[root@linuxdev ~]# yum install rpmdevtools
...
The development tools contain a lot of useful scripts, one of them I consider essential: rpmdev-setuptree. It sets up the relevant directory structure. Granted, the rpmdev tools are geared towards compiling software from their source, optionally apply patches and then create SRPMs and RPMs. RPM wasn’t designed for the purpose of packaging an Oracle database, but it can still be made to work. You can work around the size restriction using the %post hook in the SPEC file, as you will see later in this post. But I’m getting ahead of myself.
First I’m creating the RPM tree structure:
[oracle@linuxdev ~]$ rpmdev-setuptree [oracle@linuxdev ~]$ ls -l rpmbuild/ total 0 drwxr-xr-x 2 oracle oinstall 6 Oct 24 10:50 BUILD drwxr-xr-x 2 oracle oinstall 6 Oct 24 10:50 RPMS drwxr-xr-x 2 oracle oinstall 6 Oct 24 10:50 SOURCES drwxr-xr-x 2 oracle oinstall 6 Oct 24 10:50 SPECS drwxr-xr-x 2 oracle oinstall 6 Oct 24 10:50 SRPMS [oracle@linuxdev ~]$
Using another tool named rpmdev-newspec I create a new, empty SPEC file:
[oracle@linuxdev SPECS]$ rpmdev-newspec oracle_12102_jul16_psu_ojvm.spec oracle_12102_jul16_psu_ojvm.spec created; type minimal, rpm version >= 4.11. [oracle@linuxdev SPECS]$ ls -lrt total 4 -rw-r--r-- 1 oracle oinstall 338 Oct 24 10:53 oracle_12102_jul16_psu_ojvm.spec [oracle@linuxdev SPECS]$
Completing the SPEC file is easy if you know how to :) To save you the pain I put a stub together that is far from perfect, but should give you an idea. This is just a demo, the code especially lacks all (error, space, etc) checking which really should be in.
[oracle@linuxdev SPECS]$ cat oracle_12102_jul16_psu_ojvm.spec
Name: oracle_12102_jul16_psu_ojvm
Version: 0
Release: 1%{?dist}
Summary: a spec file for Oracle 12.1.0.2 with patch 23615289 (combo for 23054246 and 23177536)
Group: Application/Databases
License: Commercial
URL: https://martincarstenbach.wordpress.com/
Requires: oraclelinux-release >= 7:7.0, oracle-rdbms-server-12cR1-preinstall
%description
Oracle 12.1.0.2 EE patched with the July 2016 PSU and OJVM patch
Requires Oracle Linux 7, depends on the preinstall RPM to set up users, groups and defaults
ONLY A PROTOTYPE, USED FOR DEMONSTRATION PURPOSES ONLY
%files
%defattr(-,root,root,-)
%post
ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1
if [ -d "$ORACLE_HOME" ]; then
/usr/bin/echo "ORACLE_HOME directory $ORACLE_HOME exists-aborting";
exit 127
fi
/usr/bin/echo "cloning the oracle home now to $ORACLE_HOME"
/usr/bin/mkdir -p $ORACLE_HOME
/usr/bin/umount /mnt
/usr/bin/mount -t nfs linuxdev:/u01/nfs_export /mnt
/usr/bin/tar --gzip -xf /mnt/oracle/12.1.0.2/oracle_12102_jul16_psu_ojvm.tar.gz -C $ORACLE_HOME
/usr/bin/chown -R oracle:oinstall /u01
/usr/bin/su - oracle -c "cd $ORACLE_HOME/clone/bin && ./clone.pl ORACLE_HOME=$ORACLE_HOME ORACLE_BASE=/u01/app/oracle -defaultHomeName"
/u01/app/oraInventory/orainstRoot.sh
$ORACLE_HOME/root.sh
umount /mnt
%changelog
* Mon Oct 24 2016 Martin Bach
- initial version
Again, at the risk of repeating myself, this is just a stub and needs to be extended to include error handling and other things that we consider good coding practice. You also should check if there is sufficient space in the $ORACLE_HOME directory etc… you get the idea. Also, if you create another SPEC file for a 12c installation you currently can’t install a new Oracle Home as the path is hard coded to dbhome_1.
The script – which is the part in the %post section – is so simple it should be self-explanatory. It will be passed to /bin/sh and executed as part of the installation process.
With the SPEC file in place you can try and build the RPM. Funny enough, the rpmbuild tool is used for that:
[oracle@linuxdev SPECS]$ rpmbuild -ba oracle_12102_jul16_psu_ojvm.spec Processing files: oracle_12102_jul16_psu_ojvm-0-1.el7.x86_64 Provides: oracle_12102_jul16_psu_ojvm = 0-1.el7 oracle_12102_jul16_psu_ojvm(x86-64) = 0-1.el7 Requires(interp): /bin/sh Requires(rpmlib): rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(CompressedFileNames) <= 3.0.4-1 Requires(post): /bin/sh Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/oracle/rpmbuild/BUILDROOT/oracle_12102_jul16_psu_ojvm-0-1.el7.x86_64 Wrote: /home/oracle/rpmbuild/SRPMS/oracle_12102_jul16_psu_ojvm-0-1.el7.src.rpm Wrote: /home/oracle/rpmbuild/RPMS/x86_64/oracle_12102_jul16_psu_ojvm-0-1.el7.x86_64.rpm Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.4xSozZ + umask 022 + cd /home/oracle/rpmbuild/BUILD + /usr/bin/rm -rf /home/oracle/rpmbuild/BUILDROOT/oracle_12102_jul16_psu_ojvm-0-1.el7.x86_64 + exit 0
This command has created a Source RPM, and the RPM file to be used, as you can see here:
[oracle@linuxdev SPECS]$ ls -l ../RPMS/x86_64/ total 4 -rw-r--r-- 1 oracle oinstall 2832 Oct 26 16:24 oracle_12102_jul16_psu_ojvm-0-1.el7.x86_64.rpm [oracle@linuxdev SPECS]$ rpm -qpil --requires ../RPMS/x86_64/oracle_12102_jul16_psu_ojvm-0-1.el7.x86_64.rpm Name : oracle_12102_jul16_psu_ojvm Version : 0 Release : 1.el7 Architecture: x86_64 Install Date: (not installed) Group : Application/Databases Size : 0 License : Commercial Signature : (none) Source RPM : oracle_12102_jul16_psu_ojvm-0-1.el7.src.rpm Build Date : Wed 26 Oct 2016 16:24:02 BST Build Host : linuxdev.example.com Relocations : (not relocatable) URL : https://martincarstenbach.wordpress.com/ Summary : a spec file for Oracle 12.1.0.2 with patch 23615289 (combo for 23054246 and 23177536) Description : Oracle 12.1.0.2 EE patched with the July 2016 PSU and OJVM patch Requires Oracle Linux 7, depends on the preinstall RPM to set up users, groups and defaults ONLY A PROTOTYPE, USED FOR DEMONSTRATION PURPOSES ONLY oraclelinux-release >= 7:7.0 oracle-rdbms-server-12cR1-preinstall /bin/sh rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadIsXz) <= 5.2-1 (contains no files)
You require a “gold image” binary copy for cloning as well. I am assuming it is in place and NFS-accessible in the location specified by the %post scriptlet.
You can refer to Frits Hoogland’s post for more detail about creating the gold image: https://fritshoogland.wordpress.com/2010/07/03/cloning-your-oracle-database-software-installation/. There are probably lots of other posts out there describing the same procedure.
Installation
Here is an example install session, executed as root on a new box I created: server9.
[root@server9 ~]# rpm -ihv /tmp/oracle_12102_jul16_psu_ojvm-0-1.el7.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:oracle_12102_jul16_psu_ojvm-0-1.e################################# [100%]
cloning the oracle home now to /u01/app/oracle/product/12.1.0.2/dbhome_1
./runInstaller -clone -waitForCompletion "ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1"
"ORACLE_BASE=/u01/app/oracle" -defaultHomeName -silent -paramFile
/u01/app/oracle/product/12.1.0.2/dbhome_1/clone/clone_oraparam.ini
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 500 MB. Actual 9204 MB Passed
Checking swap space: must be greater than 500 MB. Actual 767 MB Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2016-10-26_05-03-05PM.
Please wait ...You can find the log of this install session at:
/u01/app/oraInventory/logs/cloneActions2016-10-26_05-03-05PM.log
.................................................. 5% Done.
.................................................. 10% Done.
.................................................. 15% Done.
.................................................. 20% Done.
.................................................. 25% Done.
.................................................. 30% Done.
.................................................. 35% Done.
.................................................. 40% Done.
.................................................. 45% Done.
.................................................. 50% Done.
.................................................. 55% Done.
.................................................. 60% Done.
.................................................. 65% Done.
.................................................. 70% Done.
.................................................. 75% Done.
.................................................. 80% Done.
.................................................. 85% Done.
..........
Copy files in progress.
Copy files successful.
Link binaries in progress.
Link binaries successful.
Setup files in progress.
Setup files successful.
Setup Inventory in progress.
Setup Inventory successful.
Finish Setup successful.
The cloning of OraHome1 was successful.
Please check '/u01/app/oraInventory/logs/cloneActions2016-10-26_05-03-05PM.log' for more details.
Setup Oracle Base in progress.
Setup Oracle Base successful.
.................................................. 95% Done.
As a root user, execute the following script(s):
1. /u01/app/oraInventory/orainstRoot.sh
2. /u01/app/oracle/product/12.1.0.2/dbhome_1/root.sh
.................................................. 100% Done.
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
Check /u01/app/oracle/product/12.1.0.2/dbhome_1/install/root_server9_2016-10-26_17-04-05.log
for the output of root script
The “post” script that is executed spawns a shell (/bin/sh) and executes a script stored in /var/tmp. In my case, this is the contents; pretty much exactly the %post section of the SPEC file:
[root@server9 ~]# cat /var/tmp/rpm-tmp.tZ4pYf
ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1
if [ -d "$ORACLE_HOME" ]; then
/usr/bin/echo "ORACLE_HOME directory $ORACLE_HOME exists-aborting";
exit 127
fi
/usr/bin/echo "cloning the oracle home now to $ORACLE_HOME"
/usr/bin/mkdir -p $ORACLE_HOME
/usr/bin/umount /mnt
/usr/bin/mount -t nfs linuxdev:/u01/nfs_export /mnt
/usr/bin/tar --gzip -xf /mnt/oracle/12.1.0.2/oracle_12102_jul16_psu_ojvm.tar.gz -C $ORACLE_HOME
/usr/bin/chown -R oracle:oinstall /u01
/usr/bin/su - oracle -c "cd $ORACLE_HOME/clone/bin && ./clone.pl ORACLE_HOME=$ORACLE_HOME ORACLE_BASE=/u01/app/oracle -defaultHomeName"
/u01/app/oraInventory/orainstRoot.sh
$ORACLE_HOME/root.sh
umount /mnt
[root@server9 ~]#
And that’s it! One of the slightly more complex things was to ensure that this can be installed on Oracle Linux 7 only. One way to do this is to add the oraclelinux-release package as a dependency. I had to fiddle around with it a bit as it uses the epoch field (which seems rare), and not just the version. After I worked it out I managed to enforce the release. On my Oracle Linux 6 system the RPM could not be installed:
[root@linux6 ~]# rpm -ihv /tmp/oracle_12102_jul16_psu_ojvm-1-0.el7.x86_64.rpm error: Failed dependencies: oraclelinux-release >= 7:7.0 is needed by oracle_12102_jul16_psu_ojvm-1-0.el7.x86_64
If you are now thinking of using the epoch:version to check for the oraclelinux-release on Oracle Linux 6, don’t. This is the current oraclelinux-release RPM taken from the “latest” channel in public-yum:
[oracle@linux6 ~]$ rpm -qpi --provides ./oraclelinux-release-6Server-8.0.3.x86_64.rpm Name : oraclelinux-release Relocations: (not relocatable) Version : 6Server Vendor: Oracle America Release : 8.0.3 Build Date: Wed May 11 11:51:57 2016 Install Date: (not installed) Build Host: x86-ol6-builder-05.us.oracle.com Group : System Environment/Base Source RPM: oraclelinux-release-6Server-8.0.3.src.rpm Size : 49976 License: GPL Signature : RSA/8, Wed May 11 11:44:59 2016, Key ID 72f97b74ec551f03 Summary : Oracle Linux 6 release file Description : System release and information files config(oraclelinux-release) = 6:6Server-8.0.3 oraclelinux-release = 6:6Server-8.0.3 oraclelinux-release(x86-64) = 6:6Server-8.0.3
So instead of “Requires: oraclelinux-release >= 7:7.0”, … you probably need to replace that with “Requires: oraclelinux-release >= 6:6Server-2.0” for 12c (Oracle Linux 6 update 2 or newer, according to the Linux installation guide).
Do you know if oracle-rdbms-server-release-preinstall.x86_64 packages set kernel parameters based on system configuration of mininum required configuration. Let’s say system has 96G RAM, will all the kernel parameters be set accordingly? Thanks Michael.
Hi Michael,
that’s a good question, and I think I know the answer but need to double-check. This is a great topic for a blog post!
Might be worth noting that such RPM can be used within ones own organization, but cannot generally be redistributed as Oracle Corporation does not allow redistribution of its software.
Hi, thanks for your comment. As I said this SPEC file is just a prototype and needs to be made production ready even before thinking of using the resulting RPM in house. I never intended the RPM to be out in the wild. One aspect of your comment made me curious though. I am not sure where you see Oracle software as part of the RPM? The code in the post section merely refers to a clone of an Oracle binary which is not included. It is mounted via NFS from a different host. I didn’t say it explicitly in the post but for this clone of the binaries you need a valid license (of course).
Ingenious! Yes good point my bad. I did not understand at first that the Oracle binaries were pulled from an NFS mounted gold copy.
Hi, thanks for your comment. I did not understand at first that the Oracle binaries were pulled from an NFS mounted gold copy.