Do I really have to set resource limits for Oracle in systemd unit files?

TL;DR: it is very much required to set resource limits in systemd unit files. If you’d like to learn more about potential directives to put into a unit file, please have a look at the earlier post I wrote for a more thorough explanation. I also wrote a short post about a potential unit file used to start an Oracle single-instance database (without Oracle Restart) with a couple more details.

Failing to set resource limits in systemd unit files can lead to outages.

Large pages are a good example

Without the Limit-directives added to the unit file the database(s) might refuse to start. Using large pages is a good example. Switching a database to enforce the use of large pages is easy:

SQL> ALTER SYSTEM SET use_large_pages='ONLY' SCOPE=SPFILE;

System altered.

When implementing this directive you tell Oracle not to start at all unless the entire Shared Global Area (SGA) fits into large pages. It also means the database won’t start if your configuration is wrong. Better get it right :)

How many large pages do I need?

On my host with a single Oracle database it couldn’t be easier to work out the number of large pages required. The alert.log tells me I need to allocate 2050 large pages if I want the SGA to fit completely. If there are multiple Oracle instances per host, you need to allocate the sum of all required large pages without over-allocating them (more on that later).

On my system 2050 large pages is comfortably below the mark of 70% of available memory. As a rule of thumb, don’t allocate more memory in large pages than 70% of your total available memory. The O/S, most processes and the PGA cannot use large pages. Allocating too many large pages is a sure way to create instability and it’s not easy to fix so please don’t do it.

Again, as there are no other Oracle instances on the host, defining the appropriate number of large pages is simple. To make the change permanent I need to either change /etc/sysctl.conf (not a good idea) or create a new file /etc/sysctl.d/99-oracle-large-pages.conf (preferred) containing the following directive:

vm.nr_hugepages = 2052

I was lucky and managed to allocate the necessary number of large pages at runtime saving the time of a reboot. Either way, the output of cat /proc/sys/vm/nr_hugepages needs to show the correct number of large pages available:

$ cat /proc/sys/vm/nr_hugepages
2052

Next I’m shutting the database down before removing LimitMEMLOCK from the unit file mentioned in my earlier post, and reloading the systemd daemon. As you might have imagined, the database doesn’t come up when the system reboots:

$ ps -ef | egrep "tns|smon"
root          37       2  0 09:30 ?        00:00:00 [netns]
oracle      1011       1  0 19:30 ?        00:00:00 /u01/app/oracle/product/19.0.0/dbhome_1/bin/tnslsnr LISTENER -inherit
vagrant     7829    7805  0 19:31 pts/0    00:00:00 grep -E --color=auto tns|smon

This shouldn’t be a surprise. The default memlock setting is 64kb, not quite enough for my 4 GB SGA:

$ whoami
vagrant
$ egrep "^Limit|Max locked memory" /proc/$$/limits  
Limit                     Soft Limit           Hard Limit           Units     
Max locked memory         65536                65536                bytes   

And sure enough, ${ORACLE_HOME}/rdbms/log/startup.log confirm the suspicion:

/u01/app/oracle/product/19.0.0/dbhome_1/bin/dbstart: Starting up database "ORCL"
Thu Jun 17 19:30:14 UTC 2021


SQL*Plus: Release 19.0.0.0.0 - Production on Thu Jun 17 19:30:15 2021
Version 19.11.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

SQL> Connected to an idle instance.
SQL> ORA-27125: unable to create shared memory segment
Linux-x86_64 Error: 1: Operation not permitted
Additional information: 4641
Additional information: 10485760
SQL> Disconnected

/u01/app/oracle/product/19.0.0/dbhome_1/bin/dbstart: Database instance "ORCL" warm started.

Well it wasn’t really warm started, was it ;) I had 2052 large pages available for the SGA, the problem is the shell limit: oracle isn’t allowed to lock more than the default 64k of memory in large pages since systemd doesn’t use pam_limits(8).

Restoring the LimitMEMLOCK directive fixes that problem. After changing the unit file I rebooted once more and, voila:

$ ps -ef | egrep "tns|smon"
root          37       2  0 19:37 ?        00:00:00 [netns]
oracle      1016       1  0 19:37 ?        00:00:00 /u01/app/oracle/product/19.0.0/dbhome_1/bin/tnslsnr LISTENER -inherit
oracle      2761       1  0 19:37 ?        00:00:00 ora_smon_ORCL
vagrant     4438    4412  0 19:39 pts/0    00:00:00 grep -E --color=auto tns|smon

The database is up and running, and it uses large pages:

$ grep -i ^hugepages_ /proc/meminfo
HugePages_Total:    2052
HugePages_Free:        5
HugePages_Rsvd:        3
HugePages_Surp:        0

Summary

You can use systemd on Oracle Linux 8 to start your databases along with the operating system. This is great in situations when you don’t want to/can’t use Oracle Restart and/or Oracle Real Application Clusters. Making use of systemd.directives(7) allows the database administrator to set appropriate limits for processes created by systemd as the latter doesn’t read settings in /etc/limits.d/* and /etc/limits.conf.

Advertisement