Configuring Kernel Parameters and Shell Limits

The kernel parameters and shell limits presented in this section are recommended values only as documented by Oracle. For production database systems, Oracle recommends that you tune these values to optimize the performance of the system.

On both Oracle RAC nodes, verify that the kernel parameters shown in this section are set to values greater than or equal to the recommended values. Also note that when setting the four semaphore values that all four values need to be entered on one line.

Setting Shared Memory

Shared memory allows processes to access common structures and data by placing them in a shared memory segment. This is the fastest form of inter-process communications (IPC) available, mainly due to the fact that no kernel involvement occurs when data is being passed between the processes. With shared memory, data does not need to be copied between processes.

Oracle makes use of shared memory for its Shared Global Area (SGA) which is an area of memory that is shared by all Oracle backup and foreground processes. Adequate sizing of the SGA is critical to Oracle performance because it is responsible for holding the database buffer cache, shared SQL, access paths, and so much more.

To determine all shared memory limits, use the following:

# ipcs -lm

------ Shared Memory Limits --------

max number of segments = 4096

max seg size (kbytes) = 4194303

max total shared memory (kbytes) = 1073741824

min seg size (bytes) = 1





in My Env:



[oracle@crOracle01 ~]$ ipcs -lm



------ Shared Memory Limits --------

max number of segments = 4096

max seg size (kbytes) = 67108864

max total shared memory (kbytes) = 17179869184

min seg size (bytes) = 1







Setting SHMMAX

The SHMMAX parameters defines the maximum size (in bytes) for a shared memory segment. The Oracle SGA is comprised of shared memory and it is possible that incorrectly setting SHMMAX could limit the size of the SGA. When setting SHMMAX, keep in mind that the size of the SGA should fit within one shared memory segment. An inadequate SHMMAX setting could result in the following:

ORA-27123: unable to attach to shared memory segment

You can determine the value of SHMMAX by performing the following: # cat /proc/sys/kernel/shmmax
4294967295


[oracle@crOracle01 ~]$ cat /proc/sys/kernel/shmmax
68719476736

For most Linux systems, the default value for SHMMAX is 32MB. This size is often too small to configure the Oracle SGA. The default value for SHMMAX in Enterprise Linux 5.0 is 4GB. Note that this value of 4GB is not the "normal" default value for SHMMAX in a Linux environment — Oracle Enterprise Linux 5.0 inserts the following two entries in the file /etc/sysctl.conf:

..................................................................
# Controls the maximum shared segment size, in bytes
kernel.shmmax = 4294967295
# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 268435456
..................................................................

I highly recommend removing these two values from both Oracle RAC nodes and replacing them with the recommended values document by Oracle. All recommended Oracle kernel parameter values are documented in this section.

Oracle recommends sizing the SHMMAX parameter as the minimum of (4GB - 1 byte), or half the size of physical memory (in bytes), whichever is lower. Given my nodes are configured with 2GB of physical RAM, I will configure SHMMAX to 1GB:

• You can alter the default setting for SHMMAX without rebooting the machine by making the changes directly to the /proc file system (/proc/sys/kernel/shmmax) using the following command:

  # sysctl -w kernel.shmmax=1073741823
  
       

• You should then make this change permanent by inserting the kernel parameter in the /etc/sysctl.conf startup file:

  # echo "kernel.shmmax=1073741823" >> /etc/sysctl.conf
  
       

Setting SHMMNI

We now look at the SHMMNI parameters. This kernel parameter is used to set the maximum number of shared memory segments system wide. The default value for this parameter is 4096.

You can determine the value of SHMMNI by performing the following:

# cat /proc/sys/kernel/shmmni 4096 The default setting for SHMMNI should be adequate for your Oracle RAC 11g Release 1 installation.

Setting SHMALL

Finally, we look at the SHMALL shared memory kernel parameter. This parameter controls the total amount of shared memory (in pages) that can be used at one time on the system. In short, the value of this parameter should always be at least:

ceil(SHMMAX/PAGE_SIZE)

The default size of SHMALL is 2097152 and can be queried using the following command:

# cat /proc/sys/kernel/shmall

2097152



[oracle@crOracle01 ~]$ cat /proc/sys/kernel/shmall

4294967296

The default setting for SHMALL should be adequate for our Oracle RAC 11g Release 1 installation.

(Note: The page size in Red Hat Linux on the i386 platform is 4,096 bytes. You can, however, use bigpages which supports the configuration of larger memory page sizes.)

Setting Semaphores

Now that you have configured your shared memory settings, it is time to configure the semaphores. The best way to describe a "semaphore" is as a counter that is used to provide synchronization between processes (or threads within a process) for shared resources like shared memory. Semaphore sets are supported in UNIX System V where each one is a counting semaphore. When an application requests semaphores, it does so using "sets."

To determine all semaphore limits, use the following:

# ipcs -ls



max number of arrays = 128

max semaphores per array = 250

max semaphores system wide = 32000

max ops per semop call = 32





semaphore max value = 32767

You can also use the following command:

# cat /proc/sys/kernel/sem

250     32000   32      128





Setting SEMMSL



   

The SEMMSL kernel parameter is used to control the maximum number of semaphores per semaphore set.

Oracle recommends setting SEMMSL to the largest PROCESS instance parameter setting in the init.ora file for all databases on the Linux system plus 10. Also, Oracle recommends setting the SEMMSL to a value of no less than 100.

Setting SEMMNI

The SEMMNI kernel parameter is used to control the maximum number of semaphore sets in the entire Linux system. Oracle recommends setting the SEMMNI to a value of no less than 100.

Setting SEMMNS

The SEMMNS kernel parameter is used to control the maximum number of semaphores (not semaphore sets) in the entire Linux system.

Oracle recommends setting the SEMMNS to the sum of the PROCESSES instance parameter setting for each database on the system, adding the largest PROCESSES twice, and then finally adding 10 for each Oracle database on the system.

Use the following calculation to determine the maximum number of semaphores that can be allocated on a Linux system. It will be the lesser of:

SEMMNS -or- (SEMMSL * SEMMNI)

Setting SEMOPM

The SEMOPM kernel parameter is used to control the number of semaphore operations that can be performed per semop system call.

The semop system call (function) provides the ability to do operations for multiple semaphores with one semop system call. A semaphore set can have the maximum number of SEMMSL semaphores per semaphore set and is therefore recommended to set SEMOPM equal to SEMMSL.

Oracle recommends setting the SEMOPM to a value of no less than 100.

Setting Semaphore Kernel Parameters

Finally, we see how to set all semaphore parameters using several methods. In the following, the only parameter I care about changing (raising) is SEMOPM. All other default settings should be sufficient for our example installation.

• You can alter the default setting for all semaphore settings without rebooting the machine by making the changes directly to the /proc file system (/proc/sys/kernel/sem) using the following command:

  # sysctl -w kernel.sem="250 32000 100 128"

• You should then make this change permanent by inserting the kernel parameter in the /etc/sysctl.conf startup file:

  # echo "kernel.sem=250 32000 100 128" >> /etc/sysctl.conf

Setting File Handles

When configuring the Oracle RAC nodes, it is critical to ensure that the maximum number of file handles is sufficiently large. The setting for file handles denotes the number of open files that you can have on the Linux system.

Use the following command to determine the maximum number of file handles for the entire system:

# cat /proc/sys/fs/file-max

205733







[oracle@crOracle01 ~]$ cat /proc/sys/fs/file-max

248278



[oracle@oracle01 ~]$ cat /proc/sys/fs/file-max

6553600

Oracle recommends that the file handles for the entire system be set to at least 65536.

• You can alter the default setting for the maximum number of file handles without rebooting the machine by making the changes directly to the /proc file system (/proc/sys/fs/file-max) using the following:

  # sysctl -w fs.file-max=65536

• You should then make this change permanent by inserting the kernel parameter in the /etc/sysctl.conf startup file:

  # echo "fs.file-max=65536" >> /etc/sysctl.conf

You can query the current usage of file handles by using the following: # cat /proc/sys/fs/file-nr
960 0 65536

[oracle@oracle01 ~]$ cat /proc/sys/fs/file-nr
185640 0 6553600


[oracle@crOracle01 ~]$ cat /proc/sys/fs/file-nr
1020 0 248278

The file-nr file displays three parameters: total allocated file handles, currently used file handles, and maximum file handles that can be allocated.

Note: If you need to increase the value in /proc/sys/fs/file-max, then make sure that the ulimit is set properly. Usually for 2.4 and 2.6 kernels it is set to unlimited. Verify the ulimit setting by issuing the ulimit command:

# ulimit unlimited

Setting IP Local Port Range

Configure the system to allow a local port range of 1024 through 65000.

Use the following command to determine the value of ip_local_port_range:

# cat /proc/sys/net/ipv4/ip_local_port_range
32768 61000

The default value for ip_local_port_range is ports 32768 through 61000. Oracle recommends a local port range of 1024 to 65000.

• You can alter the default setting for the local port range without rebooting the machine by making the changes directly to the /proc file system (/proc/sys/net/ipv4/ip_local_port_range) using the following command:

  # sysctl -w net.ipv4.ip_local_port_range="1024 65000"

• You should then make this change permanent by inserting the kernel parameter in the /etc/sysctl.conf startup file:

  # echo "net.ipv4.ip_local_port_range = 1024 65000" >> /etc/sysctl.conf

[oracle@oracle01 ~]$ cat /proc/sys/net/ipv4/ip_local_port_range
1024 65000

[oracle@crOracle01 ~]$ cat /proc/sys/net/ipv4/ip_local_port_range
32768 61000

Setting Shell Limits for the oracle User

To improve the performance of the software on Linux systems, Oracle recommends you increase the following shell limits for the oracle user:

Shell Limit	Item in limits.conf	Hard Limit
Maximum number of open file descriptors	nofile	65536
Maximum number of processes available to a single user	nproc	16384

To make these changes, run the following as root:

cat >> /etc/security/limits.conf <oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

EOF

cat >> /etc/pam.d/login <# Added for Oracle Shell Limits

session    required     /lib/security/pam_limits.so

session    required     pam_limits.so



ref:http://www.oracle.com/technology/pub/articles/hunter_rac11gr1_iscsi_2.html

EOF