Setting the Greenplum Recommended OS Parameters

A newer version of this documentation is available. Click here to view the most up-to-date release of the Greenplum 4.x documentation.

Setting the Greenplum Recommended OS Parameters

Greenplum requires the certain operating system (OS) parameters be set on all hosts in your Greenplum Database system (masters and segments).

In general, the following categories of system parameters need to be altered:

  • Shared Memory - A Greenplum Database instance will not work unless the shared memory segment for your kernel is properly sized. Most default OS installations have the shared memory values set too low for Greenplum Database. On Linux systems, you must also disable the OOM (out of memory) killer.
  • Network - On high-volume Greenplum Database systems, certain network-related tuning parameters must be set to optimize network connections made by the Greenplum interconnect.
  • User Limits - User limits control the resources available to processes started by a user's shell. Greenplum Database requires a higher limit on the allowed number of file descriptors that a single process can have open. The default settings may cause some Greenplum Database queries to fail because they will run out of file descriptors needed to process the query.

Linux System Settings

  • Set the following parameters in the /etc/sysctl.conf file and reboot:
    kernel.shmmax = 500000000
    kernel.shmmni = 4096
    kernel.shmall = 4000000000
    kernel.sem = 250 512000 100 2048
    kernel.sysrq = 1
    kernel.core_uses_pid = 1
    kernel.msgmnb = 65536
    kernel.msgmax = 65536
    kernel.msgmni = 2048
    net.ipv4.tcp_syncookies = 1
    net.ipv4.ip_forward = 0
    net.ipv4.conf.default.accept_source_route = 0
    net.ipv4.tcp_tw_recycle = 1
    net.ipv4.tcp_max_syn_backlog = 4096
    net.ipv4.conf.all.arp_filter = 1
    net.ipv4.ip_local_port_range = 1025 65535
    net.core.netdev_max_backlog = 10000
    net.core.rmem_max = 2097152
    net.core.wmem_max = 2097152
    vm.overcommit_memory = 2
  • Set the following parameters in the /etc/security/limits.conf file:
    * soft nofile 65536
    * hard nofile 65536
    * soft nproc 131072
    * hard nproc 131072

    For RedHat Enterprise Linux 6.x and CentOS 6.x, parameter values in the /etc/security/limits.d/90-nproc.conf file override the values in the limits.conf file. If a parameter value is set in both conf files, ensure that the parameter is set properly in the 90-nproc.conf file. The Linux module pam_limits sets user limits by reading the values from the limits.conf file and then from the 90-nproc.conf file. For information about PAM and user limits, see the documentation on PAM and pam_limits.

  • XFS is the preferred file system on Linux platforms for data storage. Greenplum recommends the following XFS mount options:

    See the manual page (man) for the mount command for more information about using that command (man mount opens the man page).

    The XFS options can also be set in the /etc/fstab file. This example entry from an fstab file specifies the XFS options.

    /dev/data /data xfs nodev,noatime,inode64,allocsize=16m 0 0
  • The Linux disk I/O scheduler for disk access supports different policies, such as CFQ, AS, and deadline.

    Greenplum recommends the following scheduler option: deadline. To specify a scheduler until the next system reboot, run the following:

    # echo schedulername > /sys/block/devname/queue/scheduler

    For example:

    # echo deadline > /sys/block/sbd/queue/scheduler

    You can specify the I/O scheduler at boot time with the elevator kernel parameter. Add the parameter elevator=deadline to the kernel command in the file /boot/grub/grub.conf, the GRUB boot loader configuration file. After adding the parameter, reboot the system.

  • Each disk device file should have a read-ahead (blockdev) value of 16384.

    To verify the read-ahead value of a disk device:

    # /sbin/blockdev --getra devname

    For example:

    # /sbin/blockdev --getra /dev/sdb

    To set blockdev (read-ahead) on a device:

    # /sbin/blockdev --setra bytes devname

    For example:

    # /sbin/blockdev --setra 16384 /dev/sdb

    See the manual page (man) for the blockdev command for more information about using that command (man blockdev opens the man page).

  • Edit the /etc/hosts file and make sure that it includes the host names and all interface address names for every machine participating in your Greenplum Database system.
  • Disable Transparent Huge Pages (THP). RedHat Enterprise Linux 6.0 or higher enables THP by default. THP degrades Greenplum Database performance. One way to disable THP is by disabling the setting in the /sys/kernel file system. This example echo command disables THP:
    $ echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

    Place the echo command in /etc/rc.local to ensure THP is disabled on each reboot.

    This cat command checks the state of THP. The output indicates that THP is disabled.

    $ cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
    always [never]

    For more information about Transparent Huge Pages, see your operating system documentation.

Mac OS X System Settings

  • Add the following to /etc/sysctl.conf:
  • Add the following line to /etc/hostconfig: