Configuring IPsec for Greenplum Database
A newer version of this documentation is available. Click here to view the most up-to-date release of the Greenplum 4.x documentation.
Configuring IPsec for Greenplum Database
This topic describes how to set up and configure Internet Protocol Security (IPsec) for a Greenplum Database cluster.
Internet Protocol Security (IPsec) is a suite of protocols that authenticate and encrypt communications at the IP network level (OSI layer 3). Using IPsec can help to prevent network attacks, such as packet sniffing, altering network packets, identity spoofing, and man-in-the-middle attacks in the Greenplum Database system.
When IPsec is enabled for Greenplum Database, a virtual private network (VPN), or tunnel, is established between every pair of hosts in the cluster and every packet exchanged between them is encrypted and sent through the tunnel. If you have n hosts in the cluster, n(n-1)/2 VPNs are needed to connect each host to every other host. You may choose to add other hosts on the network, for example ETL servers, to the IPsec configuration.
Encrypting IP traffic has a cost in network performance. To ensure suitable network bandwidth is available after IPsec configuration, use the Greenplum Database gpcheckperf utility. See the Greenplum Database Utility Guide for help with gpcheckperf. If network bandwidth is insufficient for performance and database workloads you may need to tune the configuration or use a higher bandwidth network medium.
This section describes how to set up and configure IPsec for a Greenplum cluster on Red Hat or CentOS hosts using Openswan, a popular IPsec implementation for Linux. Openswan provides user tools to enable IPsec on Linux. It uses the Internet Key Exchange (IKE) protocol and X.509 certificates to handshake and exchange session keys, and uses the netlink API to interface with the IPsec support built into the Linux kernel.
The IKE protocol allows two peers to negotiate the authentication and encryption algorithms the tunnels will use. The negotiation occurs in two phases. During phase 1, the peers perform a Diffie-Hellman key exchange to establish a secure, encrypted channel. Phase 1 must successfully complete before phase 2 begins.
During phase 2, the peers negotiate the authentication and encryption algorithms to be used with the IPsec tunnels. The result of the phase 2 negotiation is a security association (SA). It contains the source, the destination, and an instruction. The Linux kernel uses the SA to set up the connection.
- RSA public key encryption. Each host has a public and private key. Public keys are distributed to all hosts in the cluster so that any host can authenticate any other host.
- Pre-shared keys. This method is easiest to configure, but is not as secure as using RSA public key encryption.
- X.509 certificates. A certificate is issued for each host by a certificate authority (CA). The host is authenticated based on trust conferred by the CA. This is most often used when many hosts connect to a central gateway.
RSA public key encryption is the preferred method.
There are two connection modes for a connection: tunnel or transport. In tunnel mode, the entire IP packet is encrypted, including the IP headers. In transport mode, the headers are exposed. The tunnel mode is preferred for greater security.
- Internet Key Exchange (IKE) - Wikipedia article that describes the IKE protocol used to set up a security association.
- Security Association - Wikipedia article that describes the attributes and purpose of a security association.
- AES instruction set - Wikipedia article that provides an overview of the Intel Advanced Encryption Standard (AES) instruction set and lists the CPU families that support it.
- ipsec.conf(5) - man page for the ipsec.conf configuration file.
- setkey(8) - man page for the setkey utility used to manage the Security Association Database (SAD) and Security Policy Database (SPD) in the Linux kernel.
- Openswan - Red Hat Openswan package overview; applies to CentOS also.
- Host-to-Host VPN Using Openswan - Red Hat guide for creating a host-to-host VPN using Openswan; can also be used with CentOS.
Openswan may be installed using the package manager on your system, by downloading an installable package from the Openswan Web site, or by downloading and compiling source code.
Pivotal recommends that you use Openswan version 2.6.43 or later. If your package manager has an earlier version, you can download an RPM of the latest Openswan release from the Openswan Web site. You can also download Openswan source code from the Openswan Web site.
The following instructions assume you are installing Openswan on hosts running 64-bit Red Hat 6.x or CentOS 6.x.
$ sudo yum info installed openswan
If the recommended version is already installed, continue with Configuring and Verifying the Openswan Installation.
$ sudo yum remove openswan
Installing Openswan with an RPM
$ sudo yum list available openswan
$ sudo yum install -y openswan
If the recommended version is not in the repository, you can download it from the Openswan Web site at https://download.openswan.org. Browse to the /rhel6/x86_64 directory to find the RPM.
$ sudo rpm -i openswanX-version.x86_64.rpm
Installing Openswan from Source
If you cannot install Openswan with an RPM you can download the source, compile it, and install it.
- Download the Openswan source from the Openswan Web site at Openswan Web site.
- Extract the archive and review the README file to ensure that the
prerequisite packages are installed on your build machine. For
sudo yum install gmp gmp-devel gawk flex bison \ iproute2 iptables sed awk bash cut python
- Build the Openswan tools by following the instructions in the
README file. For
$ make programs $ sudo make install
Configuring and Verifying the Openswan Installation
Follow the steps in this section to configure each host and verify the Openswan installation.
- Edit /etc/sysctl.conf and modify or add the following
net.ipv4.conf.default.accept_redirects = 0 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.default.send_redirects = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.ip_forward = 1
Execute sysctl -p to reload the file.
- Restore the default SELinux security contexts for the IPsec directory by running the
# restorecon -Rv /etc/ipsec.d
- If a firewall is enabled, modify it to allow IPsec packets:
Here is an example of IPsec rules for iptables:
- UDP port 500 for the Internet Key Exchange (IKE) protocol
- UDP port 4500 for IKE NAT-Traversal
- Protocol 50 for Encapsulated Security Payload (ESP) IPsec packets
- (not recommended) Protocol 51 for Authenticated Header (AH) IPsec packets
iptables -A INPUT -p udp --sport 500 --dport 500 -j ACCEPT iptables -A OUTPUT -p udp --sport 500 --dport 500 -j ACCEPT iptables -A INPUT -p udp --sport 4500 --dport 4500 -j ACCEPT iptables -A OUTPUT -p udp --sport 4500 --dport 4500 -j ACCEPT iptables -A INPUT -p 50 -j ACCEPT iptables -A OUTPUT -p 50 -j ACCEPT
- Edit the /etc/ipsec.conf file and make the following changes:
- Change protostack from auto to
- Uncomment or add the following
include /etc/ipsec.d/*.confThis allows you to create and install a separate configuration file for each host-to-host tunnel.
- Change protostack from auto to netkey:
- Start IPsec with the service command.
# service start ipsec
- Run ipsec verify to check the IPsec installation. Python must be
installed to run this command.
$ sudo ipsec verifyThe output looks like the following:
Checking if IPsec got installed and started correctly: Version check and ipsec on-path [OK] Openswan U2.6.43/K2.6.32-504.16.2.el6.x86_64 (netkey) See `ipsec --copyright' for copyright information. Checking for IPsec support in kernel [OK] NETKEY: Testing XFRM related proc values ICMP default/send_redirects [OK] ICMP default/accept_redirects [OK] XFRM larval drop [OK] Hardware random device check [N/A] Two or more interfaces found, checking IP forwarding [OK] Checking rp_filter [ENABLED] /proc/sys/net/ipv4/conf/all/rp_filter [ENABLED] /proc/sys/net/ipv4/conf/lo/rp_filter [ENABLED] /proc/sys/net/ipv4/conf/eth0/rp_filter [ENABLED] /proc/sys/net/ipv4/conf/pan0/rp_filter [ENABLED] Checking that pluto is running [OK] Pluto listening for IKE on udp 500 [OK] Pluto listening for IKE on tcp 500 [NOT IMPLEMENTED] Pluto listening for IKE/NAT-T on udp 4500 [OK] Pluto listening for IKE/NAT-T on tcp 4500 [NOT IMPLEMENTED] Pluto listening for IKE on tcp 10000 (cisco) [NOT IMPLEMENTED] Checking NAT and MASQUERADEing [TEST INCOMPLETE] Checking 'ip' command [OK] Checking 'iptables' command [OK]Note: The result for Checking 'ip' command may be [IP XFRM BROKEN], depending on the version of iproute on your system. This can be a misdiagnosis caused by a change in IP XFRM message output between iproute versions.
- Enable starting IPsec on boot with the following
# chkconfig ipsec on
Configuring Openswan Connections
|Connection Number||Left host||Right host|
Create Host Keys
# ipsec newhostkey --output /etc/ipsec.d/ipsec.secrets --bits 4096
The key is saved in the /etc/ipsec.d/ipsec.secrets file and its attributes are set to allow access to the root user only.
# ipsec showhostkey --left # rsakey AQOW+RwpL leftrsasigkey=0sAQOW+RwpLg7CGoyywCnv+vnasGJI7... # ipsec showhostkey --right # rsakey AQOW+RwpL rightrsasigkey=0sAQOW+RwpLg7CGoyywCnv+vnasGJI7...
You will need to use this command as you configure the tunnel between each host pair.
Create a Connection Configuration File
IPsec tunnels are configured by creating a conn section in the /etc/ipsec.conf file. Because we added the include /etc/ipsec.d/*.conf directive to /etc/ipsec.conf, we can create a .conf file for each connection.
- Log in to the host that will be on the "left" side of the tunnel.
- Create a new configuration file in the /etc/ipsec.d directory.
Choose a name that includes both host names and has a .conf
extension. The following configuration file, named mdw-sdw1.conf,
configures the connection between the hosts mdw and
conn mdw-sdw1 leftid=mdw left=192.0.2.214 leftrsasigkey=0sAQOW+RwpLg7CGoyywCnv+vnasGJI7... # shortened for readability rightid=sdw1 right=192.0.2.215 rightrsasigkey=0sAQNfdDCoDte5bGaGLGkHTKa5GMRl... # shortened for readability type=tunnel authby=rsasig ike=aes192-sha2;dh20 phase2alg=aes_gcm_c-160-null auto=start
See the ipsec.conf man page for the complete list of available parameters and their default values.
The connection name in the example is mdw-sdw1.
For the leftrsasigkey use the output from running ipsec showhostkey --left on the "left" host. For rightrsasigkey use the output from running ipsec showhostkey --right on the "right" host.Following are recommendations for configuring parameters for Greenplum Database IPsec connections to obtain the best security and performance:
- Set to tunnel, the default mode.
- Set to rsasig. This is more secure than using pre-shared keys (psk).
- Set to start so that the tunnel is brought up when IPsec starts up.
- The ike parameter is used during phase 1 to authenticate
the peers and negotiate secure session keys for phase2. The parameter value is
an entry in the
cipher-hash;modpgroup, cipher-hash;modpgroup, ....
- cipher is an encryption algorithm. AES is more secure than 3DES, which is more secure than DES. AES has length of 128, 192, or 256 bits. The more bits, the stronger the encryption, but more time is required for computation.
hash is the hash algorithm. SHA2 is
stronger than SHA1, which is stronger than MD5. SHA2 is recommended, but
if SHA2 is not supported on the device, use SHA1. SHA2 is a family of
hash functions—SHA-224, SHA-256, SHA-384, SHA-512, SHA-512/224,
SHA-512/256—not all of which are supported by Openswan. To find out which
algorithms Openswan supports, run the following command after starting the
# ipsec auto -status 000 algorithm IKE hash: id=2, name=OAKLEY_SHA1, hashsize=20 000 algorithm IKE hash: id=4, name=OAKLEY_SHA2_256, hashsize=32 000 algorithm IKE hash: id=6, name=OAKLEY_SHA2_512, hashsize=64 000 algorithm ESP encrypt: id=2, name=ESP_DES, ivlen=8, keysizemin=64, keysizemax=64 000 algorithm ESP encrypt: id=3, name=ESP_3DES, ivlen=8, keysizemin=192, keysizemax=192 000 algorithm ESP encrypt: id=18, name=ESP_AES_GCM_A, ivlen=8, keysizemin=160, keysizemax=288000 algorithm ESP encrypt: id=19, name=ESP_AES_GCM_B, ivlen=12, keysizemin=160, keysizemax=288 000 algorithm ESP encrypt: id=20, name=ESP_AES_GCM_C, ivlen=16, keysizemin=160, keysizemax=288See http://en.wikipedia.org/wiki/SHA-2 for information about SHA2.
- modpgroup is the Diffie-Hellman group. The peers negotiate a shared secret using the Diffie-Hellman protocol. The Diffie-Hellman group is a set of standardized parameters the peers agree to use as the basis for their calculations. The groups are numbered, and higher numbered groups are more secure (and more compute-intensive) than lower numbered groups. Avoid the lowest numbered groups: 1 (modp768), 3 (modp1024), and 5 (modp1576), which are not considered secure. Choose a higher level group, such as dh14, dh15, dh19, dh20, dh21, or dh24.
- Set to esp, the default, to encrypt data. The ah setting creates a connection that authenticates, but does not encrypt IP packets.
- The phase2alg parameter specifies algorithms to use for encrypting and authenticating data. The format and defaults are the same as for the ike parameter.
- The AES cipher and SHA hash algorithm are more secure. For effective use of emerging 10-gigabit and 40-gigabit network devices, and to enable high speed communication channels, the AES_GCM algorithm is currently the recommended best option. To use AES_GCM, verify that the CPU supports the AES_NI instruction set. See AES instruction set for a list of CPUs that support AES_NI.
- To see if the CPU supports AES-NI, see if the aes flag is
grep aes /proc/cpuinfo
- To see if AES-N1 has been enabled, search
/proc/crypto for the
grep module /proc/crypto | sort –u
- To see if the aesni_intel kernel module is
- To specify the AES_GCM algorithm, use the following
phase2alg=aes_gcm_c-160-nullOpenswan requires adding the salt size (32 bits) to the key size (128, 192, or 256 bits). In the example above, "160" is calculated by adding a 128-bit key size to the 32 bit salt size. The other valid values are 224 and 288.
- Use scp to copy the configuration file to the "right" host. For
# scp /etc/ipsec.d/mdw-sdw1.conf sdw1:/etc/ipsec.d/
- Ensure that IPsec is started by executing the following command on both the "left"
and "right" hosts:
# ipsec service start
- Load the tunnel on both left and right hosts with the following
# ipsec auto --add mdw-sdw
- Bring up the tunnel on both left and right hosts with the following
# ipsec auto --up mdw-sdw
Test the IPsec Connection
tcdump -n -i interface_name host hostname
# tcpdump -n -i eth0 host mdw tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 08:22:10.186743 IP 192.0.2.214 > 192.0.2.215: ESP(spi=0xe56f19ea,seq=0x1), length 132 08:22:10.186808 IP 192.0.2.214 > 192.0.2.215: ICMP echo request, id 30987, seq 1, length 64 08:22:10.186863 IP 192.0.2.215 > 192.0.2.214: ESP(spi=0x4e55824c,seq=0x1), length 132 08:22:11.189663 IP 192.0.2.214 > 192.0.2.215: ESP(spi=0xe56f19ea,seq=0x2), length 132 08:22:11.189707 IP 192.0.2.214 > 192.0.2.215: ICMP echo request, id 30987, seq 2, length 64
The ESP packets verify that the IP packets are encrypted and encapsulated.
When you have connections set up between all of the hosts in the cluster and Greenplum Database is running, you can run the tcpdump command on segment hosts to observe database activity in the IPsec tunnels.