For Greenplum Database 4.3.8.0, the CREATE EXTERNAL TABLE command supports creating readable external tables that access files from Amazon S3. For information about Amazon S3, see Amazon S3.

The s3 protocol is used in a URL that specifies the location of an Amazon Simple Storage Service (Amazon S3) bucket. Amazon S3 provides secure, durable, highly-scalable object storage. This is the s3 protocol syntax for the LOCATION clause of the CREATE EXTERNAL TABLE command.

's3://S3_endpoint/bucket_name/[folder/[folder/]...]] [config=/config_file_location]'

The s3 protocol URL specifies the AWS S3 endpoint, S3 bucket name, and optional S3 folder location.

All the files in the S3 location (S3_endpoint/bucket_name/folder_location) are used as the source for the external table and must have the same format and each file must contain complete data rows. A data row cannot be split between files. Only the TEXT and CSV formats are supported. The S3 file permissions must be Open/Download and View for the S3 user ID that is accessing the files. S3 server side encryption is not supported.

For information about the CREATE EXTERNAL TABLE command, see the Greenplum Database Reference Guide. For information about using external tables, see "Loading and Unloading" data in the Greenplum Database Administrator Guide.

For Greenplum Database 4.3.8.0, the gpcrondump utility includes enhanced support for special characters. You can specify names of databases, schemes, and tables that contain these special characters.

When the name contains special characters and is specified on the command line, the name must be enclosed in double quotes ("). Double quotes are optional for names that do not contain special characters. For example, both example are valid on the command line "my#1schema".mytable. and "my#1schema"."mytable". Within the name, these special characters must be escaped with a backslash (\) : " ` $ \ .

When the name is specified in an input file, the name must not be enclosed in double quotes. Special characters do not require escaping.

For information about the utility, see the Greenplum Database Utility Guide.

These are the Greenplum Database 4.3.8.0 enhancements for the gpdbrestore utility.

• The utility supports for special characters in names of databases, schemas, and tables. You can specify names that contain these special characters.
  " ' ` ~ # $ % ^ & * ( ) _ - + [ ] { } > < \ | ; : / ? and the space character.

  Note: The characters !, comma (,), and period (.) are not supported. Also, the tab (\t) and newline (\n) characters are not supported.

  When the name contains special characters and is specified on the command line, the name must be enclosed in double quotes ("). Double quotes are optional for names that do not contain special characters. For example, both example are valid on the command line "my#1schema".mytable. and "my#1schema"."mytable". Within the name, these special characters must be escaped with a backslash (\) : " ` $ \ .

  When the name is specified in an input file, the name must not be enclosed in double quotes. Special characters do not require escaping.

• The utility adds supports for the -S option to specify schema name. All the tables that are qualified by the specified schema are restored from the backup.

  The -S option can be specified multiple times. System catalog schemas are not supported. The schema name must exist in the backup set of the database being restored. To replace the data in the schema tables with the data from backup, you can specify the --truncate option. The schema tables are truncated.

  The -S option cannot be specified with the --change-schema option

• The utility performance has been improved when restoring specific append-optimized tables from a full backup (for example, with the -T or --table-file option). The utility updates statistics only append-optimized tables that are restored. In previous releases, the utility updated statistics information for all append-optimized tables in the database, not just the tables that were restored.

For information about the utility, see the Greenplum Database Utility Guide.

These are the Greenplum Database 4.3.8.0 enhancements for Pivotal Query Optimizer.

• For Greenplum Database 4.3.8.0, Pivotal Query Optimizer can generate query plans that include bitmap index scans. In previous releases, only the legacy query optimizer supported bitmap index scans. See resolved issue 25924.
• Greenplum Database permits GPORCA to execute catalog queries that run only on the Greenplum Database master. See the server configuration parameter optimizer_enable_master_only_queries in New and Changed Parameters.

For information about Pivotal Query Optimizer, the Greenplum Database Administrator Guide. For information about the parameter, see the Greenplum Database Reference Guide.

Greenplum Database 4.3.8.0 supports the server configuration parameter optimizer_enable_master_only_queries. The parameter controls how catalog queries that run only on the Greenplum Database master are executed.

In Greenplum Database 4.3.8.0, the calculation for the server configuration parameter gp_vmem_protect_limit has been improved.

The following notes describe platform support for Greenplum Database. Please send any questions or comments to Pivotal Support at https://support.pivotal.io.

• The only file system supported for running Greenplum Database is the XFS file system. All other file systems are explicitly not supported by Pivotal.
• Greenplum Database is supported on all 1U and 2U commodity servers with local storage. Special purpose hardware that is not commodity may be supported at the full discretion of Pivotal Product Management based on the general similarity of the hardware to commodity servers.
• Greenplum Database is supported on network or shared storage if the shared storage is presented as a block device to the servers running Greenplum Database and the XFS file system is mounted on the block device. Network file systems are not supported. When using network or shared storage, Greenplum Database mirroring must be used in the same way as with local storage, and no modifications may be made to the mirroring scheme or the recovery scheme of the segments. Other features of the shared storage such as de-duplication and/or replication are not directly supported by Pivotal, but may be used with support of the storage vendor as long as they do not interfere with the expected operation of Greenplum Database at the discretion of Pivotal.
• Greenplum Database is supported when running on virtualized systems, as long as the storage is presented as block devices and the XFS file system is mounted for the storage of the segment directories.
• A minimum of 10-gigabit network is required for a system configuration to be supported by Pivotal.
• Greenplum Database is supported on Amazon Web Services (AWS) servers using either Amazon instance store (Amazon uses the volume names ephemeral[0-20]) or Amazon Elastic Block Store (Amazon EBS) storage. If using Amazon EBS storage the storage should be RAID of Amazon EBS volumes and mounted with the XFS file system for it to be a supported configuration.
• For Red Hat Enterprise Linux 7.2 or CentOS 7.2, the default systemd setting RemoveIPC=yes removes IPC connections when non-system users logout. This causes the Greenplum Database utility gpinitsystem to fail with semaphore errors. To avoid this issue, see "Setting the Greenplum Recommended OS Parameters" in the Greenplum Database Installation Guide.

An upgrade from 4.3.x to 4.3.8.0 involves stopping Greenplum Database, updating the Greenplum Database software binaries, upgrading and restarting Greenplum Database. If you are using Greenplum Extension packages, you must install and use Greenplum Database 4.3.5.0 or later extension packages. If you are using custom modules with the extensions, you must also use modules that were built for use with Greenplum Database 4.3.5.0 or later.

1. Log in to your Greenplum Database master host as the Greenplum administrative user:

   $ su - gpadmin

2. Uninstall the Greenplum Database gNet extension package if it is installed.

   The gNet extension package contains the software for the gphdfs protocol. For Greenplum Database 4.3.1 and later releases, the extension is bundled with Greenplum Database. The files for gphdfs are installed in $GPHOME/lib/hadoop.

3. Perform a smart shutdown of your current Greenplum Database 4.3.x system (there can be no active connections to the database). This example uses the -a option to disable confirmation prompts:

   $ gpstop -a

4. Run the installer for 4.3.8.0 on the Greenplum Database master host.
   When prompted, choose an installation location in the same base directory as your current installation. For example:

   /usr/local/greenplum-db-4.3.8.0

5. Edit the environment of the Greenplum Database superuser (gpadmin) and make sure you are sourcing the greenplum_path.sh file for the new installation. For example change the following line in .bashrc or your chosen profile file:

   source /usr/local/greenplum-db-4.3.0.0/greenplum_path.sh

   to:

   source /usr/local/greenplum-db-4.3.8.0/greenplum_path.sh

   Or if you are sourcing a symbolic link (/usr/local/greenplum-db) in your profile files, update the link to point to the newly installed version. For example:

   $ rm /usr/local/greenplum-db
   $ ln -s /usr/local/greenplum-db-4.3.8.0 /usr/local/greenplum-db

6. Source the environment file you just edited. For example:

   $ source ~/.bashrc

7. Run the gpseginstall utility to install the 4.3.8.0 binaries on all the segment hosts specified in the hostfile. For example:

   $ gpseginstall -f hostfile

8. Rebuild any modules that were built against the provided C language header files for use with Greenplum Database 4.3.5.0 or later (for example, any shared library files for user-defined functions in $GPHOME/lib). See your operating system documentation and your system administrator for information about rebuilding and compiling modules such as shared libraries.
9. Use the Greenplum Database gppkg utility to install Greenplum Database extensions. If you were previously using any Greenplum Database extensions such as pgcrypto, PL/R, PL/Java, PL/Perl, and PostGIS, download the corresponding packages from Pivotal Network, and install using this utility. See the Greenplum Database 4.3 Utility Guide for gppkg usage details.
10. After all segment hosts have been upgraded, you can log in as the gpadmin user and restart your Greenplum Database system:

    # su - gpadmin
    $ gpstart

11. If you are upgrading a version of Greenplum Database between 4.3.0 and 4.3.2, check your Greenplum Database for inconsistencies due to an incorrect conversion of 4.2.x append-only tables to 4.3.x append-optimized tables.
    Important: The Greenplum Database system must be started but should not be running any SQL commands while the utility is running.
    1. Run the fix_ao_upgrade.py utility with the option --report. The following is an example.

       $ $GPHOME/share/postgresql/upgrade/fix_ao_upgrade.py --host=mdw --port=5432 --report

    2. If the utility displays a list of inconsistencies, fix them by running the fix_ao_upgrade.py utility without the --report option.

       $ $GPHOME/share/postgresql/upgrade/fix_ao_upgrade.py --host=mdw --port=5432

    3. (optional) Run the fix_ao_upgrade.py utility with the option --report again. No inconsistencies should be reported.
12. For databases that contain append-optimized tables that were created from Greenplum Database 4.2.x append-only tables, run the fix_visimap_owner.sql script. The script resolves an issue associated with relations associated with append-optimized tables. For example, this command runs the script on the database testdb.

    $ psql -d testdb1 -f $GPHOME/share/postgresql/upgrade/fix_visimap_owner.sql

    The script displays this prompt that allows you to display changes to the affected relations without performing the operation.

    Dry run, without making any modifications (y/n)?

    • Enter y to list ownership changes that would have been made. The owner of the relation is not changed.
    • Enter n make the ownership changes and display the changes to relation ownership.
    Note: Pivotal recommends that you run the script during low activity period. Heavy workloads do not affect database functionality but might affect performance.
13. If you are utilizing Data Domain Boost, you have to re-enter your DD Boost credentials after upgrading from Greenplum Database 4.3.x to 4.3.8.0 as follows:

    gpcrondump --ddboost-host ddboost_hostname --ddboost-user ddboost_user
      --ddboost-backupdir backup_directory

$GPHOME/share/postgresql/upgrade/fix_visimap_owner.sql

Dry run, without making any modifications (y/n)?

fix_ao_upgrade.py {-h master_host | --host=master_host}
     {-p master_port | --port=master_port}
     [-u user | --user=user ]
     [--report] [-v | --verbose] [--help]

$GPHOME/share/postgresql/upgrade/fix_orphan_segment_tables.py

$GPHOME/share/postgresql/upgrade/fix_orphan_segment_tables.py -p port

$GPHOME/share/postgresql/upgrade/fix_orphan_segment_tables.py -p port -f orphan_tables_file_timestamp 

Upgrading Greenplum Database from 4.3.x to 4.3.8.0 on a Pivotal DCA system involves stopping Greenplum Database, updating the Greenplum Database software binaries, and restarting Greenplum Database. If you are using Greenplum Extension packages, you must install and use Greenplum Database 4.3.5.0 or later extension packages. If you are using custom modules with the extensions, you must also use modules that were built for use with Greenplum Database 4.3.5.0 or later.

1. Log in to your Greenplum Database master host as the Greenplum administrative user (gpadmin):

   # su - gpadmin

2. Download or copy the installer file to the Greenplum Database master host.
3. Uninstall the Greenplum Database gNet extension package if it is installed. For information about uninstalling a Greenplum Database extension package, see gppkg in the Greenplum Database Utility Guide.

   The gNet extension package contains the software for the gphdfs protocol. For Greenplum Database 4.3.1 and later releases, the extension is bundled with Greenplum Database. The files for gphdfs are installed in $GPHOME/lib/hadoop.

4. Perform a smart shutdown of your current Greenplum Database 4.3.x system (there can be no active connections to the database). This example uses the -a option to disable confirmation prompts:

   $ gpstop -a

5. As root, run the Pivotal DCA installer for 4.3.8.0 on the Greenplum Database master host and specify the file hostfile that lists all hosts in the cluster. If necessary, copy hostfile to the directory containing the installer before running the installer.

   This example command runs the installer for Greenplum Database 4.3.8.0 for Redhat Enterprise Linux 5.x.

   # ./greenplum-db-appliance-4.3.8.0-build-1-RHEL5-x86_64.bin hostfile

   The file hostfile is a text file that lists all hosts in the cluster, one host name per line.

6. Install Greenplum Database extension packages. For information about installing a Greenplum Database extension package, see gppkg in the Greenplum Database Utility Guide.
   Important: Rebuild any modules that were built against the provided C language header files for use with Greenplum Database 4.3.5.0 or later (for example, any shared library files for user-defined functions in $GPHOME/lib). See your operating system documentation and your system administrator for information about rebuilding and compiling modules such as shared libraries.
7. After all segment hosts have been upgraded, you can log in as the gpadmin user and restart your Greenplum Database system:

   # su - gpadmin
   $ gpstart

8. If you are upgrading a version of Greenplum Database between 4.3.0 and 4.3.2, check your Greenplum Database for inconsistencies due to an incorrect conversion of 4.2.x append-only tables to 4.3.x append-optimized tables.
   Important: The Greenplum Database system must be started but should not be running any SQL commands while the utility is running.
   1. Run the fix_ao_upgrade.py utility with the option --report. The following is an example.

      $ $GPHOME/share/postgresql/upgrade/fix_ao_upgrade.py --host=mdw --port=5432 --report

   2. If the utility displays a list of inconsistencies, fix them by running the fix_ao_upgrade.py utility without the --report option.

      $ $GPHOME/share/postgresql/upgrade/fix_ao_upgrade.py --host=mdw --port=5432

   3. (optional) Run the fix_ao_upgrade.py utility with the option --report again. No inconsistencies should be reported.
9. If you are utilizing Data Domain Boost, you have to re-enter your DD Boost credentials after upgrading from Greenplum Database 4.3.x to 4.3.8.0 as follows:

   gpcrondump --ddboost-host ddboost_hostname --ddboost-user ddboost_user
     --ddboost-backupdir backup_directory

This section describes how you can upgrade from Greenplum Database 4.2.x.x or later to Greenplum Database 4.3.8.0. For users running versions prior to 4.2.x.x of Greenplum Database, see the following:

• For Users Running Greenplum Database 4.1.x.x
• For Users Running Greenplum Database 4.0.x.x
• For Users Running Greenplum Database 3.3.x.x

Users on a release prior to 4.1.x.x cannot upgrade directly to 4.3.8.0.

1. Upgrade from your current release to 4.2.x.x (follow the upgrade instructions in the latest Greenplum Database 4.2.x.x release notes available at Pivotal Documentation).
2. Follow the upgrade instructions in these release notes for Upgrading from 4.2.x.x to 4.3.8.0.

Users on a release prior to 4.1.x.x cannot upgrade directly to 4.3.8.0.

1. Upgrade from your current release to 4.1.x.x (follow the upgrade instructions in the latest Greenplum Database 4.1.x.x release notes available on Dell EMC Support Zone).
2. Upgrade from the current release to 4.2.x.x (follow the upgrade instructions in the latest Greenplum Database 4.2.x.x release notes available at Pivotal Documentation).
3. Follow the upgrade instructions in these release notes for Upgrading from 4.2.x.x to 4.3.8.0.

Users on a release prior to 4.0.x.x cannot upgrade directly to 4.3.8.0.

1. Upgrade from your current release to the latest 4.0.x.x release (follow the upgrade instructions in the latest Greenplum Database 4.0.x.x release notes available on Dell EMC Support Zone).
2. Upgrade the 4.0.x.x release to the latest 4.1.x.x release (follow the upgrade instructions in the latest Greenplum Database 4.1.x.x release notes available on Dell EMC Support Zone).
3. Upgrade from the 4.1.1 release to the latest 4.2.x.x release (follow the upgrade instructions in the latest Greenplum Database 4.2.x.x release notes available at Pivotal Documentation).
4. Follow the upgrade instructions in these release notes for Upgrading from 4.2.x.x to 4.3.8.0.

If you experience issues during the migration process and have active entitlements for Greenplum Database that were purchased through Pivotal, contact Pivotal Support. Information for contacting Pivotal Support is at https://support.pivotal.io.

Be prepared to provide the following information:

• A completed Upgrade Procedure.
• Log output from gpmigrator and gpcheckcat (located in ~/gpAdminLogs)

Greenplum releases a number of client tool packages on various platforms that can be used to connect to Greenplum Database and the Greenplum Command Center management tool. The following table describes the compatibility of these packages with this Greenplum Database release.

Tool packages are available from Pivotal Network.

The Greenplum Database Client Tools, Load Tools, and Connectivity Tools are supported on the following platforms:

• AIX 5.3L (32-bit)
• AIX 5.3L and AIX 6.1 (64-bit)
• Apple OSX on Intel processors (32-bit)
• HP-UX 11i v3 (B.11.31) Intel Itanium (Client and Load Tools only)
• Red Hat Enterprise Linux i386 (RHEL 5)
• Red Hat Enterprise Linux x86_64 6.x (RHEL 6)
• Red Hat Enterprise Linux x86_64 (RHEL 5)
• SUSE Linux Enterprise Server x86_64 (SLES 10 and SLES 11)
• Solaris 10 SPARC32
• Solaris 10 SPARC64
• Solaris 10 i386
• Solaris 10 x86_64
• Windows 7 (32-bit and 64-bit)
• Windows Server 2003 R2 (32-bit and 64-bit)
• Windows Server 2008 R2 (64-bit)
• Windows XP (32-bit and 64-bit)

For Greenplum Database 4.3, this is the package file naming format.

pkgname-ver_pvpkg-version_gpdbrel-OS-version-arch.gppkg

This example is the package name for a postGIS package.

postgis-ossv2.0.3_pv2.0.1_gpdb4.3-rhel5-x86_64.gppkg

pkgname-ver - The package name and optional version of the software that was used to create the package extension. If the package is based on open source software, the version has format ossvversion. The version is the version of the open source software that the package is based on. For the postGIS package, ossv2.0.3 specifies that the package is based on postGIS version 2.0.3.

pvpkg-version - The package version. The version of the Greenplum Database package. For the postGIS package, pv2.0.1 specifies that the Greenplum Database package version is 2.0.1.

gpdbrel-OS-version-arch - The compatible Greenplum Database release. For the postGIS package, gpdb4.3-rhel5-x86_64 specifies that package is compatible with Greenplum Database 4.3 on Red Hat Enterprise Linux version 5.x, x86 64-bit architecture.