Pivotal Greenplum 6.11 Release Notes

A newer version of this documentation is available. Use the version menu above to view the most up-to-date release of the Greenplum 6.x documentation.

Pivotal Greenplum 6.11 Release Notes

This document contains pertinent release information about Pivotal Greenplum 6.11 releases. For previous versions of the release notes for Greenplum Database, go to Pivotal Greenplum Database Documentation. For information about Greenplum Database end of life, see Pivotal Greenplum Database end of life policy.

Pivotal Greenplum 6 software is available for download from VMware Tanzu Network.

Pivotal Greenplum 6 is based on the open source Greenplum Database project code.

Important: VMware does not provide support for open source versions of Greenplum Database. Only Pivotal Greenplum is supported by VMware.

Release 6.11.2

Release Date: 2020-10-2

Pivotal Greenplum 6.11.2 is a maintenance release that includes changes and resolves several issues.

Changed Features

Greenplum Database 6.11.2 includes these changes:

Resolved Issues

Pivotal Greenplum 6.11.2 resolves these issues:

30549 - Management and Monitoring
Greenplum excluded externally-routable loopback addresses from replication entries, which caused utilities such as gpinitstandby and gpaddmirrors to fail. This problem has been resolved.
30795 - GPORCA
Fixed a problem where GPORCA did not utilize an index scan for certain subqueries, which could lead to poor performance for affected queries.
30878 - GPORCA
If a CREATE TABLE .. AS statement was used to create a table with non-legacy (jump consistent) hash algorithm distribution from a source table that used the legacy (modulo) hash algorithm, GPORCA would distribute the data according to the value of gp_use_legacy_hashops; however, it would set the table's distribution policy hash algorithm to the value of the original table. This could cause queries to give incorrect results if the distribution policy did not match the data distribution. This problem has been resolved.
30903 / 30966 - Metrics Collector
Workfile entries were sometimes freed prematurely, which could lead to the postmaster process being reset on segments and failures in query execution, or segment PANIC. This problem has been resolved.
30928 - GPORCA
If gp_use_legacy_hashops was enabled, GPORCA could crash when generating the query plan for certain queries that included an aggregate. This problem has been resolved.
174812955 - Query Execution
When executing a long query that contained multi-byte characters, Greenplum could incorrectly truncate the query string (removing multi-byte characters) and, if log_min_duration_statement was set to 0, could subsequent write an invalid symbol to segment logs. This behavior could cause errors in gp_toolkit and Command Center. This problem has been resolved.

Release 6.11.1

Release Date: 2020-09-17

Pivotal Greenplum 6.11.1 is a maintenance release that includes changes and resolves several issues.

Changed Features

Greenplum Database 6.11.1 includes this change:

  • Greenplum Platform Extension Framework (PXF) version 5.15.1 is included, which includes changes and bug fixes. Refer to the PXF Release Notes for more information on release content and to access the PXF documentation.

Resolved Issues

Pivotal Greenplum 6.11.1 resolves these issues:

30751, 173714727 - Query Optimizer
Resolves an issue where a correlated subquery that contained at least one left or right outer join caused the Greenplum Database master to crash when the server configuration parameter optimizer_join_order was set to exhaustive2.
30880 - gpload
Fixed a problem where gpload operations would fail if a table column name included capital letters or special characters.
30901 - GPORCA
For queries that included an outer ref in a subquery, such as select * from foo where foo.a = (select foo.b from bar), GPORCA always used the results of the subquery after unnesting the outer reference. This could cause a crash or incorrect results if the subquery returned no rows, or if the subquery contained a projection with multiple values below the outer reference. To address this problem, all such queries now fall back to using the Postgres planner instead of GPORCA. Note that this behavior occurs for cases where GPORCA would have returned correct results, as well as for cases that could cause crashes or return incorrect results.
30913, 170824967 - gpfdists
A command that accessed an external table using the gpfdists protocol failed if the external table did not use an IP address when specifying a host system in the LOCATION clause of the external table definition. This issue is resolved in Greenplum 6.11.1.
174609237 - gpstart
gpstart was updated so that it does not attempt to start a standby master segment when that segment is unreachable, preventing an associated stack trace during startup.

Upgrading from Greenplum 6.x to Greenplum 6.11

Note: Greenplum 6 does not support direct upgrades from Greenplum 4 or Greenplum 5 releases, or from earlier Greenplum 6 Beta releases.

See Upgrading from an Earlier Greenplum 6 Release to upgrade your existing Greenplum 6.x software to Greenplum 6.11.0.

Release 6.11.0

Release Date: 2020-09-11

Pivotal Greenplum 6.11.0 is a minor release that includes changed features and resolves several issues.

Features

Greenplum Database 6.11.0 includes these new and changed features:

  • GPORCA partition elimination has been enhanced to support a subset of lossy assignment casts that are order-preserving (increasing) functions, including timestamp::date and float::int. For example, GPORCA supports partition elimination when a partition column is defined with the timestamp datatype and the query contains a predicate such as WHERE ts::date == '2020-05-10' that performs a cast on the partitioned column (ts) to compare column data (a timestamp) to a date.
  • PXF version 5.15.0 is included, which includes new and changed features and bug fixes. Refer to the PXF Release Notes for more information on release content and supported platforms, and to access the PXF documentation.
  • Greenplum Command Center 6.3.0 and 4.11.0 are included, which include new workload management and other features, as well as bug fixes. See the Command Center Release Notes for more information.
  • The DataDirect ODBC Drivers for Pivotal Greenplum were updated to version 07.16.0389 (B0562, U0408). This version introduces support for the following datatypes:
    Greenplum Datatype ODBC Datatype
    citext SQL_LONGVARCHAR
    float SQL_REAL
    tinyint SQL_SMALLINT
    wchar SQL_CHAR
    wvarchar SQL_VARCHAR

Resolved Issues

Pivotal Greenplum 6.11.0 resolves these issues:

30899 - Resource Groups
In some cases when running queries are managed by resource groups, Greenplum Database generated a PANIC when managing runaway queries (queries that use an excessive amount of memory) because of locking issues. This issue is resolved.
30877 - VACUUM
In some cases, running VACUUM returns ERROR: found xmin <xid> from before relfrozenxid <frozen_xid>. The error was caused when a previously run VACUUM FULL was interrupted and aborted on a query executor (QE) and corrupted catalog frozen XID information. This issue is resolved.
30870 - Segment Mirroring
In some cases, performing an incremental recovery of a Greenplum Database segment instance failed with the message requested WAL segment has already been removed because the recovery checkpoint was not created properly. This issue is resolved.
30858 - analyzedb
analyzedb failed if analyzedb attempted to update statistics for a set of tables and one of the tables was dropped and then recreated while analyzedb was running. analyzedb has been enhanced better handle the specified situation.
30845 - Query Execution
Under heavy load when running multiple queries, some queries randomly failed with the error Error on receive from seg<ID>. The error was caused when Greenplum Database encountered a divide by 0 error while managing the backend processes that are used to run queries on the segment instances. This issue is resolved.
30761 - Postgres Planner
In some cases, Greenplum Database generated a PANIC when a DROP VIEW command was cancelled from the Greenplum Command Center. The PANIC was generated when Greenplum Database did not correctly handle the visibility of the relation.
30721 - gpcheckcat
Resolved a problem where gpcheckcat would fail with Missing or extraneous entries check errors if the gp_sparse_vector extension was installed.
30637 - Query Optimizer
For some queries against partitioned tables, GPORCA did not perform partition elimination when a predicate that includes the partition column also performs an explicit cast. For example, GPORCA would not perform partition elimination when a partition column is defined with the timestamp datatype and the query contains a predicate such as WHERE ts::date == '2020-05-10' that performs a cast on the partitioned column (ts) to compare column data (a timestamp) to a date. GPORCA partition elimination has been improved to support the specified type of query. See Features.
10491 - Postgres Planner
For some queries that contain nested subqueries that do not specify a relation and also contain a nested GROUP BY clauses, Greenplum Database generated a PANIC. The PANIC was generated when Greenplum Database did not correctly manage the subquery correctly. This is an example of the specified type or query.
SELECT * FROM (SELECT * FROM (SELECT c1, SUM(c2) c2 FROM mytbl GROUP BY c1 ) t2 ) t3
  GROUP BY c2, ROLLUP((c1))
  ORDER BY 1, 2;
This issue is resolved.
10561 - Server
Greenplum Database does not support altering the datatype of a column defined as a distribution key or with a constraint. When attempting to change the datatype, the error message did not clearly indicate the cause. The error message has been altered to provide more information.
174505130 - Resource Groups
In some cases for a query managed by resource group, the resource group cancelled the query with the message Canceling query because of high VMEM usage because the resource group calculated the incorrect memory used by the query. This issue is resolved.
174353156 - Interconnect
In some cases when Greenplum Database uses proxies for interconnect communication (the server configuration parameter gp_interconnect_type is set to proxy), a Greenplum background worker process became an orphaned process after the postmaster process was terminated. This issue is resolved.
174205590 - Interconnect
When Greenplum Database uses proxies for interconnect communication (the server configuration parameter gp_interconnect_type is set to proxy), a query might have hung if the query contains multiple concurrent subplans running on the segment instances. The query hung when the Greenplum interconnect did not properly handle the communication among the concurrent subplans. This issue is resolved.
174483149 - Cluster Management - gpinitsystem
gpinitsystem now exports the MASTER_DATA_DIRECTORY environment variable before calling gpconfig, to avoid throwing warning messages when configuring system parameters on Greenplum Database appliances (DCA).

Upgrading from Greenplum 6.x to Greenplum 6.11

Note: Greenplum 6 does not support direct upgrades from Greenplum 4 or Greenplum 5 releases, or from earlier Greenplum 6 Beta releases.

See Upgrading from an Earlier Greenplum 6 Release to upgrade your existing Greenplum 6.x software to Greenplum 6.11.0.

Deprecated Features

Deprecated features will be removed in a future major release of Greenplum Database. Pivotal Greenplum 6.x deprecates:

  • The gpsys1 utility.
  • The analzyedb option --skip_root_stats (deprecated since 6.2).

    If the option is specified, a warning is issued stating that the option will be ignored.

  • The server configuration parameter gp_statistics_use_fkeys (deprecated since 6.2).
  • The server configuration parameter gp_ignore_error_table (deprecated since 6.0).

    To avoid a Greenplum Database syntax error, set the value of this parameter to true when you run applications that execute CREATE EXTERNAL TABLE or COPY commands that include the now removed Greenplum Database 4.3.x INTO ERROR TABLE clause.

  • Specifying => as an operator name in the CREATE OPERATOR command (deprecated since 6.0).
  • The Greenplum external table C API (deprecated since 6.0).

    Any developers using this API are encouraged to use the new Foreign Data Wrapper API in its place.

  • Commas placed between a SUBPARTITION TEMPLATE clause and its corresponding SUBPARTITION BY clause, and between consecutive SUBPARTITION BY clauses in a CREATE TABLE command (deprecated since 6.0).

    Using this undocumented syntax will generate a deprecation warning message.

  • The timestamp format YYYYMMDDHH24MISS (deprecated since 6.0).

    This format could not be parsed unambiguously in previous Greenplum Database releases, and is not supported in PostgreSQL 9.4.

  • The createlang and droplang utilities (deprecated since 6.0).
  • The pg_resqueue_status system view (deprecated since 6.0).

    Use the gp_toolkit.gp_resqueue_status view instead.

  • The GLOBAL and LOCAL modifiers when creating a temporary table with the CREATE TABLE and CREATE TABLE AS commands (deprecated since 6.0).

    These keywords are present for SQL standard compatibility, but have no effect in Greenplum Database.

  • Using WITH OIDS or oids=TRUE to assign an OID system column when creating or altering a table (deprecated since 6.0).
  • Allowing superusers to specify the SQL_ASCII encoding regardless of the locale settings (deprecated since 6.0).

    This choice may result in misbehavior of character-string functions when data that is not encoding-compatible with the locale is stored in the database.

  • The @@@ text search operator (deprecated since 6.0).

    This operator is currently a synonym for the @@ operator.

  • The unparenthesized syntax for option lists in the VACUUM command (deprecated since 6.0).

    This syntax requires that the options to the command be specified in a specific order.

  • The plain pgbouncer authentication type (auth_type = plain) (deprecated since 4.x).

Migrating Data to Greenplum 6

Note: Greenplum 6 does not support direct upgrades from Greenplum 4 or Greenplum 5 releases, or from earlier Greenplum 6 Beta releases.

See Migrating Data from Greenplum 4.3 or 5 for guidelines and considerations for migrating existing Greenplum data to Greenplum 6, using standard backup and restore procedures.

Known Issues and Limitations

Pivotal Greenplum 6 has these limitations:

  • Upgrading a Greenplum Database 4 or 5 release, or Greenplum 6 Beta release, to Greenplum 6 is not supported.
  • MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems.
  • Greenplum 6 is not supported for installation on DCA systems.
  • Greenplum for Kubernetes is not yet provided with this release.

The following table lists key known issues in Pivotal Greenplum 6.x.

Table 1. Key Known Issues in Pivotal Greenplum 6.x
Issue Category Description
N/A Backup/Restore Restoring the Greenplum Database backup for a table fails in Greenplum 6 versions earlier than version 6.10 when a replicated table has an inheritance relationship to/from another table that was assigned via an ALTER TABLE ... INHERIT statement after table creation.
Workaround: Use the following SQL commands to determine if Greenplum Database includes any replicated tables that inherit from a parent table, or if there are replicated tables that are inherited by a child table:
SELECT inhrelid::regclass FROM pg_inherits,
  gp_distribution_policy dp
WHERE inhrelid=dp.localoid AND dp.policytype='r';
SELECT inhparent::regclass FROM pg_inherits,
  gp_distribution_policy dp
WHERE inhparent=dp.localoid AND dp.policytype='r';

If these queries return any tables, you may choose to run gprestore with the -–on-error-continue flag to not fail the entire restore when this issue is hit. Or, you can specify the list of tables returned by the queries to the -–exclude-table-file option to skip those tables during restore. You must recreate and repopulate the affected tables after restore.

N/A Spark Connector This version of Greenplum is not compatible with Greenplum-Spark Connector versions earlier than version 1.7.0, due to a change in how Greenplum handles distributed transaction IDs.
N/A PXF Starting in 6.x, Greenplum does not bundle cURL and instead loads the system-provided library. PXF requires cURL version 7.29.0 or newer. The officially-supported cURL for the CentOS 6.x and Red Hat Enterprise Linux 6.x operating systems is version 7.19.*. Greenplum Database 6 does not support running PXF on CentOS 6.x or RHEL 6.x due to this limitation.

Workaround: Upgrade the operating system of your Greenplum Database 6 hosts to CentOS 7+ or RHEL 7+, which provides a cURL version suitable to run PXF.

29703 Loading Data from External Tables Due to limitations in the Greenplum Database external table framework, Greenplum Database cannot log the following types of errors that it encounters while loading data:
  • data type parsing errors
  • unexpected value type errors
  • data type conversion errors
  • errors returned by native and user-defined functions
LOG ERRORS returns error information for data exceptions only. When it encounters a parsing error, Greenplum terminates the load job, but it cannot log and propagate the error back to the user via gp_read_error_log().

Workaround: Clean the input data before loading it into Greenplum Database.

30594 Resource Management Resource queue-related statistics may be inaccurate in certain cases. VMware recommends that you use the resource group resource management scheme that is available in Greenplum 6.
30522 Logging Greenplum Database may write a FATAL message to the standby master or mirror log stating that the database system is in recovery mode when the instance is synchronizing with the master and Greenplum attempts to contact it before the operation completes. Ignore these messages and use gpstate -f output to determine if the standby successfully synchronized with the Greenplum master; the command returns Sync state: sync if it is synchronized.
30537 Postgres Planner The Postgres Planner generates a very large query plan that causes out of memory issues for the following type of CTE (common table expression) query: the WITH clause of the CTE contains a partitioned table with a large number partitions, and the WITH reference is used in a subquery that joins another partitioned table.

Workaround: If possible, use the GPORCA query optimizer. With the server configuration parameter optimizer=on, Greenplum Database attempts to use GPORCA for query planning and optimization when possible and falls back to the Postgres Planner when GPORCA cannot be used. Also, the specified type of query might require a long time to complete.

170824967 gpfidsts For Greenplum Database 6.x, a command that accesses an external table that uses the gpfdists protocol fails if the external table does not use an IP address when specifying a host system in the LOCATION clause of the external table definition. This issue is resolved in Greenplum 6.11.1.
n/a Materialized Views By default, certain gp_toolkit views do not display data for materialized views. If you want to include this information in gp_toolkit view output, you must redefine a gp_toolkit internal view as described in Including Data for Materialized Views.
168957894 PXF The PXF Hive Connector does not support using the Hive* profiles to access Hive transactional tables.

Workaround: Use the PXF JDBC Connector to access Hive.

168548176 gpbackup When using gpbackup to back up a Greenplum Database 5.7.1 or earlier 5.x release with resource groups enabled, gpbackup returns a column not found error for t6.value AS memoryauditor.
164791118 PL/R PL/R cannot be installed using the deprecated createlang utility, and displays the error:
createlang: language installation failed: ERROR:
no schema has been selected to create in
Workaround: Use CREATE EXTENSION to install PL/R, as described in the documentation.
N/A Greenplum Client/Load Tools on Windows The Greenplum Database client and load tools on Windows have not been tested with Active Directory Kerberos authentication.

Differences Compared to Open Source Greenplum Database

Pivotal Greenplum 6.x includes all of the functionality in the open source Greenplum Database project and adds:
  • Product packaging and installation script
  • Support for QuickLZ compression. QuickLZ compression is not provided in the open source version of Greenplum Database due to licensing restrictions.
  • Support for data connectors:
    • Greenplum-Spark Connector
    • Greenplum-Informatica Connector
    • Greenplum-Kafka Integration
    • Greenplum Streaming Server
  • Data Direct ODBC/JDBC Drivers
  • gpcopy utility for copying or migrating objects between Greenplum systems
  • Support for managing Greenplum Database using Pivotal Greenplum Command Center
  • Support for full text search and text analysis using Pivotal GPText
  • Greenplum backup plugin for DD Boost
  • Backup/restore storage plugin API