Pivotal Greenplum 6.4 Release Notes
A newer version of this documentation is available. Use the version menu above to view the most up-to-date release of the Greenplum 6.x documentation.
Pivotal Greenplum 6.4 Release Notes
This document contains pertinent release information about Pivotal Greenplum Database 6.4 releases. For previous versions of the release notes for Greenplum Database, go to Pivotal Greenplum Database Documentation. For information about Greenplum Database end of life, see Pivotal Greenplum Database end of life policy.
Pivotal Greenplum 6 software is available for download from the Pivotal Greenplum page on Pivotal Network.
Pivotal Greenplum 6 is based on the open source Greenplum Database project code.
Release Date: 2020-02-11
Pivotal Greenplum 6.4.0 is a maintenance release that includes changed features and resolves several issues.
Greenplum Database 6.4.0 includes these changed features:
- DISCARD ALL is not supported. The command returns a message that states that the command is not supported and to consider alternatives such as DEALLOCATE ALL or DISCARD TEMP. See DISCARD.
- Greenplum Database resource groups support automatic query termination when resource
group global shared memory is enabled. For resource groups that use global shared
memory, Greenplum Database gracefully cancels running queries that are managed by those
resource groups when the queries consume a large portion of the global shared memory. At
this point, the queries would have already consumed available resource group slot memory
and group shared memory. Greenplum Database cancels queries in order of memory used,
from highest to lowest until the percentage of utilized global shared memory is below
the percentage specified by the server configuration parameter runaway_detector_activation_percent.
Global shared memory is enabled for resource groups that are configured to use the vmtracker memory auditor, such as admin_group and default_group when the sum of the MEMORY_LIMIT attribute values configured for all resource groups is less than 100. If resource groups are not configured to use global shared memory pool, the resource group automatic query termination feature is not enabled.
For information about resource groups, see Using Resource Groups.
- Greenplum Database supports creating the standby master or mirror segment instances on hosts that are in a different subnet from the master and primary segment instance hosts. Mirror segment instances can also be moved to hosts that are in a different subnet.
Pivotal Greenplum 6.4.0 resolves these issues:
- 30209, 170762049 - gpaddmirrors
- The gpaddmirrors utility failed to add mirror segments to a Greenplum Database system, when the mirror segments are in a different subnet from the primary segments. This issue is resolved. Now Greenplum Database supports mirror segments in a different subnet from the primary segments.
- 30266 - Vacuum
- The vacuum workflow changed in Greenplum Database 6 to dispatch once per auxiliary table in addition to the dispatch for the main table. This could cause performance problems and permission verification failures on auxiliary tables. This is fixed. The VACUUM command again dispatches once per segment.
- 30282 - Resource Management
- For some queries managed by Greenplum Database resource groups, the query failed due to an out of memory condition. This caused a segment failure and other issues. This issue is resolved. Resource groups have been enhanced to handle queries that consume a large amount of memory. See Changed Features.
- 30299 - Server: Execution
- For some queries against a partitioned table with multiple column indexes that perform a dynamic index scan, Greenplum Database generated a SIGSEGV. The error occurred when Greenplum Database did not correctly manage dynamic index scan pointer memory. This issue is resolved.
- 30325 - Query Optimizer
- For some queries that reference a view that is defined with a CTE (common table expression) query, and the main query also contains a predicate that is a subquery that references the view, GPORCA returns an error that states could not open temporary file. The error was caused by incorrect predicate pushdown during preprocessing. This issue is resolved.
- 30359 - Server: Segment Mirroring
- A panic occurred during crash recovery for a mirror when replaying WAL records for a transaction that created an append-optimized table, truncated the table, and then aborted the transaction. This issue is resolved.
- 30354 - DISCARD
- DISCARD TEMP might not drop temporary tables on segment instances. This issue is resolved.
- 30360 - Server: Security
- Greenplum Database incorrectly logged the message time constraints added on superuser role every time a superuser role was checked. Now the message is logged only when time constraints are added to a superuser role.
- 30366 - Server: Execution
- Using GRANT commands on partitioned tables across segments would cause the system to PANIC due to alterations in the cached plan. This issue has been resolved.
- 30371 - Server: Execution
- The to_timestamp() function did not return an error for an out of range value. For example, select to_timestamp('20200340123456','YYYYMMDDHH24MISS'); returned a valid date and time 2020-04-09 12:34:56-07. This issue is resolved. Now the function returns an error.
- 30387 - Server: DML
- After completing the execution of a user-defined function that changed the value of a server configuration parameter, Greenplum Database restored the original parameter value on the query dispatcher, but did not synchronize and restore the value to the query executors. This issue is resolved.
- 170787232 - Server: Query Dispatcher
- A Query Dispatcher (QD) would run into local deadlock for some prepared statements that execute an UPDATE or DELETE command. This issue has been fixed by taking into consideration the server configuration parameter gp_enable_global_deadlock_detector.
Upgrading to Greenplum 6.4.0
See Upgrading from an Earlier Greenplum 6 Release to upgrade your existing Greenplum 6.x software to Greenplum 6.4.0.
Deprecated features will be removed in a future major release of Greenplum Database. Pivotal Greenplum 6.x deprecates:
- The analzyedb option --skip_root_stats (deprecated
If the option is specified, a warning is issued stating that the option will be ignored.
- The server configuration parameter gp_statistics_use_fkeys (deprecated since 6.2).
- The following PXF configuration properties (deprecated since 6.2):
- The PXF_USER_IMPERSONATION, PXF_PRINCIPAL, and PXF_KEYTAB settings in the pxf-env.sh file. You can use the pxf-site.xml file to configure Kerberos and impersonation settings for your new Hadoop server configurations.
- The pxf.impersonation.jdbc property setting in the jdbc-site.xml file. You can use the pxf.service.user.impersonation property to configure user impersonation for a new JDBC server configuration.
- The server configuration parameter gp_ignore_error_table
(deprecated since 6.0).
To avoid a Greenplum Database syntax error, set the value of this parameter to true when you run applications that execute CREATE EXTERNAL TABLE or COPY commands that include the now removed Greenplum Database 4.3.x INTO ERROR TABLE clause.
- Specifying => as an operator name in the CREATE OPERATOR command (deprecated since 6.0).
- The Greenplum external table C API (deprecated since 6.0).
Any developers using this API are encouraged to use the new Foreign Data Wrapper API in its place.
- Commas placed between a SUBPARTITION TEMPLATE clause and its
corresponding SUBPARTITION BY clause, and between consecutive
SUBPARTITION BY clauses in a CREATE TABLE
command (deprecated since 6.0).
Using this undocumented syntax will generate a deprecation warning message.
- The timestamp format YYYYMMDDHH24MISS (deprecated since 6.0).
This format could not be parsed unambiguously in previous Greenplum Database releases, and is not supported in PostgreSQL 9.4.
- The createlang and droplang utilities (deprecated since 6.0).
- The pg_resqueue_status system view (deprecated since 6.0).
Use the gp_toolkit.gp_resqueue_status view instead.
- The GLOBAL and LOCAL modifiers when
creating a temporary table with the CREATE TABLE and
CREATE TABLE AS commands (deprecated since 6.0).
These keywords are present for SQL standard compatibility, but have no effect in Greenplum Database.
- The Greenplum Platform Extension Framework (PXF) HDFS profile names
for the Text, Avro, JSON, Parquet, and SequenceFile data formats
(deprecated since 5.16).
Refer to Connectors, Data Formats, and Profiles in the PXF Hadoop documentation for more information.
- Using WITH OIDS or oids=TRUE to assign an OID system column when creating or altering a table (deprecated since 6.0).
- Allowing superusers to specify the SQL_ASCII encoding
regardless of the locale settings (deprecated since 6.0).
This choice may result in misbehavior of character-string functions when data that is not encoding-compatible with the locale is stored in the database.
- The @@@ text search operator (deprecated since 6.0).
This operator is currently a synonym for the @@ operator.
- The unparenthesized syntax for option lists in the VACUUM command
(deprecated since 6.0).
This syntax requires that the options to the command be specified in a specific order.
- The plain pgbouncer authentication type (auth_type = plain) (deprecated since 4.x).
Migrating Data to Greenplum 6
See Migrating Data from Greenplum 4.3 or 5 for guidelines and considerations for migrating existing Greenplum data to Greenplum 6, using standard backup and restore procedures.
Known Issues and Limitations
Pivotal Greenplum 6 has these limitations:
- Upgrading a Greenplum Database 4 or 5 release, or Greenplum 6 Beta release, to Pivotal Greenplum 6 is not supported.
- MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems.
- Greenplum 6 is not supported for installation on DCA systems.
- Greenplum for Kubernetes is not yet provided with this release.
The following table lists key known issues in Pivotal Greenplum 6.x.
|N/A||Metrics collector||Setting the gpcc_enable_query_profiling configuration
parameter to on may cause Greenplum Database to crash. This affects Greenplum
Database versions 6.3, 6.4, and 6.5.
Workaround: Install Greenplum Command Center 6.2.0 and upgrade the metrics_collector extension with the gppkg package included with Greenplum Command Center 6.2.0.
|N/A||PXF||Starting in 6.x, Greenplum does not bundle cURL and instead
loads the system-provided library. PXF requires cURL version 7.29.0
or newer. The officially-supported cURL for the CentOS 6.x and Red
Hat Enterprise Linux 6.x operating systems is version 7.19.*. Greenplum Database 6
does not support running PXF on CentOS 6.x or RHEL 6.x due to this limitation.
Workaround: Upgrade the operating system of your Greenplum Database 6 hosts to CentOS 7+ or RHEL 7+, which provides a cURL version suitable to run PXF.
|N/A||Logging||In some cases, it is safe to ignore certain FATAL
messages that Greenplum Database writes to the logs:
|29703||Loading Data from External Tables||Due to limitations in the Greenplum Database external table framework,
Greenplum Database cannot log the following types of errors that it encounters while
Workaround: Clean the input data before loading it into Greenplum Database.
|30522||Logging||Greenplum Database may write a FATAL message to the standby master or mirror log stating that the database system is in recovery mode when the instance is synchronizing with the master and Greenplum attempts to contact it before the operation completes. Ignore these messages and use gpstate -f output to determine if the standby successfully synchronized with the Greenplum master; the command returns Sync state: sync if it is synchronized.|
|170824967||gpfidsts||For Greenplum Database 6.x, a command that accesses an external table that uses the gpfdists protocol fails if the external table does not use an IP address when specifying a host system in the LOCATION clause of the external table definition.|
|n/a||Materialized Views||By default, certain gp_toolkit views do not display data for materialized views. If you want to include this information in gp_toolkit view output, you must redefine a gp_toolkit internal view as described in Including Data for Materialized Views.|
|168689202||PXF||PXF fails to run any query on Java 11 that specifies a
Hive* profile due to this Hive known issue:
ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer.
Workaround: Run PXF on Java 8 or use the PXF JDBC Connector to access Hive.
|168957894||PXF||The PXF Hive Connector does not support using the
Hive* profiles to access Hive transactional tables.
Workaround: Use the PXF JDBC Connector to access Hive.
|169200795||Greenplum Stream Server||When loading Kafka data into Greenplum Database in UPDATE and MERGE modes, GPSS requires that a MAPPING exist for each column name identified in the MATCH_COLUMNS and UPDATE_COLUMNS lists.|
|170202002||Greenplum-Kafka Integration||Updating the METADATA:SCHEMA property and restarting a previously-run load job could cause gpkafka to re-read Kafka messages published to the topic, and load duplicate messages into Greenplum Database.|
|168548176||gpbackup||When using gpbackup to back up a Greenplum Database 5.7.1 or earlier 5.x release with resource groups enabled, gpbackup returns a column not found error for t6.value AS memoryauditor.|
|164791118||PL/R||PL/R cannot be installed using the deprecated createlang
utility, and displays the
createlang: language installation failed: ERROR: no schema has been selected to create inWorkaround: Use CREATE EXTENSION to install PL/R, as described in the documentation.
|30437||JDBC Driver||JDBC cached query plans do not store the partition selector parameter that is
required for performing partition elimination. If you create a JDBC prepared
statement that operates against a partitioned table, partition elimination is
performed the first 5 times the query is executed. After that point, the JDBC driver
may choose to cache the query plan, in which case partition elimination is no longer
performed and the query may suffer from extremely degraded
Workaround: Use the DataDirect JDBC driver version 5.1.4.000270 (F000450.U000214) or later, introduced in Greenplum 6.1, and set the prepareThreshold connection parameter to a very large value. For example: jdbc:Pivotal:greenplum://<ip>:<port>;DatabaseName=<name>;prepareThreshold=1000000000 See PrepareThreshold in the DataDirect documentation.
|9460||CREATE UNIQUE INDEX||When you create a unique index on a partitioned table, Greenplum Database does not check to ensure that the index contains the table partition keys, which are required to enforce uniqueness. Workaround: To avoid duplicate rows, manually specify partition keys when executing CREATE UNIQUE INDEX on a partitioned table.|
|N/A||Greenplum Client/Load Tools on Windows||The Greenplum Database client and load tools on Windows have not been tested with Active Directory Kerberos authentication.|
Differences Compared to Open Source Greenplum Database
- Product packaging and installation script
- Support for QuickLZ compression. QuickLZ compression is not provided in the open source version of Greenplum Database due to licensing restrictions.
- Support for data connectors:
- Greenplum-Spark Connector
- Greenplum-Informatica Connector
- Greenplum-Kafka Integration
- Greenplum Stream Server
- Data Direct ODBC/JDBC Drivers
- gpcopy utility for copying or migrating objects between Greenplum systems
- Support for managing Greenplum Database using Pivotal Greenplum Command Center
- Support for full text search and text analysis using Pivotal GPText
- Greenplum backup plugin for DD Boost
- Backup/restore storage plugin API (Beta)