VMware Tanzu Greenplum 6.17 Release Notes
VMware Tanzu Greenplum 6.17 Release Notes
This document contains pertinent release information about VMware Tanzu Greenplum Database 6.17 releases. For previous versions of the release notes for Greenplum Database, go to VMware Tanzu Greenplum Database Documentation. For information about Greenplum Database end of life, see VMware Tanzu Greenplum Database end of life policy.
VMware Tanzu Greenplum 6 software is available for download from the VMware Tanzu Greenplum page on VMware Tanzu Network.
VMware Tanzu Greenplum 6 is based on the open source Greenplum Database project code.
Release Date: 2021-10-2
VMware Tanzu Greenplum 6.17.7 is a maintenance release that resolves a single issue.
The fix for issue 11308, added in Greenplum 6.16, introduced a regression that could cause incorrect results for queries having a bitmap index condition. This problem was resolved in 6.17.7 by reverting the original fix.
Release Date: 2021-9-25
VMware Tanzu Greenplum 6.17.6 is a maintenance release that resolves a single issue.
Incorrect bitmap index page data was written to WAL logs, resulting in bitmap index corruption after failover to mirror segments or after crash recovery on primaries. This issue has been resolved.
Release Date: 2021-9-22
VMware Tanzu Greenplum 6.17.5 is a maintenance release that resolves a single issue.
Postgres Query Planner
A previous fix to resolve duplicate sequence values (178253995) resulted in a performance regression. An incomplete hash key in the fix caused most hash table lookups to fail, which increased the time required to insert data having columns with sequence values. The problem was resolved by ensuring that the hash key contains complete information.
Release Date: 2021-9-21
VMware Tanzu Greenplum 6.17.4 is a maintenance release that resolves several issues.
VMware Tanzu Greenplum 6.17.4 resolves these issues:
A memory allocation function failed to handle a null pointer exception, which could lead to a system panic when creating the memory pools on Greenplum segments. This issue was resolved, by raising an exception if the memory allocation fails.
- A query plan with a nestled loop join and a dynamic table/index scan on the inner side of the join could cause an out of memory exception during a high concurrency workload. This was caused by an inefficient use of executor memory in the dynamic table/index scan operators. This issue has been resolved.
Segment failover caused bitmap index corruption and returned the error Invalid page in block 0 of relation. This was caused by the metapage not being present in the shared buffers. This issue has been resolved.
Release Date: 2021-8-30
VMware Tanzu Greenplum 6.17.3 is a maintenance release that resolves several issues.
VMware Tanzu Greenplum 6.17.3 resolves these issues:
In some cases, when processing queries that contained inner joins, the optimizer failed to process statistics, which resulted in a PANIC. This has been resolved.
- 31726, 31696
- A lock table corruption issue caused the Greenplum Database master segment to PANIC if a query waiting for a resource queue lock was cancelled, or if the query errored with a deadlock report. This issue is resolved.
Release Date: 2021-8-6
VMware Tanzu Greenplum 6.17.2 is a maintenance release that resolves several issues.
VMware Tanzu Greenplum 6.17.2 resolves these issues:
- This fix creates a single backout script when gpinitsystem runs so it is possible to clean up all segment directories in case of an error.
The gpcheckperf output displayed a NOTICE about deprecated flags for gpnetbench, however these flags are still used in netperf. This fix removes the NOTICE message.
- The command gpcheckcat now has a --help option.
For large clusters gpconfig returned the error filedescriptor out of range in select() due to the liveness check being performed on all hostnames associated with each Database ID. This fix addresses this issue by performing liveness check only on unique hostnames.
This version introduces a new type extension pljavat which only installs a language handler for the trusted language pljava and not the untrusted language pljavau.
- For queries that insert from a replicated table into a distributed table, the query optimizer could generate a plan that produced incorrect results. Specifically, for operations involving a SEQUENCE function on the replicated table, the result was not guaranteed to be identical across different segments. This issue was resolved by identifying these scenarios and redistributing tuples from a single segment, rather than exploiting the advantages of replicated table.
- The query optimizer generated a better plan with optimizer_join_order set to "greedy" in comparison to the default value of "exhaustive." This occurred because the default join order algorithm was partial to finding a plan that exploited dynamic partition elimination (DPE) and sometimes ignoring cases where both static and dynamic partition elimination would improve plan performance. This was done to prevent increasing the optimization time significantly at the expense of a sub-optimal plan. To avoid generating sub-optimal plans, the query optimizer now generates both partition elimination options but only for the greedy algorithm, regardless of the join order chosen. This helps to generate some lower-cost alternatives while incurring only a moderate increase in optimization time (15%).
- Creating a temp table with DISTRIBUTED RANDOMLY from a DISTRIBUTED REPLICATED table put all records into one instance when the query optimizer was enabled. This issue was resolved by adding the necessary logic to insure that, when inserting from a replicated table into a randomly-distributed table, the query optimizer randomizes the tuples to data skew.
- GPORCA could generate wrong results if a query used both a WITH clause and a join condition across different output columns of the same derived tables. The problem occurred because the distribution policy created for the WITH clause subquery was incorrectly propagated. GPORCA has resolved this problem by pushing properties into the WITH clause only when all columns come from the same derived table.
- Incorrectly estimating cardinality for casted columns resulted in a bad join order, which could lead to a query that did not finish. This issue was resolved by updating the cardinality estimation model when the input histogram is not well formed, as is the case when columns are casted.
- Resolved an issue where a database server process was not properly holding the partitionLock and ResQueueLock after releasing a resource queue lock following a deadlock error, which resulted in a segmentation fault.
- Resolved an issue where gpload would fail with column names that used uppercase or mixed-case characters. gpload now automatically adds double quotes to column names that are not already quoted in the YAML control file.
Release Date: 2021-7-23
VMware Tanzu Greenplum 6.17.1 is a maintenance release that resolves several issues.
VMware Tanzu Greenplum 6.17.1 resolves these issues:
- Fixed a problem where the Postgres Planner failed to consider aggregates on the entry flow. This could cause queries to fail with: ERROR: MIN/MAX subplan has unexpected flowtype .
During a left outer join query, with an index on the join column, the Query Optimizer explored a plan search space that hit an exception, was unable to generate a plan, and reverted to the Postgres Planner. This issue has been fixed by changing the requested collocation property, and eliminating the invalid search space.
- The query optimizer added an unnecessary Explicit Redistribute Motion when generating a plan for queries using DELETE on a randomly distributed table. This issue has been resolved.
Resolves a data corruption issue that could occur with concurrent DML queries on AO/AOCO tables when Greenplum Database was not aware of committed transactions that started after snapshot initiation.
Release Date: 2021-7-9
VMware Tanzu Greenplum 6.17.0 is a minor release that includes feature changes and resolves several issues.
Greenplum Database 6.17.0 includes these new and changed features:
- The PXF version 6.1.0 distribution is available with this release; you can download it from the Release Download directory named Greenplum Platform Extension Framework on VMware Tanzu Network. Refer to the PXF documentation for information about this release and for installation and upgrade instructions.
- The gprecoverseg, gpaddmirrors, and gpmovemirrors utilities now include a -b option to specify the maximum number of segments per host to operate on in parallel.
- The gpecheckcat utility now permits users to skip one or more tests, with the new -s option. In addition, the -R option now accepts a comma-separated list of of multiple tests to run.
- The Progress DataDirect JDBC Driver v6.0.0+181 is included in this release. See the Readme file for a list of new features available in v6.0.0
VMware Tanzu Greenplum 6.17.0 resolves these issues:
Resolved an issue where the message, An exception was encountered during the execution of a statement was unable to be suppressed. Users can now control the verbosity of the exception message with the log_min_messages GUC.
- Resolved an out of memory condition that could occur because the server held certain data contexts longer than necessary.
- When altering a role from superuser to non-superuser, the role resource group was not changing from admin_group to default_group. This fix addresses this issue.
- Resolved an issue where issuing a UPDATE/DELETE statement while a VACUUM operation was running returned the error tuple concurrently updated, due to an out of date snapshot.
- Fixed the error unrecognized parameter "appendoptimized" that was reported when using the appendoptimized clause for a table partition definition.
- Resolved a bug that caused the database to crash when running queries against tables that had not been redistributed after an expansion.
- Resolved an issue where a CTAS statement referencing a table created with the DISTRIBUTED REPLICATED clause was generating incorrect results with the Legacy Planner.
- For partitioned tables with indexes, ORCA would cause a PANIC while trying to create an index plan, when root and leaf partitions did not have the same underlying dropped column structure. This issue has been resolved.
Running queries with the GPORCA query optimizer that used an INNER JOIN with an EXCEPT clause was causing a fallback to Legacy Planner and reporting the error: No plan has been computed for required properties. This issue has been resolved.
- Resolved an issue where running queries with a LEFT OUTER JOIN were producing incorrect results as it was being replaced with an INNER JOIN in the subquery plan.
- Resolved an issue where running a query using UNION ALL and an external table was returning incorrect results with GPORCA query optimizer, due to not all information from the external table being gathered.
- The ORCA Query Optimizer was unnecessarily opting out of certain optimizations when it encountered a NO SQL function. It no longer does that.
- Introduced a fix to verify that materialize does not project in ORCA Query Optimizer.
- Resolved an issue with array overflow for function argument types which was causing memory corruption and a database crash with ORCA Query Optimizer.
- Resolved a bug in the implementation of Left Outer Joins that was causing a database crash when using ORCA Query Optimizer.
- Resolved an issue where gprecoverseg -r -- when it called gprescoverseg internally a second time -- did not correctly pass on the -B and -b options.
- Resolved an issue where gprecoverseg was updating the pg_hba.conf file on all primaries regardless of whether those primaries needed to be updated.
- Resolved an issue where gpmovemirrors -B was calling gprecoverseg with an excessively large number of parallel processes.
- 31338, 31351
- Resolved an issue where old mirror directories were not being removed from old tablespace directories after a mirror was moved using gpmovemirrors.
- Resolved an issue where gprecoverseg was printing out empty pg_rewind progress lines for mirrors.
gprecoverseg no longer adds pg_hba.conf entries if they are already present in the file.
For all GUCs with a vartype of string, you may no longer enclose the value you pass to gpconfig -c in single quotes.
- Resolved an issue where gprecoverseg -p was reporting an error despite the operation being successful.
- Resolved an issue where gprecoverseg -p would error out when master could not connect to an individual segment host.
Postgres Query Planner
Resolved an issue where an error occurred when the planner was executing a query that performed a UNION ALL between a replicated table and a subquery with an explicit sequence nextval.
- Resolved an issue where the sequence executor node was generating duplicate sequence values.
Upgrading from Greenplum 6.x to Greenplum 6.17
See Upgrading from an Earlier Greenplum 6 Release to upgrade your existing Greenplum 6.x software to Greenplum 6.17.
Deprecated features will be removed in a future major release of Greenplum Database. VMware Tanzu Greenplum 6.x deprecates:
- The gpsys1 utility.
- The analzyedb option --skip_root_stats (deprecated
If the option is specified, a warning is issued stating that the option will be ignored.
- The server configuration parameter gp_statistics_use_fkeys (deprecated since 6.2).
- The server configuration parameter gp_ignore_error_table (deprecated
To avoid a Greenplum Database syntax error, set the value of this parameter to true when you run applications that execute CREATE EXTERNAL TABLE or COPY commands that include the now removed Greenplum Database 4.3.x INTO ERROR TABLE clause.
- Specifying => as an operator name in the CREATE OPERATOR command (deprecated since 6.0).
- The Greenplum external table C API (deprecated since 6.0).
Any developers using this API are encouraged to use the new Foreign Data Wrapper API in its place.
- Commas placed between a SUBPARTITION TEMPLATE clause and its
corresponding SUBPARTITION BY clause, and between consecutive
SUBPARTITION BY clauses in a CREATE TABLE command
(deprecated since 6.0).
Using this undocumented syntax will generate a deprecation warning message.
- The timestamp format YYYYMMDDHH24MISS (deprecated since 6.0).
This format could not be parsed unambiguously in previous Greenplum Database releases, and is not supported in PostgreSQL 9.4.
- The createlang and droplang utilities (deprecated since 6.0).
- The pg_resqueue_status system view (deprecated since 6.0).
Use the gp_toolkit.gp_resqueue_status view instead.
- The GLOBAL and LOCAL modifiers when creating a
temporary table with the CREATE TABLE and CREATE TABLE
AS commands (deprecated since 6.0).
These keywords are present for SQL standard compatibility, but have no effect in Greenplum Database.
- Using WITH OIDS or oids=TRUE to assign an OID system column when creating or altering a table (deprecated since 6.0).
- Allowing superusers to specify the SQL_ASCII encoding regardless of the
locale settings (deprecated since 6.0).
This choice may result in misbehavior of character-string functions when data that is not encoding-compatible with the locale is stored in the database.
- The @@@ text search operator (deprecated since 6.0).
This operator is currently a synonym for the @@ operator.
- The unparenthesized syntax for option lists in the VACUUM command
(deprecated since 6.0).
This syntax requires that the options to the command be specified in a specific order.
- The plain pgbouncer authentication type (auth_type = plain) (deprecated since 4.x).
Migrating Data to Greenplum 6
Greenplum 6 supports direct upgrades, using gpupgrade, from Greenplum 5.x releases to Greenplum 6.x. For more information, see the gpupgrade documentation. See also Migrating Data from Greenplum 4.3 or 5 for guidelines and considerations for migrating existing Greenplum data to Greenplum 6, using standard backup and restore procedures.
Known Issues and Limitations
VMware Tanzu Greenplum 6 has these limitations:
- Upgrading a Greenplum Database 4 release to VMware Tanzu Greenplum 6 is not supported. Upgrading a Greenplum Database 5.x release to VMware Tanzu Greenplum 6 is supported via gpupgrade. For more information, see the gpupgrade documentation.
- MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems.
- Greenplum for Kubernetes is not yet provided with this release.
The following table lists key known issues in VMware Tanzu Greenplum 6.x.
|11308||Server||A bitmap index scan running concurrently with an index INSERT on a full bitmap page, occasionally fails to read the correct tid.|
|31010||Query Optimizer||A view created in Greenplum Database 5.28.3 or older that
specified an external table in the FROM clause,
and that was migrated to Greenplum Database 6.x, always falls
back to the Postgres Planner when queried.
Workaround: If you migrated a view from Greenplum Database 5.28.3 or earlier, and the view specified an external table in the FROM clause, you must drop and recreate the view in Greenplum 6.x to ensure that the Query Optimizer is exercised when you query the view.
|N/A||Backup/Restore||Restoring the Greenplum Database backup for a table fails in
Greenplum 6 versions earlier than version 6.10 when a replicated
table has an inheritance relationship to/from another table that
was assigned via an ALTER TABLE ... INHERIT
statement after table creation.
Workaround: Use the following SQL commands to determine if Greenplum Database includes any replicated tables that inherit from a parent table, or if there are replicated tables that are inherited by a child table:
SELECT inhrelid::regclass FROM pg_inherits, gp_distribution_policy dp WHERE inhrelid=dp.localoid AND dp.policytype='r'; SELECT inhparent::regclass FROM pg_inherits, gp_distribution_policy dp WHERE inhparent=dp.localoid AND dp.policytype='r';
If these queries return any tables, you may choose to run gprestore with the -–on-error-continue flag to not fail the entire restore when this issue is hit. Or, you can specify the list of tables returned by the queries to the -–exclude-table-file option to skip those tables during restore. You must recreate and repopulate the affected tables after restore.
|N/A||Spark Connector||This version of Greenplum is not compatible with Greenplum-Spark Connector versions earlier than version 1.7.0, due to a change in how Greenplum handles distributed transaction IDs.|
|N/A||PXF||Starting in 6.x, Greenplum does not bundle cURL and instead
loads the system-provided library. PXF requires cURL version 7.29.0
or newer. The officially-supported cURL for the CentOS 6.x and Red
Hat Enterprise Linux 6.x operating systems is version 7.19.*. Greenplum Database 6
does not support running PXF on CentOS 6.x or RHEL 6.x due to this limitation.
Workaround: Upgrade the operating system of your Greenplum Database 6 hosts to CentOS 7+ or RHEL 7+, which provides a cURL version suitable to run PXF.
|29703||Loading Data from External Tables||Due to limitations in the Greenplum Database external table framework,
Greenplum Database cannot log the following types of errors that it encounters while
Workaround: Clean the input data before loading it into Greenplum Database.
|30594||Resource Management||Resource queue-related statistics may be inaccurate in certain cases. VMware recommends that you use the resource group resource management scheme that is available in Greenplum 6.|
|30522||Logging||Greenplum Database may write a FATAL message to the standby master or mirror log stating that the database system is in recovery mode when the instance is synchronizing with the master and Greenplum attempts to contact it before the operation completes. Ignore these messages and use gpstate -f output to determine if the standby successfully synchronized with the Greenplum master; the command returns Sync state: sync if it is synchronized.|
|30537||Postgres Planner||The Postgres Planner generates a very large query plan that causes out of
memory issues for the following type of CTE (common table expression) query: the
WITH clause of the CTE contains a partitioned table with a large
number partitions, and the WITH reference is used in a subquery
that joins another partitioned table.
Workaround: If possible, use the GPORCA query optimizer. With the server configuration parameter optimizer=on, Greenplum Database attempts to use GPORCA for query planning and optimization when possible and falls back to the Postgres Planner when GPORCA cannot be used. Also, the specified type of query might require a long time to complete.
|170824967||gpfdists||For Greenplum Database 6.x, a command that accesses an external table that uses the gpfdists protocol fails if the external table does not use an IP address when specifying a host system in the LOCATION clause of the external table definition.|
|n/a||Materialized Views||By default, certain gp_toolkit views do not display data for materialized views. If you want to include this information in gp_toolkit view output, you must redefine a gp_toolkit internal view as described in Including Data for Materialized Views.|
|168957894||PXF||The PXF Hive Connector does not support using the Hive*
profiles to access Hive transactional tables.
Workaround: Use the PXF JDBC Connector to access Hive.
|168548176||gpbackup||When using gpbackup to back up a Greenplum Database 5.7.1 or earlier 5.x release with resource groups enabled, gpbackup returns a column not found error for t6.value AS memoryauditor.|
|164791118||PL/R||PL/R cannot be installed using the deprecated createlang
utility, and displays the
createlang: language installation failed: ERROR: no schema has been selected to create inWorkaround: Use CREATE EXTENSION to install PL/R, as described in the documentation.
|N/A||Greenplum Client/Load Tools on Windows||The Greenplum Database client and load tools on Windows have not been tested with Active Directory Kerberos authentication.|
Differences Compared to Open Source Greenplum Database
- Product packaging and installation script
- Support for QuickLZ compression. QuickLZ compression is not provided in the open source version of Greenplum Database due to licensing restrictions.
- Support for data connectors:
- Greenplum-Spark Connector
- Greenplum-Informatica Connector
- Greenplum-Kafka Integration
- Greenplum Streaming Server
- Data Direct ODBC/JDBC Drivers
- Greenplum PostGIS Extension
- gpcopy utility for copying or migrating objects between Greenplum systems
- Support for managing Greenplum Database using VMware Tanzu Greenplum Command Center
- Support for full text search and text analysis using VMware Tanzu GPText
- Greenplum backup plugin for DD Boost
- Backup/restore storage plugin API