Release Notes

Release Notes

The Tanzu Greenplum Backup and Restore utilities are released and updated independently of the Tanzu Greenplum release (from Greenplum Backup and Restore version 1.13.0).

This topic includes:

Version 1.20.4

Release Date: May 07, 2021

Greenplum Backup and Restore version 1.20.4 is a maintenance release that fixes one issue.

Supported Platforms

Greenplum Backup and Restore is compatible with these Tanzu Greenplum versions:

  • Tanzu Greenplum 4.3.22 and later
  • Tanzu Greenplum 5.5.0 and later
  • Tanzu Greenplum 6.0.0 and later

Software Component Versions

This release includes the following utilities:

See Data Domain Boost for Data Domain Boost support information.

gpbackup v1.20.4

Resolved Issues

  • [31436] : Resolves an issue in which user-defined functions defined with EXECUTE ON INITPLAN were missing the Execute on attribute value after a restore.

gpbackup_helper v1.20.4

No changes since Tanzu Greenplum Backup and Restore version 1.14.0.

gprestore v1.20.4

No changes since Tanzu Greenplum Backup and Restore version 1.20.3

gpbackup_manager v1.3.1

No changes since Greenplum Backup and Restore version 1.20.0.

Backup/Restore Plugin API

No changes since Greenplum Backup and Restore version 1.19.0.

gpbackup_ddboost_plugin v1.5.0

No changes since Greenplum Backup and Restore version 1.19.0.

gpbackup_s3_plugin v1.6.0

No changes since Greenplum Backup and Restore version 1.20.2.

Version 1.20.3

Release Date: March 11, 2021

Greenplum Backup and Restore version 1.20.3 is a maintenance release that fixes two issues.

Supported Platforms

Greenplum Backup and Restore is compatible with these Tanzu Greenplum versions:

  • Tanzu Greenplum 4.3.22 and later
  • Tanzu Greenplum 5.5.0 and later
  • Tanzu Greenplum 6.0.0 and later

Software Component Versions

This release includes the following utilities:

See Data Domain Boost for Data Domain Boost support information.

gpbackup v1.20.3

Resolved Issues

  • [176969167] : Resolves an issue where gpbackup would generate multiple index entries for a table where the combined length of the column name and table name exceeds 63 characters
  • [177050466]: For views containing anyarray type casts in the view definition, which cannot be restored, gpbackup now detects and generates an error message similar to: Detected anyarray type cast in view definition for View 'public.fooview': Drop the view or recreate the view without explicit array type casts. For workaround suggestions, refer to the Known Issues.

gpbackup_helper v1.20.3

No changes since Tanzu Greenplum Backup and Restore version 1.14.0.

gprestore v1.20.3

No changes since Tanzu Greenplum Backup and Restore version 1.20.2

gpbackup_manager v1.3.1

No changes since Greenplum Backup and Restore version 1.20.0.

Backup/Restore Plugin API

No changes since Greenplum Backup and Restore version 1.19.0.

gpbackup_ddboost_plugin v1.5.0

No changes since Greenplum Backup and Restore version 1.19.0.

gpbackup_s3_plugin v1.6.0

No changes since Greenplum Backup and Restore version 1.20.2.

Version 1.20.2

Release Date: January 22, 2021

Greenplum Backup and Restore version 1.20.2 is a maintenance release that fixes several issues.

Supported Platforms

Greenplum Backup and Restore is compatible with these Tanzu Greenplum versions:

  • Tanzu Greenplum 4.3.22 and later
  • Tanzu Greenplum 5.5.0 and later
  • Tanzu Greenplum 6.0.0 and later

Software Component Versions

This release includes the following utilities:

See Data Domain Boost for Data Domain Boost support information.

gpbackup v1.20.2

Resolved Issues

  • [31036] Querying the internal catalogs of external tables with error logging enabled could generate an error similar to ERROR: Canceling query because of high VMEM usage. This issue has been resolved.
  • [175696310] Resolves an issue where gpbackup could omit the LOG ERRORS PERSISTENTLY option during the backup of external tables configured with persistent error logging, in Tanzu Greenplum 5.x and above.
  • [175453403] When a SIGTERM event occurred during a backup operation with --single-data-file, the Greenplum Database segments could end up with uncleaned residual gpbackup_helper processes. This issue has been resolved.
  • [175323370] Resolves a misleading error message occuring when a provided backup timestamp did not exist in the supplied --backup-dir directory.
  • [31040/176387978] Addresses an issue in v1.20.1 where the number of locks aquired by gpbackup, when using the --jobs flag, could result in [CRITICAL]:-ERROR: out of shared memory (SQLSTATE 53200). Also resolved a deadlock issue in v.1.20.0 and earlier, between COPY and outside statements such as TRUNCATE, DROP, and ALTER when using the --jobs flag. For further details, see the --jobs section in the gpbackup reference page.

gpbackup_helper v1.20.2

No changes since Tanzu Greenplum Backup and Restore version 1.14.0.

gprestore v1.20.2

Changed Feature

  • The gprestore application name, used in system catalog views like pg_stat_activity, now includes both the backup and restore time stamps. The new name helps identify separate restore operations that are running concurrently. For example, in gprestore_20201103085529_20201103085630, 20201103085529 is the backup time stamp, and 20201103085630 the restore timestamp.

Resolved Issues

  • [175453403] When a SIGTERM event occurred during a restore operation with --single-data-file, the Greenplum Database segments could end up with uncleaned residual gpbackup_helper processes. This issue has been resolved.

gpbackup_manager v1.3.1

No changes since Greenplum Backup and Restore version 1.20.0.

Backup/Restore Plugin API

No changes since Greenplum Backup and Restore version 1.19.0.

gpbackup_ddboost_plugin v1.5.0

  • This version supports Data Domain OS 7.3 and Data Domain OS 7.2.

No feature changes since Greenplum Backup and Restore version 1.19.0.

gpbackup_s3_plugin v1.6.0

New Features

  • [34] Introduces two tuning parameters, restore_max_concurrent_requests and restore_multipart_chunksize, which can be set in the plugin configuration yaml file. For more details see S3 Storage Plugin Configuration File Format.

Version 1.20.1

Release Date: December 4, 2020

Warning: The 1.20.1 release has been removed from Tanzu Network due to a discovered regression. When running gpbackup with the --jobs flag, the available shared memory of the node might be exhausted, and gpbackup might fail with the following error: [CRITICAL]:-ERROR: out of shared memory (SQLSTATE 53200). If you're using version 1.20.1, use gpbackup without the --jobs flag; otherwise downgrade to version 1.20.0

Greenplum Backup and Restore version 1.20.1 was released as a maintenance release.

Supported Platforms

Greenplum Backup and Restore is compatible with these Tanzu Greenplum versions:

  • Tanzu Greenplum 4.3.22 and later
  • Tanzu Greenplum 5.5.0 and later
  • Tanzu Greenplum 6.0.0 and later

Software Component Versions

This release includes the following utilities:

See Data Domain Boost for Data Domain Boost support information.

gpbackup v1.20.1

Resolved Issues

  • [31036] Querying the internal catalogs of external tables with error logging enabled could generate an error similar to ERROR: Canceling query because of high VMEM usage. This issue has been resolved.
  • [31040] Occasionally, when backing up using the --jobs flag, a deadlock issue would occur between COPY and outside statements such as TRUNCATE, DROP, and ALTER. This deadlock issue has been resolved; now each job worker acquires ACCESS SHARE locks on each table involved in the backup operation.
  • [175696310] Resolves an issue where gpbackup could omit the LOG ERRORS PERSISTENTLY option during the backup of external tables configured with persistent error logging, in Tanzu Greenplum 5.x and above.
  • [175453403] When a SIGTERM event occurred during a backup operation with --single-data-file, the Greenplum Database segments could end up with uncleaned residual gpbackup_helper processes. This issue has been resolved.
  • [175323370] Resolves a misleading error message occuring when a provided backup timestamp did not exist in the supplied --backup-dir directory.

gpbackup_helper v1.20.1

No changes since Tanzu Greenplum Backup and Restore version 1.14.0.

gprestore v1.20.1

Changed Feature

  • The gprestore application name, used in system catalog views like pg_stat_activity, now includes both the backup and restore time stamps. The new name helps identify separate restore operations that are running concurrently. For example, in gprestore_20201103085529_20201103085630, 20201103085529 is the backup time stamp, and 20201103085630 the restore timestamp.

Resolved Issues

  • [175453403] When a SIGTERM event occurred during a restore operation with --single-data-file, the Greenplum Database segments could end up with uncleaned residual gpbackup_helper processes. This issue has been resolved.

gpbackup_manager v1.3.1

No changes since Greenplum Backup and Restore version 1.20.0.

Backup/Restore Plugin API

No changes since Greenplum Backup and Restore version 1.19.0.

gpbackup_ddboost_plugin v1.5.0

  • This version supports Data Domain OS 7.3 and Data Domain OS 7.2.

No feature changes since Greenplum Backup and Restore version 1.19.0.

gpbackup_s3_plugin v1.6.0

New Features

  • [34] Introduces two tuning parameters, restore_max_concurrent_requests and restore_multipart_chunksize, which can be set in the plugin configuration yaml file. For more details see S3 Storage Plugin Configuration File Format.

Version 1.20.0

Release Date: October 30, 2020

Pivotal Greenplum Backup and Restore version 1.20.0 is a minor release.

Supported Platforms

Pivotal Greenplum Backup and Restore is compatible with these Greenplum Database versions:

  • Pivotal Greenplum Database 4.3.22 and later
  • Pivotal Greenplum Database 5.5.0 and later
  • Pivotal Greenplum Database 6.0.0 and later

Software Component Versions

This release includes the following utilities:

See Data Domain Boost for Data Domain Boost support information.

gpbackup v1.20.0

New Features

  • The backup email notification subject line now includes the backup status, Success or Failure.
  • The beginning of the gpbackup output now includes the Greenplum Database version and the gpbackup version. For example:
    20200815:13:23:40 gpbackup:gpadmin:idb585.howard.ms.com:380498-[INFO]:-gpbackup version = 1.20.0 
    20200815:13:23:40 gpbackup:gpadmin:idb585.howard.ms.com:380498-[INFO]:-Greenplum Database Version =  6.1.2
    20200815:13:23:40 gpbackup:gpadmin:idb585.howard.ms.com:380498-[INFO]:-Starting backup of database opstech_dw 
    . . . . 

Resolved Issues

  • [MPP-30881] Backups would occasionally fail with the message [CRITICAL]:- dial unix <hostname>.com/.s.PGSQL.7047: connect: not a directory. The error was a postgres connection initialization issue caused by the postgres connector library (pgx). The pgx module has been updated and the issue has been resolved.

gpbackup_helper v1.20.0

No changes since version 1.14.0.

gprestore v1.20.0

New Feature

  • Introduces a new gprestore --run-analyze flag which collects current statistics for the restored tables on the root partition during the restore operation.
  • [30879] gprestore now restores SEQUENCE values if you provide the --data-only option and the SEQUENCE data is available in the backup. This applies to all restore operations that specify --data-only, including incremental backups.
  • The beginning of the gprestore output now includes the Greenplum Database version and the gprestore version. For example:
    20200815:13:23:40 gprestore:gpadmin:idb585.howard.ms.com:380498-[INFO]:-gpbackup version = 1.18.0 
    20200815:13:23:40 gprestore:gpadmin:idb585.howard.ms.com:380498-[INFO]:-gprestore version = 1.20.0 
    20200815:13:23:40 gprestore:gpadmin:idb585.howard.ms.com:380498-[INFO]:-Greenplum Database Version = 6.1.2
  • The restore email notification subject line now includes the restore status, Success or Failure.

Resolved Issues

  • [174260883] Adding the last backup details to the backup_history.yaml file was causing performance delays. The process has been optimized and this timing issue has been resolved.
  • [174561013] When the --truncate-table flag and --jobs flag were used together, the TRUNCATE was not taking advantage of parallelism. This issue has been resolved.
  • [174916912] Resolves a gprestore regression issue where --include-schema and --exclude-table were incorrectly set as incompatible.

gpbackup_manager v1.3.0

No changes since version 1.19.0.

Backup/Restore Plugin API

No changes since version 1.19.0.

gpbackup_ddboost_plugin v1.5.0

No changes since version 1.19.0.

gpbackup_s3_plugin v1.5.0

New Features

  • [30822] Introduces a new S3 plugin configuration file option http_proxy which allows access to an S3 location via a proxy. For further details see S3 Storage Plugin Configuration File Format.
  • [30952] Introduces two tuning parameters, backup_max_concurrent_requests and backup_multipart_chunksize which can be set in the plugin configuration yaml file. You can now fine tune the backup partition (referred to as chunk) size to backup large multi GB tables without exceeding the 10,000 parts Amazon S3 limitation. The backup_max_concurrent_requests allows you to increase the gpbackup concurrency setting within each Greenplum Database segment. For more details see S3 Storage Plugin Configuration File Format.

Resolved Issues

  • [30952] A gpbackup job to an S3 storage location would fail when the table size exceeded the 10,000 multi part upload limit imposed by the Amazon S3. Two S3 plugin configuration parameters have been introduced in this release, backup_max_concurrent_requests and backup_multipart_chunksize, which allow to overcome this restriction.

Data Domain Boost

This release of gpbackup and gprestore supports Data Domain Boost for backup on Red Hat Enterprise Linux. This table lists the supported versions of DDOS, Data Domain Boost SDK, and BoostFS.

Table 1. Data Domain Boost Compatibility
DDOS Data Domain Boost BoostFS
7.3 7.3 7.3
7.2 7.2 7.2
7.1 7.1 7.0
6.2 3.5 1.3
6.1 (all versions) 3.4 1.1
Note: In addition to the DDOS versions listed in the previous table, gpbackup and gprestore support all minor patch releases (fourth digit releases) later than the certified version.

Known Issues

  • When attempting to restore data from Greenplum 4.3.x/5.x/6.x (prior to Greenplum v6.14.1 and v5.28.6), views with anyarray typecasts are not restorable. From Tanzu Greenplum Backup and Restore v1.20.3, gpbackup generates an error when such views are detected.

    Workaround: On the original cluster, identify any views that exhibit this symptom, using a query similar to:

    SELECT relname AS anyarray_views 
    FROM pg_class WHERE relkind = 'v' AND oid >= 16384 AND (pg_get_viewdef(oid) 
    LIKE '%::anyarray%') IS TRUE; 

    Then select option 1 or 2:

    1. Using the backup set with the errors, run gprestore with the --on-error-continue flag. The affected views will not be re-created. After the restore completes, re-create each VIEW using your original VIEW definition.
    2. Drop each affected VIEW on the original Greenplum cluster and recreate it on the new target cluster, using the original VIEW definition. Take another backup of the older cluster and run gprestore on the new cluster.
  • When attempting to restore data from Greenplum 4.3.x/5.x into Greenplum 6.x, if the restore involved a table distributed by character(n) or char(n) using the legacy bpchar hash operator, the restore would fail. This issue has been resolved in Greenplum Database 6.12. Upgrade to the latest Greenplum release to avoid this issue.
  • Running gpbackup with the --include-table or --include-table-file options to back up a set of tables incorrectly backs up user-defined functions and protocols. This will be resolved in gpbackup v1.18.1

    Workaround: When running gprestore to restore the tables from the backup, use the same --include-table or --include-table-file options that were used to create the backup.

  • A gprestore operation with the --incremental option (Beta) recreates existing postdata objects. The operation consequently errors out with an already exists message.
  • gprestore does not support restoring a backup that contains partitioned tables where the table is created with a non-reserved keyword that is used as a partition name. For example, the non-reserved keyword at is used as a partition name in this SUBPARTITION TEMPLATE clause fragment:
    ... 
    SUBPARTITION TEMPLATE
      ( SUBPARTITION "at" VALUES ('usa'),
    ...

    gpbackup backs up the partitioned table, but gprestore returns an error when attempting to restore the table.

    Before performing a backup with gpbackup, you must ensure that partitioned tables do not use any of these non-reserved keywords as a partition name:

    ADD, ALTER, ALWAYS, AT, ATTRIBUTE, CATALOG, COMMENTS, CONFIGURATION, CONFLICT, CONTINUE, CURRENT, DATA, DAY, DENY, DEPENDS, DICTIONARY, DISCARD, DOCUMENT, DXL, EVENT, EXTENSION, FAMILY, FILESPACE, FILTER, FULLSCAN, FUNCTIONS, HOUR, IDENTITY, IGNORE, IMPORT, INITPLAN, INLINE, LABEL, LEAKPROOF, LOCKED, LOGGED, MAPPING, MATERIALIZED, METHOD, MINUTE, MONTH, NOCREATEEXTTABLE, OFF, ORDERED, ORDINALITY, OVER, PARALLEL, PARSER, PASSING, PLANS, POLICY, PROGRAM, RANDOMLY, READABLE, READS, RECURSIVE, REF, REFRESH, REJECT, REPLICATED, ROOTPARTITION, SECOND, SEQUENCES, SERVER, SKIP, SNAPSHOT, SQL, STANDALONE, STRIP, TABLES, TEXT, TRANSFORM, TYPES, UNLOGGED, VALIDATION, VARYING, VIEWS, WEB, WHITESPACE, WITHIN, WITHOUT, WRAPPER, WRITABLE, XML, YEAR, YES

  • A gprestore operation using the --redirect-schema option fails if gprestore attempts to restore an index in a schema and the name of the index is the name of the schema followed by a '.' (period). For example, this CREATE INDEX command creates the index named test. on a table in the schema test. The index is in the schema test.

    CREATE INDEX "test." ON test.mytbl USING btree (i);
    If the index and table are backed up with gpbackup, restoring the backup with this gprestore command fails because gprestore fails to restore the test. index.
    gprestore --timestamp <timestamp> --redirect-schema foo2
  • Beginning with versions 4.3.33 and 5.19, Greenplum Database checks that the distribution key for a table is a prefix of any unique index key. This policy is not in place for Greenplum Database versions before 4.3.33 or 5.19, and it is possible to create a backup of a database from one of these earlier versions with unique indexes that do not comply with the policy. When you restore such a backup to a Greenplum Database version that does enforce the policy:
    • If the unique index is for a primary key constraint, Greenplum Database automatically modifies the table's distribution policy if the table has no data.
    • In other cases, creating a unique index with a key that does not begin with the table's distribution key fails.

    This issue affects restoring backups made from Greenplum Database 4.3.32.0, 5.18.0, or earlier to a Greenplum Database version 4.3.33.0, 5.19.0, 6.0, or later system. The issue affects the gprestore, gpdbrestore, and pg_restore utilities.

Differences Compared to Open Source Greenplum Backup and Restore

Greenplum Backup and Restore includes all of the functionality in the open source Greenplum Backup github repository and S3 storage plugin repository and adds:

  • Greenplum backup plugin for DD Boost
  • Greenplum gpbackup_manager utility
  • Greenplum Backup and Restore installation file in gppkg format

Release Numbering Conventions

The Greenplum Backup and Restore distribution release number indicates the type of the release.

  • The first number is the gpbackup/gprestore major release number. For example, given the release number, 1.16.0, the major release number is 1.
  • The second number is the minor release number. Given the release number 1.16.0, the minor release number is 16. This number increments when new features are added to the gpbackup/gprestore utilities.
  • The third number is the maintenance release number. This number increments when the gpbackup/gprestore utilities included in the distribution contain fixes without new features, or if one or more components included in the distribution, such as gpbackup_manager and the backup storage plugins, have been updated.

The release versions of the components included in the distribution, such as gpbackup_manager and the backup storage plugins, are separate from the distribution version, but follow the same numbering scheme as the distribution.