Checking for Failed Segments
Checking for Failed Segments
With mirroring enabled, you can have failed segment instances in the system without interruption of service or any indication that a failure has occurred. You can verify the status of your system using the gpstate utility, by examing the contents of the gp_segment_configuration catalog table, or by checking log files.
Check for failed segments using gpstate
The gpstate utility provides the status of each individual component of a Greenplum Database system, including primary segments, mirror segments, master, and standby master.
$ gpstate -eIf the utility lists Segments with Primary and Mirror Roles Switched, the segment is not in its preferred role (the role to which it was assigned at system initialization). This means the system is in a potentially unbalanced state, as some segment hosts may have more active segments than is optimal for top system performance.
Segments that display the Config status as Down indicate the corresponding mirror segment is down.
See Recovering from Segment Failures for instructions to fix this situation.
Check for failed segments using the gp_segment_configuration table
To get detailed information about failed segments, you can check the gp_segment_configuration catalog table. For example:$ psql postgres -c "SELECT * FROM gp_segment_configuration WHERE status='d';"
$ gpstate -m
Check for failed segments by examining log files
Log files can provide information to help determine an error's cause. The master and segment instances each have their own log file in log of the data directory. The master log file contains the most information and you should always check it first.
Use the gplogfilter utility to check the Greenplum Database log files for additional information. To check the segment log files, run gplogfilter on the segment hosts using gpssh.
To check the log files
- Use gplogfilter to check the master log file for
WARNING, ERROR, FATAL or
PANIC log level messages:
$ gplogfilter -t
- Use gpssh to check for WARNING,
ERROR, FATAL, or PANIC log level
messages on each segment instance. For
example:
$ gpssh -f seg_hosts_file -e 'source /usr/local/greenplum-db/greenplum_path.sh ; gplogfilter -t /data1/primary/*/log/gpdb*.log' > seglog.out