Monitoring the Database Backup and Streaming Replication
You can monitor every database backup and replication component running on the server, including:
Streaming replicationBase backupWAL archive (continuous archive)Monitoring Database Backup and Replication from the Notifications Pane
To monitor database backup and replication from the NOTIFICATIONS page:
In the global administration view, select Main > Notifications in the navigation pane.The NOTIFICATIONS page is displayed.
Any alerts related to database backup or streaming replication are displayed.
Monitoring Database Backup and Replication from the Server Task Manager
If streaming replication is set up, you can monitor it from the Server Task Manager. The CTERA Portal runs a task every few minutes to verify that replication is working as expected. If any issues are detected, the task fails, and the CTERA Portal displays an appropriate notification in the NOTIFICATIONS page, and also sends an email alert to the portal administrators.
Note: The task runs the ctera.sh replication_status command and analyzes the output. For more information see Monitoring Database Backup and Replication from the Server Console.
To monitor database backup and replication from the Server Task Manager:
In the global administration view, select Main > Servers in the navigation pane.The SERVERS page is displayed, listing all the servers for the CTERA Portal.
3 Click the replicating secondary server name.
The server window is displayed with the server name as the window title.
4 Click the DB Replication option.
The portal reports the status of its scheduled base backups and transaction log archiving process, as well as additional metrics to help detect when database replication falls behind due to lags in the process. In the event that replication falls behind, portal administrators are notified via email. The relevant email templates are Replication setup failed and Replication has errors.
Monitoring Database Backup and Replication from the Server Console
You can view the status of each database backup and replication component.
To monitor database backup and replication from the server console:
Note: This procedure can be performed on both the master server and on secondary servers.
1 Using SSH, log in as root to the master or secondary server.
2 Run the following command: ctera-storage-util.sh extend_db_archive_pool device
3 Run the following command: ctera.sh replication_status | json_reformat
The output is displayed in JSON format. For example:
{
"streaming_replication": {
"status": "ok",
"lastSuccess": "256KB"
},
"base_backup": {
"status": "ok",
"lastSuccess": "none"
},
"wal_archive": {
"status": "failed",
"lastSuccess": "3 hours"
}
}
Both the master and secondary servers have the same output structure. However, the fields have slightly different meanings:
Field | Master Server | Secondary Server |
|---|---|---|
streaming_replication | ||
status | Indicates whether a secondary server is streaming from the master server: ok – Streaming replication is running. failed – Streaming replication failed. not configured – Streaming replication is not configured on this server. | Indicates whether this server is streaming from the master server: ok – Streaming replication is running. failed – Streaming replication failed. |
lastSuccess | The difference in size (in KB) between the last sent WAL segment and the secondary server’s last checkpoint, a point in the WAL logs at which all data files have been written to the disk. In other words, the amount of data the master database sent to the secondary database, which has not been processed by the secondary database. | The difference in size (in KB) between the last received WAL segment and the secondary server’s last checkpoint, a point in the WAL logs at which all data files have been written to the disk. In other words, the amount of data the master database sent to the secondary database, but which has not been processed by the secondary database. If the value is none, then the secondary database is up to date. |
base_backup | ||
status | Indicates whether the last base backup was completed successfully: ok – Last base backup completed successfully. failed – Last base backup failed. not configured – Base backup is not configured for this server. This means that continuous archiving was not configured on the server. | Indicates whether the last base backup was completed successfully: ok – Last base backup completed successfully. failed – Last base backup failed. |
lastSuccess | The last time a successful backup was run (in days). If the value is none, the status field is ok, and the last base backup was completed successfully. | |
wal_archive | ||
status | Indicates whether an WAL was successfully archived in the past hour: ok – WAL directory was archived successfully in the past hour. failed – WAL directory was not archived in the past hour. This does not indicate a problem. You can change the status to ok, by running a manual command that forces archiving. For more information, contact CTERA support. not configured – WAL archiving, that is continuous archiving, is not configured on this server. Note: Archiving occurs after 16MB of data has been written or during a system restart. | Indicates whether a WAL was successfully synchronized in the past hour: ok – WAL directory was synchronized successfully in the past hour. failed – WAL directory was not synchronized in the past hour. |
lastSuccess | The last time a WAL log was archived successfully (in hours). If the status field's value is ok, then none is output. | |
4 Run the following command on the slave server to view the slave archive synchronization log: ctera-portal-log.sh replication [-f]
Include the -f flag to display the log in the command window. Otherwise, the log is displayed in vim.