Project:Check Backups

Backups are currently saved in the same deployment server running the portal services (mardi02 for production and mardi03 for staging).

This is not optimal because a failure in the server could mean losing access to the backups.

Currently there is no automated workflow to monitor the backups and to save a copy of them outside of the server. As a result, the following manual tasks should be conducted

1) Check that backups are being created

ssh mardi02
cd portal-compose/backup
ls -alh

There should always be 8 portal_db_backup_<date>.gz files, corresponding to the database backup with an approximate size of ~30 GB. There should be one backup file for each of the 8 previous days and there should not be large fluctuations in the size of these backups.

There should also be 8 images_<date>.gz files with a size in the order of MB.

If some of the files, specially corresponding to the latest date, are smaller or missing, it might indicate problems in the backup process. If this is the case, it should be investigated why backups are failing. A possible cause of the problem is missing space in the server. To debug the problem one should start by checking the logs in the mardi-backup container.

2) Save backup file in an external storage

Once in a while, it is important to download a copy of the backup saved in the server into an external machine.

It is specially important to do that if the server starts experiencing any other sort of problem.

This can be achieved with sftp or scp.

If using sftp just navigate the local directory where you want to save the backup.

cd ~/local_backup_directory

Connect to mardi02 or mardi03 via sftp

sftp username@mardi02

Navigate to the backup directory and download the latest portal_db_backup_<date>.gz file:

cd portal-compose/backup
get portal_db_backup_<date>.gz