Check backup errors and monitor status

[tj57]

It would be nice to check backup errors and monitor backup status to healthchecks.io in cloud or self-hosted using curl or wget.

https://stackoverflow.com/questions/6341321/how-to-check-if-postgresql-backup-was-successful

https://healthchecks.io/docs/bash/
https://healthchecks.io/docs/signaling_failures/

[prodrigestivill]

You already have an API using the curl "http://localhost:$HEALTHCHECK_PORT/" where you can have last backup result.

Is this what you are requiring?

[snyssen]

Hi, I'm reviving this issue because this is a feature I need too.

The linked pull request is perfect for my use case, but it does not seem to have made it into the final code for some reason?

An even better option imho would be to do something similar to this restic container I also use, which allows end users to provide a pre- and post-backup script. This could for example be useful for monitoring execution time as well as simply whether the backup fails or not. I could try and implement it myself then make a pull request here if you want.

[prodrigestivill]

Hello @snyssen, does the merged PR #47 closes this issue?

[snyssen]

Hi @prodrigestivill, thank you for your time!

Personally I would have preferred a more flexible solution with pre- and post-backup hooks as used in restic-backup-docker, so for example:

if [ -f "/hooks/pre-backup.sh" ]; then
    echo "Starting pre-backup script ..."
    /hooks/pre-backup.sh
fi
[...]
if [ -f "/hooks/post-backup.sh" ]; then
    echo "Starting post-backup script ..."
    /hooks/post-backup.sh $backupRC # $backupRC refers to the status code returned by the backup operation
fi

Meaning end users can input any arbitrary scripts they want be including them as volumes in their Docker implementation:

[...]
volumes:
    - ./hooks:/hooks

I prefer this feature because it allows me to not only ping healthchecks.io with status codes, but it also allows me to use its ability to measure execution time by pinging it on backup start and end. Here are my scripts for example:

• pre-backup.sh

#! /bin/sh
curl --retry 3 https://hc-ping.com/"${HEALTHCHECK_ID}"/start

• post-backup.sh

#! /bin/sh
curl --retry 3 https://hc-ping.com/"${HEALTHCHECK_ID}"/"$1" # $1 is the status code returned by the restic command

---

Having said that, the proposed solution is already pretty good for ensuring the backup is actually run on time, but I find it is still missing the ability to tell whether the backup was successful or not. I am willing to try and implement a solution on my end and then let you review it if you want, but I would like to know first if you would prefer that I try and implement my proposed solution (pre- and post-backup scripts), or if I should rather stick with the current implementation and simply provide an additionnal variables to indicate whether the backup was successful or not (so that variable can be used to build the $WEBHOOK_URL or the $WEBHOOK_EXTRA_ARGS ones).

[snyssen]

I realized I maybe gave a bit too much information, so I will clarify with this TL;DR:

PR #47 is good enough for me, except that it lacks the ability to signal whether the backup was successful or not, as it only signals whether it was run or not.

[prodrigestivill]

Closed after implementing #47 as separate default/example script.
Now more scripts can be added in the hooks folder and they are called for each diferent cases as documented in the README.

[snyssen]

Now that's perfect! Thanks a bunch!