[bareos-users] differential backup misbehaviour (since/accurate)

[p***@citylink.dinoex.sub.org]

Hi all,

I am using a filesystem backup with
monthly full,
weekly differential,
daily incremental.

This works solidly for more than ten years already. The pool gets expired after a few months, and in order to keep some backups longer, I do periodic migrate tasks which move the "full" backups to another pool shortly before they would expire (and quarterly "full" to a third pool later).

bareos-dir: Backup Level: Differential, since=2018-09-16 03:57:32

But in fact, if it finds a migration which has run more recent than the last "full" backup, it seems to use the original start time of that migrated job as the since value, i.e. it will backup many more files than required (all files changed within a couple of months, instead of only those changed within this month).
Nevertheless, it appears that in the log the correct since value is reported, i.e. the start time of the most recent regular "full" backup.

Further investigation shows, if I remove the "Accurate=yes" option from jobdefs, then the misbehaviour disappears. So it seems, the true problem is not the "since" value, but the Accurate feature retrieves the file data (including file change timestamps) from the wrong job.

I do not want Accurate to do such things anyway; I only want it to properly record deleted files (i.e. test on existence only). Therefore I have removed the "accurate" option from the fileset options definition.
I could not find any documentation on what would be the default behaviour, if jobdefs(Accurate=yes) and this fileset option is *not set* - but it seemed to work that way.

Looking into the source, I was able to identify the issue: some query to find recent backups for the "accurate" feature gets sorted on JobId primarily. I changed this -rough&dirty- and it seems to solve the issue.

--- src/cats/postgresql_queries.inc.orig 2018-10-21 20:22:51.357673000 +0200
+++ src/cats/postgresql_queries.inc 2018-10-21 20:25:44.564357000 +0200
@@ -497,8 +497,7 @@
"AND Type = 'B' "
"AND StartTime < '%s' "
"AND FileSet.FileSetId = %s "
-"ORDER BY Job.JobId DESC, "
- "Job.JobTDate DESC "
+"ORDER BY Job.JobTDate DESC "
"LIMIT 1 "
,

[p***@citylink.dinoex.sub.org]

I forgot to mention: my code (and the patch above) is currently on version 17.2.5

ORDER BY Job.JobId DESC, Job.JobTDate DESC LIMIT 1

Job.JobId is primary key (always distinct), so the second sort on Job.JobTDate is without effect.
I might suppose that the original author mistakenly reversed the sort clause. Writing it as

ORDER BY Job.JobTDate DESC, Job.JobId DESC LIMIT 1

would give some logical sense. I would suggest that this gets changed in the master.