[bareos-users] Consolidating from AI scheme using volume in error state

[d***@opcode.cl]

Hi everyone, sorry by my bad english.

Currently I'm consolidating an always incremental scheme job and I have a lot
of previous jobs that are candidate to consolidate, say 4, 9, 10, 16, 19,
22,... Currently Bareos ask me for job nº16 that is in state "Error", and as a
error I've deleted that file, so no way to recover it. Is correct that
situation? I mean, if is in error state, backup file is inconsistent so there
is no reason to use it. Any way to skip it?

Thanks in advance, image attached.

[Dan]

Your screenshot shows a VirtualFull job, not a consolidation.

If I understand correctly, you deleted the file from your storage system that is associated with volume patriciodesktop-ai-0016. It is not a good practice to delete a file from the storage system unless you have deleted the associated volume from the catalog. There is data on that volume, which is why Bareos is asking for it during the virtual full. If you click on the volume name it will show you each job that has written to that volume and the number of files and bytes.

If you can't recover the file, the only recourse I know is to delete volume patriciodesktop-ai-0016 from bconsole. That will allow the Virtual Full to complete, but obviously with the data that was on that disk missing from the VirtualFull backup. More importantly, the files that were on that volume and are no longer recoverable with Bareos will be backed up again the next time that client performs an AI backup.

Dan

[d***@opcode.cl]

Hi thanks for reply,

Post by Dan
Your screenshot shows a VirtualFull job, not a consolidation.

Well, is a VirtualFull job while running consolidating

Post by Dan
If I understand correctly, you deleted the file from your storage system that is associated with volume patriciodesktop-ai-0016. It is not a good practice to delete a file from the storage system unless you have deleted the associated volume from the catalog. There is data on that volume, which is why Bareos is asking for it during the virtual full. If you click on the volume name it will show you each job that has written to that volume and the number of files and bytes.

What I'm wondering is why bareos need failed jobs to consolidate... ?

Post by Dan
If you can't recover the file, the only recourse I know is to delete volume patriciodesktop-ai-0016 from bconsole. That will allow the Virtual Full to complete, but obviously with the data that was on that disk missing from the VirtualFull backup. More importantly, the files that were on that volume and are no longer recoverable with Bareos will be backed up again the next time that client performs an AI backup.

"Will be backed up again next time..... " if I don't remove job from catalog???

Post by Dan
Dan

Patricio

[Dan]

Patricio -

1) AI Consolidation and VirtualFull backups are different things for different purposes. The VirtualFull doesn't consolidate anything. It creates a snapshot of the current AI dataset that can be archived for long term retention.

2) This is the first mention you have made of a FAILED job. Assuming the was a failed job, that would not necessarily mean that there is no valid data on the volume. For instance, a job will fail if it successfully backs up all of the data but fails to write a bootstrap file. The bottom line is that the Bareos catalog has a list of jobs and files that it has backed up to that volume and it is trying to read them and include them in the VirtualFull backup.

3) If you delete the volume from the catalog, Bareos will delete the jobs and files that were associated with the volume. There should not be a reason to also delete the job. Now it is possible that your job spanned multiple volumes and I don't know off hand whether Bareos will delete the job from the catalog or only those files from the job that are on the deleted volume. If you do not delete the volume from the catalog, Bareos will not back up those missing files again unless they change, so any 'static' files will be unrecoverable forever. Maybe somebody else know the answer about a job spanning volumes. If that was the case with you and you want to be conservative because you don't know how it behaves, then you can first delete all jobs that are listed on the volume and then delete the volume. In fact, if you delete the jobs and leave the volume in the catalog Bareos will purge and recycle the volume and recreate the file on your storage system.

Hope that helps.

Dan

[d***@opcode.cl]

Dan,

Thanks for your help :-)

Patricio

Post by Dan
Patricio -
1) AI Consolidation and VirtualFull backups are different things for different purposes. The VirtualFull doesn't consolidate anything. It creates a snapshot of the current AI dataset that can be archived for long term retention.
2) This is the first mention you have made of a FAILED job. Assuming the was a failed job, that would not necessarily mean that there is no valid data on the volume. For instance, a job will fail if it successfully backs up all of the data but fails to write a bootstrap file. The bottom line is that the Bareos catalog has a list of jobs and files that it has backed up to that volume and it is trying to read them and include them in the VirtualFull backup.
3) If you delete the volume from the catalog, Bareos will delete the jobs and files that were associated with the volume. There should not be a reason to also delete the job. Now it is possible that your job spanned multiple volumes and I don't know off hand whether Bareos will delete the job from the catalog or only those files from the job that are on the deleted volume. If you do not delete the volume from the catalog, Bareos will not back up those missing files again unless they change, so any 'static' files will be unrecoverable forever. Maybe somebody else know the answer about a job spanning volumes. If that was the case with you and you want to be conservative because you don't know how it behaves, then you can first delete all jobs that are listed on the volume and then delete the volume. In fact, if you delete the jobs and leave the volume in the catalog Bareos will purge and recycle the volume and recreate the file on your storage system.
Hope that helps.
Dan