[bareos-users] AI Always consolidation

[Dakota Pilot]

For reasons that made sense at the time (but may not now - see below) I created a backup location on a file server for Bareos to store server and workstation backups. There are three sub directories in this backup are - servers, Windows workstations, Linux Workstations.

The Always Incremental works fine and keeps creating the incremental backups so I'm trying to get consolidation (and after that VLT storage) working. Each of the backup jobs for servers, workstations archives to the correct subdirectory and that works fine. Windows stuff goes to the Windows workstation directory, Linux stuff to it's directory.

Since a consolidate job takes ALL AI jobs and does it's consolidation based on the AI jobs where does it store the consolidated jobs. We can define an archive location for the consolidated job so does it use that or does it use the location given in each AI job?

When I set up the directory structure (which can be changed) it seemed right to separate the server, windows, and Linux workstation backups but thinking about it does it make sense? Given that we don't work directly with these backups but go through bconsole or webui and Bareos stores it's data in a database should I be using just one directory to back things up. I'd appreciate thoughts and input on this.

Thanks.

[Dan]

Dakota -

The consolidate job executes on the pool. If you have defined different pools for your servers, linux workstations, and windows workstations, then I presume you can just run 3 separate consolidate jobs, one for each pool, and maintain the separation you want. I say presume because I haven't tested that. I tend to think you've overcomplicated things by maintaining the separation that you have. I might be tempted to separate the server data if I was using tape as my primary backup medium because it might significantly improve restore performance on streaming media if the data was not intermingled. For random access (disk) primary backup I just don't care.

[Dakota Pilot]

That was the missing piece of the puzzle - that consolidate operates on the pool. That helps immensely.

I think you are correct in that I've overcomplicated it. When I set it up I wasn't familiar with Bareos so I set things up as you might do for storing files. I can change it and will to just use one directory for storing backups.

Thank you.

[Dakota Pilot]

So, from what I'm seeing I think that a consolidate job operates on ALL pools that are Always Incremental? It appears to me that when I run my consolidate job it checks all the pools I have setup to do AI (three of them). Am I right about this or did I miss something.

I have three jobs backing up windows, linux workstations, and one other that backs up some files from a server.

So one consolidate job will do all the pools?

[Dakota Pilot]

Post by Dan
Dakota -
The consolidate job executes on the pool. If you have defined different pools for your servers, linux workstations, and windows workstations, then I presume you can just run 3 separate consolidate jobs, one for each pool, and maintain the separation you want. I say presume because I haven't tested that. I tend to think you've overcomplicated things by maintaining the separation that you have. I might be tempted to separate the server data if I was using tape as my primary backup medium because it might significantly improve restore performance on streaming media if the data was not intermingled. For random access (disk) primary backup I just don't care.

From what I've tested one consolidate job runs on all pools. I created two consolidate type jobs - one for Linux, one for Windows. I have one workstation in each right now. I can run either and the job looks at both workstations so I assume it's taking the pool for Windows, running the consolidate against it and then repeating for Windows so one Consolidate job might do it. Here's the output.

09-Aug 18:29 bareos-dir JobId 1430: Start Consolidate JobId 1430, Job=WinWSCons.2018-08-09_18.29.32_28
09-Aug 18:29 bareos-dir JobId 1430: Looking at always incremental job ws-1-linux2
09-Aug 18:29 bareos-dir JobId 1430: ws-1-linux2: considering jobs older than 08-Aug-2018 18:29:34 for consolidation.
09-Aug 18:29 bareos-dir JobId 1430: ws-1-linux2: less incrementals than required to consolidate without full, not consolidating
09-Aug 18:29 bareos-dir JobId 1430: Looking at always incremental job Ranger
09-Aug 18:29 bareos-dir JobId 1430: Ranger: considering jobs older than 08-Aug-2018 18:29:34 for consolidation.
09-Aug 18:29 bareos-dir JobId 1430: Ranger: less incrementals than required to consolidate without full, not consolidating
09-Aug 18:29 bareos-dir JobId 1430: BAREOS 17.2.4 (21Sep17): 09-Aug-2018 18:29:34
JobId: 1430
Job: WinWSCons.2018-08-09_18.29.32_28
Scheduled time: 09-Aug-2018 18:29:32
Start time: 09-Aug-2018 18:29:34
End time: 09-Aug-2018 18:29:34
Termination: Consolidate OK

[Dan]

Dakota -

You are correct. Per the user guide ...

"The Consolidate job evaluates all jobs congured with Always Incremental = yes. When a job is
selected for consolidation, all job runs are taken into account, independent of the pool and storage where
they are located."

Dan

[Dakota Pilot]

I saw that but after you comment about maybe running three consolidate jobs I wasn't sure so I tested it. Good to know anyway. Working on getting VLT working now.