[bareos-users] Scratch pool and multiple catalogs. How can I deal with this

[Alejandro Hamann]

Hello.

I have a doubt about the BareOS Scratch pool.

According to manual, "when Bareos needs a new Volume for writing and it cannot find one, it will look in the Scratch pool, and if it finds an available Volume, it will move it out of the Scratch pool into the Pool currently being used by the job".

So far so good. I've tried it and it works perfectly for me.


But due to the large size that the catalog is taking (and the need to maintain the file list for many, many years), I decided to try a new configuration, in which I contain multiple catalogs instead of just one.

Then, after a clean installation, and already configured about six different catalogs, with a few blank tapes in the autochanger, my first problem starts with the label barcode command.
BareOS no longer only asks for the pool to send the new tapes to, but in addition to that, it requires you to indicate which catalog to use.

I would have thought that since the Scratch pool is present in all the catalogs, BareOS would have some capacity to "share" that particular pool with all the catalogs. But then I see that this is not the case. Only one Scratch pool of a single catalog contains all the new volumes / tapes, and therefore, I can only use those tapes in that particular catalog.

So, how can I do to have all the tapes available, regardless of the catalog I'm using? It should be managed "manually", deciding which ones manually send tapes which Scratch pool which catalog?

And since we are, how can I "move" a tape from a pool Scratch from one catalog to the Scratch pool from another catalog? (apparently, label and relabel are not useful for this purpose once they have been used with a particular catalog).

If anyone could help me with this, I would appreciate it a lot.


Regards!

[Alejandro Hamann]

Returning to this after a while, I answer to myself about how I solved this, and it remains as a reference for those who have been presented with the same concern.
The problem is how to handle this when you have multiple catalogs. Since you can not define a single Scratch pool for multiple catalogs (if you do not specify which catalog to use in the Scratch Pool definition, Bareos assigns one automatically). What I did was define a Scratch Pool for each catalog, and indicate explicitly, for each pool defined, which Scratch Pool should use.
For example; Assume that I have three catalogs defined: Catalog-A, Catalog-B and Catalog-C, as follows:
============================================
"/etc/bareos/bareos-dir.d/catalog/Catalog-A"
============================================
Catalog {
Name = "Catalog-A"
Description = "First catalog: Catalog A"
DB Driver = postgresql
DB Name = bareos_Catalog-A
DB User = bareos
# echo bareospassword | sha224sum
DB Password = 1a58321bf4ec7a4da07e6e87d3b011149a61b9d7b6bf7d2434769bf1
}


============================================
"/etc/bareos/bareos-dir.d/catalog/Catalog-B"
============================================
Catalog {
Name = "Catalog-B"
Description = "Second catalog: Catalog B"
DB Driver = postgresql
DB Name = bareos_Catalog-B
DB User = bareos
# echo bareospassword | sha224sum
DB Password = 1a58321bf4ec7a4da07e6e87d3b011149a61b9d7b6bf7d2434769bf1
}


============================================
"/etc/bareos/bareos-dir.d/catalog/Catalog-C"
============================================
Catalog {
Name = "Catalog-C"
Description = "Third catalog: Catalog C"
DB Driver = postgresql
DB Name = bareos_Catalog-C
DB User = bareos
# echo bareospassword | sha224sum
DB Password = 1a58321bf4ec7a4da07e6e87d3b011149a61b9d7b6bf7d2434769bf1
}



Then, first define the Scratch Pool for each catalog, as follows:
==========================================================
"/etc/bareos/bareos-dir.d/pool/Scratch for Catalog-A.conf"
==========================================================
Pool {
Name = "Scratch for Catalog-A"
Catalog = Catalog-A
Pool Type = Scratch
# I am using IBM cleaning cartridges, which come
# with the barcode printed with the prefix CL
Cleaning Prefix = CL
}


==========================================================
"/etc/bareos/bareos-dir.d/pool/Scratch for Catalog-B.conf"
==========================================================
Pool {
Name = "Scratch for Catalog-B"
Catalog = Catalog-B
Pool Type = Scratch
# I am using IBM cleaning cartridges, which come
# with the barcode printed with the prefix CL
Cleaning Prefix = CL
}


==========================================================
"/etc/bareos/bareos-dir.d/pool/Scratch for Catalog-C.conf"
==========================================================
Pool {
Name = "Scratch for Catalog-C"
Catalog = Catalog-C
Pool Type = Scratch
# I am using IBM cleaning cartridges, which come
# with the barcode printed with the prefix CL
Cleaning Prefix = CL
}



So now, with the Scratch Pools already defined, it only remains to specify what the Scratch Pool will be for each pool, as follows:
===========================================================
"/etc/bareos/bareos-dir.d/pool/Some pool of Catalog-A.conf"
===========================================================
Pool {
Name = "Some pool of Catalog-A"
Description = "A pool of tapes, using Catalog-A"
Catalog = Catalog-A
Pool Type = Backup
Storage = Tape
Cleaning Prefix = CL
Maximum Block Size = 524288
Catalog Files = yes
Recycle = yes
Scratch Pool = "Scratch for Catalog-A"
Recycle Pool = "Scratch for Catalog-A"
Volume Use Duration = 1 week
Volume Retention = 1 week
Job Retention = 1 week
File Retention = 1 week
}


==============================================================
"/etc/bareos/bareos-dir.d/pool/File pool using Catalog-B.conf"
==============================================================
Pool {
Name = "File pool using Catalog-B"
Description = "A pool of backups on files, using Catalog-B"
Catalog = Catalog-B
Pool Type = Backup
Storage = File
Catalog Files = yes
Volume Use Duration = 1 week 3 days
Volume Retention = 1 week 3 days
Job Retention = 1 week 3 days
File Retention = 1 week 3 days
Recycle = yes
Scratch Pool = "Scratch for Catalog-B"
Recycle Pool = "Scratch for Catalog-B"
}


====================================================================
"/etc/bareos/bareos-dir.d/pool/WORM-Like Tapes using Catalog-C.conf"
====================================================================
Pool {
Name = "WORM-Like Tapes using Catalog-C"
Description = "Unique tape backups, forever, using Catalog-C"
Catalog = Catalog-C
Pool Type = Backup
Storage = Tape
Cleaning Prefix = CL
Maximum Block Size = 524288
Catalog Files = yes
Volume Use Duration = 100 years
Volume Retention = 100 years
Job Retention = 100 years
File Retention = 100 years
Maximum Volume Jobs = 1
# Here, the first thing that one would think about,
# is that it does not make much sense to have a Scratch Pool
# for a backup that will last 100 years, however, it
# is convenient to send the newly pre-labeled tapes to the
# Scratch Pool, so that when a job that will use this pool,
# Bareos has where to look for the new tapes.
Scratch Pool = "Scratch for Catalog-C"
#Recycle = yes
#Recycle Pool = "Scratch for Catalog-C"
}


Now, for the tapes, from bconsole, you can label a series of tapes by sending them to the corresponding Scratch Pool for each catalog; for example:
***@bareos-test:~# bconsole
Connecting to Director localhost:9101
1000 OK: bareos-dir Version: 18.2.4rc1 (25 Sep 2018)
Secure connection with cipher ECDHE-PSK-CHACHA20-POLY1305

Enter a period to cancel a command.

*label storage=Tape pool="Scratch of Catalog-A" slots=1-6 barcodes
*label storage=Tape pool="Scratch of Catalog-C" slots=7-10,21 barcodes
*q

With this I managed to solve my problem. Bareos searches for tapes in the Scratch Pool, and since a Scratch Pool was defined for each catalog, and it was made explicit for each pool of each catalog, which Scratch Pool corresponded, the problem was solved.

Regards!