Justin Vrooman
2015-12-11 15:34:54 UTC
I am attempting to do a disk-to-disk-to-tape job from my NetApp to local storage on the backup server and then off to a tape drive attached to my backup server. I have the disk-to-disk portion working using NDMP but I am unable to get the disk-to-tape migration to work. Here is how I have everything setup:
+==================================================+
+ bareos-dir.conf +
+==================================================+
#
# A NDMP Client.
#
Client {
Name = filer
Address = filer.egr.msu.edu
Port = 10000
Protocol = NDMPv4 # Need to specify protocol before password as protocol determines password encoding used.
Auth Type = MD5 # Clear == Clear Text, MD5 == Challenge protocol
Username = "root" # username of the NDMP user on the DATA AGENT e.g. storage box being backuped.
Password = "password1" # password of the NDMP user on the DATA AGENT e.g. storage box being backuped.
}
Fileset {
Name = "NDMP Test Fileset"
Include {
Options {
meta = "USER=root"
}
File = /vol/test_vol
}
}
JobDefs {
Name = "NDMPTestJobDef"
Type = Backup
Protocol = NDMP
Level = Full
Client = filer
Backup Format = dump
FileSet = "NDMP Test Fileset"
Schedule = "WeeklyCycle"
Storage = NDMPFile
Messages = Standard
Pool = NDMPFile
Priority = 10
#Write Bootstrap = "/var/opt/bareos/run/bareos/%c.bsr"
}
#
# A NDMP Backup Job using the JobDef above.
#
Job {
Name = "NDMPTestJob"
JobDefs = "NDMPTestJobDef"
Level = Full
}
#
# Define the migration job
#
Job {
Name = "NDMPMigrateAll"
Type = Migrate
Level = Full
Messages = Standard
Selection Type = Volume
Selection Pattern = "."
Pool = NDMPFile
Maximum Concurrent Jobs = 4
}
# (why?)
# Seperate Pool for NDMP data so upgrading of Jobs works and selects the right storage.
#
Pool {
Name = NDMPFile
Pool Type = Backup
#Recycle = yes
#AutoPrune = yes
Next Pool = Tape
# This Storage line overrides any specified in the Job. Recommended here.
Storage = NDMPFile
#Volume Retention = 365 days # one year
#Maximum Volume Bytes = 50G # Limit Volume size to something reasonable
#Maximum Volumes = 100 # Limit number of Volumes in Pool
# The value deï¬ned by this directive in the bareos-dir.conf ï¬le is the default value used when a Volume is created. Once the volume is created, changing the value in the bareos-dir.conf ï¬le will not change what is stored for the Volume. To change the value for an existing Volume you must use the update command in the Console.
# This makes a new volume file per backup job, just to try it in testing
Maximum Volume Jobs = 1
Label Format = "NDMPFull-" # Volumes will be labeled "Full-<volume-id>"
}
#
# Tape pool definition
#
Pool {
Name = Tape
Pool Type = Backup
# AutoPrune = yes
# Recycle = yes
Storage = LTO5Drive
}
#
# "Same storage daemon (as File) but via NDMP protocol."
# We link via the PairedStorage config option the Bareos SD instance definition to a NDMP TAPE AGENT.
#
Storage {
Name = NDMPFile
Address = hostname.egr.msu.edu # N.B. Use a fully qualified name here
# DECS: because the netapp needs to connect to us!
Port = 10000
Protocol = NDMPv4 # Need to specify protocol before password as protocol determines password encoding used.
Auth Type = Clear # Clear == Clear Text, MD5 == Challenge protocol
Username = ndmp # username of the NDMP user on the TAPE AGENT e.g. the Bareos SD but accessed via the NDMP protocol.
Password = password2 # password of the NDMP user on the TAPE AGENT e.g. the Bareos SD but accessed via the NDMP protocol.
Device = FileStorage
Media Type = File
PairedStorage = File
}
#
# Definition of LTO-5 tape storage device
#
Storage {
Name = LTO5Drive
Address = hostname
Password = "password3"
Device = "Drive-1" # same as Device in Storage daemon
Media Type = LTO-5 # same as MediaType in Storage daemon
}
+==================================================+
+ bareos-sd.conf +
+==================================================+
Storage { # definition of myself
Name = hostname-sd
Maximum Concurrent Jobs = 20
# remove comment from "Plugin Directory" to load plugins from specified directory.
# if "Plugin Names" is defined, only the specified plugins will be loaded,
# otherwise all storage plugins (*-sd.so) from the "Plugin Directory".
#
# Plugin Directory = /usr/lib64/bareos/plugins
# Plugin Names = ""
NDMP Enable = yes
NDMP Log Level = 10
NDMP Snooping = yes
}
#
# List Directors who are permitted to contact Storage daemon
#
Director {
Name = hostname-dir
Password = "password3"
}
#
# Restricted Director, used by tray-monitor to get the
# status of the storage daemon
#
Director {
Name = hostname-mon
Password = "password4"
Monitor = yes
}
#
# Devices supported by this Storage daemon
# To connect, the Director's bareos-dir.conf must have the same Name and MediaType.
#
Device {
Name = FileStorage
Media Type = File
Archive Device = /VL
LabelMedia = yes; # lets Bareos label unlabeled media
Random Access = yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
}
#
# include additional configuration files, if required
#
# eg.
#@/etc/bareos/bareos-sd.d/device-gluster.conf
# Tape Drives
#----------------------------------
Autochanger {
Name = Autochanger
Device = Drive-1, Drive-2, Drive-3, Drive-4
Changer Command = "/usr/lib/bareos/scripts/mtx-changer %c %o %S %a %d"
Changer Device = /dev/changer
}
Device {
Name = Drive-1
Drive Index = 0
Media Type = LTO-5
Archive Device = /dev/nst3 # Normal archive device
Autochanger = yes
LabelMedia = no;
AutomaticMount = yes;
AlwaysOpen = yes;
Fast Forward Space File = no;
BSF at EOM = yes;
TWO EOF = yes;
}
Device {
Name = Drive-2
Drive Index = 1
Media Type = LTO-5
Archive Device = /dev/nst2 # Normal archive device
Autochanger = yes
LabelMedia = no;
AutomaticMount = yes;
AlwaysOpen = yes;
Fast Forward Space File = no;
BSF at EOM = yes;
TWO EOF = yes;
}
Device {
Name = Drive-3
Drive Index = 2
Media Type = LTO-5
Archive Device = /dev/nst1 # Normal archive device
Autochanger = yes
LabelMedia = no;
AutomaticMount = yes;
AlwaysOpen = yes;
Fast Forward Space File = no;
BSF at EOM = yes;
TWO EOF = yes;
}
Device {
Name = Drive-4
Drive Index = 3
Media Type = LTO-5
Archive Device = /dev/nst0 # Normal archive device
Autochanger = yes
LabelMedia = no;
AutomaticMount = yes;
AlwaysOpen = yes;
Fast Forward Space File = no;
BSF at EOM = yes;
TWO EOF = yes;
}
#
# Send all messages to the Director,
# mount messages also are sent to the email address
#
Messages {
Name = Standard
director = hostname-dir = all
}
Ndmp {
Name = 10amd64-default-job-01-dir # Can be any name but normally you should use the name of the Director here.
Username = ndmp # Same username as you specified in the NDMPFile storage definition.
Password = password2 # Same password as you specified in the NDMPFile storage definition.
# MD5 failed, so....
AuthType = Clear # Clear == Clear Text, MD5 == Challenge protocol
Log Level = 5
}
With all of this. The NDMP backup to disk works. The disk to tape backup then fails as follows:
+==================================================+
+ Excerpt from Log +
+==================================================+
11-Dec 10:26 hostname-dir JobId 166: The following 1 JobId was chosen to be migrated: 124
11-Dec 10:26 hostname-dir JobId 166: Automatically selected Catalog: MyCatalog
11-Dec 10:26 hostname-dir JobId 166: Using Catalog "MyCatalog"
11-Dec 10:26 hostname-dir JobId 167: Migration using JobId=124 Job=NDMPTestJob.2015-12-10_14.19.35_04
11-Dec 10:26 hostname-dir JobId 167: Bootstrap records written to /var/lib/bareos/hostname-dir.restore.1.bsr
11-Dec 10:26 hostname-dir JobId 166: Job queued. JobId=167
11-Dec 10:26 hostname-dir JobId 166: Migration JobId 167 started.
11-Dec 10:26 hostname-dir JobId 166: Bareos hostname-dir 15.2.2 (16Nov15):
Build OS: x86_64-redhat-linux-gnu redhat Red Hat Enterprise Linux Server release 6.5 (Santiago)
Current JobId: 166
Current Job: NDMPMigrateAll.2015-12-11_10.26.43_04
Catalog: "MyCatalog" (From Default catalog)
Start time: 11-Dec-2015 10:26:45
End time: 11-Dec-2015 10:26:46
Elapsed time: 1 sec
Priority: 10
Termination: Migration -- OK
11-Dec 10:26 hostname-dir JobId 167: Start Migration JobId 167, Job=NDMPMigrateAll.2015-12-11_10.26.46_05
11-Dec 10:26 hostname-dir: ERROR in authenticate.c:232 Failed ASSERT: store->password.encoding == p_encoding_md5
Then the director crashes. Can anyone see any problems?
Also, As a test I have tried doing a local disk-to-disk-to tape backup and that sort of silently "fails" reporting "Migration -- OK" after 0 seconds and not copying anything to tape. The director does not crash in this instance. Can someone tell me where I am going wrong or offer suggestions of things to try?
+==================================================+
+ bareos-dir.conf +
+==================================================+
#
# A NDMP Client.
#
Client {
Name = filer
Address = filer.egr.msu.edu
Port = 10000
Protocol = NDMPv4 # Need to specify protocol before password as protocol determines password encoding used.
Auth Type = MD5 # Clear == Clear Text, MD5 == Challenge protocol
Username = "root" # username of the NDMP user on the DATA AGENT e.g. storage box being backuped.
Password = "password1" # password of the NDMP user on the DATA AGENT e.g. storage box being backuped.
}
Fileset {
Name = "NDMP Test Fileset"
Include {
Options {
meta = "USER=root"
}
File = /vol/test_vol
}
}
JobDefs {
Name = "NDMPTestJobDef"
Type = Backup
Protocol = NDMP
Level = Full
Client = filer
Backup Format = dump
FileSet = "NDMP Test Fileset"
Schedule = "WeeklyCycle"
Storage = NDMPFile
Messages = Standard
Pool = NDMPFile
Priority = 10
#Write Bootstrap = "/var/opt/bareos/run/bareos/%c.bsr"
}
#
# A NDMP Backup Job using the JobDef above.
#
Job {
Name = "NDMPTestJob"
JobDefs = "NDMPTestJobDef"
Level = Full
}
#
# Define the migration job
#
Job {
Name = "NDMPMigrateAll"
Type = Migrate
Level = Full
Messages = Standard
Selection Type = Volume
Selection Pattern = "."
Pool = NDMPFile
Maximum Concurrent Jobs = 4
}
# (why?)
# Seperate Pool for NDMP data so upgrading of Jobs works and selects the right storage.
#
Pool {
Name = NDMPFile
Pool Type = Backup
#Recycle = yes
#AutoPrune = yes
Next Pool = Tape
# This Storage line overrides any specified in the Job. Recommended here.
Storage = NDMPFile
#Volume Retention = 365 days # one year
#Maximum Volume Bytes = 50G # Limit Volume size to something reasonable
#Maximum Volumes = 100 # Limit number of Volumes in Pool
# The value deï¬ned by this directive in the bareos-dir.conf ï¬le is the default value used when a Volume is created. Once the volume is created, changing the value in the bareos-dir.conf ï¬le will not change what is stored for the Volume. To change the value for an existing Volume you must use the update command in the Console.
# This makes a new volume file per backup job, just to try it in testing
Maximum Volume Jobs = 1
Label Format = "NDMPFull-" # Volumes will be labeled "Full-<volume-id>"
}
#
# Tape pool definition
#
Pool {
Name = Tape
Pool Type = Backup
# AutoPrune = yes
# Recycle = yes
Storage = LTO5Drive
}
#
# "Same storage daemon (as File) but via NDMP protocol."
# We link via the PairedStorage config option the Bareos SD instance definition to a NDMP TAPE AGENT.
#
Storage {
Name = NDMPFile
Address = hostname.egr.msu.edu # N.B. Use a fully qualified name here
# DECS: because the netapp needs to connect to us!
Port = 10000
Protocol = NDMPv4 # Need to specify protocol before password as protocol determines password encoding used.
Auth Type = Clear # Clear == Clear Text, MD5 == Challenge protocol
Username = ndmp # username of the NDMP user on the TAPE AGENT e.g. the Bareos SD but accessed via the NDMP protocol.
Password = password2 # password of the NDMP user on the TAPE AGENT e.g. the Bareos SD but accessed via the NDMP protocol.
Device = FileStorage
Media Type = File
PairedStorage = File
}
#
# Definition of LTO-5 tape storage device
#
Storage {
Name = LTO5Drive
Address = hostname
Password = "password3"
Device = "Drive-1" # same as Device in Storage daemon
Media Type = LTO-5 # same as MediaType in Storage daemon
}
+==================================================+
+ bareos-sd.conf +
+==================================================+
Storage { # definition of myself
Name = hostname-sd
Maximum Concurrent Jobs = 20
# remove comment from "Plugin Directory" to load plugins from specified directory.
# if "Plugin Names" is defined, only the specified plugins will be loaded,
# otherwise all storage plugins (*-sd.so) from the "Plugin Directory".
#
# Plugin Directory = /usr/lib64/bareos/plugins
# Plugin Names = ""
NDMP Enable = yes
NDMP Log Level = 10
NDMP Snooping = yes
}
#
# List Directors who are permitted to contact Storage daemon
#
Director {
Name = hostname-dir
Password = "password3"
}
#
# Restricted Director, used by tray-monitor to get the
# status of the storage daemon
#
Director {
Name = hostname-mon
Password = "password4"
Monitor = yes
}
#
# Devices supported by this Storage daemon
# To connect, the Director's bareos-dir.conf must have the same Name and MediaType.
#
Device {
Name = FileStorage
Media Type = File
Archive Device = /VL
LabelMedia = yes; # lets Bareos label unlabeled media
Random Access = yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
}
#
# include additional configuration files, if required
#
# eg.
#@/etc/bareos/bareos-sd.d/device-gluster.conf
# Tape Drives
#----------------------------------
Autochanger {
Name = Autochanger
Device = Drive-1, Drive-2, Drive-3, Drive-4
Changer Command = "/usr/lib/bareos/scripts/mtx-changer %c %o %S %a %d"
Changer Device = /dev/changer
}
Device {
Name = Drive-1
Drive Index = 0
Media Type = LTO-5
Archive Device = /dev/nst3 # Normal archive device
Autochanger = yes
LabelMedia = no;
AutomaticMount = yes;
AlwaysOpen = yes;
Fast Forward Space File = no;
BSF at EOM = yes;
TWO EOF = yes;
}
Device {
Name = Drive-2
Drive Index = 1
Media Type = LTO-5
Archive Device = /dev/nst2 # Normal archive device
Autochanger = yes
LabelMedia = no;
AutomaticMount = yes;
AlwaysOpen = yes;
Fast Forward Space File = no;
BSF at EOM = yes;
TWO EOF = yes;
}
Device {
Name = Drive-3
Drive Index = 2
Media Type = LTO-5
Archive Device = /dev/nst1 # Normal archive device
Autochanger = yes
LabelMedia = no;
AutomaticMount = yes;
AlwaysOpen = yes;
Fast Forward Space File = no;
BSF at EOM = yes;
TWO EOF = yes;
}
Device {
Name = Drive-4
Drive Index = 3
Media Type = LTO-5
Archive Device = /dev/nst0 # Normal archive device
Autochanger = yes
LabelMedia = no;
AutomaticMount = yes;
AlwaysOpen = yes;
Fast Forward Space File = no;
BSF at EOM = yes;
TWO EOF = yes;
}
#
# Send all messages to the Director,
# mount messages also are sent to the email address
#
Messages {
Name = Standard
director = hostname-dir = all
}
Ndmp {
Name = 10amd64-default-job-01-dir # Can be any name but normally you should use the name of the Director here.
Username = ndmp # Same username as you specified in the NDMPFile storage definition.
Password = password2 # Same password as you specified in the NDMPFile storage definition.
# MD5 failed, so....
AuthType = Clear # Clear == Clear Text, MD5 == Challenge protocol
Log Level = 5
}
With all of this. The NDMP backup to disk works. The disk to tape backup then fails as follows:
+==================================================+
+ Excerpt from Log +
+==================================================+
11-Dec 10:26 hostname-dir JobId 166: The following 1 JobId was chosen to be migrated: 124
11-Dec 10:26 hostname-dir JobId 166: Automatically selected Catalog: MyCatalog
11-Dec 10:26 hostname-dir JobId 166: Using Catalog "MyCatalog"
11-Dec 10:26 hostname-dir JobId 167: Migration using JobId=124 Job=NDMPTestJob.2015-12-10_14.19.35_04
11-Dec 10:26 hostname-dir JobId 167: Bootstrap records written to /var/lib/bareos/hostname-dir.restore.1.bsr
11-Dec 10:26 hostname-dir JobId 166: Job queued. JobId=167
11-Dec 10:26 hostname-dir JobId 166: Migration JobId 167 started.
11-Dec 10:26 hostname-dir JobId 166: Bareos hostname-dir 15.2.2 (16Nov15):
Build OS: x86_64-redhat-linux-gnu redhat Red Hat Enterprise Linux Server release 6.5 (Santiago)
Current JobId: 166
Current Job: NDMPMigrateAll.2015-12-11_10.26.43_04
Catalog: "MyCatalog" (From Default catalog)
Start time: 11-Dec-2015 10:26:45
End time: 11-Dec-2015 10:26:46
Elapsed time: 1 sec
Priority: 10
Termination: Migration -- OK
11-Dec 10:26 hostname-dir JobId 167: Start Migration JobId 167, Job=NDMPMigrateAll.2015-12-11_10.26.46_05
11-Dec 10:26 hostname-dir: ERROR in authenticate.c:232 Failed ASSERT: store->password.encoding == p_encoding_md5
Then the director crashes. Can anyone see any problems?
Also, As a test I have tried doing a local disk-to-disk-to tape backup and that sort of silently "fails" reporting "Migration -- OK" after 0 seconds and not copying anything to tape. The director does not crash in this instance. Can someone tell me where I am going wrong or offer suggestions of things to try?
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+***@googlegroups.com.
To post to this group, send email to bareos-***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+***@googlegroups.com.
To post to this group, send email to bareos-***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.