Duplication between DataDomains partially failing
Hello, we are currently facing strange behaviour during the duplication process between two Dell Data Domains. Basically, there are on source location 3 VMs with respective VMware backups that are duplicated to the target location. All settings, software, hardware are same, there are no differences but the duplication job is successful only for 2 VMs, the last one always fails and is not possible to duplicate in any time. Job error log: Critical bpdm (pid=11720) sts_copy_extent failed: error 2060046 plugin error Critical bpdm (pid=11720) image copy failed: error 2060046: plugin error Error bpdm (pid=11720) cannot copy image from disk, bytesCopied = 0 Error bpduplicate (pid=205144) host MediaServer 1 backup id VMabc123 optimized duplication failed, media write error (84). Error bpduplicate (pid=205144) Duplicate of backupid XXXXXX failed, media write error (84). Error bpduplicate (pid=205144) Status = no images were successfully processed. I've tried to used different Media server, new storage unit, restarting the duplication chain, restarting every single part of the hardware infrastructure, nothing helped, situation remained same - two VMs are always successfully duplicated, one keeps failing. What should be check in such case as a next step? Thank you. Tomas
Duplication of data to new STU
Hey guys, I have a question how to duplicate all backup images from one MSDP to another. We have one appliance where backups are goin via SLPs and now we want to add new appliance to our environment for duplication purposes. How can we easly duplicate all data? If we add duplication to SLP it will only duplicate new images right? We need to copy also old images as this is requirement from business. Br, PiotrDuplication: nbstlutil report numbers
Hello, I would like to ask you or better clarify command "nbstlutil report". What exactly is considered in the number of copies? Only "images to be duplicated" or "images to be duplicated together with images that are already duplicated and waiting e.g. for expiration"? From nbstlutil report -lifecycle xxxx I got following: Backlog of incomplete SLP Copies In Process (Storage Lifecycle State: 2): Number of copies: 4028 Total expected size 22996802 MB SLP Name: (state) Number of copies: 4028 Size: 22996802 MB Becasue when I want to cancel whole processing via "nbstlutil cancel -lifecycle xxxx" I got only 1876 images instead of total 4028 images, that's why I'm wondering what's included in the "total number". Thank you for any clarification. Tom
Duplications failed with code 191 - NetBackup 8.1
Dear contributors, Hoping that you are well, I write this post because I have a problem with Duplications, I am new to this NetBackup world, so maybe it is a simple mistake. Unfortunately all Duplications have begun to fail .. I need help. - NetBackup 5230 - Appliance ver. 3.1 - NetBackup ver. 8.1 LOG: _____________________________________________________________________ Jan 30, 2020 2:09:12 PM - requesting resource LCM_CLESCNBU20010-hcart-robot-tld-1 Jan 30, 2020 2:09:12 PM - Info nbrb (pid=281640) Limit has been reached for the logical resource LCM_CLESCNBU20010-hcart-robot-tld-1 Jan 30, 2020 2:58:55 PM - granted resource LCM_CLESCNBU20010-hcart-robot-tld-1 Jan 30, 2020 2:58:55 PM - started process RUNCMD (pid=304107) Jan 30, 2020 2:58:56 PM - begin Duplicate Jan 30, 2020 2:58:56 PM - requesting resource CLESCNBU20010-hcart-robot-tld-1 Jan 30, 2020 2:58:56 PM - requesting resource @aaaab Jan 30, 2020 2:58:56 PM - reserving resource @aaaab Jan 30, 2020 2:58:56 PM - awaiting resource CLESCNBU20010-hcart-robot-tld-1. No drives are available. Jan 30, 2020 2:58:56 PM - ended process 0 (pid=304107) Jan 30, 2020 2:59:25 PM - awaiting resource CLESCNBU20010-hcart-robot-tld-1. Waiting for resources. Reason: Drives are in use, Media server: CLESCNBU20010.dys.corp, Robot Type(Number): TLD(1), Media ID: N/A, Drive Name: N/A, Volume Pool: Pool_Semanal, Storage Unit: CLESCNBU20010-hcart-robot-tld-1, Drive Scan Host: N/A, Disk Pool: N/A, Disk Volume: N/A Jan 30, 2020 2:59:26 PM - awaiting resource CLESCNBU20010-hcart-robot-tld-1. No drives are available. Jan 30, 2020 3:15:21 PM - awaiting resource CLESCNBU20010-hcart-robot-tld-1. Waiting for resources. Reason: Drives are in use, Media server: CLESCNBU20010.dys.corp, Robot Type(Number): TLD(1), Media ID: N/A, Drive Name: N/A, Volume Pool: Pool_Semanal, Storage Unit: CLESCNBU20010-hcart-robot-tld-1, Drive Scan Host: N/A, Disk Pool: N/A, Disk Volume: N/A Jan 30, 2020 3:15:23 PM - awaiting resource CLESCNBU20010-hcart-robot-tld-1. No drives are available. Jan 30, 2020 3:15:39 PM - awaiting resource CLESCNBU20010-hcart-robot-tld-1. Waiting for resources. Reason: Drives are in use, Media server: CLESCNBU20010.dys.corp, Robot Type(Number): TLD(1), Media ID: N/A, Drive Name: N/A, Volume Pool: Pool_Semanal, Storage Unit: CLESCNBU20010-hcart-robot-tld-1, Drive Scan Host: N/A, Disk Pool: N/A, Disk Volume: N/A Jan 30, 2020 3:15:39 PM - awaiting resource CLESCNBU20010-hcart-robot-tld-1. No drives are available. Jan 30, 2020 3:26:28 PM - resource @aaaab reserved Jan 30, 2020 3:26:28 PM - granted resource WC0179 Jan 30, 2020 3:26:28 PM - granted resource HP.ULTRIUM7-SCSI.001 Jan 30, 2020 3:26:28 PM - granted resource CLESCNBU20010-hcart-robot-tld-1 Jan 30, 2020 3:26:28 PM - granted resource MediaID=@aaaab;DiskVolume=PureDiskVolume;DiskPool=dp_disk_CLESCNBU20010;Path=PureDiskVolume;StorageServer=CLESCNBU20010.dys.corp;MediaServer=CLESCNBU20010.dys.corp Jan 30, 2020 3:26:29 PM - Info bpduplicate (pid=304107) window close behavior: Suspend Jan 30, 2020 3:26:29 PM - Info bptm (pid=314191) start Jan 30, 2020 3:26:29 PM - started process bptm (pid=314191) Jan 30, 2020 3:26:30 PM - Info bptm (pid=314191) start backup Jan 30, 2020 3:26:30 PM - Info bpdm (pid=314250) started Jan 30, 2020 3:26:30 PM - started process bpdm (pid=314250) Jan 30, 2020 3:26:30 PM - Info bpdm (pid=314250) reading backup image Jan 30, 2020 3:26:30 PM - Info bpdm (pid=314250) using 30 data buffers Jan 30, 2020 3:26:30 PM - Info bpdm (pid=314250) requesting nbjm for media Jan 30, 2020 3:26:30 PM - Info bptm (pid=314191) media id WC0179 mounted on drive index 1, drivepath /dev/nst0, drivename HP.ULTRIUM7-SCSI.001, copy 2 Jan 30, 2020 3:26:30 PM - Info bptm (pid=314191) INF - Waiting for positioning of media id WC0179 on server CLESCNBU20010.dys.corp for writing. Jan 30, 2020 3:26:32 PM - begin reading Jan 30, 2020 3:38:11 PM - end reading; read time: 0:11:39 Jan 30, 2020 3:38:11 PM - begin reading Jan 30, 2020 3:49:03 PM - end reading; read time: 0:10:52 Jan 30, 2020 3:49:03 PM - begin reading Jan 30, 2020 3:59:49 PM - end reading; read time: 0:10:46 Jan 30, 2020 3:59:49 PM - begin reading Jan 30, 2020 4:11:49 PM - end reading; read time: 0:12:00 Jan 30, 2020 4:11:49 PM - begin reading Jan 30, 2020 4:22:58 PM - end reading; read time: 0:11:09 Jan 30, 2020 4:22:58 PM - begin reading Jan 30, 2020 4:36:42 PM - end reading; read time: 0:13:44 Jan 30, 2020 4:36:42 PM - begin reading Jan 30, 2020 4:46:01 PM - end reading; read time: 0:09:19 Jan 30, 2020 4:46:01 PM - begin reading Jan 30, 2020 4:56:42 PM - end reading; read time: 0:10:41 Jan 30, 2020 4:56:42 PM - begin reading Jan 30, 2020 5:06:46 PM - end reading; read time: 0:10:04 Jan 30, 2020 5:06:46 PM - begin reading Jan 30, 2020 5:06:46 PM - Critical bpdm (pid=314250) Storage Server Error: (Storage server: PureDisk:CLESCNBU20010.dys.corp) PdvfsRead: Failed to read from spoold (Connection reset by peer). Ensure storage server services are running and operational. V-454-19 Jan 30, 2020 5:06:46 PM - Critical bpdm (pid=314250) sts_read_image failed: error 2060019 error occurred on network socket Jan 30, 2020 5:06:46 PM - Critical bpdm (pid=314250) image read failed: error 2060019: error occurred on network socket Jan 30, 2020 5:06:46 PM - Error bpdm (pid=314250) cannot read image from disk, Invalid argument Jan 30, 2020 5:06:48 PM - Error bptm (pid=314191) media manager terminated by parent process Jan 30, 2020 5:06:51 PM - Info CLESCNBU20010.dys.corp (pid=314250) StorageServer=PureDisk:CLESCNBU20010.dys.corp; Report=PDDO Stats for (CLESCNBU20010.dys.corp): read: 460800078 KB, CR received: 342190618 KB, CR received over FC: 0 KB, dedup: 0.0% Jan 30, 2020 5:07:19 PM - Error bpduplicate (pid=304107) host CLESCNBU20010.dys.corp backup id dysf-ex10mbx1.dys.corp_1580004201 read failed, media read error (85). Jan 30, 2020 5:07:19 PM - Error bpduplicate (pid=304107) host CLESCNBU20010.dys.corp backupid dysf-ex10mbx1.dys.corp_1580004201 write failed, media manager killed by signal (82). Jan 30, 2020 5:07:20 PM - Error bpduplicate (pid=304107) Duplicate of backupid dysf-ex10mbx1.dys.corp_1580004201 failed, media manager killed by signal (82). Jan 30, 2020 5:07:20 PM - Error bpduplicate (pid=304107) Status = no images were successfully processed. Jan 30, 2020 5:07:20 PM - end Duplicate; elapsed time 2:08:24 no images were successfully processed (191) _____________________________________________________________________ Please indicate if you need any more details.
Disk staging to tape causes status 196 on regular policies
Weekly full backups are staged to an EMC Data Domain basic disk staging storage unit. Staging to a 6 drive LTO4 robot is triggered by a schedule at 3PM which takes more than the sheduled window to complete. Tape-only policies that are sceduled at 11PM fail with status 196 because all drives are occupied by the staging. How can I ensure that the staging has less priority than regular backup policies? Is this recommended? I want to avoid tweeking with policy priorities which are all at 0. The staging schedule has a priority of 90000 which makes no sense to me.SLP not duplicating
Hi. So we got a problem with our SLP doing duplications. We are currently running Netbackup 7.6.1, one masterserver, and 3 media servers. Windows 2008 R2 We are running all duplications to tape, a dell TL4000. We run 7 different SLP's where 2 of these duplicates to tape. Here comes our problem, one of these two SLP jobs wont duplicate the data to tape. It just keeps putting the data in backlog. The backupjob runs fine, but when its time for the SLP to run duplication, it nevere starts. nbstlutil stlilist -image_incomplete -U Image fil4.hedmark.org_1510305239 for Lifecycle DAS102-ADV-Year is IN_PROCESS Copy to nb-media1-hcart-robot-tld-0 of type DUPLICATE is NOT_STARTED The problem is, the backupjob is done, but it says "IN_PROCESS". I've tried the following: Cancled all jobs running on the SLP. Then manually run them again. Delete the SLP and created it again. Booted all master and media servers. When running the duplication manually from "Catalog" it works fine. And the other SLP runs just fine, duplicates to tape no problem. Any ideas?
"Postpone creation" function for SLP disappears after setting
Hi, First I would like to thank you all for this forum: you saved my day so many times during last years! This is however my first post, and it's regarding an odd behaviour of SLP definition and processing for wich I haven't found any related report. As many other customers I'm using NetBackup v 7.7.2 on a Windows Server 2008 R2 cluster environment (as Master) with two Windows Server 2008 R2 hosts as Media and a third Windows Server 2012 ad another Media with dedicated backup pools. Associated with the two "main" Media Servers are two MSDP disk pool sized about 20TB each, serving as a temporary stage area (from a minumum of 3 to a maximum of 8 days) before defined SLPs duplicate the backup images to tape for the remaining retention periods. Almost all SLPs are responsible for a second duplication to a separate tape pool for each backup image, in order to copy data for Disaster Recovery purpose. So the workflow should run like that: - Take the backup image and write it to MSDP pool with retention time X => COPY1 - Immediately after that, as per SLP window, duplicate the backup image to the DR tape pool with retention time Z (tapes are automatically counted and ejected during the next working day) => COPY3 - After time X, before primary copy expiration, duplicate the backup image to the PROD tape pool with retention time Z => COPY2 During the past few years (and incarnations) of NBU this strategy worked well, as intended: every SLP had the flag set on "Postpone creation of this copy until the source copy is about to expire" for every COPY2 duplication type. Recently I noticed that all COPY2 duplication types, inside all defined SLP, had that flag unsetted. Thinking of someone's misleading action, I personally flagged that option again (only for the COPY2 duplication job types). After about ten minutes, I checked SLP definitions, and all those settings were gone... I repeated the "flag" operation again, and after some time the setting was gone again. The worst thing is that this is not only a "display" issue, because SLP are not working as intended too: without the "Postpone creation of this copy until the source copy is about to expire" option, duplications for COPY2 starts in conjunction with duplications for COPY3, immediately after production of COPY1, resulting in higher SLP backlog and drives/tapes workload. Attached is a screen of one of those odd-acting SLPs, with evidence on the option property that simply refuses to stay flagged... Any suggestions? Thanks again and sorry for my bad english ^^' M.Backup Exec server cannot access the specified storage device (tape library) in CAS/MBES environment
We have many backup policies in our environment and almost all of them are having the same configuration. It may sound odd but we have one backup for each VM in this environment. All of them are having a weekly job and daily incremental job. As the second stage, each of these jobs is linked to a duplication job which duplicates the correspondent backup set to a DR open storage device. Every first Monday of the month we get the last full backup set and duplicate it to LTO5 tapes and keep them for long retention as monthly backups. The tape library is attached to one of the MBES servers. The problem is that there are a few jobs that cannot see the tape library during monthly tape duplications on each month. The jobs that cannot access the tape library are completely random. So, if one backup policy cannot see the tape library in one month it may work fine next month. Refreshing the Backup Exec services on CAS and two MBESs have not helped in our case. The only way we used to get the full backup sets duplicated to tape was to manually find the backup sets in the deduped OST and duplicated them to the tape library. Just recently, I have found a work around to make the backup policies which have trouble accessing the library to find the device and kick off the monthly duplication. However, after a few jobs, they go to the same state and I have to do the same process again until all of the backup policies are duplicated to tape successfully. We also get "ODBC access error. Possible lost connection to database or unsuccessful access to catalog index in the database" error message on CAS. Apparently, this happens when there is a communication drop out between CAS and one of the MBES servers. I guess this might be causing the monthly duplication issue we are facing in the environment. This is already escalated to Veritas support and is still under investigation. My question is if anybody knows a permanent fix for this issue?