Forum Discussion

johnmath99's avatar
johnmath99
Level 3
6 years ago

NBU 8.1.1 and Data Domain VE 6.1.2.5 Failing Replication

I have an issue with replication between two sets of media servers and DDs, I've had them working when replication to different targets but can't figure out whats going on here.  I suspect I've missed something obvious as I've had loads working previously but this one I can't track down:

Activity Log Output:

07-Feb-2019 16:19:42 - requesting resource  LCM_*Remote*Master*
07-Feb-2019 16:19:42 - granted resource  LCM_*Remote*Master*
07-Feb-2019 16:19:42 - started process RUNCMD (pid=11570)
07-Feb-2019 16:19:42 - requesting resource  @aaaal
07-Feb-2019 16:19:42 - reserving resource @aaaal
07-Feb-2019 16:19:42 - resource @aaaal reserved
07-Feb-2019 16:19:42 - granted resource  MediaID=@aaaal;DiskVolume=NBUVWOFFMED003_01;DiskPool=NBUVAOFFDD003_NBUVWOFFMED003_01;Path=NBUVWOFFMED003_01;StorageServer=NBUVAOFFDD003-dd;MediaServer=nbuvwoffmed003-bck
07-Feb-2019 16:19:42 - Info nbreplicate (pid=11570) Suspend window close behavior is not supported for nbreplicate
07-Feb-2019 16:19:42 - Info nbreplicate (pid=11570) window close behavior: Continue processing the current image
07-Feb-2019 16:19:43 - Info bpdm (pid=2664) started
07-Feb-2019 16:19:43 - started process bpdm (pid=2664)
07-Feb-2019 16:19:46 - Info nbuvwoffmed003-bck (pid=2664) Using OpenStorage to replicate backup id TENVWZVM001_1549481690, media id @aaaal, storage server NBUVAOFFDD003-dd, disk volume NBUVWOFFMED003_01
07-Feb-2019 16:19:46 - Info nbuvwoffmed003-bck (pid=2664) Replicating images to target storage server 10.169.68.69, disk volume NBUVWOFFMED003_01_REP
07-Feb-2019 16:19:47 - Error bpdm (pid=2664) wait failed: error 174
07-Feb-2019 16:19:47 - Error bpdm (pid=2664) <async> cancel failed: error 2060001: one or more invalid arguments
07-Feb-2019 16:19:47 - Error bpdm (pid=2664) copy cancel failed: error 174
07-Feb-2019 16:19:48 - Error nbreplicate (pid=11570) ReplicationJob::Replicate: Replication failed for backup id TENVWZVM001_1549481690: media write error (84)
07-Feb-2019 16:19:48 - Replicate failed for backup id TENVWZVM001_1549481690 with status 84
no images were successfully processed  (191)

 

VxLogView Output

07/02/19 16:19:42.237 [RecoverableJob::registerForEmail] (ID:0x7f5ac8007458 CTX:0x7f5ac8007458) Registered for email(RecoverableJob.cpp:3054)
07/02/19 16:19:42.237 V-116-215 [CommandJob::run] jobid=19534 submitted to nbjm for processing
07/02/19 16:19:42.240 [CommandJob::CommandJob] policy display name: SLP_SP_MED003_7DAYS(CommandJob.cpp:62)
07/02/19 16:19:42.241 [JobInstance::state] (0x7f084c007570) state=0(JobInstance.cpp:355)
07/02/19 16:19:42.244 V-118-227 [ResBroker_i::requestResources] received resource request ID {2CBC112E-2AF4-11E9-97C3-EB316D1D4F92}, priority 0, secondary priority 0, description COMMAND_JOB_19534
07/02/19 16:19:42.244 [ResBroker_i::requestResources] RequestSeq:
Request=0 provider=NamedResourceProvider resourcename=LCM_*Remote*Master* CountedResourceRequest { max=2048 }
07/02/19 16:19:42.265 V-118-226 [ResBroker_i::evaluateOne] Evaluating request ID {2CBC112E-2AF4-11E9-97C3-EB316D1D4F92}
07/02/19 16:19:42.265 V-118-146 [ProviderManager::allocate] NamedResourceProvider returned Allocation Granted for request ID {2CBC112E-2AF4-11E9-97C3-EB316D1D4F92}
07/02/19 16:19:42.267 [ResBroker_i::evaluateOne] resources found for request, requestId={2CBC112E-2AF4-11E9-97C3-EB316D1D4F92}, userId=jobid=19534, birthTime=1549556382, timeToFirstEval=0 sec, timeToSuccessOrFailure=0 sec, numberOfTimesEvaluated=1
07/02/19 16:19:42.267 V-118-109 [ResBroker_i::passOne] resources found for request ID {2CBC112E-2AF4-11E9-97C3-EB316D1D4F92}, following allocations are granted:
07/02/19 16:19:42.267 V-118-228 [ResBroker_i::passOne] allocation ID {2CBF9BBE-2AF4-11E9-B61C-CBD1E34B86DD}, group ID {00000000-0000-0000-0000-000000000000}, provider NamedResourceProvider, resource LCM_*Remote*Master*
07/02/19 16:19:42.267 [ResBroker_i::passOne] allocation: (Allocation: id={2CBF9BBE-2AF4-11E9-B61C-CBD1E34B86DD} provider=NamedResourceProvider resourcename=LCM_*Remote*Master* masterserver=NBUVLOFFMAS001-bck.capitahosting.co.uk groupid={00000000-0000-0000-0000-000000000000} userSequence=-1 userid="jobid=19534" firstuserid="jobid=19534" named resource allocation)
07/02/19 16:19:42.268 V-118-249 [CallbackQueue::resourcesAvailable] added allocations to the callback queue for request ID {2CBC112E-2AF4-11E9-97C3-EB316D1D4F92}
07/02/19 16:19:42.268 V-118-250 [CallbackQueue::notifyClientAvailable] queueing notification of availability of resources for request ID {2CBC112E-2AF4-11E9-97C3-EB316D1D4F92}
07/02/19 16:19:42.268 V-118-256 [CorbaCall_notifyClientAvailable::execute] sending availability of resources to nbjm for request ID {2CBC112E-2AF4-11E9-97C3-EB316D1D4F92}
07/02/19 16:19:42.269 [JobInstance::state] (0x7f084c007570) state=1(JobInstance.cpp:355)
07/02/19 16:19:42.269 [JobInstance::state] (0x7f084c007570) state=1(JobInstance.cpp:355)
07/02/19 16:19:42.269 V-117-296 [CommandJob::start] unable to get media server resource attribute
07/02/19 16:19:42.583 [JobInstance::state] (0x7f084c007570) state=5(JobInstance.cpp:355)
07/02/19 16:19:42.829 V-118-227 [ResBroker_i::requestResources] received resource request ID {2D1566FC-2AF4-11E9-9E88-646B3452986C}, priority 0, secondary priority 0, description COMMAND_JOB_19534
07/02/19 16:19:42.829 [ResBroker_i::requestResources] RequestSeq:
Request=0 provider=ReservationGroupProvider resourcename=ReservationGroupProvider RB::ReservationGroupRequest groupid={00000000-0000-0000-0000-000000000000}
SubRequest=0 provider=DriveOperationProvider resourcename=MEDIA RESOURCE userSequence=0 MediaRequest { mediaId=@aaaal mediaServer= mediaKey=0 userReservationId= assignedTime=0 client= usageType=29 mustBeNdmp=no driveName= drivePath= mediaPool= robotNumber=-1 slotNumber=-1 density=-1 ndmpControlHost= failIfNoMedia=yes externalFile= mediaType=0 mediaSubType=6 REQ_IS_EARI=true NDMPHostName= }
SubRequest=1 provider=ReservationGroupProvider resourcename=@aaaal userSequence=1 MediaReservationRequest { mountCount=1 mediaId=@aaaal }
07/02/19 16:19:42.843 V-118-226 [ResBroker_i::evaluateOne] Evaluating request ID {2D1566FC-2AF4-11E9-9E88-646B3452986C}
07/02/19 16:19:42.844 [reserveMedia] INITIATING:
07/02/19 16:19:42.844 [reserveMedia] masterServer = NBUVLOFFMAS001-bck.capitahosting.co.uk, jobType = 29, shared = 1
07/02/19 16:19:42.844 [reserveMedia] MediaRequest_Record: MediaServer = , PreferredMediaServer = , MediaId = @aaaal, MediaKey = 0, AssignedTime = 0, DriveName = , DrivePath = , PoolName = , Density = -1, RequiredNdmpTapeServer = , PreferredNdmpTapeServer = , MediaType = 0, MediaSubType = 6, StuNameForDupWrite = , FeatureNumber = -1
07/02/19 16:19:42.848 [reserveMedia] reservationKey = 20086
07/02/19 16:19:42.848 [reserveMedia] MediaRequest_Record: MediaServer = , PreferredMediaServer = , MediaId = @aaaal, MediaKey = 11, AssignedTime = 0, DriveName = , DrivePath = , PoolName = , Density = -1, RequiredNdmpTapeServer = , PreferredNdmpTapeServer = , MediaType = 0, MediaSubType = 6, StuNameForDupWrite = , FeatureNumber = -1
07/02/19 16:19:42.848 [reserveMedia] EXIT INFO:
07/02/19 16:19:42.848 [reserveMedia] reserved_dv_list = (11, 0)
07/02/19 16:19:42.848 [reserveMedia] EXIT STATUS = 0 (EMM_ERROR_Success, Success)

07/02/19 16:19:42.848 V-118-213 [EMMAccess::reserveMedia] request to reserve media (Master Server NBUVLOFFMAS001-bck.capitahosting.co.uk, JobType 29, Media ID @aaaal, Media Type 0, Media Sub Type 6, Shared Reservation true) returned 0
07/02/19 16:19:42.849 [allocateMedia] INITIATING:
07/02/19 16:19:42.849 [allocateMedia] masterServer = NBUVLOFFMAS001-bck.capitahosting.co.uk, client = , jobType = 29, capabilityFlags = 32800, fatPipePreference = 0, statusOnly = 0
07/02/19 16:19:42.849 [allocateMedia] MediaRequest_Record: MediaServer = , PreferredMediaServer = , MediaId = @aaaal, MediaKey = 0, AssignedTime = 0, DriveName = , DrivePath = , PoolName = , Density = -1, RequiredNdmpTapeServer = , PreferredNdmpTapeServer = , MediaType = 0, MediaSubType = 6, StuNameForDupWrite = , FeatureNumber = -1
07/02/19 16:19:42.849 [allocateMedia] reserved_dv_list = (11, 0)
07/02/19 16:19:42.849 [EMMAccess::reserveMedia] media reserved: mediaKey=11, reservationKey=20086
07/02/19 16:19:42.860 [fill_bsr] ALLOCATION: key = 20087, media server = nbuvwoffmed003-bck, mds_flags = 0, scsi_protection = 0, preempt_reservation = 0, preempted_ms_key = 0
07/02/19 16:19:42.860 [fill_bsr] MEDIA: id = @aaaal, key = 11
07/02/19 16:19:42.860 [fill_bsr] DRIVE: name = *NULL*, path = , secondary_path = , key = 0, ndmp_host_key = 0
07/02/19 16:19:42.860 [fill_bsr] STU: name = *NULL*, master server = *NULL*
07/02/19 16:19:42.860 [fill_bsr] DISKGROUP: id = NBUVAOFFDD003_NBUVWOFFMED003_01, key = 6
07/02/19 16:19:42.860 [fill_bsr] DISKVOLUME: media_id = @aaaal, key = 11
07/02/19 16:19:42.860 [fill_bsr] DISKMOUNTPOINT: mount_point = NBUVWOFFMED003_01, key = 8
07/02/19 16:19:42.860 [allocateMedia] EXIT INFO:
07/02/19 16:19:42.860 [allocateMedia] rbActions = 0
07/02/19 16:19:42.860 [allocateMedia] reserved_dv_list = (11, 0)
07/02/19 16:19:42.860 [allocateMedia] EXIT STATUS = 0 (EMM_ERROR_Success, Success)

07/02/19 16:19:42.874 V-118-79 [EMMAccess::QueryMachineConfig] query of machine configuration value COMMON_SERVER_FOR_DUP for machine NBUVLOFFMAS001-bck.capitahosting.co.uk, type 3 returned 2001045
07/02/19 16:19:42.874 V-118-164 [GroupProvider::allocateOne] created Resource Group ID {2D1C881A-2AF4-11E9-A8EB-1886616B296C}
07/02/19 16:19:42.874 V-118-146 [ProviderManager::allocate] ReservationGroupProvider returned Allocation Granted for request ID {2D1566FC-2AF4-11E9-9E88-646B3452986C}
07/02/19 16:19:42.874 [ResBroker_i::evaluateOne] resources found for request, requestId={2D1566FC-2AF4-11E9-9E88-646B3452986C}, userId=jobid=19534, birthTime=1549556382, timeToFirstEval=0 sec, timeToSuccessOrFailure=0 sec, numberOfTimesEvaluated=1
07/02/19 16:19:42.874 V-118-109 [ResBroker_i::passOne] resources found for request ID {2D1566FC-2AF4-11E9-9E88-646B3452986C}, following allocations are granted:
07/02/19 16:19:42.874 V-118-228 [ResBroker_i::passOne] allocation ID {2D1C8DBA-2AF4-11E9-9633-78DF7D2CD9BB}, group ID {2D1C881A-2AF4-11E9-A8EB-1886616B296C}, provider ReservationGroupProvider, resource aResourceGroup
07/02/19 16:19:42.874 [ResBroker_i::passOne] allocation: (Allocation: id={2D1C8DBA-2AF4-11E9-9633-78DF7D2CD9BB} provider=ReservationGroupProvider resourcename=aResourceGroup masterserver=NBUVLOFFMAS001-bck.capitahosting.co.uk groupid={2D1C881A-2AF4-11E9-A8EB-1886616B296C} userSequence=-1 userid="jobid=19534" firstuserid="jobid=19534" (ReservationGroupAllocation: groupId={2D1C881A-2AF4-11E9-A8EB-1886616B296C}))
07/02/19 16:19:42.874 V-118-228 [ResBroker_i::passOne] allocation ID {2D1C8B30-2AF4-11E9-93FE-35335A4AB1DC}, group ID {2D1C881A-2AF4-11E9-A8EB-1886616B296C}, provider ReservationGroupProvider, resource @aaaal
07/02/19 16:19:42.874 [ResBroker_i::passOne] allocation: (Allocation: id={2D1C8B30-2AF4-11E9-93FE-35335A4AB1DC} provider=ReservationGroupProvider resourcename=@aaaal masterserver=NBUVLOFFMAS001-bck.capitahosting.co.uk groupid={2D1C881A-2AF4-11E9-A8EB-1886616B296C} userSequence=1 userid="jobid=19534" firstuserid="jobid=19534" (MediaReservation: mountCount=1 reservationKey=20086 request=(MediaRequest: mediaId=@aaaal mediaServer= mediaKey=11 userReservationId= assignedTime=0 client= usageType=0 mustBeNdmp=no driveName= drivePath= mediaPool= robotNumber=-1 slotNumber=-1 density=-1 ndmpControlHost= failIfNoMedia=no externalFile= mediaType=0 mediaSubType=6 isNdmp=true isTirRestore=true isFlashbackupRestore=false isBlockMapRead=true isCatalogBackup=true isGcsCatalogBackup=true isVMWare=false isLifeCycle=false preferVtlToDirectAttachedTape=false)))
07/02/19 16:19:42.874 V-118-228 [ResBroker_i::passOne] allocation ID {2D1B916C-2AF4-11E9-B687-7877F50BB4E0}, group ID {2D1C881A-2AF4-11E9-A8EB-1886616B296C}, provider ReservationGroupProvider, resource @aaaal
07/02/19 16:19:42.874 [ResBroker_i::passOne] allocation: (Allocation: id={2D1B916C-2AF4-11E9-B687-7877F50BB4E0} provider=ReservationGroupProvider resourcename=@aaaal masterserver=NBUVLOFFMAS001-bck.capitahosting.co.uk groupid={2D1C881A-2AF4-11E9-A8EB-1886616B296C} userSequence=0 userid="jobid=19534" firstuserid="jobid=19534" (Media_Drive_Allocation_Record: allocationKey=20087 (Media_Drive_Record: MediaKey=11 MediaId=@aaaal MediaServer=nbuvwoffmed003-bck DriveKey=0 DriveName= PrimaryPath= PoolName= RobotNum=0 RobotType=0 MediaTypeName= DriveTypeName= NdmpControlHost= RetentionLevel=0 PolicyType=2 JobType=29 MasterServer=NBUVLOFFMAS001-bck.capitahosting.co.uk) (Storage_Unit_Record: STU= STUType=0 MasterServer= MediaServer= RobotType=0 RobotNumber=0 Density=0 OnDemandOnly=0 ConcurrentJobs=0 ActiveJobs=0 MaxMultiplexing=0 NdmpAttachHost= AbsolutePath=) (Bptm_Strings_Record: 0="MEDIADB 1 20087 @aaaal 11 *NULL* 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0" 1="VOLUME 1 @aaaal 11 *NULL* *NULL* *NULL* *NULL* 0 0 0 0 0 {00000000-0000-0000-0000-000000000000} 0" 2="DRIVE 4 *NULL* 0 *NULL* *NULL* 0 0 0 0 0 0 0 0 *NULL* *NULL* *NULL* *NULL* 0 0 0 0 0 0" 3="STORAGE 1 *NULL* 0 0 0 0 0 0 nbuvwoffmed003-bck nbuvwoffmed003-bck *NULL*" 4="DISKGROUP 0 10 DataDomain 16 NBUVAOFFDD003-dd 31 NBUVAOFFDD003_NBUVWOFFMED003_01" 5="DISKVOLUME 0 6 @aaaal 17 NBUVWOFFMED003_01 0" 6="DISKMOUNTPOINT 0 17 NBUVWOFFMED003_01" ) TpReqFileName=) useFatPipeTransport=false diskPath=NBUVWOFFMED003_01 diskGroupName=NBUVAOFFDD003_NBUVWOFFMED003_01 storageServerType=DataDomain diskGroupStorageServerName=NBUVAOFFDD003-dd failureCode=0)
07/02/19 16:19:42.877 V-118-249 [CallbackQueue::resourcesAvailable] added allocations to the callback queue for request ID {2D1566FC-2AF4-11E9-9E88-646B3452986C}
07/02/19 16:19:42.878 V-118-250 [CallbackQueue::notifyClientAvailable] queueing notification of availability of resources for request ID {2D1566FC-2AF4-11E9-9E88-646B3452986C}
07/02/19 16:19:42.879 V-118-256 [CorbaCall_notifyClientAvailable::execute] sending availability of resources to nbjm for request ID {2D1566FC-2AF4-11E9-9E88-646B3452986C}
07/02/19 16:19:48.201 [JobInstance::state] (0x7f084c007570) state=5(JobInstance.cpp:355)
07/02/19 16:19:48.201 [CommandJob::~CommandJob] (dump Command Job):
**************************
* CommandJob *
(jobid=19534 jobtype=20)
**************************
JobType = 20
PolicyName = SLP_Internal_Policy
PolicType = 36
PolicyDisplayName = SLP_SP_MED003_7DAYS
SchedName = Default_24x7_Window
SchedType = 2
Priority = 0
HostToStartJobOn = NBUVLOFFMAS001-bck.capitahosting.co.uk
MaxJobsPerPolicy = 2147483647
Resources(count) = 1
CommandPath = /usr/openv/netbackup/bin/admincmd/nbreplicate
CommandName = nbreplicate
CommandArgs = -classic_air -rcn 102 -cn 1 -priority 0 -slp_name SP_MED003_7DAYS -wcopt SFN -Bidfile /usr/openv/var/global/nbstserv/bid_file_rs_1549556381_0_0
(CommandJob.cpp:103)
07/02/19 16:19:48.203 V-116-216 [RecoverableJob::doJobCompletion] jobid=19534 completed with status 191
07/02/19 16:19:48.204 [PemTask::taskComplete] (ID:0x7f5ac8007458 CTX:0x7f5ac8007458) Active subtask count=0(PemTask.cpp:530)
07/02/19 16:19:48.204 [Info] V-116-14 CLIENT NONE POLICY SLP_Internal_Policy SCHED Default_24x7_Window EXIT STATUS 191 (no images were successfully processed) VBRF 1 191
07/02/19 16:19:48.204 [Error] V-116-85 backup of client NONE exited with status 191 (no images were successfully processed)
07/02/19 16:19:48.205 V-118-102 [ResBroker_i::releaseOne] releasing allocation ID {2D1B916C-2AF4-11E9-B687-7877F50BB4E0}
07/02/19 16:19:48.205 [GroupProvider::release(mdar)] releasing media/drive allocation, part of group (groupId={2D1C881A-2AF4-11E9-A8EB-1886616B296C})
07/02/19 16:19:48.210 [GroupProvider::releaseDriveMediaForSpanning] updating drive/media for ResourceGroup (groupId={2D1C881A-2AF4-11E9-A8EB-1886616B296C}), releasing allocation (id={2D1B916C-2AF4-11E9-B687-7877F50BB4E0})
07/02/19 16:19:48.211 [EMMProvider::release(mdar)] received release of mediaId=@aaaal, driveName=, STU=
07/02/19 16:19:48.221 [downStuCnts] INITIATING:
07/02/19 16:19:48.221 [downStuCnts] allocationKey = 20087
07/02/19 16:19:48.221 [downStuCnts] reserved_dv_list = (11, 0)
07/02/19 16:19:48.221 [downStuCnts] EXIT INFO:
07/02/19 16:19:48.221 [downStuCnts] reserved_dv_list = (11, 0)
07/02/19 16:19:48.221 [downStuCnts] EXIT STATUS = 0 (EMM_ERROR_Success, Success)

07/02/19 16:19:48.221 V-118-217 [EMMAccess::downStuCnts] request to down STU counts (STU , Allocation Key 20087) returned 0
07/02/19 16:19:48.227 [deallocate] INITIATING:
07/02/19 16:19:48.227 [deallocate] deallocType = 1
07/02/19 16:19:48.227 [deallocate] allocationKey = 20087
07/02/19 16:19:48.227 [deallocate] reserved_dv_list = (11, 0)
07/02/19 16:19:48.233 [deallocate] EXIT INFO:
07/02/19 16:19:48.233 [deallocate] rbActions = 0
07/02/19 16:19:48.233 [deallocate] reserved_dv_list = (11, 0)
07/02/19 16:19:48.233 [deallocate] EXIT STATUS = 0 (EMM_ERROR_Success, Success)

07/02/19 16:19:48.234 V-118-215 [EMMAccess::deallocate] request to deallocate resources (Allocation Key 20087, Deallocation Type 1) returned 0
07/02/19 16:19:48.234 [EMMAccess::deallocate] deallocated resource (allocationKey=20087)
07/02/19 16:19:48.240 [EMMProvider::release(reservation)] received release of media reservation, reservationKey=20086
07/02/19 16:19:48.242 [releaseMedia] INITIATING:
07/02/19 16:19:48.242 [releaseMedia] reservationKey = 20086
07/02/19 16:19:48.242 [releaseMedia] reserved_dv_list = (11, 0)
07/02/19 16:19:48.252 [releaseMedia] EXIT INFO:
07/02/19 16:19:48.252 [releaseMedia] reserved_dv_list = (11, 0)
07/02/19 16:19:48.252 [releaseMedia] EXIT STATUS = 0 (EMM_ERROR_Success, Success)

07/02/19 16:19:48.252 V-118-214 [EMMAccess::releaseMedia] request to release media (Reservation Key 20086) returned 0
07/02/19 16:19:48.252 [EMMAccess::releaseMedia] released media reservation (reservationKey=20086)
07/02/19 16:19:48.287 V-118-102 [ResBroker_i::releaseOne] releasing allocation ID {2CBF9BBE-2AF4-11E9-B61C-CBD1E34B86DD}
07/02/19 16:19:48.290 V-118-102 [ResBroker_i::releaseOne] releasing allocation ID {2D1C8DBA-2AF4-11E9-9633-78DF7D2CD9BB}
07/02/19 16:19:48.290 [GroupProvider::releaseGroup] marking ResourceGroup (groupId={2D1C881A-2AF4-11E9-A8EB-1886616B296C}) to be deleted when all resources are released
07/02/19 16:19:48.290 [GroupProvider::releaseGroup] deleting ResourceGroup (groupId={2D1C881A-2AF4-11E9-A8EB-1886616B296C})
07/02/19 16:19:48.290 V-118-165 [GroupProvider::releaseGroup] deleted Resource Group ID {2D1C881A-2AF4-11E9-A8EB-1886616B296C}
07/02/19 16:19:48.292 V-118-102 [ResBroker_i::releaseOne] releasing allocation ID {2D1C8B30-2AF4-11E9-93FE-35335A4AB1DC}
07/02/19 16:19:48.292 [GroupProvider::release] release of media reservation for ResourceGroup (groupId={2D1C881A-2AF4-11E9-A8EB-1886616B296C}) that no longer exists

  • johnmath99's avatar
    johnmath99
    6 years ago

    So, as it turnas out it wasn't NFS at all, it was config on the DDs.  I had set the encryption on the DD filesystem but not for encryption in flight during replication.  So, anyone encountering the same errors as shown above you may want to look at your DDs or DDVAs and ensure that encryption in-flight replication is enabled as well as encryption at rest:

    ddboost file-replication option set encryption enabled

5 Replies

  • It seems that bpdm (pid=2664) on media server nbuvwoffmed003-bck  reported status 174.

    Do you have bpdm log on this server?
    Level 3 should be good. 

      • Marianne's avatar
        Marianne
        Level 6

        This looks like DataDomain issue:

        09:02:51.139 [2260.5216] <16> 19968:bptm:2260:nbuvwoffmed003-bck: [8D4:1460] ddpi_remote_fileop() failed operation: 2, Err: 5004-nfs remote file op failed (nfs: No such file or directory)
        09:02:51.139 [2260.5216] <16> 19968:bptm:2260:nbuvwoffmed003-bck: E:\Program Files\Veritas\NetBackup\bin\\ost-plugins\libstspiDataDomain.dll:stspi_async_cancel_v11 remote fileop 'stop' [DDErrNo = 5004 (nothing matched)]
        09:02:51.139 [2260.5216] <16> 19968:bptm:2260:nbuvwoffmed003-bck: [8D4:1460] ddpi_imagesetop() failed operation 4 on 19968:nbuvloffmas001-bck.capitahosting.co.uk, Err: 5008-nfs imageset op failed (nfs: Invalid argument)
        09:02:51.139 [2260.5216] <16> 19968:bptm:2260:nbuvwoffmed003-bck: E:\Program Files\Veritas\NetBackup\bin\\ost-plugins\libstspiDataDomain.dll:stspi_async_cancel_v11 imageset cancel failed to imageset [DDErrNo = 5008 (invalid argument)] for 19968:nbuvloffmas001-bck.capitahosting.co.uk

        Similar post over here:

        https://vox.veritas.com/t5/NetBackup/AIR-with-DataDomain-replication-job-191/td-p/711075