cannot write data to socket, 10054

[GreenTree]

I'm running Windows 2008, enterprise on master and media servers.  Backups won't write to tape drives.

All jobs writing to the tape library will fail with the same error:

6/14/2011 12:14:24 PM - requesting resource LCM_hostname002-hcart-robot-tld-0
6/14/2011 12:14:30 PM - Info nbrb(pid=2848) Limit has been reached for the logical resource LCM_hostname002-hcart-robot-tld-0   
6/14/2011 2:58:14 PM - granted resource LCM_hostname002-hcart-robot-tld-0
6/14/2011 2:58:14 PM - started process RUNCMD (8336)
6/14/2011 2:58:15 PM - begin Duplicate
6/14/2011 2:58:15 PM - ended process 0 (8336)
6/14/2011 2:58:16 PM - requesting resource hostname002-hcart-robot-tld-0
6/14/2011 2:58:16 PM - requesting resource @aaaam
6/14/2011 2:58:16 PM - reserving resource @aaaam
6/14/2011 2:58:24 PM - awaiting resource hostname002-hcart-robot-tld-0 - No drives are available
6/14/2011 3:38:37 PM - reserved resource @aaaam
6/14/2011 3:38:37 PM - granted resource 000392
6/14/2011 3:38:37 PM - granted resource IBM.ULTRIUM-TD4.002
6/14/2011 3:38:37 PM - granted resource hostname002-hcart-robot-tld-0
6/14/2011 3:38:37 PM - granted resource MediaID=@aaaam;DiskVolume=PureDiskVolume;DiskPool=SN_Disk_Pool;Path=PureDiskVolume;StorageServer=hos...
6/14/2011 3:38:41 PM - started process bptm (9708)
6/14/2011 3:39:04 PM - started process bpdm (8316)
6/14/2011 3:39:05 PM - started process bptm (7136)
6/14/2011 3:39:06 PM - started process bpdm (8932)
6/14/2011 3:39:11 PM - begin reading
6/14/2011 3:42:18 PM - Error bpdm(pid=8932) cannot write data to socket, 10054      
6/14/2011 3:42:35 PM - Error bpduplicate(pid=8336) host hostname003 backup id Y004_1307627938 read failed, socket write failed (24). 
6/14/2011 3:42:35 PM - Error bpduplicate(pid=8336) host hostname002 backupid Y004_1307627938 write failed, termination requested by administrator (150). 
6/14/2011 3:42:35 PM - Error bpduplicate(pid=8336) Duplicate of backupid Y004_1307627938 failed, termination requested by administrator (150).  
6/14/2011 3:42:36 PM - Error bpduplicate(pid=8336) Status = no images were successfully processed.     
6/14/2011 3:42:38 PM - end Duplicate; elapsed time: 00:44:23
no images were successfully processed(191)

It appreas that it can read the data but cannot write.

Need help Please!!!

[GreenTree]

It's running NBU 7.0

[Marianne]

Test a small backup to Storage Unit hostname002-hcart-robot-tld-0 to ensure hardware functionality and network comms.

Please give more details about duplication job configuration - I see 3 hostnames playing a role here:

StorageServer=hostname004

MediaServer=hostname003

Destination STU=hostname002

Before we troubleshoot the duplication job, please verify basic comms between hostname002 and hostname003. Create a test policy to backup hostname003 as a client to STU hostname002-hcart-robot-tld-0.

Please also provide output of the following:

bpstulist -L

(open cmd as Administrator, cd <install-path>\veritas\netbackup\bin\admincmd)

[GreenTree]

Ran a test backup to Storage Unit hostname002-hcart-robot-tld-0.  Did it through a policy and it ran successfully.  So I think the hardware and network is working fine.

As for the duplication configuration;

Master Sever: hostname001

Media servers: hostname002, hostname003, hostname178, hostname012, hotname011

Puredisk servers: Hhostname001, Hhostname002, Hhostname004, Hhostname005

All clients are connected to the media servers, hostname178, hostname012, hotname011 which are backed up to Puredisk Storage Pool 1 (Puredisk content router:Hhotname001 and Hhostname002)

Then the data gets deduped to Puredisk Storage Pool 2 (Puredisk content router:Hhostname004, Hhostname005)

Final deduplication from Puredisk Storage Pool 2 to the tape library through the media servers, hostname002 and hostname003. 

This is the part where all is failing.

[GreenTree]

Label:                hostname002-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      hotname002
Number of Drives:     10
On Demand Only:       no
Density:              hcart (6)
Robot Type/Number:    TLD (8) / 0
Max Fragment Size:    1048576
Max MPX/drive:        6

Label:                hostname003-hcart-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      hotname003
Number of Drives:     10
On Demand Only:       no
Density:              hcart (6)
Robot Type/Number:    TLD (8) / 0
Max Fragment Size:    1048576
Max MPX/drive:        1

Label:                Master_Disk_Storage
Storage Unit Type:    Disk
Media Subtype:        Basic (1)
Host Connection:      hotname001
Concurrent Jobs:      1
On Demand Only:       yes
Path:                 "E:\disk_storage"
Robot Type:           (not robotic)
Max Fragment Size:    524288
Max MPX:              1
Stage data:           no
Block Sharing:        no
File System Export:   no
High Water Mark:      98
Low Water Mark:       80
Ok On Root:           no

Label:                Storage_Unit_001
Storage Unit Type:    Disk
Media Subtype:        DiskPool (6)
Host Connection:      hostname178
Host Connection:      hostname012
Host Connection:      hostname011
Concurrent Jobs:      10
On Demand Only:       yes
Robot Type:           (not robotic)
Max Fragment Size:    51200
Max MPX:              1
Block Sharing:        yes
File System Export:   yes
Ok On Root:           no
Disk Pool:            Disk_Pool_001

Label:                Storage_Unit_002
Storage Unit Type:    Disk
Media Subtype:        DiskPool (6)
Host Connection:      hostname003
Host Connection:      hostname002
Concurrent Jobs:      10
On Demand Only:       yes
Robot Type:           (not robotic)
Max Fragment Size:    51200
Max MPX:              1
Block Sharing:        yes
File System Export:   yes
Ok On Root:           no
Disk Pool:            Disk_Pool_002