Backup job is not writing any data

[Planters]

Hi ,

One of the server backup is starting as per schedule, however no data is written onto tapes.

Netbackup Master server-> 6.5.6 - OS->Windows 2003

Client server-> 6.5.6 - OS->Windows 2008 R2

Backup Job willbe active and the operation says the jobs is writting, but data flow is not happening at all. Not even single KB data is written for this perticular server.

I have attached bpbkar and bpcd logs for reference, kinldy go through this and let me know how to fix this issue.

Please let me know if more details required.

[Marianne]

bpcd log shows successful connection from and to media servers except for VPW0INS04.generalgrowth.com - client does not have Server entry for this media server.

In bpbkar we see that it took the client more than 10 minutes to start sending data:

5:02:00.620 PM: [7416.1808] <4> dos_backup::V_VerifyFileList: INF - UBS Local Type for 'C:\*.*' --> 10020002
5:14:00.534 PM: [7416.5860] <16> dtcp_write: TCP - failure: send socket (336) (TCP 10054: Connection reset by peer)

By this time bpbrm on the media server has probably timed out.

Confirm on master server thaqt VSS is used for WOFB on this client:

Host Properties -> Master -> Client Attributes
Add client name if it does not exist in client list.
In Windows Open File Backup tab, select VSS.

Create bpfis log folder to verify snapshot at next backup attempt.

PS: 

Are you aware that NBU 6.x reached EOSL more than a year ago?
You NEED to upgrade!
LOTS of fixes in NBU 7.x for W2008 clients...

[Deb_Wilmot]

The bptm and bpbrm logs to correlate with the bpbkar and bpcd would be helpful.

[Planters]

Client is not ready for upgradation, hence no choice.

Open file backup option is already selected as VSS.

I have attached bpfis, bpbrm and bptm logs .

Please let me know if any informations required.

[Planters]

bpfis logs shows some error:  INF - EXIT STATUS 12: file open failed.

Does this related with VSS?

[Marianne]

Are the latest set of logs taken from the same backup attempt as the ones in your opening post?

Are the 4 line in bpfis the ONLY entries in bpfis log?

We will need client logs (bpbkar and bpfis) with higher logging levels as the information currently logged in these files is not enough to tell us what is wrong. Increase logging level on client to 5 and retry the backup.

Post a full set of logs:
Media server: bptm and bpbrm
Client: bpfis and bpbkar

PS: You seem to have ignored my reference to NBU 6.x EOSL?

[Planters]

Yes only few lines are there in the bpfis logs.

I have generated the logs by increasing the logging level to 5. All logs which i have provided was keeping the logging level 5

My Client doesnt want to upgrade the NBU. i have proposed before one year and still proposing, no luck

[Marianne]

Nope! bpbkar log is at level 0.

2:23:35.427 PM: [9440.7220] <4> ov_log::OVInit: INF - Starting log file: C:\Program Files\Veritas\NetBackup\logs\BPBKAR\111413.LOG

2:23:35.427 PM: [9440.7220] <4> ov_log::OVInit: GENERAL Log Level: 0

bptm is also level 0:

bptm: INITIATING (VERBOSE = 0)

and so is bpbrm:

bpbrm main: INITIATING (VERBOSE = 0)

Timestamps in bptm and bpbrm also do not correspond with timestamps in client logs.

We need a full set of logs from the same backup attempt. Only client logs need to be at level 5.

Your client should not expect any support while they are running an unsupported NBU version.
If annual maintenance is up to date, NBU upgrade is free.

[Planters]

Please find the logs attached, however bpfis log is not generated.

[Marianne]

Is this a new client that has never been backed up before? Or existing client that worked before?

Client is simply not generating backup stream. bpbkar gives no indication of what is happening in almost 5 hours of no activity (between 12:04 and 04:56):

12:04:20.213 PM: [4352.3300] <2> tar_backup::V_SetupProcessContinue: TAR - CONTINUE BACKUP received
12:04:20.432 PM: [4352.3300] <2> tar_backup_tfi::setupFileDirectives: TAR - backup filename = ALL_LOCAL_DRIVES
12:04:20.432 PM: [4352.4084] <4> tar_base::keepaliveThread: INF - keepalive thread is active with an interval of 60 seconds
12:04:20.572 PM: [4352.3300] <2> tar_backup_vxbsa::add: INF - called with '/'
12:04:20.572 PM: [4352.3300] <4> dos_backup::V_VerifyFileSystem: INF - Added: C:\
12:04:20.572 PM: [4352.3300] <4> dos_backup::V_VerifyFileSystem: INF - Added: D:\
12:04:20.572 PM: [4352.3300] <4> dos_backup::V_VerifyFileSystem: INF - Added: Shadow Copy Components:\
12:04:20.572 PM: [4352.3300] <4> dos_backup::V_VerifyFileSystem: INF - Added: System State:\
4:56:24.685 PM: [4352.4084] <16> dtcp_write: TCP - failure: send socket (332) (TCP 10054: Connection reset by peer)
4:56:24.685 PM: [4352.4084] <16> dtcp_write: TCP - failure: attempted to send 1 bytes
4:56:24.685 PM: [4352.4084] <16> tar_base::keepaliveThread: INF - keepalive thread abnormal exit :14

We can see that ALL_LOCAL_DRIVES is in Backup Selection - presumably without 'Allow Multiple Data Streams' enabled in Attributes?
Have you tried with with 'Allow Multiple Data Streams' selected?

We need to understand what is happening on the Client at the same time that the backup is kicking off.
The fact that no snapshot is attempted is weird, because all NBU settings say that snapshot should be taken:

-WOFB_enabled 

-WOFB_fim 1 - Use Microsoft's Volume Shadow Service (VSS)

-WOFB_usage 0 -  Individual Drive Snapshot

-WOFB_error 1 - Disable Snapshot and Continue 

Can you see in Windows Event Viewer logs if VSS snapshot is attempted? Or any VSS errors?

Check output of these commands on the client:

vssadmin list providers

vssadmin list writers

Test VSS snapshots using steps in this TN: http://www.symantec.com/docs/TECH47808

Let us know the result.

[Planters]

I found VSS error in the eventmanager which is mentioned below.

Volume Shadow Copy Service error: Unexpected error calling routine ConvertStringSidToSid(S-1-5-21-1209217424-826892672-922709458-51927.bak).  hr = 0x80070539, The security ID structure is invalid.
.

Operation:
   OnIdentify event
   Gathering Writer Data

Context:
   Execution Context: Shadow Copy Optimization Writer
   Writer Class Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
   Writer Name: Shadow Copy Optimization Writer
   Writer Instance ID: {4441fedb-1216-4256-8c4e-01d316b6c13b}

Also i have tried vssadmin list writers, which gave me below message.

U:\>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Waiting for responses.
These may be delayed if a shadow copy is being prepared.

[Marianne]

Check Microsoft Support site for VSS hotfixes or else log a Support call with Microsoft.

NetBackup is 'hanging' because VSS snapshot is not working.

You may want to disable WOFB in Master Server Client Attributes for this client until the VSS problem is resolved.