08-24-2018 12:32 PM
Hello,
Our master servers are linux 8.0, as are our media servers. The clients experiencing the issue are windows boxes. For the last three or four days, a few of our backup jobs are failing for clients using shadow copy. It's not all or them, just a few. Sometimes they're successful (partially--"1" code), but mostly they fail. Even the "successful" completions are showing a variation of "ERR - Unable to backup System State or Shadow Copy. Please check the state of VSS and associated Writers."
We've stopped/started NB services on the boxes. We've had our Windows SAs check things out--VSS is working, as are the writers.
Here is an example from the job details of the error:
Aug 22, 2018 8:22:49 AM - Info bpbrm (pid=21085) reading file list for client
Aug 22, 2018 8:22:50 AM - Info bpbrm (pid=21085) starting bpbkar on client
Aug 22, 2018 8:22:50 AM - connecting
Aug 22, 2018 8:22:50 AM - connected; connect time: 0:00:00
Aug 22, 2018 8:22:52 AM - Info bpbkar (pid=1684) Backup started
Aug 22, 2018 8:22:52 AM - Info bpbrm (pid=21085) bptm pid: 21126
Aug 22, 2018 8:22:52 AM - Info bpbkar (pid=1684) change time comparison:<disabled>
Aug 22, 2018 8:22:52 AM - Info bpbkar (pid=1684) archive bit processing:<enabled>
Aug 22, 2018 8:22:52 AM - Info bptm (pid=21126) start
Aug 22, 2018 8:22:52 AM - Info bptm (pid=21126) using 262144 data buffer size
Aug 22, 2018 8:22:52 AM - Info bptm (pid=21126) setting receive network buffer to 262144 bytes
Aug 22, 2018 8:22:52 AM - Info bptm (pid=21126) using 512 data buffers
Aug 22, 2018 8:22:54 AM - Info bptm (pid=21126) start backup
Aug 22, 2018 8:22:54 AM - Info bptm (pid=21126) backup child process is pid 21127
Aug 22, 2018 8:22:54 AM - begin writing
Aug 22, 2018 8:56:34 AM - Error bpbrm (pid=21085) from client client.company.com: ERR - Unable to backup System State or Shadow Copy. Please check the state of VSS and associated Writers.ERR - Unable to backup System State or Shadow Copy. Please check the state of VSS and associated Writers.ERR - Unable to backup System State or Shadow Copy. Please check the state of VSS and associated Writers.ERR - Unable to backup System State or Shadow Copy. Please check the state of VSS and associated Writers.INF - Estimate:-1 -1
Aug 22, 2018 8:56:34 AM - Error bpbrm (pid=21085) from client client.company.com: ERR - Unable to backup System State or Shadow Copy. Please check the state of VSS and associated Writers.INF - EXIT STATUS 69: invalid filelist specification
Aug 22, 2018 8:56:34 AM - Warning bpbrm (pid=21085) from client client.company.com: WRN - archive bits will NOT be cleared
Aug 22, 2018 8:56:35 AM - Critical bpbrm (pid=21085) unexpected termination of client client.company.com
Aug 22, 2018 8:56:35 AM - Error bptm (pid=21127) system call failed - Connection reset by peer (at ../child.c.1280)
Aug 22, 2018 8:56:35 AM - Error bptm (pid=21127) unable to perform read from client socket, connection may have been broken
Aug 22, 2018 8:56:38 AM - Info bptm (pid=21126) EXITING with status 42 <----------
Aug 22, 2018 8:56:38 AM - Error bpbrm (pid=21085) could not send server status message
Aug 22, 2018 8:56:39 AM - Info bpbkar (pid=1684) done. status: 42: network read failed
Aug 22, 2018 8:56:39 AM - end writing; write time: 0:33:45
---
Sometimes the error code is a 42, or 13, or 69
Solved! Go to Solution.
11-28-2018 07:45 AM
Unfortunately, there was never a valid solution. The issue cleared up on its own (at least no one ever claimed they did anything). Magic!
08-24-2018 01:40 PM
Whenever you see this:
>Error bpbrm (pid=21085) from client...
... Look at the client logs for more information. In this case, do the following:
- Turn up the logging levels on the client. On the Logging screen set the general level to 5. On the Windows Client / Client Settings screen, set the level to 2. Make sure the bpbkar and bpfis files exist in NetBackup\logs.
- Run a backup.
- Check the bpfis and bpkar logs for information about the error and the sequence of actions leading up to it.
- Check the Windows application event log for VSS errors.
08-30-2018 08:05 AM
1) Did you check VSS Writers state? https://www.veritas.com/support/en_US/article.000037440 ?
2) Did you check if some bpbkar32 processes are still running on the client?
09-11-2018 08:41 AM
Guys, thanks for your responses. Yes, I checked the writer, I didn't get any errors. Right now I have support checking things out.
09-11-2018 09:23 AM
09-11-2018 09:48 AM
Also, check how much free space is available on C:\? And if there is a limit on the size of shadow copy.
I had simiar failures on a couple of machines and found an event under Event Viewer > Windows Logs > System which indicated that the size limit of shadow copy was reached and it couldn't continue and the backup kept failing. Increased shadown copy limit/remove it, and the backup worked after that.
11-28-2018 07:45 AM
Unfortunately, there was never a valid solution. The issue cleared up on its own (at least no one ever claimed they did anything). Magic!