12-06-2014 05:47 PM
Ok guys I am going to go ahead and apologize... I am a novice with Backup Exec,
I have a server running Windows Server 2003 Small Business. This server is the primary server for this company. I have a daily back up that runs every night.
I have server mounted single tape drive that uses 4mm tapes. They are 30GB tapes (I believe). We have 10 tapes that are manually changed out daily, one for each day. The tapes are numbered 1 - 10. ie. tape #1 - Monday week 1, tape #2 - Tuesday week 1, tape #6 - Tuesday week 2, etc. The backups have been performing successfully every night for months. When I go back and review the Job History, I see that the Job History shows the media to be tape #4 for the last few months on every back up that was performed. The tapes are being changed every day, yet Backup Exec thinks the media is the same. I was unaware of this since the backups were being performed successfully.
I noticed today when attempting to do a restore from a previous tape, that changing tapes was not recognized unless the media services were restarted. If I take out tape #2 put in tape #6, tape #6 is not recognized as online media unless the media services were restarted.
Now I need to perform a restore from a tape other than tape #4. I need to perform the restore from tape #2. When I put tape #2 into the tape drive and attempt the restore, it does not recognize the data that was successfully backed up to that tape. It references data that was backed up months ago. That is likely the last time the media services were restarted and tape #2 was in the tape drive.
All the backups were successful and the back up size was incrementally larger on every back. Therefore I know the backups are being written to the tapes.
We had a catastrophic failure this week and I need to perform a restore from earlier this week. I attempted this with no luck.
Please help!!! Any input anyone can offer would be greatly appreciated. If there is anything I can do to retrieve this data off of tape #4?
Thanks for any help you can offer.
Matt
12-08-2014 02:33 AM
If you are changing tapes you need to run an inventory job to identify which tape is in the drive. If you do not run an inventory job we will think the presvious tape is inserted.
A library may need a scan or inventory depending on configuration, a stans alon (non-Library) tape drive will always need an inventory
If you truncate your catalogs, you may need to run a catalog job as well.
Note: It is possible that the reason a service restart fixes the issue as someone has previously enabled an option to run an inventory on restart.
12-08-2014 04:10 AM
12-08-2014 05:37 AM
...try the following:
1. Stop the BE services along with the SQL service;
2. Rename the Data & Catalogs folders to Data.old & Catalogs.old;
3. Restart the BE services which recreates these 2 folders;
4. Put in the tape you want to restore from, and run an Inventory and then Catalog job;
5. Run the restore;
6. If successful, stop the BE services again, and delete the Data & Catalogs folders;
7. Rename the Data.old and Catalogs.old folders to Data & Catalogs, and then start up the BE services again.
Report back with an update.
Thanks!
12-08-2014 06:09 AM
OK I just re-read some of your original post.
If Backup Exec thinks the tape identifiers (IDs) are the same it will get muddled up and you may have to start with an empty database to do a restore.
Backup Exec uses media guids that are stored in the header/format information on the tape itself and they should be unique for every tape, if they are not unique then BE will think that the same tape is being used even though you are changing tapes.
As a quick test if you label (this erases data so only do it with a tape you are not worried about the content of) one of the affected tapes with a strange name (and let the label job finish), and then put in one of the other affected tapes and run an inventory if this other tape then shows the same name without you actually labelling it then you almost certainly have some tape ID conflicts
The only fix for this is to reformat the tapes using hardware vendor suplied utilities which we believe will randomize the identifiers. Backup Exec cannot do this itself. BTW we don't know why a small number of customers have reported this type of problem, although it is possible that it is something to do with how the tapes were first formatted by the manufacturers.
However to restore from tapes in this state (obviously don't reformat them) you will probably have to retire and delete all media records for that tape label in the Backup Exec admin console (and possibly follow CraigVs suggestion as well) - Be very careful with CraigVs suggestion as if using BE 2012 or 2014, then the retention of disk based backup sets are held in the catalogs, so emptying the catalogs folder can have a dangerous effect against backup set retention
If unsure they I's suggest it might be a good idea to log a formal support case for us to look into it further.
12-08-2014 07:00 AM
Craig thanks for the suggestion. Please see below. The Media Services would not restart once I renamed the Data and Catalog folders. I renamed the Catalog folders over the weekend with no issue fo the Media Services restarting. The issue must be with the Data folder being renamed. Attached is a screen shot from the Event Viewer in regards to the Media Services failing to start. Below is the log file that SQL created when attempting to start the Media Services.
2014-12-08 08:16:39.78 Server Microsoft SQL Server 2005 - 9.00.3042.00 (Intel X86)
Feb 9 2007 22:47:07
Copyright (c) 1988-2005 Microsoft Corporation
Express Edition on Windows NT 5.2 (Build 3790: Service Pack 2)
2014-12-08 08:16:39.78 Server (c) 2005 Microsoft Corporation.
2014-12-08 08:16:39.78 Server All rights reserved.
2014-12-08 08:16:39.78 Server Server process ID is 2352.
2014-12-08 08:16:39.78 Server Authentication mode is WINDOWS-ONLY.
2014-12-08 08:16:39.78 Server Logging SQL Server messages in file 'C:\Program Files\Microsoft SQL BackupExec\MSSQL.2\MSSQL\LOG\ERRORLOG'.
2014-12-08 08:16:39.78 Server This instance of SQL Server last reported using a process ID of 4316 at 12/8/2014 8:13:58 AM (local) 12/8/2014 2:13:58 PM (UTC). This is an informational message only; no user action is required.
2014-12-08 08:16:39.78 Server Registry startup parameters:
2014-12-08 08:16:39.78 Server -d C:\Program Files\Microsoft SQL BackupExec\MSSQL.2\MSSQL\DATA\master.mdf
2014-12-08 08:16:39.78 Server -e C:\Program Files\Microsoft SQL BackupExec\MSSQL.2\MSSQL\LOG\ERRORLOG
2014-12-08 08:16:39.78 Server -l C:\Program Files\Microsoft SQL BackupExec\MSSQL.2\MSSQL\DATA\mastlog.ldf
2014-12-08 08:16:39.79 Server SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required.
2014-12-08 08:16:39.79 Server Detected 2 CPUs. This is an informational message; no user action is required.
2014-12-08 08:16:39.93 Server Using dynamic lock allocation. Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node. This is an informational message only. No user action is required.
2014-12-08 08:16:39.95 Server Database mirroring has been enabled on this instance of SQL Server.
2014-12-08 08:16:39.96 spid5s Starting up database 'master'.
2014-12-08 08:16:40.32 spid5s Recovery is writing a checkpoint in database 'master' (1). This is an informational message only. No user action is required.
2014-12-08 08:16:40.56 spid5s SQL Trace ID 1 was started by login "sa".
2014-12-08 08:16:40.61 spid5s Starting up database 'mssqlsystemresource'.
2014-12-08 08:16:40.62 spid5s The resource database build version is 9.00.3042. This is an informational message only. No user action is required.
2014-12-08 08:16:40.92 spid8s Starting up database 'model'.
2014-12-08 08:16:40.93 spid5s Server name is 'JCS1\BKUPEXEC'. This is an informational message only. No user action is required.
2014-12-08 08:16:40.93 spid5s Starting up database 'msdb'.
2014-12-08 08:16:41.12 Server A self-generated certificate was successfully loaded for encryption.
2014-12-08 08:16:41.12 Server Server is listening on [ 'any' <ipv4> 1130].
2014-12-08 08:16:41.12 Server Server local connection provider is ready to accept connection on [ \\.\pipe\SQLLocal\BKUPEXEC ].
2014-12-08 08:16:41.12 Server Server named pipe provider is ready to accept connection on [ \\.\pipe\MSSQL$BKUPEXEC\sql\query ].
2014-12-08 08:16:41.12 Server Dedicated administrator connection support was not started because it is not available on this edition of SQL Server. This is an informational message only. No user action is required.
2014-12-08 08:16:41.12 Server The SQL Network Interface library could not register the Service Principal Name (SPN) for the SQL Server service. Error: 0x54b, state: 3. Failure to register an SPN may cause integrated authentication to fall back to NTLM instead of Kerberos. This is an informational message. Further action is only required if Kerberos authentication is required by authentication policies.
2014-12-08 08:16:41.12 Server SQL Server is now ready for client connections. This is an informational message; no user action is required.
2014-12-08 08:16:41.46 spid8s Clearing tempdb database.
2014-12-08 08:16:43.17 spid8s Starting up database 'tempdb'.
2014-12-08 08:16:43.87 spid5s Recovery is complete. This is an informational message only. No user action is required.
2014-12-08 08:16:43.87 spid11s The Service Broker protocol transport is disabled or not configured.
2014-12-08 08:16:43.87 spid11s The Database Mirroring protocol transport is disabled or not configured.
2014-12-08 08:16:43.96 spid11s Service Broker manager has started.
2014-12-08 08:21:18.64 Server Server resumed execution after being idle 243 seconds: user activity awakened the server. This is an informational message only. No user action is required.
12-08-2014 07:09 AM
Colin,
Thanks for the post.
"The only fix for this is to reformat the tapes using hardware vendor suplied utilities which we believe will randomize the identifiers." This would be something that was supplied by the hardware vendor of the tape drive? How would I go about finding this software, just contact the hardware vendor? Also, is there a specific functionality I should reference when contacting the hardware vendor about this?
Also, how do I go about opening up a formal support ticket?
Thanks.
Matt
12-09-2014 12:31 AM
I would suggest starting at the hardware vendor's website and seeing if they have any utilitites (HP Libray and Tape Tools for instance)
You would need a support agreement with Symantec to log an actual case with us and details of how to log a case would have then been provided.
If you are not sure if you have an agreement to need to talk to someone about arranging for one than I believe our customer care teams can help
http://www.symantec.com/en/uk/support/assistance_care.jsp
12-09-2014 12:32 AM
You shoudl only have renamed the catalog folder and then created an empty one. The data folders is critical to service startup as it contains the database.
12-09-2014 12:33 AM
Oh and I hope you have BackuP Exec 2012 (or 2014) and NOT 12.x as 12.x is End Of Life so there would not be much point in arranging a support agreement. (2012 is technically 14.0)