media server with MSDP downgrade from 7.5 to 7.1 possible?

[Dolphix]

Hi

We upgraded our environment yesterday to NB 7.5, from 7.1.03, and all uppgraded MSDP pools are not working now.... We added a new MSDP server and that is working so the upgrade seems to have screwed up the MSDP config or we are facing a bug of some sort. These backup servers also have a tape drives connected and those backups works fine which makes it even more strange...  

I have opened a support case but the progress is very limited.

My question does anyone know if you can downgrade a media server with MSDP from 7.5 to 7.1.0.x?

I am not sure it will solve my issues but it might be the next step in the troubleshooting plan...

Kind Regards

Dolphix

[Mark_Solutions]

What exactly do you mean by "it is not working"

What exactly doesn't work?

The upgrade upgrades disk storage units so I have my doubts about a downgrade and you would certainly need Symantec's advice on that

A number of issues have been found - one of which is connected to storage unit upgrades stopping SLPs from running

See the late breaking news section:

http://www.symantec.com/docs/TECH178334

[mph999]

Interestingly - just heard about another case like this ...

You can't just downgrade - 7.1 to 7.5 is a 'major' upgrade.

You need to log a call with Symantec for this one ...  You might want to get the storage.d  log (I think) ready ...

Martin

[Dolphix]

Hi Mark 

Sorry for not giving so much details but by not working, I mean we cant run backups or restores from the MSDP and if we try to do an inventory that fails as well with an RDSM error unable to communicate with the server and if we run nbdevconfig or any other command to get or set details on the device it fails, with a similar RDSM error. 

I had a look at the late breaking news first thing after we discovered the errors but nothing fits our problem... 

Martin, do you know if they have solved that issue? in that case do you have a symantec ticket i can guide our support guy with beacuse I think he is in the deep end at the pool right now and probably need all the help he can get! 

Kind Regards

Dolphix

[mph999]

Not solved at the moment.

Can't give out any case numbers, very sorry. (+ I don't actually have it), it was just a query on an email I saw.

Martin

[Dolphix]

Martin 

If you receive any updates on that email I would appreciate if you send an update how they solved the problem. 

//Andreas

[Mark_Solutions]

Just a quick check .. if these are WIndows MSDP Server did you do the memory setting shown in the release notes? Dont know if it will help but just wanted to check:

After you upgrade to NetBackup 7.5, verify that the following shared memory
values are set in the storage_path\etc\puredisk\agent.cfg file:
SharedMemoryEnabled=1
SharedMemoryBufferSize=262144
SharedMemoryTimeout=3600

Then, restart both the NetBackup Deduplication Manager (spad.exe) and the
NetBackup Deduplication Engine (spoold.exe).

Also mention of IPV6 not being liked now - again just worth a check you are only using IPV4

[Dolphix]

Hi Mark

Correct windows MSDP servers, with 36 GB memory but we run windows standard so we can only use 32GB. byt we only have a 32 TB dedup area.

The agent settings are correct as the once you posted, and IPV6 is not enabled(disselected in the GUI on the only network card available).

//Andreas

[Mark_Solutions]

Oh well - worth a shot!

Only other things from the 7.5 de-dupe guide:

Error message Server not found, invalid command parameter
RDSM has encountered an issue with STS where
the server was not found: getStorageServerInfo
Failed to create storage server ssname, invalid
command parameter
Example
Possible root causes:
■ When you configured the storage server, you selected a media
server that runs an unsupported operating system. All media
servers in your environment appear in the Storage Server
Configuration Wizard; be sure to select only a media server that
runs a supported operating system.
■ If you used the nbdevconfig command to configure the storage
server, you may have typed the host name incorrectly. Also, case
matters for the storage server type, so ensure that you use
PureDisk for the storage server type.

Guess this is probably a bug though! - just odd that backups and duplications do work!

[Dolphix]

backups, restores and duplications do not work I had a typo in my previous message.. (was can is now can't)

the back-line support suspect some kind of corruption in the tlog so we are running splogscan at the moment.. we will see what come out of that but it seems strange that we got the same error on 4 media servers...  

[Mark_Solutions]

I wonder if you just need to wait 24 hours for its internal cleanup to have run - had this after an appliance upgrade once - not sure how you can fire it off manually - how long has this been going on for?

#EDIT# see you said Yesterday - so guess it should have cleared by now

[Dolphix]

we upgraded about 28 hours ago, and it is status-quo here so I guess this is a diffrent issue. Has anyone experienced a successfull upgrade of a windows msdp to 7.5? Four out of four failed here so has anyone else been more lucky or did I do something wrong during the upgrade?

//Dolphix

[Mark_Solutions]

I guess I have to ask .... did you follow the upgrade guide?

1. editing the server.conf file for the cache size?

2. ran the cleanup command to start phase 2 of the metadata migration before doing anything else? - did this finish with a zero status?

3. What does install_path\NetBackup\bin\admincmd\nbemmcmd -listsettings -brief -machinename masterservername say the SLP_DSSU_MIGRATION_STATE is?

[Dolphix]

1. we did not edit this as we had less than 1 million images

2 post upgrade tasks complete with a minor warning as it complained that we did not have an catalog backup policy, no other warnings or errors

SLP_DSSU_MIGRATION_STATE=1

so I think the master server is OK, the issue is probably on the media servers...

[Mark_Solutions]

Can you confirm that

bpgetconfig LIST_FS_IMAGE_HEADERS

returns NO

Not a clustered Master is it?

If you run: nbemmcmd -listhosts

Do any media servers appear as foreign?

[Dolphix]

the command returns NO and you are able to connect to all media server from gui and they are displayed when running nbemmcmd -listhosts, we are able to run tape backup to the "only" and most vital thing is that you can't do anything with the MSDP pools...

[Mark_Solutions]

Ok - cant come up with anything else off the top of my head

Worth taking a look through the logs on the Media Server to see what you can spot

Also - and I know it is a drag! 0- but worth running an All Log Entries report for the last 29 hours and just scan down it to see if it says something about the Media Servers / MSDP Pools that may give a clue

[Mark_Solutions]

One thing that does come to mind (though the notes say that it should not stop backups and duplications working) is the "invalid SLP check".

Worth just opening up your SLPs to see if they have been marked as invalid

Post your All Log Entries report from just before the upgrade to an hour or two after you upgraded the Media Servers so that we can take a look

[PeteWall]

For MSDP upgrade issues, I'd look in either the install logs for anything suspicious or enable pdplugin logging for a backup session.

Here's how to enable pdplugin logging (log files can pile up quick):

Open <INSTALLPATH>\NetBackup\bin\ost-plugins\pd.conf
Uncomment the following lines: 
DEBUGLOG = C:\pdplugin.log (This path can be changed) 
LOGLEVEL = [0 - 10] 

This should capture any errors during the backup to MSDP.

[PeteWall]

Can you also paste the output of:

<INSTALLPATH>\pdde\Crcontrol.exe --getmode