cancel
Showing results for 
Search instead for 
Did you mean: 

Old master server cancelling jobs on new master server.

BAatSynthes
Level 3

Hi

I recently created a new master server for 7.6.0.1 and migrated all of my existing clients and media servers over to it.  We still have the old master server, but all of the services are disabled.  Everything is working fine in the new system, but if I restart services on the old master, it cancels all running jobs in the new environment.

In researching this I found this article:

http://www.symantec.com/business/support/index?page=content&id=TECH50484

 

I have run registry searches on the clients, media servers and the new master server and deleted any references to the old master, but jobs still get canceled.  It's not a huge issue, but some times we need to fire up the old server to do a restore.  Is there somewhere else on a Windows server that might be point back to the old master, besides the registry?

Any help would be greatly appreciated.  This is making me a little crazy.

Thanks

Brian

 

 

2 ACCEPTED SOLUTIONS

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

It seems that TECH50484 is 100% correct in identifying the cause of the problem, but that it is falling short with providing a real solution.

If the old master does not know about the media servers in the new environment it cannot and will not probe media servers for drive status and subsequently kill backups.

I firmly belief that decommissioning the media servers from the old master will solve your problem.

If the old master stil have devices configured for media servers, remove them with this command to prevent old master attempting to connect to media server for removal:

nbemmcmd -deletealldevices -machinename <server_name> -machinetype media

Decommission media servers with:

nbdecommission -oldserver <media-server>  -newserver <master>

Let us know how it goes...

View solution in original post

Mark_Solutions
Level 6
Partner Accredited Certified

Ah - good point Marianne - but i guess if all images need to be deleted then there is not much left to keep!

So maybe as a first stage do this for each media server:

bpimage -newserver mastername -olderserver mediaservername

Then delete all STU's etc, then do the deletehost - at least all of the images are retained that way

 

View solution in original post

18 REPLIES 18

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Have you done a similar registry search on the old master?

Is hostname the same as new master or totally different? 
Can we assume different IP addresses?

Run 'nbemmcmd -listhosts -verbose' as well as 'nbemmcmd -getemmserver' on both masters and compare output.

BAatSynthes
Level 3

I did search the old master.  The only references I can find is where I listed the old master server as the alternate media server for media servers (for restores).

The new master has a new and different name, but it has the IP address that was originally assigned to the old master server.  I needed to keep that address on the active NBU server to get through the firewall to some of my DMZ clients.

I can't run those commands on the old master for the moment, we have few backups running, but I will as soon as I can.

Thanks!

 

 

revarooo
Level 6
Employee

Why would you restart services on the old master??

Also do they share the same hostname? Shouldn't have two identical hostnames/IPs on the same network

BAatSynthes
Level 3

Revaroo,

We started a new master server with a different name and a new catalog at the beginning of the year.  We put the IP address of the original master server on the new master to maintain our access through the firewall.  The old master server was assigned a differtent IP.  

We kept all the same clients and media servers.  We did that by adding the new master server through client properties, then promoting it to master.  Then we deleted the old master through client properties.

We keep the old master server around to do restores from before 2014, but whenever we start it's services, it cancels everything running on the new master.   

BAatSynthes
Level 3

Marianne,

When I run this on the old master server, it says my media servers are still pointed at it, but when I run it from the new master, it says everything points to the new server.  Where does this command get it's info? (I changed the names because I can't post machine names to the internet)

 
nbemmcmd -listhosts -verbose
NBEMMCMD, Version: 7.5.0.4
The following hosts were found:
mediaserver3.synthes.com
        ClusterName = ""
        MachineName = "mediaserver3.synthes.com"
        FQName = "mediaserver3.synthes.com"
        LocalDriveSeed = ""
        MachineDescription = ""
        MachineFlags = 0x5
        MachineNbuType = media (1)
        MachineState = active for disk jobs (12)
        MasterServerName = "oldmastername.synthes.com"
        NetBackupVersion = 7.5.0.4 (750400)
        OperatingSystem = windows (11)
        ScanAbility = 5
mediaserver4.synthes.com
        ClusterName = ""
        MachineName = "mediaserver4.synthes.com"
        FQName = "mediaserver4.synthes.com"
        LocalDriveSeed = ""
        MachineDescription = ""
        MachineFlags = 0x5
        MachineNbuType = media (1)
        MachineState = not reachable by master (4)
        MasterServerName = "oldmastername.synthes.com"
        NetBackupVersion = 7.5.0.4 (750400)
        OperatingSystem = windows (11)
        ScanAbility = 5
SSO1.synthes.com
        ClusterName = ""
        MachineName = "SSO1.synthes.com"
        FQName = "SSO1.synthes.com"
        LocalDriveSeed = ""
        MachineDescription = ""
        MachineFlags = 0x5
        MachineNbuType = media (1)
        MachineState = not reachable by master (4)
        MasterServerName = "oldmastername.synthes.com"
        NetBackupVersion = 7.5.0.4 (750400)
        OperatingSystem = windows (11)
        ScanAbility = 5
us01bac0002
        ClusterName = ""
        MachineName = "us01bac0002"
        FQName = "us01bac0002.synthes.com"
        LocalDriveSeed = ""
        MachineDescription = ""
        MachineFlags = 0x5
        MachineNbuType = media (1)
        MachineState = not reachable by master (4)
        MasterServerName = "oldmastername.synthes.com"
        NetBackupVersion = 7.5.0.4 (750400)
        OperatingSystem = hpux (4)
        ScanAbility = 5
SSO2
        ClusterName = ""
        MachineName = "SSO2"
        FQName = "SSO2.synthes.com"
        LocalDriveSeed = ""
        MachineDescription = ""
        MachineFlags = 0x5
        MachineNbuType = media (1)
        MachineState = active for disk jobs (12)
        MasterServerName = "oldmastername.synthes.com"
        NetBackupVersion = 7.5.0.4 (750400)
        OperatingSystem = windows (11)
        ScanAbility = 5
oldmastername.synthes.com
        ClusterName = ""
        MachineName = "oldmastername.synthes.com"
        FQName = "oldmastername.synthes.com"
        GlobalDriveSeed = "VEND:#.:PROD:#.:IDX"
        LocalDriveSeed = ""
        MachineDescription = ""
        MachineFlags = 0x5
        MachineNbuType = master (3)
        MachineState = active for tape and disk jobs (14)
        NetBackupVersion = 7.5.0.4 (750400)
        OperatingSystem = windows (11)
        ScanAbility = 5

 

revarooo
Level 6
Employee

Sounds odd. Do the media servers on the new server have entries in the bp.conf/registry of the old server as a SERVER entry or EMMSERVER entry by any chance?

 

BAatSynthes
Level 3

I've searched the registry on these servers and deleted any references to the old master server I could find.  It's very strange.

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Please check hosts files on media servers as well - there should be no references to old-master-hostname.
If you are you are using DNS, double-check forward and reverse lookup.

The problem that I see is that your old master still have media servers listed.
You need to decommission all media servers on the old master and remove all references to media servers that are now functioning in the new environment.
Al image ownership should be transferred to old master and should be the only SERVER and/or Media Server name known by it.

What does nbemmcmd -getemmserver on new master show?
Ensure that all media servers point to new master as EMM server.

My recommendation when media servers need to move to a new master, is to first delete all devices on old master, decommission media server on old master to transfer image ownership, then uninstall NBU on media server, followed by fresh installation under new master and then device config.

 

PS:
the TN that you found (TECH50484) actually tells you what the problem is: 

Reason:
The old master server will ask for a report drive on the media server and finds out that a job is running and is not listed in his EMM database. It will send an abort signal to the media server to cancel the job.

 

The old master can only ask media servers to report on drives if old master still knows about those media servers. You therefore need to remove all references to media servers on the old master by decommissioning them.

BAatSynthes
Level 3

Thanks Marianne.  I will try decommissioning the media servers from the old master.  I was a little hesitant about doing that because I wanted to be able to rollback to the old master if things didn't work out with the new.  And now that the new environment is up and running (as long as the old one is down), I have a hard time bringing the old master up long enough to make any changes to it.  I think I will disable old master's NIC and do the decomm via iLO.  I thought removing the old master's name from the server list on everything else would be enough, but obviously that was not correct.  

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

It seems that TECH50484 is 100% correct in identifying the cause of the problem, but that it is falling short with providing a real solution.

If the old master does not know about the media servers in the new environment it cannot and will not probe media servers for drive status and subsequently kill backups.

I firmly belief that decommissioning the media servers from the old master will solve your problem.

If the old master stil have devices configured for media servers, remove them with this command to prevent old master attempting to connect to media server for removal:

nbemmcmd -deletealldevices -machinename <server_name> -machinetype media

Decommission media servers with:

nbdecommission -oldserver <media-server>  -newserver <master>

Let us know how it goes...

Mark_Solutions
Level 6
Partner Accredited Certified

I would just go for running this on the old master against each of your media servers :

nbemmcmd -deletehost -machinename mediaservername -machinetype media

That should stop it!

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

NBU will not allow media servers with valid images, devices, stus, etc to be deleted (which seems to be the case looking at the Op's previous post).

That is why I suggested nbdecommission...

Mark_Solutions
Level 6
Partner Accredited Certified

Ah - good point Marianne - but i guess if all images need to be deleted then there is not much left to keep!

So maybe as a first stage do this for each media server:

bpimage -newserver mastername -olderserver mediaservername

Then delete all STU's etc, then do the deletehost - at least all of the images are retained that way

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Seems to me all images need to be kept for restores... This is why old master is still around...

So yes, transfer all images or use nbdecommission which should remove devices, STUs, transfer images and remove EMM entries.

BAatSynthes
Level 3

It looks like both you have part of the solution.  I tried Marianne's suggestion of running nbdecommission, but that kept failing with a message about failed to delete the host.  Then I tried Marks bpimage command and after that nbdecommission is working.  It's still running at this point, so I'll let you know if it is the ultimate solution a little later.  We have a restore waiting to run on the old master right now.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

You will still need to get rid of media server entries. bpimage will transfer image ownership, but will not remove other components that keeps media server in the picture.

If nbdecommission is failing, we need to understand which part of the process is failing, so that you can manually remove all components:
devices, STUs, etc.

Try this command to list references to media server:

 nbdecommission -oldserver <hostname> -list_ref 

Mark_Solutions
Level 6
Partner Accredited Certified

Does your old master have any devices (tape drives?) - if not then when you get rid of the media servers how will you do the restore?

And if you do have tape drives are they dedicated to this system and not also used by the other system?

BAatSynthes
Level 3

Decomming the media servers worked.  We did this upgrade method 3 years ago, and the instructions in TECH50484 worked then.  Not sure what the difference is this time, but I'm glad to get past it.