Forum Discussion

osvaldo_olmedo's avatar
7 years ago

Unable to start vmd on master server

Hello, I have recently upgrade NBU to 8.1.1 in VCS environment. After a couple of days, we realized that vmd was not running on master server (all devices are configured in media servers). When we try to start vmd starting ltid we receive the following error: "Failed to validate this host with the EMM server". Everything is working fine, except that we are not able to run inverntory robots or work with storage devices from the console (for inventory we just run vmupdate from the media server).

When vmd is configured in debug mode, we see the following in the log:

10:47:43.550 [25100532] <4> vmd: INITIATING
10:47:43.550 [25100532] <2> mm_getnodename: cached_hostname shortnamehostname.com.uy, cached_method 3
10:47:43.565 [25100532] <2> mm_getnodename: (6) hostname shortnamehostname.com.uy (from cached_hostname)
10:47:43.565 [25100532] <4> vmd: Host name is shortnamehostname.com.uy
10:47:45.590 [25100532] <4> vmd: emmlib_initialize bkserverag, 1556, <status 0>
10:47:45.590 [25100532] <2> mm_getnodename: (0) hostname shortnamehostname.com.uy (from cached_hostname)
10:47:45.618 [25100532] <16> emmlib_UpdateHostEx: (0) UpdateMachine failed, emmError = 2001071, nbError = 0
10:47:45.618 [25100532] <16> AddAndVerifyHost: Failed to update this host. EMM error code = 2001071
10:47:45.618 [25100532] <16> AddAndVerifyHost: (-) Translating EMM_ERROR_MachineAlreadyExistAsMismatchedType(2001071) to 136 in the Media context
10:47:45.618 [25100532] <16> vmd: Failed updating host information in EMM
10:47:45.619 [25100532] <16> vmd: terminating - the operation requested has failed (163)
10:47:45.619 [25100532] <16> uninitialize: (-) Invalid Connection ID.
10:47:45.619 [25100532] <16> emmlib_uninitialize: (0) Failed to release EMM session and database objects
10:47:45.619 [25100532] <16> vmd: terminating - daemon terminated (7)

 

The weird thing is that in vmd log the hostname is shown as fqn name however the hostname is listed using the shortname (bpclntcm -hn hostname)

 

Any help or comment will be very welcome

 

Thanks and best regards

4 Replies

  • Please show us output of these commands on the active master server node:

    bpclntcmd -self

    nbemmcmd -listhosts -verbose

    Contents of bp.conf

    Extremely important that we distinguish between physical and virtual hostnames in a clustered environment, so, if you need to replace actual hostnames, please use names where we can clearly see difference between nodename and virtual hostname. 
    e.g:
    node1  
    node1.fqdn
    virtmast
    virtmast.fqdn
    etc.

    PS:
    Are you aware that NetBackup 8.1.2 and later does not support Master or Media Server on AIX?

    • osvaldo_olmedo's avatar
      osvaldo_olmedo
      Level 4

      Marianne,

       

      Thanks for your reply, here the outputs:

       

      root@bkmasterag) /> bpclntcmd -self
      yp_get_default_domain failed: (6) internal yp server or client error
      NIS does not seem to be running: (1) args to yp function are bad
      gethostname() returned: bkserverag
      host bkserverag: bkserverag at 172.26.151.125
      aliases:     bkserverag     172.26.151.125
      getfqdn(bkserverag) returned: bkserverag.corp.acme.com.uy
      you have mail in /usr/spool/mail/root

       

      (root@bkmasterag) /> nbemmcmd -listhosts -verbose
      NBEMMCMD, Version: 8.1.1
      The following hosts were found:
      bkserverag
              MachineName = "bkserverag"
              FQName = "bkserverag.corp.acme.com.uy"
              MachineDescription = ""
              MachineNbuType = server (6)
      bkserverag
              ClusterName = ""
              MachineName = "bkserverag"
              FQName = "bkserverag.corp.acme.com.uy"
              GlobalDriveSeed = "VEND:#.:PROD:#.:IDX"
              LocalDriveSeed = ""
              MachineDescription = ""
              MachineFlags = 0x17
              MachineNbuType = master (3)
              MachineState = active for disk jobs (12)
              NetBackupVersion = 8.1.1.0 (811000)
              OperatingSystem = rs6000 (5)
              ScanAbility = 5
      bkserverpar
              ClusterName = ""
              MachineName = "bkserverpar"
              FQName = "bkserverpar.corp.acme.com.uy"
              LocalDriveSeed = ""
              MachineDescription = ""
              MachineFlags = 0x15
              MachineNbuType = media (1)
              MachineState = active for tape and disk jobs (14)
              MasterServerName = "bkserverag"
              NetBackupVersion = 8.1.1.0 (811000)
              OperatingSystem = rs6000 (5)
              ScanAbility = 5
      smediaag
              ClusterName = ""
              MachineName = "smediaag"
              FQName = "smediaag.corp.acme.com.uy"
              LocalDriveSeed = ""
              MachineDescription = ""
              MachineFlags = 0x17
              MachineNbuType = media (1)
              MachineState = active for disk jobs (12)
              MasterServerName = "bkserverag"
              NetBackupVersion = 8.1.1.0 (811000)
              OperatingSystem = windows (11)
              ScanAbility = 5
      bkserverpar
              MachineName = "bkserverpar"
              FQName = "bkserverpar.corp.acme.com.uy"
              MachineDescription = "PureDisk"
              MachineFlags = 0x2
              MachineNbuType = ndmp (2) (storage_server)
      svnxparndmp
              MachineName = "svnxparndmp"
              FQName = "svnxparndmp.corp.acme.com.uy"
              MachineDescription = ""
              MachineFlags = 0
              MachineNbuType = ndmp (2)
      bkmediaag.corp.acme.com.uy
              ClusterName = ""
              MachineName = "bkmediaag.corp.acme.com.uy"
              FQName = "bkmediaag.corp.acme.com.uy"
              LocalDriveSeed = ""
              MachineDescription = ""
              MachineFlags = 0x17
              MachineNbuType = media (1)
              MachineState = active for tape and disk jobs (14)
              MasterServerName = "bkserverag"
              NetBackupVersion = 8.1.1.0 (811000)
              OperatingSystem = rs6000 (5)
              ScanAbility = 5
      smediapar
              ClusterName = ""
              MachineName = "smediapar"
              FQName = "smediapar.corp.acme.com.uy"
              LocalDriveSeed = ""
              MachineDescription = ""
              MachineFlags = 0x17
              MachineNbuType = media (1)
              MachineState = active for disk jobs (12)
              MasterServerName = "bkserverag"
              NetBackupVersion = 8.1.1.0 (811000)
              OperatingSystem = windows (11)
              ScanAbility = 5
      COFC-CZJ6340BBN01
              MachineName = "COFC-CZJ6340BBN01"
              FQName = "COFC-CZJ6340BBN01"
              MachineDescription = "hp-StoreOnceCatalyst"
              MachineFlags = 0x2
              MachineNbuType = ndmp (2) (storage_server)
      COFC-CZJ64301XG01
              MachineName = "COFC-CZJ64301XG01"
              FQName = "COFC-CZJ64301XG01"
              MachineDescription = "hp-StoreOnceCatalyst"
              MachineFlags = 0x2
              MachineNbuType = ndmp (2) (storage_server)
      COFC-hpAgraciada
              MachineName = "COFC-hpAgraciada"
              FQName = "COFC-hpAgraciada"
              MachineDescription = "hp-StoreOnceCatalyst"
              MachineFlags = 0x2
              MachineNbuType = ndmp (2) (storage_server)
      COFC-hpParaguay
              MachineName = "COFC-hpParaguay"
              FQName = "COFC-hpParaguay"
              MachineDescription = "hp-StoreOnceCatalyst"
              MachineFlags = 0x2
              MachineNbuType = ndmp (2) (storage_server)
      svwvcenter5.corp.acme.com.uy
              MachineName = "svwvcenter5.corp.acme.com.uy"
              FQName = "svwvcenter5.corp.acme.com.uy"
              MachineDescription = ""
              MachineNbuType = virtual_machine (10)
      bkmediaag.corp.ue.com.uy
              MachineName = "bkmediaag.corp.ue.com.uy"
              FQName = "bkmediaag.corp.ue.com.uy"
              MachineDescription = ""
              MachineNbuType = foreign_media (13)
      svwveritasmsdp
              ClusterName = ""
              MachineName = "svwveritasmsdp"
              FQName = "svwveritasmsdp.corp.acme.com.uy"
              LocalDriveSeed = ""
              MachineDescription = ""
              MachineFlags = 0x17
              MachineNbuType = media (1)
              MachineState = active for disk jobs (12)
              MasterServerName = "bkserverag"
              NetBackupVersion = 8.1.1.0 (811000)
              OperatingSystem = windows (11)
              ScanAbility = 5
      lvwnetintapp
              MachineName = "lvwnetintapp"
              FQName = "lvwnetintapp.corp.acme.com.uy"
              MachineDescription = ""
              MachineNbuType = remote_master (11)
      svwveritasevault
              MachineName = "svwveritasevault"
              FQName = "svwveritasevault.corp.acme.com.uy"
              MachineDescription = "PureDisk"
              MachineFlags = 0x2
              MachineNbuType = ndmp (2) (storage_server)
      svwveritasmsdp
              MachineName = "svwveritasmsdp"
              FQName = "svwveritasmsdp.corp.acme.com.uy"
              MachineDescription = "PureDisk"
              MachineFlags = 0x2
              MachineNbuType = ndmp (2) (storage_server)
      COFC-hpAgraciada2
              MachineName = "COFC-hpAgraciada2"
              FQName = "COFC-hpAgraciada2"
              MachineDescription = "hp-StoreOnceCatalyst"
              MachineFlags = 0x2
              MachineNbuType = ndmp (2) (storage_server)
      smediaag2.corp.acme.com.uy
              ClusterName = ""
              MachineName = "smediaag2.corp.acme.com.uy"
              FQName = "smediaag2.corp.acme.com.uy"
              LocalDriveSeed = ""
              MachineDescription = ""
              MachineFlags = 0x17
              MachineNbuType = media (1)
              MachineState = active for disk jobs (12)
              MasterServerName = "bkserverag"
              NetBackupVersion = 8.1.1.0 (811000)
              OperatingSystem = windows (11)
              ScanAbility = 5
      Command completed successfully.

      ----------------------------------------------------------------------------------------------------

       

      bp.conf:

      SERVER = bkserverag
      SERVER = bkmasterag
      SERVER = bkmasterag.corp.acme.com.uy
      SERVER = bkmasterpar
      SERVER = bkmasterpar.corp.acme.com.uy
      SERVER = bkserverpar
      SERVER = bkserverpar.corp.acme.com.uy
      SERVER = bkmediaag.corp.acme.com.uy
      SERVER = pc149806
      SERVER = pc149801
      SERVER = pc154046
      SERVER = pc154036
      SERVER = pc149819
      SERVER = pvwsopdis
      SERVER = svwveritasops
      SERVER = pc148636
      SERVER = pc147171
      SERVER = pc147076
      SERVER = svnxparndmp
      SERVER = svwtareasprog
      SERVER = smediaag
      SERVER = smediapar
      SERVER = svwdevapp
      SERVER = svwveritasmsdp
      SERVER = svwnetctrlmprd
      SERVER = pvwsyasanadm
      SERVER = smediaag2.corp.acme.com.uy
      SERVER = smediaag2
      CLIENT_NAME = bkmasterag
      CLUSTER_NAME = bkserverag
      ALLOW_MEDIA_OVERWRITE = DBR
      ALLOW_MEDIA_OVERWRITE = TAR
      ALLOW_MEDIA_OVERWRITE = CPIO
      ALLOW_MEDIA_OVERWRITE = ANSI
      ALLOW_MEDIA_OVERWRITE = AOS/VS
      ALLOW_MEDIA_OVERWRITE = MTF1
      ALLOW_MEDIA_OVERWRITE = RS-MTF1
      ALLOW_MEDIA_OVERWRITE = BE-MTF1
      #VERBOSE = 5
      SERVER_SENDS_MAIL = YES
      CLIENT_CONNECT_TIMEOUT = 1800
      CLIENT_READ_TIMEOUT = 10000
      BPSTART_TIMEOUT = 9000
      SLAVE_CONNECT_TIMEOUT = 3600
      ALLOW_NON_RESERVED_PORTS = YES
      RANDOM_PORTS = NO
      CLIENT_RESERVED_PORT_WINDOW = 900 999
      SERVER_RESERVED_PORT_WINDOW = 800 899
      BPEND_TIMEOUT = 3600
      KEEP_JOBS_HOURS = 720
      KEEP_JOBS_SUCCESSFUL_HOURS = 720
      # WAIT_IN_QUEUE = YES
      # QUEUE_ON_ERROR = YES
      INCOMPLETE_JOB_CLEAN_INTERVAL = 1
      # BPTM_QUERY_TIMEOUT = 960
      EMMSERVER = bkserverag
      VXDBMS_NB_DATA = /usr/openv/db/data
      MEDIA_UNMOUNT_DELAY = 180
      INCOMPLETE_BKUP_JOB_CLEAN_INTERVAL = 1
      BPDM_VERBOSE = 0
      MEDIA_SERVER = bkserverpar
      MEDIA_SERVER = bkmediaag.corp.acme.com.uy
      MEDIA_SERVER = smediaag
      MEDIA_SERVER = smediapar
      MEDIA_SERVER = svwveritasmsdp
      BPRD_VERBOSE = 0
      BPSCHED_VERBOSE = 5
      # NBPEM_VERBOSE = 5
      # NBJM_VERBOSE = 5
      # NBRB_VERBOSE = 5
      BPTM_VERBOSE = 0
      BPBRM_VERBOSE = 0
      CLIENT_PORT_WINDOW = 0 0
      USE_VXSS = PROHIBITED
      SPS_REDIRECT_ALLOWED = dag2 smailmrpar2
      SPS_REDIRECT_ALLOWED = dag2 smailmrag2
      SPS_REDIRECT_ALLOWED = dag1 smailmrag1
      SPS_REDIRECT_ALLOWED = dag1 smailmrpar1
      SPS_REDIRECT_ALLOWED = dag2 SMAILMRPAR1
      SPS_REDIRECT_ALLOWED = DAG svwmail1
      SPS_REDIRECT_ALLOWED = DAG svwmail2
      SPS_REDIRECT_ALLOWED = DAG svwmail3
      SPS_REDIRECT_ALLOWED = DAG svwmail4
      SPS_REDIRECT_ALLOWED = DAG svwmail5
      SPS_REDIRECT_ALLOWED = DAG svwmail6
      JOB_PRIORITY = 0 0 90000 90000 90000 90000 85000 85000 80000 80000 80000 80000 75000 75000 70000 70000 50000 50000 45000 0 0 0 0 0
      AUTHENTICATION_DOMAIN = bkserverag "ADDED AUTOMATICALLY" PASSWD bkserverag 0
      AUTHORIZATION_SERVICE = bkserverag 0
      HOST_CACHE_TTL = 3600
      FORCE_RESTORE_MEDIA_SERVER = bkserverag bkmediaag.corp.acme.com.uy
      TRUSTED_MASTER = lvwnetintapp.corp.acme.com.uy
      TRUSTED_MASTER = lvwnetintapp
      BPCD_WHITELIST_PATH = /openv/scripts/scripts_ctrlm/LOG/bpdup_ag.ls
      BPCD_WHITELIST_PATH = /openv/scripts/scripts_ctrlm/LOG/bpdup_par.ls
      VM_PROXY_SERVER = smediaag
      VM_PROXY_SERVER = smediapar
      #ENABLE_NBCURL_VERBOSE = 1
      VERBOSE = 0
      TELEMETRY_UPLOAD = YES
      WEBSVC_GROUP = nbwebgrp
      WEBSVC_USER = nbwebsvc
      VXSS_SERVICE_TYPE = INTEGRITYANDCONFIDENTIALITY

      -----------------------------------------------------------------------------------

       

      Yes, we are aware of the AIX supported version, we will migrate to other platfom the master server

      Thanks and best regards

       

      Osvaldo

       

      • Mehul_Pal's avatar
        Mehul_Pal
        Level 5

        your emm server is showing as EMMSERVER = bkserverag

        Could you please check your /etc/host file and also do nslookup for the same host name.