Forum Discussion

robertngara's avatar
robertngara
Level 3
14 years ago

network protocol error (39)

I get this error when i run the command vmoprcmd -d on my media server. what could be wrong? Also the tape drives are in AVR state instead of TLD. I am running Netbackup 6.5.6. My master server is running Sles 11 and my media server has AIX 5.3.

7.1.x and Earlier
Backup and Recovery
Configuring
NetBackup
  • Marianne's avatar
    Marianne
    14 years ago

    Do 'nbemmcmd -listhosts -verbose' on master

    and 'bpps -x' on media server.

    Is media server perhaps behind firewall? Port 1556 (PBX) open in both directions?

    Where is the robotic control? If not master, you need to ensure PBX comms (bi-directional) between media server and robot control host as well.

    Proper forward and reverse lookup of host/IP in all directions? If FQDN is used anywhere, it must be able to resolve everywhere.

    Easiest is to add /etc/hosts entries on master and media server (and robot control host) with short and FQDN, e.g

    on master:

    10.10.10.1 master master.domain loghost
    10.10.10.2 media media.domain

    on media

    10.10.10.2 media media.domain loghost
    0.10.10.1 master master.domain

     

    Add appropriate entries if robot control host is another media server.

     

  • Yasuhisa_Ishika's avatar
    Yasuhisa_Ishika
    Level 6
    14 years ago
    How is the output of "nbemmcmd -listhosts", "bpps - x"? How about "vmoprcmd" on your master server?
  • robertngara's avatar
    robertngara
    Level 3
    14 years ago

    hi yasuhisa

     

    its a new setup.

    do you want me to run thr nbemmcmd and bpps commands on the master or media server?

    here is the output of vmoprcmd on the master server:

    sivls008:/usr/openv/volmgr/bin # ./vmoprcmd

                               HOST STATUS
    Host Name                                  Version   Host Status
    =========================================  =======   ===========
    sivls008                                   656000    ACTIVE
    mp2rs018                                   650000    OFFLINE
    sivrs055                                   656000    ACTIVE
                                    PENDING REQUESTS

                                        <NONE>
                                      DRIVE STATUS
    Drive Name               Label   Ready  RecMID  ExtMID  Wr.Enbl.  Type
        Host                       DrivePath                            Status
    =============================================================================
    IBM.ULTRIUM-TD3.000      No      No                     No        hcart3
        sivls008                   /dev/nst2                            TLD
    IBM.ULTRIUM-TD3.001      No      Yes            SV0043  Yes       hcart3
        sivls008                   /dev/nst3                            ACTIVE
    IBM.ULTRIUM-TD3.002      Yes     Yes    SV0034  SV0034  Yes       hcart3
        sivls008                   /dev/nst1                            ACTIVE
    IBM.ULTRIUM-TD3.003      No      No                     No        hcart3
        sivls008                   /dev/nst0                            TLD
    IBM.ULTRIUM-TD3.004      No      No                     No        hcart3
        sivrs055                   /dev/rmt4.1                          AVR
    IBM.ULTRIUM-TD3.005      No      No                     No        hcart3
        sivrs055                   /dev/rmt7.1                          AVR
    IBM.ULTRIUM-TD3.006      No      No                     No        hcart3
        sivrs055                   /dev/rmt6.1                          AVR
    IBM.ULTRIUM-TD3.007      No      No                     No        hcart3
        sivrs055                   /dev/rmt5.1                          AVR
    sivls008:/usr/openv/volmgr/bin #
     
  • Marianne's avatar
    Marianne
    Level 6
    14 years ago

    Do 'nbemmcmd -listhosts -verbose' on master

    and 'bpps -x' on media server.

    Is media server perhaps behind firewall? Port 1556 (PBX) open in both directions?

    Where is the robotic control? If not master, you need to ensure PBX comms (bi-directional) between media server and robot control host as well.

    Proper forward and reverse lookup of host/IP in all directions? If FQDN is used anywhere, it must be able to resolve everywhere.

    Easiest is to add /etc/hosts entries on master and media server (and robot control host) with short and FQDN, e.g

    on master:

    10.10.10.1 master master.domain loghost
    10.10.10.2 media media.domain

    on media

    10.10.10.2 media media.domain loghost
    0.10.10.1 master master.domain

     

    Add appropriate entries if robot control host is another media server.

     

  • robertngara's avatar
    robertngara
    Level 3
    14 years ago

    here is output of bpps -x from the media server:

    [root@sivrs055 /usr/openv/netbackup/bin]# ./bpps -x
    NB Processes
    ------------
        root 335920      1   0 16:17:16      -  0:00 /usr/openv/netbackup/bin/bpcompatd
        root 381078      1   0 16:17:18      -  0:00 /usr/openv/netbackup/bin/nbsl
        root 454766      1   0 16:17:18      -  0:03 /usr/openv/netbackup/bin/nbsvcmon
        root 471110      1   0 16:17:17      -  0:17 /usr/openv/netbackup/bin/nbrmms


    MM Processes
    ------------
        root 311508 438372   0 16:17:23      -  0:00 tldd
        root 364620      1   0 16:17:25      -  0:00 tldcd
        root 438372      1   0 16:17:15      -  0:01 /usr/openv/volmgr/bin/ltid
        root 479262 438372   0 16:17:25      -  0:03 avrd
        root 499718      1   0 16:17:16      -  0:00 vmd

    Shared Symantec Processes
    -------------------------
        root 258190      1   0   Oct 12      -  0:00 /opt/VRTSpbx/bin/pbx_exchange
    [root@sivrs055 /usr/openv/netbackup/bin]#
     
     
    and nbemmcmd -listhosts from the master server:
     
    sivls008:/usr/openv/netbackup/bin/admincmd # ./nbemmcmd -listhosts
    NBEMMCMD, Version:6.5.6
    The following hosts were found:
    server          sivls008
    master          sivls008
    media           mp2rs018
    media           sivrs055
    Command completed successfully.
    sivls008:/usr/openv/netbackup/bin/admincmd #
     
     
    yes i am using the /etc/hosts file for the host names resolution.
     
    there is no firewall and the port 1556 is open as i can telnet: telnet server-name 1556.
     
  • robertngara's avatar
    robertngara
    Level 3
    14 years ago

    hi Marianne

     

    something interesting here:

    I have defined my media server as sivrs055 but from the /usr/openv/netbackup/remote_versions folder i am getting a FQDN (sivrs055.eskom.co.za) and the file doesnt contain the netbackup version. its had 0.

     

  • Marianne's avatar
    Marianne
    Level 6
    14 years ago

    Where is the robot control? I see tldcd is running on this media server meaning it's the robot control host?

     

    Please paste output of your device config on this server:

    /usr/openv/volmgr/bin/tpconfig -l

     

    How did you do device config - manually or via the GUI wizard?

    Enable Media Manager verbose logging by adding VERBOSE line to /usr/openv/volmgr/bin/vm.conf.

    Stop/start NBU daemons.

    Ensure syslog is enabled on the AIX media server to enable media manager logging to /var/adm/messages (the syslogd daemon must be active).

     

     

  • robertngara's avatar
    robertngara
    Level 3
    14 years ago

    Hi Marianne

    Yes the robot control is on that media server. There is an HBA connected to the VTL from the media server.

    Here is the output of the tpconfig -l command:

     

    [root@sivrs055 /usr/openv/volmgr/bin]# ./tpconfig -l
    Device Robot Drive       Robot                    Drive                Device
    Type     Num Index  Type DrNum Status  Comment    Name                 Path
    robot      1    -    TLD    -       -  -          -                    /dev/ovpass2
      drive    -    0 hcart3    1      UP  -          IBM.ULTRIUM-TD3.004  /dev/rmt4.1
      drive    -    1 hcart3    4      UP  -          IBM.ULTRIUM-TD3.005  /dev/rmt7.1
      drive    -    2 hcart3    3      UP  -          IBM.ULTRIUM-TD3.006  /dev/rmt6.1
      drive    -    3 hcart3    2      UP  -          IBM.ULTRIUM-TD3.007  /dev/rmt5.1
    [root@sivrs055 /usr/openv/volmgr/bin]#
     

     

    I did configure the devices using GUI and it picked the 4 drives and one robot.

     

    I have added VERBOSE line in the vm.conf file.

     

     

  • Marianne's avatar
    Marianne
    Level 6
    14 years ago

    Looks like PBX level comms is fine from master to media server (GUI wizard worked) but problematic when initiated from the media server.

    Have you tried the FQDN aliases in /etc/hosts as I've suggested previously?

    You are going to need the logging that I've mentioned in the previous post as well as the following NBU logs:

    /usr/openv/volmgr/debug/daemon

    /usr/openv/volmgr/debug/reqlib

    When you create these log directories, also add the following line to vm.conf (to prevent /usr filling up):

    DAYS_TO_KEEP_LOGS = 3

    Verify that syslogd is running and that system messages are logged to /var/adm/messages. Stop/start NBU.

  • robertngara's avatar
    robertngara
    Level 3
    14 years ago

    Hi Marianne

    This is what i am getting from /usr/openv/volmgr/debug/reqlib, check the line in bold:

     

    [root@sivrs055 /usr/openv/volmgr/debug/reqlib]# more log.101410
    14:18:56.516 [225336] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
    14:18:56.516 [225336] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: vmd
    14:18:56.522 [225336] <2> get_long: (2) premature end of file (byte 1)
    14:18:56.522 [225336] <16> getrequestack: unable to get request ack string: There is an input or output error. (5), stat=-9
    14:18:56.522 [225336] <16> vmdbm_terminate: unable to send request to server:  network protocol error (39)
    [root@sivrs055 /usr/openv/volmgr/debug/reqlib]#
     

  • Marianne's avatar
    Marianne
    Level 6
    14 years ago

    So it seems I was right in previous post? Media Server initiated comms are not getting to the master. You need to check that all is well at TCP/IP comms level.

    Have you tried to alias FQDN in /etc/hosts as I've suggested initially?