09-17-2020 08:36 AM
Hello,
The 70% my job are failed, because the error 2074 disk is down, the dp is up but the dv is down
Disk Pool Name : dp_disk_lprpmedialevel01
Disk Type : PureDisk
Disk Volume Name : PureDiskVolume
Disk Media ID : @aaaal
Total Capacity (GB) : 34305,92
Free Space (GB) : 28205,36
Use% : 17
Status : DOWN
Flag : ReadOnWrite
Flag : AdminUp
Flag : InternalDown
Num Read Mounts : 0
Num Write Mounts : 1
Cur Read Streams : 0
Cur Write Streams : 0
Num Repl Sources : 0
Num Repl Targets : 0
I use :
./nbdevconfig -changestate -stype PureDisk -dp dp_disk_lprpmedialevel01 -dv PureDiskVolume -state UP
But the error persist, can you help please.
Regards
Solved! Go to Solution.
09-29-2020 01:11 PM
Hello,
Support veritas, today solved the problem, in the media server with problem "medialevel01", all the dir in the path "/Backup" they were in nobody:nobody, so the support:
- change all dir to root:root :
chown root:root *
- change privileges to /Backup/etc:
chmod 755 etc
- change privileges to file contentrouter.cfg:
chmod 644 puredisk/contentrouter.cfg
- change more dir:
chmod 750 log
chmod 755 var
chmod 750 tmp
- then execute commands.
/usr/openv/pdde/pdconfigure/etc/init.d/RedHat/pdservice start spoold
/usr/openv/pdde/pdcr/bin/crcontrol --getmode
./nbdevquery -listdv -stype PureDisk -U (now this command show the disk volume and storage server up)
Disk Pool Name : dp_disk_lprpmedialevel01
Disk Type : PureDisk
Disk Volume Name : PureDiskVolume
Disk Media ID : @aaaal
Total Capacity (GB) : 34305.92
Free Space (GB) : 28398.99
Use% : 17
Status : UP
Flag : ReadOnWrite
Flag : AdminUp
Flag : InternalUp
Num Read Mounts : 0
Num Write Mounts : 1
Cur Read Streams : 0
Cur Write Streams : 0
Num Repl Sources : 0
Num Repl Targets : 0
I see the job now work fine.
Regards
09-17-2020 10:41 AM
Hi
Is the volume mounted on the mount point? Do you see this LUN on the OS?
What is this OS, from where is storage - local disks, SAN ?? Any errors on OS level for any LUN etc...
09-17-2020 11:09 AM
Hi
Is the volume mounted on the mount point? Do you see this LUN on the OS?
/dev/mapper/Backup-bkp 35T 6.0T 29T 18% /Backup
What is this OS, from where is storage - local disks, SAN ?? Any errors on OS level for any LUN etc..
Red Hat Enterprise Linux Server release 7.8 (Maipo)
Nothing happend.
09-17-2020 12:15 PM
hi
hmm maybe restart of nbu as per https://www.veritas.com/support/en_US/article.100002747
can you ping that media server, is name resolution working well?
09-17-2020 02:07 PM
Hi,
I test that tech note and not working, yeah ping for ip and dns work fine.
I try the option reset too (with this disk volume UP), but after five minute the disk volume are DOWN again
Regards
09-18-2020 02:06 AM
Hello
This is weird...
Can you share outcomes from these cmd's
tpconfig -dsh -all_hosts
nbdevquery -listdp -dp dp_disk_lprpmedialevel01 -stype PureDisk -U
nbemmcmd -listhosts -verbose
cat /usr/openv/netbackup/bp.conf
cat /usr/openv/netbackup/version
can you write to this mount point ie: touch /Backup/testing_write
09-22-2020 07:34 AM
Hello,
tpconfig -dsh -all_hosts (what does this command)
nbdevquery -listdp -dp dp_disk_lprpmedialevel01 -stype PureDisk -U
Disk Pool Name : dp_disk_lprpmedialevel01
Disk Pool Id : dp_disk_lprpmedialevel01
Disk Type : PureDisk
Status : UP
Flag : Patchwork
Flag : Visible
Flag : OpenStorage
Flag : SingleStorageServer
Flag : CopyExtents
Flag : AdminUp
Flag : InternalUp
Flag : LifeCycle
Flag : CapacityMgmt
Flag : FragmentImages
Flag : Cpr
Flag : FT-Transfer
Flag : OptimizedImage
Raw Size (GB) : 34305,92
Usable Size (GB) : 34305,92
High Watermark : 95
Low Watermark : 75
Num Volumes : 1
Max IO Streams : -1
Comment :
Storage Server : lprpmedialevel01.resp.derco.cl (DOWN)
nbemmcmd -listhosts -verbose
lprpmedialevel01.resp.derco.cl
ClusterName = ""
MachineName = "lprpmedialevel01.resp.derco.cl"
FQName = "lprpmedialevel01.resp.derco.cl"
LocalDriveSeed = ""
MachineDescription = ""
MachineFlags = 0x77
MachineNbuType = media (1)
MachineState = active for disk jobs (12)
MasterServerName = "lprvnetbkp01.resp.derco.cl"
NetBackupVersion = 8.1.1.0 (811000)
OperatingSystem = linux (16)
ScanAbility = 5
lprpmedialevel01.resp.derco.cl
MachineName = "lprpmedialevel01.resp.derco.cl"
FQName = "lprpmedialevel01.resp.derco.cl"
MachineDescription = "PureDisk"
MachineFlags = 0x2
MachineNbuType = ndmp (2) (storage_server)
cat /usr/openv/netbackup/bp.conf
[root@lprpmedialevel01 var]# cat /usr/openv/netbackup/bp.conf
SERVER = lprvnetbkp01.resp.derco.cl
SERVER = lprpmedialevel01.resp.derco.cl
CLIENT_NAME = lprpmedialevel01.resp.derco.cl
CONNECT_OPTIONS = localhost 1 0 2
CONNECT_OPTIONS = lprvsappibd01.resp.derco.cl 0 1 2
USE_VXSS = PROHIBITED
EMMSERVER = lprvnetbkp01.resp.derco.cl
HOST_CACHE_TTL = 3600
CLI_GA_RET_LOGS_DURATION = 0
TELEMETRY_UPLOAD = YES
cat /usr/openv/netbackup/version
[root@lprpmedialevel01 var]# cat /usr/openv/netbackup/version
HARDWARE LINUX_RH_X86
VERSION NetBackup 8.1.1
RELEASEDATE Sat Feb 03 23:26:51 CST 2018
BUILDNUMBER 0103
can you write to this mount point ie: touch /Backup/testing_write
Yes i create a prueba.txt
Regards.
09-22-2020 08:30 AM
Please also share output of 'bpps -x' on this media server.
You may also want to check MSDP logs for errors:
spad, spoold, storaged.log.
(Log file location: https://www.veritas.com/content/support/en_US/doc/25074086-131900563-0/v95643184-131900563)
09-22-2020 08:37 AM
Hello @Marianne
[root@lprpmedialevel01 bin]# ./bpps -x
NB Processes
------------
root 3484 1 0 Sep17 ? 00:00:24 /usr/openv/netbackup/bin/vnetd - proxy inbound_proxy -number 0
root 3487 1 0 Sep17 ? 00:00:38 /usr/openv/netbackup/bin/vnetd - proxy outbound_proxy -number 0
root 3488 1 0 Sep17 ? 00:00:02 /usr/openv/netbackup/bin/vnetd - proxy http_tunnel -number 0
root 3492 6995 0 12:26 ? 00:00:00 /usr/openv/netbackup/bin/admincm d/bpstsinfo -DPSPROXY
root 4218 1 0 Sep17 ? 00:00:11 /usr/openv/netbackup/bin/vnetd - standalone
root 4667 1 0 Sep17 ? 00:00:03 /usr/openv/netbackup/bin/bpcd -s tandalone
root 4764 1 0 Sep17 ? 00:00:27 /usr/openv/netbackup/bin/nbdisco
root 6995 1 0 Sep17 ? 00:02:10 /usr/openv/netbackup/bin/nbrmms
root 7310 1 0 Sep17 ? 00:01:53 /usr/openv/netbackup/bin/nbsl
root 8135 1 0 Sep17 ? 00:00:37 /usr/openv/netbackup/bin/nbsvcmo n
root 27281 1 0 Sep17 ? 00:02:42 /usr/openv/pdde/pdcr/bin/spad
MM Processes
------------
root 6667 1 0 Sep17 ? 00:00:13 vmd
Shared Veritas Processes
-------------------------
root 2622 1 0 Sep17 ? 00:00:01 /opt/VRTSpbx/bin/pbx_exchange
[root@lprpmedialevel01 bin]# ./bpps -x
NB Processes
------------
root 3484 1 0 Sep17 ? 00:00:24 /usr/openv/netbackup/bin/vnetd -proxy inbound_proxy -number 0
root 3487 1 0 Sep17 ? 00:00:38 /usr/openv/netbackup/bin/vnetd -proxy outbound_proxy -number 0
root 3488 1 0 Sep17 ? 00:00:02 /usr/openv/netbackup/bin/vnetd -proxy http_tunnel -number 0
root 3492 6995 0 12:26 ? 00:00:00 /usr/openv/netbackup/bin/admincmd/bpstsinfo -DPSPROXY
root 4218 1 0 Sep17 ? 00:00:11 /usr/openv/netbackup/bin/vnetd -standalone
root 4667 1 0 Sep17 ? 00:00:03 /usr/openv/netbackup/bin/bpcd -standalone
root 4764 1 0 Sep17 ? 00:00:27 /usr/openv/netbackup/bin/nbdisco
root 6995 1 0 Sep17 ? 00:02:10 /usr/openv/netbackup/bin/nbrmms
root 7310 1 0 Sep17 ? 00:01:53 /usr/openv/netbackup/bin/nbsl
root 8135 1 0 Sep17 ? 00:00:37 /usr/openv/netbackup/bin/nbsvcmon
root 27281 1 0 Sep17 ? 00:02:42 /usr/openv/pdde/pdcr/bin/spad
MM Processes
------------
root 6667 1 0 Sep17 ? 00:00:13 vmd
Shared Veritas Processes
-------------------------
root 2622 1 0 Sep17 ? 00:00:01 /opt/VRTSpbx/bin/pbx_exchange
I attached the log.
Thanks for your help very much.
Regards
09-22-2020 08:53 AM
There is no spoold process running.
I see lots of errors in the spoold log. I honestly don't know what it means.
MAybe time to log a Support call with Veritas?
09-22-2020 09:42 AM
I have a case in veritas, the support try with webex, and use many commands, but the still do not solve it, i try with:
/usr/openv/pdde/pdcr/bin/./spoold and not erro appear, but when i see the process in cli or gui the stop
[root@lprpmedialevel01 spad]# /usr/openv/pdde/pdcr/bin/./crcontrol --dsstat 1
Error: -1: NetConnectByAddr: Failed to connect to host: Connection refused (111)
Error: -1: NetConnectByAddr: Failed to connect to spoold on port 10082 using the following interface(s): [ ::1 ] (Connection refused) Ensure storage server services are running and operational. V-454-92
Error: 53: Could not establish a connection to ::1:10082: connect failed (Connection refused)
Error : Connection failed connection actively refused. Note that the content router needs to be running to get a connection.
So i dont know where else to ask for help.
Regards
09-29-2020 01:11 PM
Hello,
Support veritas, today solved the problem, in the media server with problem "medialevel01", all the dir in the path "/Backup" they were in nobody:nobody, so the support:
- change all dir to root:root :
chown root:root *
- change privileges to /Backup/etc:
chmod 755 etc
- change privileges to file contentrouter.cfg:
chmod 644 puredisk/contentrouter.cfg
- change more dir:
chmod 750 log
chmod 755 var
chmod 750 tmp
- then execute commands.
/usr/openv/pdde/pdconfigure/etc/init.d/RedHat/pdservice start spoold
/usr/openv/pdde/pdcr/bin/crcontrol --getmode
./nbdevquery -listdv -stype PureDisk -U (now this command show the disk volume and storage server up)
Disk Pool Name : dp_disk_lprpmedialevel01
Disk Type : PureDisk
Disk Volume Name : PureDiskVolume
Disk Media ID : @aaaal
Total Capacity (GB) : 34305.92
Free Space (GB) : 28398.99
Use% : 17
Status : UP
Flag : ReadOnWrite
Flag : AdminUp
Flag : InternalUp
Num Read Mounts : 0
Num Write Mounts : 1
Cur Read Streams : 0
Cur Write Streams : 0
Num Repl Sources : 0
Num Repl Targets : 0
I see the job now work fine.
Regards