SLP changes from 7.1 to 7.5

[Genericus]

I just upgraded from 7.1 to 7.5.0.3 - there are major changes in how SLP are processed - not the least is you MUST get the EEB or the SLP using alternate read server won't start! (EEB_NetBackup_7.5.0.3_PET2719199_SET2858615_EEB1)

in 7.1 to see incomplete SLP images the command "bpimagelist -L -idonly -hoursago 120 -stl_incomplete" would instantly respond with a sortable list that showed me output like this:

Time: Mon Jul 23 06:01:32 2012   ID: oracleserver1_1343037692   UBAK (2)

Time: Mon Jul 23 06:01:30 2012   ID: oracleserver1_1343037690   UBAK (2)

Time: Mon Jul 23 06:01:15 2012   ID: oracleserver2_1343037675   UBAK (2)

Time: Mon Jul 23 06:01:14 2012   ID: oracleserver2_1343037674   UBAK (2)

Time: Mon Jul 23 06:01:13 2012   ID: oracleserver3_1343037673   UBAK (2)

Time: Mon Jul 23 05:57:35 2012   ID: nas05_1343037455   INCR (1)

Time: Mon Jul 23 05:53:18 2012   ID: nas05_1343037198   INCR (1)

Time: Mon Jul 23 05:44:34 2012   ID: nas05_1343036674   INCR (1)

Time: Mon Jul 23 05:43:42 2012   ID: nas05_1343036622   INCR (1)

Time: Mon Jul 23 05:40:32 2012   ID: nas05_1343036432   INCR (1)

Time: Mon Jul 23 05:39:42 2012   ID: unixserver_1343036382   INCR (1)

Time: Mon Jul 23 05:39:16 2012   ID: windows_1343036356   INCR (1)

Very easy to read and understand, and a simple "wc" on the list would give me a count of how many in process.

The designated command "nbstlutil stilist -stl_incomplete" on the other hand, took minutes to run, and the output is incomprehensible.

After upgrading to 7.5, I find they have phased out the bpimagelist command and I am forced to use the long running nbstltuil command.

There is a new command nbstlutil report, which gives a nice output, but lacks time information.

My question is - 7.5 SLP admins - how are you monitoring your SLP policies?

[mph999]

Odd, on my 7.5.0.3 system the command runs fine ...

root@womble  $ bpimagelist -L -idonly -hoursago 120 -stl_incomplete

Time: Fri Jul 27 20:09:48 2012   ID: womble_1343416188   FULL (0)

root@womble  $ cat /usr/openv/netbackup/version

HARDWARE SOLARIS

VERSION NetBackup 7.5.0.3

RELEASEDATE Tue Jun 05 21:50:33 CDT 2012

BUILDNUMBER 20120605

and from the bpimage man page 

     -stl_incomplete

        Reports only the images that the storage lifecycle has

        not completely processed. This option cannot be used with

        the stl_complete option.

martin

[Genericus]

nbstlutil report indicates that I have many incomplete SLP

bpimagelist -stl_incomplete seems to only show SLP not processed by nbstserv, once they are scanned they no longer show...

all my old aliases are no good, boo hoo

The OpCenter report is good, I may go with that...

For example - I run the bpimagelist -L -idonly -hoursago 120 -stl_incomplete | wc and get output of 3 incomplete jobs

nbstlutil report shows:

Total:                                                      2420    64553403 MB

a small discrepancy...

[mph999]

Hmm, you're right ...

root@womble  $ nbstlutil stlilist

V7.5.0 I womble_1343416188 SLP1 3

V7.5.0 I womble_1343422531 SLP1 2

V7.5.0 C diskpool1 1 1

root@womble  $ bpimagelist -stl_incomplete

no entity was found

So, once the SLP move to state 2, bpimagelist doesn't show it ...

Dinners ready, back shortly

Martin

[Genericus]

OpCenter has a SLP report, with good information.

I am used to being able to get a concise and speedy summary of what my SLP looks like, just need to get used to being forced to login to OPCenter and running a report, instead of having an instant OS alias.

The bpimagelist command response was just so much faster than the nbstlutil report is now...

[thesanman]

nbstlutil report isn't new to v7.5, I have been using it under v7.1.0.x for a while now. 

I don't believe it's 100% accurate IF (say) lifecycle processing is disabled but that I can deal with.  Once a policy is re-enabled the inforation looks ok to me.

I use nbstlutil stlilist -U -image_incomplete for an accurate picture of what is still to run.  The -U formats it nicely.

[mph999]

Wasn't aware that had changed, but then, I've never used bpimagelist with that option.  Tech support, at least in the uk just use nbstlutil list / stlilist .

Don't suppose you happen to have an example of the old against new - just save me a job, if not no worries, I'll find a 7.1 system and have a play.

Martin

[Genericus]

I guess I am spoiled - my old command (bpimagelist -L -idonly -hoursago 120 -stl_incomplete)

#1 - responded back almost instantly, and

#2 - gave me an output that easily indicated my oldest incomplete SLP, with the backup date and the backupid.

I just ran the command nbstlutil report, the output took 61 seconds to generate on my master server.

So, problem #1 the recommended solution is way slower than my old way.

problem #2 is that the output is not very helpful.

This is the output of nbstlutil stlilist -U -image_incomplete | more
When were these backups done? I guess I need to learn to read epoch time at a glance?

# nbstlutil stlilist -U -image_incomplete | more
Image inorap03-backup_1343441485 for Lifecycle LTO5-NORA3-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343446283 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343446393 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343446562 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343446697 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343446788 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343447340 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343447515 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343447602 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343447774 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
Image sql2k5-dev0_1343447958 for Lifecycle LTO5-DBA-8wk-Vault03 is IN_PROCESS
 

I did find an Opcenter report that I can drill down and find the job, which gives me better information because it includes the job id - so I can filter for that in my activity monitor. Only it is a long, multistep process and no where near as simple and efficient as any of my old ways.

[loori]

the runtime is really bad, about thrice the runtime of the 7.1 version

we check the slp-state via nagios, the slp-report is useless for me

I use bpdbm to convert the timestamp and then date to get a more useful format

here's the script:

#!/bin/bash

NBU=/usr/openv/netbackup/bin/admincmd/nbstlutil

if [[ $1 == "-H" || $1 == "--help" || $# -lt 2 ]]; then
        echo "$0 WARNLEVEL CRITICAL"
        echo "jeweils in Sekunden."
        exit 0
fi

IMAGE=`$NBU stlilist -image_incomplete -U | head -1`
OLDEST_IMAGE_NOT_ON_TAPE=$(expr `date '+%s'` - `echo $IMAGE | cut -f 2 -d "_" | cut -f 1 -d " "`)
OLDEST_IMAGE_NOT_ON_TAPE_ID=`echo $IMAGE | cut -f 2 -d " " `
OLDEST_IMAGE_NOT_ON_TAPE_TS1=`/usr/openv/netbackup/bin/bpdbm -ctime ${OLDEST_IMAGE_NOT_ON_TAPE_ID#*_}`
OLDEST_IMAGE_NOT_ON_TAPE_TS1=${OLDEST_IMAGE_NOT_ON_TAPE_TS1#*= }
OLDEST_IMAGE_NOT_ON_TAPE_TS=`date --date="$OLDEST_IMAGE_NOT_ON_TAPE_TS1" +'%Y-%m-%d %H:%M:%S' `


if [ $OLDEST_IMAGE_NOT_ON_TAPE -ge $2 ]; then
        echo "CRITICAL - Oldest Image ($OLDEST_IMAGE_NOT_ON_TAPE_ID,$OLDEST_IMAGE_NOT_ON_TAPE_TS) not on tape is older than $2|age=$OLDEST_IMAGE_NOT_ON_TAPE;$1;$2"
        exit 2
elif [ $OLDEST_IMAGE_NOT_ON_TAPE -ge $1 ]; then
        echo "WARNING - Oldest Image ($OLDEST_IMAGE_NOT_ON_TAPE_ID,$OLDEST_IMAGE_NOT_ON_TAPE_TS) not on tape is older than $1|age=$OLDEST_IMAGE_NOT_ON_TAPE;$1;$2"
        exit 1
elif [ $OLDEST_IMAGE_NOT_ON_TAPE -lt $1 ]; then
        echo "OK - Oldest Image ($OLDEST_IMAGE_NOT_ON_TAPE_ID,$OLDEST_IMAGE_NOT_ON_TAPE_TS) not on tape looks good|age=$OLDEST_IMAGE_NOT_ON_TAPE;$1;$2"
        exit 0
else
        echo "UNKNOWN - what happened here? huh?"
        exit 3
fi
 

[Genericus]

My nbstlutil commands take 2 minutes to run! The old bpimagelist comand was instant.

-stl_incomplete is INCORRECT!

What is up with in process status ?

I have a backup of my cmhnasa server from Monday the 10th, still showing up as in process even after it has completed copies.

This SLP writes one backup then makes a copy to tape. Both are complete, but the fragments are "active" and the SLP state is 2 (IN_PROCESS)

This completed a while ago, the SLP scan has run multiple times since then, and it is still in process.

# nbstlutil list -U -backupid cmhnasa_1347294052
Image:
 Master Server            : lorenzo
 Backup ID                : cmhnasa_1347294052
 Client                   : cmhnasa
 Backup Time              : 1347294052 (Mon Sep 10 12:20:52 2012)
 Policy                   : UNIX.NDMP_cmhnasA.PROD
 Client Type              : 19
 Schedule Type            : 0
 Storage Lifecycle Policy : LTO5-CMH-8wk-Vault04
 Storage Lifecycle State  : 2 (IN_PROCESS)
 Time In Process          : 1347357529 (Tue Sep 11 05:58:49 2012)
 Data Classification ID   : (none specified)
 Version Number           : 8
 OriginMasterServer       : (none specified)
 OriginMasterServerID     : 00000000-0000-0000-0000-000000000000
 Import From Replica Time : 0 (Wed Dec 31 19:00:00 1969)
 Required Expiration Date : 0 (Wed Dec 31 19:00:00 1969)
 Created Date Time        : 1347294052 (Mon Sep 10 12:20:52 2012)

 Copy:
   Master Server       : lorenzo
   Backup ID           : cmhnasa_1347294052
   Copy Number         : 1
   Copy Type           : 0 (BACKUP)
   Expire Time         : 1347639652 (Fri Sep 14 12:20:52 2012)
   Expire LC Time      : 1347639652 (Fri Sep 14 12:20:52 2012)
   Try To Keep Time    : 1347639652 (Fri Sep 14 12:20:52 2012)
   Residence           : med02np-hcart2-edlb-cmhnasb
   Copy State          : 3 (COMPLETE)
   Job ID              : 1979371
   Retention Type      : 0 (FIXED)
   MPX State           : 0 (FALSE)
   RetryCount          : 0
   Last Retry Time     : 1347357008 (Tue Sep 11 05:50:08 2012)
   Source              : 0
   Destination ID      : (none specified)
   Last Retry Time     : 1347357008
   Replica:            : 0
   DataFormat          : 3 (DF_NDMP)
   SLP Index           : 1

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : -1
     Resume Count      : 0
     Media ID          : B00268
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 119091200
     Delete Header     : 0
     Fragment ID       : B00268
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : 1
     Resume Count      : 0
     Media ID          : B00288
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 241468309504
     Delete Header     : 0
     Fragment ID       : B00288
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : 2
     Resume Count      : 0
     Media ID          : B00065
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 276954218496
     Delete Header     : 0
     Fragment ID       : B00065
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : 3
     Resume Count      : 0
     Media ID          : B00390
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 282545750016
     Delete Header     : 0
     Fragment ID       : B00390
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : 4
     Resume Count      : 0
     Media ID          : B00224
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 265684713472
     Delete Header     : 0
     Fragment ID       : B00224
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : 5
     Resume Count      : 0
     Media ID          : B00389
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 272349921280
     Delete Header     : 0
     Fragment ID       : B00389
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : 6
     Resume Count      : 0
     Media ID          : B00381
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 271644557312
     Delete Header     : 0
     Fragment ID       : B00381
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : 7
     Resume Count      : 0
     Media ID          : B00364
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 275305988096
     Delete Header     : 0
     Fragment ID       : B00364
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 1
     Fragment Number   : 8
     Resume Count      : 0
     Media ID          : B00268
     Media Server      : med02np
     Storage Server    : (none specified)
     Media Type        : 3 (NDMP)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 228883628032
     Delete Header     : 0
     Fragment ID       : B00268
     Snap MountHost    : (none specified)
     Media Description : (none specified)

 Copy:
   Master Server       : lorenzo
   Backup ID           : cmhnasa_1347294052
   Copy Number         : 2
   Copy Type           : 1 (DUPLICATE)
   Expire Time         : 2147483647 (Mon Jan 18 22:14:07 2038)
   Expire LC Time      : 2147483647 (Mon Jan 18 22:14:07 2038)
   Try To Keep Time    : 1352132452 (Mon Nov  5 11:20:52 2012)
   Residence           : med04np-LTO5
   Copy State          : 3 (COMPLETE)
   Job ID              : 1985692
   Retention Type      : 0 (FIXED)
   MPX State           : 0 (FALSE)
   RetryCount          : 2
   Last Retry Time     : 1347445543 (Wed Sep 12 06:25:43 2012)
   Source              : 1
   Destination ID      : (none specified)
   Last Retry Time     : 1347445543
   Replica:            : 0
   DataFormat          : 3 (DF_NDMP)
   SLP Index           : 2

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 2
     Fragment Number   : -1
     Resume Count      : 0
     Media ID          : E01675
     Media Server      : med04np
     Storage Server    : (none specified)
     Media Type        : 2 (RMEDIA)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 119085056
     Delete Header     : 0
     Fragment ID       : E01675
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 2
     Fragment Number   : 1
     Resume Count      : 0
     Media ID          : E01671
     Media Server      : med04np
     Storage Server    : (none specified)
     Media Type        : 2 (RMEDIA)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 1073740840960
     Delete Header     : 0
     Fragment ID       : E01671
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 2
     Fragment Number   : 2
     Resume Count      : 0
     Media ID          : E01671
     Media Server      : med04np
     Storage Server    : (none specified)
     Media Type        : 2 (RMEDIA)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 929575141376
     Delete Header     : 0
     Fragment ID       : E01671
     Snap MountHost    : (none specified)
     Media Description : (none specified)

   Fragment:
     Master Server     : lorenzo
     Backup ID         : cmhnasa_1347294052
     Copy Number       : 2
     Fragment Number   : 3
     Resume Count      : 0
     Media ID          : E01675
     Media Server      : med04np
     Storage Server    : (none specified)
     Media Type        : 2 (RMEDIA)
     Media Sub-Type    : 1 (BASIC)
     Fragment State    : 1 (ACTIVE)
     Fragment Size     : 111521103872
     Delete Header     : 0
     Fragment ID       : E01675
     Snap MountHost    : (none specified)
     Media Description : (none specified)

# nbstlutil stlilist -image_incomplete -U | head -1
Image cmhnasa_1347294052 for Lifecycle LTO5-CMH-8wk-Vault04 is IN_PROCESS
 

1347294052 = Mon Sep 10 12:20:52 2012