Forum Discussion

Michael_De_Simo's avatar
Michael_De_Simo
Level 3
20 years ago

Find out what jobs *should* have run

Hi,
Simple question: I need to know what jobs should have run in the past, specifically last night.


Details, background etc.
We are dealing with a problem where we have "phantom" jobs. These jobs queue up but will never run - ever. The problem is there is no way to differentiate these jobs from jobs that should be run. We have a separate team tracking that issue specifically. My task is to try and figure out if the jobs that were supposed to have run actually ran. But, since I am joining this effort late, am somewhat behind the curve and did not get a list of all the masters this could be happening on. I had new server added to my list this morning that had the problem last night. In the future, like for tonight's window, I know I can use bpsched -predict to find out what runs tonight then run bperror tomorrow and with some fancy footwork determine if everything that should have run did (and did that for the servers in my list who of course did not exhibit the problem).

Thanks,
Mike D.Message was edited by:
Michael De Simone
7.1.x and Earlier
Backup and Recovery
NetBackup

3 Replies

  • zippy's avatar
    zippy
    Level 6
    20 years ago
    Mike,

    On the Master run.

    it will email it to you.

    If its a window box then convert it.

    Have fun.

    Jim D

    #!/usr/bin/ksh

    #This script is used for giving daily statistics.The cronjob is set to run on a daily basis.
    #Script created by James Dunn
    #set -vx
    #Set variables

    TODAY=`date +%a`
    HOST=`hostname`
    CURRDATE=`date +%m%d%y`
    OUTFILE=/var/tmp/out2
    OUTFILE1=/var/tmp/out1
    OUTFILE2=/var/tmp/out2
    OUTFILE3=/var/tmp/out3
    ADMINCMD=/usr/openv/netbackup/bin/admincmd
    BPIMAGELIST=/usr/openv/netbackup/bin/admincmd/bpimagelist
    ERRORLIST=/usr/openv/netbackup/bin/admincmd/bperror
    GOODIE=/usr/openv/netbackup/bin/goodies


    # Clear /tmp/out2 file first
    /bin/rm -rf $OUTFILE $OUTFILE1 $OUTFILE2 $OUTFILE3

    if ; then
    HARDWARE=`head -1 /usr/openv/netbackup/version | cut -f2 -d" "`
    else
    echo "/usr/openv/netbackup/version not found"
    exit 1
    fi

    for i in $HOST
    do

    echo "
    " >> $OUTFILE
    $ADMINCMD/bperror -U -backstat -s info -hoursago 24 1>> $OUTFILE 2>&1
    echo "
    Recently Used Media:
    " >> $OUTFILE
    $BPIMAGELIST -A -media -hoursago 24 1>>$OUTFILE 2>&1

    mailx -s "SDP's Backup list for `date +%x`" root < $OUTFILE


    $ERRORLIST -U -problems -hoursago 24 1>>$OUTFILE1 2>&1


    mailx -s "SDP's Backup errors for `date +%x`" root < $OUTFILE1

    $ADMINCMD/bperror -U -backstat -by_statcode >> $OUTFILE2
    mailx -s "Company Backup by CODE for `date +%x`" net_backup < $OUTFILE2

    $ADMINCMD/bpcatlist -server $HOST -client $HOST -since-days 1 >>$OUTFILE3
    mailx -s "ORACLE SNAP BACKUPS for `date +%x`" oracle_snap < $OUTFILE3

    done
  • Michael_De_Simo's avatar
    Michael_De_Simo
    Level 3
    20 years ago
    James,
    This script will let me know what errors I had, media used etc. It does not tell me what should have run. Perhaps I should elaborate more. When we have the "phantom" jobs they look like normal queued jobs (same clients and policies etc.) but they will never run and do not go away until NB is recycled. When it is recycled they fail with a status 50 but this is not picked up by bperror. The issue is that during this time normal jobs will queue and run but when everything fails during a recycle we want to double check that everything that should have been scheduled actually did get scheduled. To do this I need to know what the schedule was and compare that to what does end up in bperror - which is only normal jobs. I can do this proactively with bpsched -predict but, that can only look to the future. I need to be able to look at a schedule, not the results (well I do need that but I already have it), for the past.

    Thanks for your help,
    michael
  • Stumpr2's avatar
    Stumpr2
    Level 6
    20 years ago
    We are dealing with a problem where we have "phantom" jobs. These jobs queue up but will never run - ever.

    Yeah...I'm having this problem too. I think it is related to a problem we are having with bprd dying on the weekend. Perhaps my master server is just overloaded. I still haven't fixed the problem and stop/starting netbackup is frowned upon as we have 7x24 tape drive usage.

    Kluge fix: I create a list of clients that are stuck in a queued state and then run the list through a loop doing
    bpimagelist -U -client $CLIENTNAME -hoursago 72