I have a Windows 2008 R2 server with two drives (C & E) and a HP 2 drive MSL4048 library. I would like to backup up the E drive using both drives that are in the MSL4048 library to improve backup performance (reduce the time needed to back it up). I tried the following in my Daily Backup policy (which is Full / Incremental): NEW_STREAM E:\Directory1 E:\Directory2 NEW_STREAM E:\Directory3 E:\Directory4 etc... When the Full backup portion of the policy ran, I only got the 2nd stream in my backup (ie. E:\Directory3, E:\Directory4, etc...) Nothing from the 1st stream. What am I missing? Am I trying to do something that is not allowed because it is on a single drive? Can someone send me a link with examples on how to use the NEW_STREAM directive? I read the Admin manual and it looked straight forword. Thanks in advance.

All, Thanks so much for every ones help, especially Marianne. As it turns out, streams were the least problem. It was various items including lack of understanding on how NetBackup works in general. The final solution to my problem was to configure the Host Properties -> Global Attributes -> Max Jobs per Client from a default value of 1 to 4 and Host Properties -> Client Attributes -> Max Data Streams from default of 1 to 4. Both drives, inside of the MSL4048, are now servicing each stream and my backup times have been reduced by over 50%. The knowledge you have shared with me will enhance the further reduction in my backup times. Not sure who should get credit for the solution but my problem is offically resolved. Thanks again...

Examples of how to use new_stream

Marianne
Level 6
12 years ago
That is exactly how I use NEW_STREAM.

Important to ensure that 'Allow multiple data streams' is selected in attributes.

It is also important to make this change when Full schedule is due to prevent Incrementals from running as Full.

Have you perhaps checked with 'nbpemreq -predict ....' to ensure that policy shows 2 streams?

Was a parent and a child stream started for this backup or just a single job?

Can you please check if any STREAMS files were created under client's images folder on the master?
BigMo
Level 3
12 years ago
Marianne,

This may have more to do with my lack of understanding how Netbackup 7.5 works rather than a lack of understanding about NEW_STREAMS.

Yes, Allow multiple data streams is selected. Limit jobs per policy is not (meaning unlimited). I have 2 storage units that the NetBackup Wizard added, <ServerName>-hcart3 and <ServerName>-hcart3-robot-tld-1. Both are set to Enable Multiplexing with 2 max concurrent write drives and 4 max streams per drive.

After further review, I noticed that both parts of my Daily Backup policy (Incremental and Full) ran at the same time:
(Note: I took the default policy items when I created the Daily Backup policy which includes 2 schedules, a incremental and a full. This may be the real source of my problem. Initially I tried to create just an incremental backup job but it would not run and complained about not having a full backup.)

1044 Backup Done 0 Daily_Backup Full <client> <media server> 4/3/2013 10:00:13 PM 4/4/2013 3:26:46 AM 05:26:33 904179080 788026
1043 Backup Done 0 Daily_Backup Cumulative-Inc <client> <media server> 4/3/2013 10:00:13 PM 4/3/2013 10:04:16 PM 00:04:03 4539848 6
1042 Backup Done 0 Daily_Backup - <client> <media server> 4/3/2013 10:00:00 PM 4/4/2013 3:26:56 AM 05:26:56

From looking at the details of each job it looks like NEW_STREAM one did an incremental backup of directories 1 & 2 and NEW_STREAM two did a
full backup of directories 3,4,5 etc.
That is NOT what I expected. I expected tape drive 1 to service NEW_STREAM 1 and tape drive 2 to service NEW_STREAM 2.

Here is a copy of my Daily_Backup Streams file from the images dir.

T 0 0
1 0 Daily_Backup Full 0,0,604800 0 0 E:\directory1
1 0 Daily_Backup Full 0,0,604800 0 0 E:\directory2
2 1365041048 Daily_Backup Full 0,0,604800 0 0 E:\directory3
2 1365041048 Daily_Backup Full 0,0,604800 0 0 E:\directory4
2 1365041048 Daily_Backup Full 0,0,604800 0 0 E:\directory5
2 1365041048 Daily_Backup Full 0,0,604800 0 0 E:\directory6
2 etc...
1 1365040813 Daily_Backup Cumulative-Inc 4,0,86400 0 0 E:\directory1
1 1365040813 Daily_Backup Cumulative-Inc 4,0,86400 0 0 E:\directory2
2 0 Daily_Backup Cumulative-Inc 4,0,86400 0 0 E:\directory3
2 0 Daily_Backup Cumulative-Inc 4,0,86400 0 0 E:\directory4
2 0 Daily_Backup Cumulative-Inc 4,0,86400 0 0 E:\directory5
2 etc...

This looks like what I expected (tape drive 1 to service NEW_STREAM 1 and tape drive 2 to service NEW_STREAM 2)
I didn't expect both schedules to run. The policy had just the E:\ listed before I started using the NEW_STREAM directive and the full backup would run weekly and the incremental ran the other days. Never simultaneously.

Thanks for your prompt and insightful comments.
Marianne
Level 6
12 years ago

I noticed that both parts of my Daily Backup policy (Incremental and Full) ran at the same time

This should not have happened.

Please show us your Policy config with this command on the Master:

bppllist <policy-name> -U

as well as recent backup details for this client:

bpimagelist -client <client-name> -d 04/01/2013 -U
BigMo
Level 3
12 years ago
VxSS authentication failed (116) when I ran bppllist <policy-name> -U

and VxSS authentication failed with bpimagelist -client <client-name> -d 04/01/2013 -U

Not sure what to do next. Have a great weekend.

Thanks... Look forward to your reply.
Marianne
Level 6
12 years ago
Seems you have NBAC configured?

You will need to 'login' before running NBU commands:

bpnbat -login
BigMo
Level 3
12 years ago
Yes, I have NBAC configured but have never used it so I am extremely unfamiliar with it. I hope to use it to configure an end user for restores (and only restores).

On to your previous questions:

C:\Program Files\Veritas\NetBackup\bin\admincmd>bppllist -U
Policy:            1Manual_Backup
Policy:            CDriveBackup
Policy:            Catalog_Backup
Policy:            Daily_Backup
Policy:            LMS_Archive
Policy:            Monthly_Backup
Policy:            Weekly_Backup

C:\Program Files\Veritas\NetBackup\bin\admincmd>bpimagelist -client client-name -d 04/01/2013 -U

Backed Up         Expires       Files       KB C Sched Type   On Hold Index Status Policy
---------------- ---------- -------- -------- - ------------ ------- ------------ ------------
04/08/2013 04:03 INFINITY       1643 1764466 N Full Backup 0       0            Catalog_Backup
04/08/2013 04:01 INFINITY         15   236076 N Full Backup 0       0            Catalog_Backup
04/07/2013 22:03 04/13/2013     1639 13355481 N Cumulative I 0       0            Daily_Backup
04/07/2013 16:35 INFINITY       1663 1764119 N Full Backup 0       0            Catalog_Backup
04/07/2013 16:32 INFINITY         15   236080 N Full Backup 0       0            Catalog_Backup
04/06/2013 22:00 05/02/2013   850305 3671528133 Y Full Backup 0       0            Weekly_Backup
04/06/2013 04:04 INFINITY       1636 1769114 N Full Backup 0       0            Catalog_Backup
04/06/2013 04:01 INFINITY         15   236076 N Full Backup 0       0            Catalog_Backup
04/05/2013 22:03 04/11/2013     1488 10376952 N Cumulative I 0       0            Daily_Backup
04/05/2013 20:44 04/11/2013     1485 10312629 N Cumulative I 0       0            Daily_Backup
04/05/2013 20:36 INFINITY       1656 1768575 N Full Backup 0       0            Catalog_Backup
04/05/2013 20:34 INFINITY         15   236080 N Full Backup 0       0            Catalog_Backup
04/04/2013 22:00 04/10/2013    63834 4056409943 N Full Backup 0       0            Daily_Backup
04/04/2013 04:03 INFINITY       1635 1864068 N Full Backup 0       0            Catalog_Backup
04/04/2013 04:01 INFINITY         15   236076 N Full Backup 0       0            Catalog_Backup
04/03/2013 22:04 04/09/2013   788026 904179080 N Full Backup 0       0            Daily_Backup
04/03/2013 22:00 04/09/2013        6 4539848 N Cumulative I 0       0            Daily_Backup
04/03/2013 08:04 INFINITY       1648 1767957 N Full Backup 0       0            Catalog_Backup
04/03/2013 08:02 INFINITY         15   236516 N Full Backup 0       0            Catalog_Backup
04/03/2013 07:20 04/09/2013    35287 55631651 N Cumulative I 0       0            Daily_Backup
04/03/2013 04:02 INFINITY       1561 1763355 N Full Backup 0       0            Catalog_Backup
04/03/2013 04:00 INFINITY         15   235688 N Full Backup 0       0            Catalog_Backup
04/02/2013 22:00 04/08/2013    35246 47576549 N Cumulative I 0       0            Daily_Backup
04/02/2013 12:09 INFINITY       1554 1758972 N Full Backup 0       0            Catalog_Backup
04/02/2013 11:51 INFINITY         16   236391 N Full Backup 0       0            Catalog_Backup
04/02/2013 11:04 INFINITY       1545 1758732 N Full Backup 0       0            Catalog_Backup
04/02/2013 10:58 INFINITY         16   235707 N Full Backup 0       0            Catalog_Backup
04/01/2013 16:34 04/27/2013   852446 3695707291 Y Full Backup 0       0            Weekly_Backup

Thanks (and a special Thanks for tolerating my ignorance....)
Rusty_Major
Level 4
12 years ago
Without being able to see what your policy looks like, I would hazard a guess that they are running concurrently because the windows for both schedules are not exactly the same. Take a look at these and make sure they overlap at the same exact times, otherwise you can have situations where you do have an INC and a Full running at the same time on the same day.
BigMo
Level 3
12 years ago
There are 2 schedules inside my Daily_Backup policy (Incremental & Full) both start at 10PM and close at 4AM

I couldn't figure out how to get an export from the GUI policy so I ran you a bppllist Daily_Backup -L. Don't know if this info will help...
Policy Name:       Daily_Backup
Options:           0x0
template:          FALSE
audit_reason:         ?
Names:             (none)
Policy Type:       MS-Windows (13)
Active:            yes
Effective date:    08/11/2011 06:48:50
Client Compress:   no
Follow NFS Mnts:   yes
Backup netwrk drvs:yes
Collect TIR info:      no
Mult. Data Stream: yes
Perform Snapshot Backup:   no
Snapshot Method:           (none)
Snapshot Method Arguments: (none)
Perform Offhost Backup:    no
Backup Copy:               0
Use Data Mover:            no
Data Mover Type:           2
Use Alternate Client:      no
Alternate Client Name:     (none)
Use Virtual Machine:      0
Hyper-V Server Name:     (none)
Enable Instant Recovery:   no
Policy Priority:   0
Max Jobs/Policy:   12
Disaster Recovery: 0
Collect BMR Info: no
Keyword:           (none specified)
Data Classification:       -
Residence is Storage Lifecycle Policy:    no
Client Encrypt:    no
Checkpoint:        no
Residence:         <ServerName>-hcart3-robot-tld-1
Volume Pool:       Daily
Server Group:      *ANY*
Granular Restore Info: no
Exchange Source attributes:              no
Exchange 2010 Preferred Server: (none defined)
Application Discovery:      no
Discovery Lifetime:      28800 seconds
ASC Application and attributes: (none defined)
Generation:      42
Ignore Client Direct: no
Enable Metadata Indexing: no
Index server name: NULL
Use Accelerator: no
Client/HW/OS/Pri/DMI:   <Server+DomainName> Windows-x64 Windows2008 0 0 0 0 ?
Include:           NEW_STREAM
Include:           E:\directory1
Include:           E:\directory2
Include:           NEW_STREAM
Include:           E:\directory3
Include:           E:\directory4
Include:           E:\directory5
Include:           etc...
Schedule:              Full
Type:                FULL (0)
Frequency:           7 day(s) (604800 seconds)
    EXCLUDE DATE 0 - <List of Exclude Dates>
Maximum MPX:         1
Synthetic:           0
Checksum Change Detection: 0
PFI Recovery:        0
Retention Level:     0 (144 hours)
u-wind/o/d:          0 0
Incr Type:           DELTA (0)
Alt Read Host:       (none defined)
Max Frag Size:       0 MB
Number Copies:       1
Fail on Error:       0
Residence:           (specific storage unit not required)
Volume Pool:         (same as policy volume pool)
Server Group:        (same as specified for policy)
Residence is Storage Lifecycle Policy:         0
Schedule indexing:     0
Daily Windows:
   Day         Open       Close       W-Open     W-Close
   Sunday      022:00:00 028:00:00   022:00:00 028:00:00
   Monday      022:00:00 028:00:00   046:00:00 052:00:00
   Tuesday     022:00:00 028:00:00   070:00:00 076:00:00
   Wednesday   022:00:00 028:00:00   094:00:00 100:00:00
   Thursday    022:00:00 028:00:00   118:00:00 124:00:00
   Friday      022:00:00 028:00:00   142:00:00 148:00:00
   Saturday    022:00:00 028:00:00   166:00:00 172:00:00 004:00:00
Schedule:              Cumulative-Inc
Type:                CINC (4)
Frequency:           1 day(s) (86400 seconds)
    EXCLUDE DATE 0 - <List of Exclude Dates>

Maximum MPX:         1
Synthetic:           0
Checksum Change Detection: 0
PFI Recovery:        0
Retention Level:     0 (144 hours)
u-wind/o/d:          0 0
Incr Type:           DELTA (0)
Alt Read Host:       (none defined)
Max Frag Size:       0 MB
Number Copies:       1
Fail on Error:       0
Residence:           (specific storage unit not required)
Volume Pool:         (same as policy volume pool)
Server Group:        (same as specified for policy)
Residence is Storage Lifecycle Policy:         0
Schedule indexing:     0
Daily Windows:
   Day         Open       Close       W-Open     W-Close
   Sunday      022:00:00 028:00:00   022:00:00 028:00:00
   Monday      022:00:00 028:00:00   046:00:00 052:00:00
   Tuesday     022:00:00 028:00:00   070:00:00 076:00:00
   Wednesday   022:00:00 028:00:00   094:00:00 100:00:00
   Thursday    022:00:00 028:00:00   118:00:00 124:00:00
   Friday      022:00:00 028:00:00   142:00:00 148:00:00
   Saturday    000:00:00 000:00:00
Andy_Welburn
Level 6
12 years ago
Two changes I would make, which may or may not help you resolve some of your issues.

I presume from the fact that the frequencies you have set that you want to run the incrementals daily (exc Saturday) & the fulls once a week? (sorry, don't like -L on bppllist for working out backup windows -U is much friendlier!!)

In which case:

make sure your backup windows *only* open on the day(s) you want (i.e. it looks like your full schedules have windows open every day)

make your frequency less than 1 day for your incrementals & less than 7 days for your fulls e.g. you could essentially get away with a frequency of 7 hours for both (just larger than your backup window) - this would stop any possible 'schedule creep'.

Other than that your settings for streams etc seem ok at first glance.

***EDIT***

Making those frequency changes should also allow you more leeway to re-run failed jobs or test changes without them affecting when your scheduled jobs actually start.
BigMo
Level 3
12 years ago
Marianne asked for the bppllist -U and I forgot to put the policy name in the string. Here is the -U for your reference.

You may want to review my other comments on how this policy is running. When the full daily backup runs it only does the 2nd stream not the first. I want both streams to run and use both drives to reduce the backup time, but that is not happening. The incremental are working the way I want but not the full. I have not clue as to why.

I also noticed my Weekly backup with 2 streams (c:\ & e:\) ran only the 1st stream (e:\) on 4-6 but not the 2nd stream (c:\). On 4-8 it backed up the 2nd stream (c:\) but not the 1st stream (e:\) - Looks like I don't understand quite a bit. Note: 4-6 Weekly backup was re-run because of no window available on the original date.

I have adjusted the Daily_Backup to reflect your suggestions.

What Marianne Asked for.... Policy Name:       Daily_Backup

Policy Type:         MS-Windows
Active:              yes
Effective date:      08/11/2011 06:48:50
Backup network drvs: yes
Collect TIR info:    no
Mult. Data Streams: yes
Client Encrypt:      no
Checkpoint:          no
Policy Priority:     0
Max Jobs/Policy:     12
Disaster Recovery:   0
Collect BMR info:    no
Residence:           <Server>-hcart3-robot-tld-1
Volume Pool:         Daily
Server Group:        *ANY*
Keyword:             (none specified)
Data Classification:       -
Residence is Storage Lifecycle Policy:    no
Application Discovery:      no
Discovery Lifetime:      28800 seconds
ASC Application and attributes: (none defined)

Granular Restore Info: no
Ignore Client Direct: no
Enable Metadata Indexing: no
Index server name: NULL
Use Accelerator: no
HW/OS/Client: Windows-x64   Windows2008   <Server FQDN>

Include: NEW_STREAM
            E:\Directory1
            E:\Directory2
            NEW_STREAM
            E:\Directory3
            E:\Directory4
            E:\Directory5
            E:\ etc...

Schedule:              Full
    Type:                Full Backup
    Frequency:           every 7 days
    Maximum MPX:         1
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     0 (144 hours)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
      EXCLUDE DATE 0 - 01/05/2013
      EXCLUDE DATE 1 - etc...
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Sunday     22:00:00 --> Monday     04:00:00
          Monday     22:00:00 --> Tuesday    04:00:00
          Tuesday    22:00:00 --> Wednesday 04:00:00
          Wednesday 22:00:00 --> Thursday   04:00:00
          Thursday   22:00:00 --> Friday     04:00:00
          Friday     22:00:00 --> Saturday   04:00:00
          Saturday   22:00:00 --> Sunday     04:00:00

Schedule:              Cumulative-Inc
    Type:                Cumulative Incremental Backup
    Frequency:           every 1 day
    Maximum MPX:         1
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     0 (144 hours)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
      EXCLUDE DATE 0 - 01/05/2013
      EXCLUDE DATE 1 - etc...

    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Sunday     22:00:00 --> Monday     04:00:00
          Monday     22:00:00 --> Tuesday    04:00:00
          Tuesday    22:00:00 --> Wednesday 04:00:00
          Wednesday 22:00:00 --> Thursday   04:00:00
          Thursday   22:00:00 --> Friday     04:00:00
          Friday     22:00:00 --> Saturday   04:00:00
          Saturday   22:00:00 --> Sunday     04:00:00

Forum Discussion

Examples of how to use new_stream

Related Content

NEW_STREAM with wildcards

DA Search Term examples

Clearwell Loadfile/opticon example

Example policy for synthetic backups

EV Move Archive - speed / transfer rate example