cancel
Showing results for 
Search instead for 
Did you mean: 

Why is NetBackup limited to one new backup job per second?

sdo
Moderator
Moderator
Partner    VIP    Certified

From my limited perspective, I cannot see why we cannot have two backup images, one named CLIENTA_123456789 and another named CLIENTB_123456789, i.e. two backups start at the same time, i.e. both have the same trailing 'ctime' component.

In fact, it must be possible to import an image from another environment with a different clientname, but the same ctime.

So, is there a fundamental reason why two backup jobs for different clients cannot start at the same time?

4 ACCEPTED SOLUTIONS

Accepted Solutions

Nicolai
Moderator
Moderator
Partner    VIP   

From the Symantec NetBackup Backup Planning and Performance Tuning Guide Release 7.5 and Release 7.6

http://www.symantec.com/docs/DOC7449

Limits to scalability:

Background: A NetBackup domain is defined as a number of NetBackup media servers and client computers under the control of a single NetBackup master server. A NetBackup domain may span multiple sites and datacenters . A single NetBackup capacity planning 41 Designing your backup system site or datacenter may consist of more than one NetBackup domain. Within a NetBackup domain, each backup job receives an identifier that includes a unique 10-digit UTC timestamp. As each timestamp must be unique, only one backup job can be launched per second, regardless of the resources available. As a result, a single NetBackup domain can launch a maximum of 86,400 backup jobs per day. In most situations, constraints within the hardware resources and infrastructure are likely to prevent the server from reaching the 86,400 limit. If a domain does not allow sufficient jobs, an environment can be configured to support multiple NetBackup domains

View solution in original post

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

But.......

 

From the Best Practice Performance guide its says

 

"Number of jobs not to exceed 30000 within 24 hrs"

 

So even if it could do more than 1 per second, it clearly can't sustain it.

View solution in original post

Andrew_Madsen
Level 6
Partner

To paraphrase Nicolai's post: In a given domain you can start multiple jobs per client if it was possible to start more than one job a second then since the ctime is only to the second you could have multiple images with the same backupid and different content. This would be a bad thing since the backupid is a record key in the database.

When you import an image the client is different so the backupid would be different even though the ctime portion of the backupid is the same. It could be possible to rewrite the code to query the image database to see if a backupid has been assigned to a specific client but by the time you would get a response and work through the response a second would have passed any way so why not just limit it to one a second and be done with it?

View solution in original post

mph999
Level 6
Employee Accredited

 

From the Best Practice Performance guide its says

"Number of jobs not to exceed 30000 within 24 hrs"

So even if it could do more than 1 per second, it clearly can't sustain it"

 

I'd say 30 000 was an upper limit ... but I suspect it depends who you ask.

Some of us start getting nervous if numbers above 25000 are mentioned ...

The theroetical lilmit of 86400 jobs, as far as I know has never been reached.

I heard on the grapevine that engineering had a go once and got close, but couldn't quite manage 86400.

M

View solution in original post

16 REPLIES 16

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

I would say its probably the way that NBPEM/BPDBM queues / handsout the image numbers. And it would appear to be a serial process. Now serving 1404594003, Take a number and get in the queue :)

sdo
Moderator
Moderator
Partner    VIP    Certified

Ok - but how come we can have images for different clients with the same ctime if we import.  So it doesn't appear to be an image database restriction.

Is that 'now serving' a reference to BeetleJuice?  :p

Or, can I ask, in which log can I see 'ctimes' being handed-out/assigned/selected ?

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Hahaha, yes I can visualise that :p

 

It wont be an issue having mulitple clients with the same time stamp. Dont think the catalog would self destruct.

You can have instances of the same time on different clients if they were issued there id's in different domains. I'm sure there are multiple machines serving number 1404594003 out there in the world :p

Nicolai
Moderator
Moderator
Partner    VIP   

From the Symantec NetBackup Backup Planning and Performance Tuning Guide Release 7.5 and Release 7.6

http://www.symantec.com/docs/DOC7449

Limits to scalability:

Background: A NetBackup domain is defined as a number of NetBackup media servers and client computers under the control of a single NetBackup master server. A NetBackup domain may span multiple sites and datacenters . A single NetBackup capacity planning 41 Designing your backup system site or datacenter may consist of more than one NetBackup domain. Within a NetBackup domain, each backup job receives an identifier that includes a unique 10-digit UTC timestamp. As each timestamp must be unique, only one backup job can be launched per second, regardless of the resources available. As a result, a single NetBackup domain can launch a maximum of 86,400 backup jobs per day. In most situations, constraints within the hardware resources and infrastructure are likely to prevent the server from reaching the 86,400 limit. If a domain does not allow sufficient jobs, an environment can be configured to support multiple NetBackup domains

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

But.......

 

From the Best Practice Performance guide its says

 

"Number of jobs not to exceed 30000 within 24 hrs"

 

So even if it could do more than 1 per second, it clearly can't sustain it.

revarooo
Level 6
Employee

Another point, what would happen if you initiated the same policy twice if you could have the same ctime value in the backup id? You may have 2 processes fighting to overwrite the same backup id - NOT GOOD!

Andrew_Madsen
Level 6
Partner

To paraphrase Nicolai's post: In a given domain you can start multiple jobs per client if it was possible to start more than one job a second then since the ctime is only to the second you could have multiple images with the same backupid and different content. This would be a bad thing since the backupid is a record key in the database.

When you import an image the client is different so the backupid would be different even though the ctime portion of the backupid is the same. It could be possible to rewrite the code to query the image database to see if a backupid has been assigned to a specific client but by the time you would get a response and work through the response a second would have passed any way so why not just limit it to one a second and be done with it?

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

So SDO, to summarize it for....... Just drink the Kool-Aid, lol

Nicolai
Moderator
Moderator
Partner    VIP   

@revaroo - database backup like SAP using multiple stream, each job has different backup ID even thou they are born at the same time. If it was a issue, I am pretty certain we would discuss it much more ...

mph999
Level 6
Employee Accredited

 

From the Best Practice Performance guide its says

"Number of jobs not to exceed 30000 within 24 hrs"

So even if it could do more than 1 per second, it clearly can't sustain it"

 

I'd say 30 000 was an upper limit ... but I suspect it depends who you ask.

Some of us start getting nervous if numbers above 25000 are mentioned ...

The theroetical lilmit of 86400 jobs, as far as I know has never been reached.

I heard on the grapevine that engineering had a go once and got close, but couldn't quite manage 86400.

M

sdo
Moderator
Moderator
Partner    VIP    Certified

Hey everyone - thanks for the super answers.

Now that servers are getting very capable indeed, I was going to ask the question... I wonder if engineering will, at some point, update the code to allow different clients to start backups at the same time?

...but, upon reflection, even a very capable master server can only have so many network ports open, and can only ingest so much 'file catalog' data at once - so the question regarding having more than one client start during the same 'second of time' is heading towards slightly nonsensical.  :)

 

mph999
Level 6
Employee Accredited

... heading towards slightly nonsensical.  :

That is the crux of the matter TBH.

 

 

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Think of the poor backup admin that has to sift through 86400 jobs daily.....

Nicolai
Moderator
Moderator
Partner    VIP   

One divine female TA  can do that in ½ a hour :)

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Bwhahaha

mph999
Level 6
Employee Accredited

Not me, mine would all work perfectly .... :0-