zpool compression for performance improvements

[edla_shravya]

Hi Everyone,

I have a query regarding zpool compression for performance improvements . Can you please clarify?

Available statistics of the bptm debug logs to the last seven days show serious waits and

delays on daily and weekly duplication jobs. On the "waited for full buffer" side, there are

average delays up to 3 sec. each wait event indicating slow read performance on the

DSSU. On the "waited for empty buffer" side, the values are not critical. To sum up: It

looks like the duplication jobs suffer from slow DSSU reading.

The zpool based file system under  uses compression. Of course, this

setting slows down read and write performance. Furthermore, the current compression

ratio is about 2, thus saving 50% storage.

! should the file system compression should be disabled temporarily in order to check for performance

improvements. If we disable the compression what are the impacts?

Policies

[Marianne]

I seriously doubt that filesystem compression is supported or recommended for NBU disk storage unit.

It seems you are lucky that anything is working at this stage. 
I found this old TN: http://www.symantec.com/docs/TECH160238  
The assumption in your case is that files need to be uncompressed before they can be read for duplication.
 

You should rather consider NBU dedupe pool (MSDP) instead of filesystem compression.

[edla_shravya]

If we use MSDP performance can be improved?

[Marianne]

Let us forget about NBU for a minute.

Compression and performance have never been 'friends'.

You cannot expect any kind of performance on a busy filesystem with lots of read and writes where compression is enabled.
Compression should only be enabled on inactive filesystems or filesystems with low activity.

If you have a need for space saving, you should look at dedupe, BUT you need a media server with serious resources (memory and cpu) as well as fast disk.
MSDP requirements are listed in NetBackup Deduplication Guide  

If you are short on disk space in this DSU, then add more space so that you can disable filesystem compression.

 

[edla_shravya]

Its mentioned in the tuning guide that

The debug log files introduce additional overhead and have a small effect on

the overall performance of NetBackup. This effect is more noticeable for a high

verbose level setting. Normally, you should not need to run with debug logging

enabled on a production system.

So can you please suggest whether we should not run with debug logging

[Marianne]

Level 0 log (default) is good for day-to-day troubleshooting and will not affect performance.

Only increase logging levels when asked by Symantec for a support case.

[edla_shravya]

Thanks a lot for your response.

On the "waited for full buffer" side, there are

average delays up to 3 sec. each wait event indicating slow read performance on the

DSSU.

Can you please suggest how to overcome this?

[Marianne]

Have you disabled compression yet? 1st step will be to ensure good IO at OS/filesystem level. Have you configured buffer sizes on the media server? Do this when you are satisfied OS level IO is good.

[edla_shravya]

No Compression has not been disabled.Yes we have configured buffer sizes on the media server

[Marianne]

As per my previous post - you need to do it the other way round. 

Disable compression first of all. 
ONLY when IO at OS level is good should you look at buffer settings.

[edla_shravya]

If we use  DSSU as temporary staging storage can we make use of image multiplexing during

duplication on the final destination storage unit.

[Marianne]

Multiplexing plays no role in duplication/staging.

Disk Staging Relocation Behavior: http://www.symantec.com/docs/TECH44719

[edla_shravya]

The high water mark configured for the disk staging storage unit is set to    88% instead of 98% ie default value.  The low water mark default 80%, currently set to 75%. Will this have any impacts? the current percentage of DSSU usage is only 29%

[Marianne]

HWM and LWM has nothing to do with staging performance.

Disk Staging Cleanup Behavior: http://www.symantec.com/docs/TECH66149 

You will agree that your query about filesystem compression and performance has been answered many times over, right?

[edla_shravya]

Thanks a lot. can we set HWM & LWM to 88% and 75% or default values are suggested?

[Marianne · Accepted solution]

Please read through the cleanup TN. 

HWM and LWM is 100% your decision. The defaults are normally fine for most users.
I can only imagine that compression will cause issues/confusion because it will affect the actual filesystem usage and because filles need to be uncompressed before they can be read for duplication and restore purposes. 

My initial recommendation stands - disable compression.

You keep on changing the subject and does not seem to accept that your performance issue is primarily caused by compression. I have nothing more to say about this subject.

Good luck!