VxFS Performance problem after HPUX/Veritas upgrade
An an old machine - model "9000/800/rp4440 " (4 PA-RISC 8800 processors (1000 MHz, 64 MB)) with 16GB we have upgraded HPUX from 11.23 to 11.31 - Veritas from 3.5 to 4.1. Now, from time to time we have system with very high response time. Basically, all normal operations (login, directory listing, ... ) take very, very long time, even though the CPU usage during that period is very low. Also, memory system is occuped only 50%, pagging is not used at all, I/O seems not to be overloaded too. Sometimes this situation is solved after some period of time without intervention, but during working hour machine has to be restarted, because we can't wait too much such situation during the day. One of measureable issues we have detecetd is slow execution of lstat64 system call on vxfs partitions/mount points. We used TUSC to perform a 'll' command on a directory with 15k files. NFS mount points are not affected. For example: 06:31:46 [ls -l /test ]{2035012} (0.000054) lstat64("/test/FILE_00249395_20130301063957.Z", 0x77ff0468) = 0 st_dev: 64 0x000003 st_ino: 42172 st_mode: S_IFREG|0640 st_nlink: 1 st_rdev: 0 0x000000 st_size: 0 st_blksize: 8192 st_blocks: 0 st_uid: 0 st_gid: 3 st_aclv: 0 st_acl: 0 st_fstype: 10 st_atime: Thu Feb 26 15:29:17 2015 st_mtime: Thu Feb 26 15:29:17 2015 st_ctime: Thu Feb 26 15:29:17 2015 06:32:29 [ls -l /test ]{2035012} lstat64(0x40001888, 0x77ff0468) [running] 06:32:29 [ls -l /test ]{2035012} (42.357834) lstat64("/test/FILE_00249396_20130301064257.Z", 0x77ff0468) = 0 st_dev: 64 0x000003 st_ino: 42173 st_mode: S_IFREG|0640 st_nlink: 1 st_rdev: 0 0x000000 st_size: 0 st_blksize: 8192 st_blocks: 0 st_uid: 0 st_gid: 3 st_aclv: 0 st_acl: 0 st_fstype: 10 st_atime: Thu Feb 26 15:29:17 2015 st_mtime: Thu Feb 26 15:29:17 2015 st_ctime: Thu Feb 26 15:29:17 2015 06:32:29 [ls -l /test ]{2035012} (0.000063) lstat64("/test/FILE_00249397_20130301064557.Z", 0x77ff0468) = 0 As you can notice, there is a 42seconds of delay during the execution lstat64 system call. This is observed on a local vxfs drive (Veritas 4.1 is installed). The whole directory listing 'll' sometimes takes 15min, sometimes even 20min. When system is fresh reboted it takes few seconds. Just to note. System did not change in hardware or softvare after the upgrade. All issues staretd after the upgrade. I don't know if someone had similar issues after the upgrade to 11.31 (Veritas 4.1). If you can get a hint what should be checked I would appreaciate very much.
Announcing Storage Foundation High Availability Customer Forum 2012
We are excited to announce Storage Foundation High Availability Customer Forum 2012, a free learning event by the engineers for the engineers. Registration is open, register now to get on the priority list. The forum will take place at our Mountain View, CA offices on March 14th and 15th 2012. Join your peers and our engineers for two days of learning and knowledge sharing. The event features highly technical sessions tohelp you get more out of your days. Learn and share best practices from your peers in the industry and build a long lasting support network in the process Become a Power User by significantly increasing your troubleshooting and diagnostic skills as well as your product knowledge Engage with the engineers who architected and wrote the code for the products Please see here for event agenda and sessions details. More questions? See our events page.