I found this Bug Report that describes an issue with hung catalog backups on Windows, but the issue is solved with 6.5.4...
http://seer.entsupport.symantec.com/docs/323222.htm
Also found TechNote for AIX master - nothing for Solaris....
All I can suggest is that you enable bpbkar log on the master and examine the log when the backup hangs.