If Backup Exec is installed into the "virtual" machine then there are no compatibility issues. It will still function as it does in a normal OS.
Keep in mind you will not be able to use Backup Exec to backup the VMWare server since it is running its own propietary OS which BE remote agent cannot be installed onto. But inside the virtual machines can be protected.