Not sure of the cause of your problem although you might want to telnet/ssh onto the Netapp and see what file sizes are reported at the file system level on the Netapp.
Of bigger concern, however, is if you have a VMware Virtual VMFS Datastore and Virtual Machines inside the LUN on the NetApp, then you have the wrong solution.
What you should be doing is presenting the NetApp LUN holding the VMFS Datastore to both the Media server and the ESX Hosts and then using SAN Transport to backup the VMs (with the backup storage device attached to the media server)
If you backup the LUN using the NDMP option then all you can restore is the whole of the LUN, there is NO granularity for individual VMs within the Datatsore and also no granularity down to individual file or e-mail level(GRT) within each VM. You are basically left with break one server in the datastore/LUN restore the whole datastore/LUN whihc might serverely impact any recovery process.
EDIT: you will of course need Virtual agent licneses and not the NDMP option license to implement the more appropriate solution.
EDIT2: The maintenance license is just there to warn you for what you might need to review your support, it does not affect the operation of backup Exec as long as you have the product licenses. We do have a defect about it reading incorrectly in one of our service packs that should be fixed in the next servcie pack.
