The problem with virtualizing a media server is access to tape devices - it is fine if you are only messing with backup to disk and in fact the support teams do often run virtual media servers for testing. However about the only virtual machine access to a tape drive that is supported is based on a VMware document that states that you must be using an Adaptec SCSI controller (so no SAN and no SAS controllers) and there is no support on a Xenserver for physical hardware access by BE.
Note this does not meamn you won't be able to get it to work, just means we don't test it so can't support you should you have problems