Hi Darren,
You need a Remote Agent on a server to back it up if it's not your media server. it won't work otherwise.
If this server is no longer needed on the backup job, you can go into your selection list and remove it from there, along with checking your inclusion/exclusion list to make sure that there are no references there.
Have you checked to make sure that the service account isn't locked out?
Can you also post the exact error?
Laters!