I see that a 'Remote Agent for Windows' comes with a 'Agent for database and applications'
No. It is the other way round. The Agent for Applications and Databases licence includes the Agent for Windows licence. If you buy the Agent for Windows licence, you would not be able to back up any applications/databases.
do I need the Agent for Windows for the HOST OS on the remote server?
No. You just need the Agent for VMware and Hyper-V licence for the host which also entitles you to Agent for Windows licence for any number of VM's on the licenced host. Again, this does not allow you to backup any applications/databases in any VM. You would need an Agent for Applications and Databases licence for each application/database in each VM.