In order to configure a particular network interface to be used, you will need to use the Backup template to specify network adapter, otherwise the backup job would not know which NIC to use, and will go by whatever Windows takes. The IP set on remote agent is just used for advertising itself, it has no control over which NIC, the media server would use. You may try setting a NIC under "Network and security" tab in backup job properties and also specifiying the setting "use any available network route for remote agents not bound to..... ". That way it would use the specified NIC for the remote agents that are working on that network and for others it would use default route as taken by Windows. So, you would also not require setting up separate policies for this.
Please mark it a solution, if this is useful.
Thanks