You still need to publish the EV Virtual directory for other EV operations to work from Outlook anywhere. Whatever you are saying is applicable to OWA external access, incase of OWA if you dont want AE and Search externally then there is no need to publish EV VD.
In the same guide, if you look at the Figure shown for Exchange 2010, you will understand that all EV requests are direct to EV Server:
The Enterprise Vault client contacts Enterprise Vault as follows:
■ By default, the client first attempts to contact the default Enterprise Vault
server that hosts the archive.
■ If that is unavailable, the client uses the RPC overHTTPproxyURLconfigured
in the Enterprise Vault Exchange Desktop policy.
■ If no URL is specified in the Enterprise Vault policy, then the client uses the
Exchange proxy URL configured in the Outlook profile.
With direct connections, all the Enterprise Vault servers that host the archives
must be accessible to the internal and external clients. If you do not want to publish
multiple Enterprise Vault servers to external clients, then you can use an
Enterprise Vault server as a proxy server. A client connects to the Enterprise
Vault proxy server, and the proxy server forwards the requests to the Enterprise
Vault server that hosts the archive.
