Is the Vault Service Account been added to the Domain Admins groups? This error is commonly caused by a security setting within MSSQL that forces all connections by Domain Administrators to be via a trusted connection(SSL).
Something else to try is to disable or remove the Enhanced Security settings because you have to add the site to the Vault Service Account profile, not the account that your accessing it from.
Something else to verify is that you do not have large icons enabled within your Display Settings. The Enterprise Vault forms only have icons that support the normal 96 DPI settings. Large or Medium icons will cause the icons to show up as normal ...