These are the steps you can take to perform this operation:
1. Install BE 12.0 on the new server with default selection, let the database go to default instance created by Backup Exec.
2. Use following document to move database to the new Backup Exec installation:
http://support.veritas.com/docs/286093
3. Perform an upgrade of BE 12.0 to BE 2010.
4. After step 3, your database will be in the default instance of BE. Now, we need to move it to SQL 2005 instance. YOu can use the following document to do this:
http://support.veritas.com/docs/275658
Make sure to backup your database, before carrying out these steps.
Please mark it a solution, if this is useful.
Thanks