Really need more detail to be honest, Slow retrievals can be for so many reasons...
Slow Storage or communication issues with the storage (Such as a centera or items on NetBackup)
Could be that the SQL Server has a query thats blocking other queries or making them run slower or the SQL Servers CPU is pegged at 100% and queries for the items are taking a really long time to complete.
It could be that the EV Server itself is 100% because of the EVConverterSandBox.exe converting items from their Word doc formats etc in to Text or HTML.
To be honest though, to me it sounds like its an issue with MSMQ
I think what you need to do is just dtrace some processes and retrieve a few items and record how fast they go and look at the dtrace to see what is really taking so long