Have you looked at your wait stats?  Although you haven't provided any detail I can guess that such high latency could be caused by too much IO read:  not enough spindles and/or datafiles in the database for the amount of requests that need ot be serviced.

How is your TempDB set up?