I can think of 3 solutions:

1) Hire a professional to help you tune the poorly performing database. It seems that you are not a SQL Server guru so I bet there are lots of things that could be improved. This is NOT a personal attack, just an observation based on years of experience.

2) Upgrade (at significant expense) to SQL Server 2008 Enterprise Edition and then take advantage of the Resource Governor. Note that you cannot 'govern' IO, which is a limitation of this subsystem.

3) Have 2 instances on the server and put the 'bad' database on one of them and then restrict that instances CPU and Memory. Expect exceedingly poor performance of that database however, but it could free up sufficient resources to allow the other database to perform acceptably.

Note that you should try to put the 'bad' database on separate IO subsytem if at all possible in any event.