If the COM object is hung up in-process then restarting SQL is the only way to clear it.

I found this http://support.persits.com/show.asp?code=PS051122152 it seems to use sp_oadestroy to destroy thing other than the object, I've not seen that before. I find a secondary similar reference http://www.cknotes.com/?p=437

Also you really should set the max memory for you SQL Server, I would think in the 14-15GB range, depending on what else is running on that box perhaps as low as 13GB..

CEWII