This is a bug with SQL server. HEre is the workaround.

1. Add another file to the same file group

ALTER DATABASE DBNAME

ADD FILE

(

NAME = Data_File_Name,

FILENAME = [X:\SQLData\FileName.ndf],

SIZE = 1MB,

FILEGROWTH = 10%

)TO FILEGROUP [FileGroupName];

2. Empty the first file

DBCC SHRINKFILE(First_Data_File,EMPTYFILE )

3. Now you can Remove the original file and the new file you added and the file group

ALTER DATABASE DBNAME REMOVE FILE First_File;

ALTER DATABASE DBNAME REMOVE FILE 2nd_File;

ALTER DATABASE DBNAME REMOVE FILEGROUP FileGroupName;

Hope this helps.