I'm sharing the answer that i found in the hopes it helps someone else.

I too had a stubborn filegroup that could not be removed. All the files had been removed from the filegroup, the filegroup had been removed from the partition scheme, and it was not holding service broker information. It was truly an empty filegroup yet SQL Server thought it was not empty and thus would not allow the filegroup to be deleted.

Using the following query i found a record in sys.destination_data_spaces for the filegroup i wished to delete and this record was preventing me from deleting the filegroup.

select

partition_scheme_id,destination_id,a.data_space_id,name,b.data_space_id

from sys.destination_data_spaces a right join sys.filegroups b

on a.data_space_id = b.data_space_id

Why did this occur? I could be wrong but my theory is this:

I had been massaging the partitions quite a bit, adding some new splits to the partition schema then removing some partitions by merging as a result there was gaps in the list of data_space_id's like so

1

2

3

4

6

9

10

11

id's 5 and 7 and 8 had been deleted, id 11 was the filegroup i was wanting to delete but could not.

The resolution:

I added 4 new filegroups and then added these filegroups to partition scheme which created ids 5,7,9, and 12. This cause SQL Server to remove id 11 from sys.destination_data_spaces and then i was able to delete the filegroup with no problem.