I actually found an easy solution to this fix since I had the same issue.

I was trying to remove the filegroup [fg_LMeterDetail_13] but got the "cannot be removed because it is not empty" error. This filegroup's only sin was being associated with a partition scheme. This filegroup had no data.

So after finding there's no opposite to NEXT USED, I experimented and found that you can issue another NEXT USED but pointing it to the previous filegroup:

ALTER PARTITION SCHEME ps_LMeterDetail

NEXT USED [fg_LMeterDetail_12]

After that, I was able to issue the REMOVE FILEGROUP which then automatically removed it from the partition scheme

ALTER DATABASE [WACOE] REMOVE

FILEGROUP [fg_LMeterDetail_13]

Voila! It worked!

You're welcome everyone.