Good question.

My problem is I am not certain that this implementation of the feature is very good. I can find where people would want to remove the in-memory filegroups. Needing to restore the database in order to remove the filegroups is a complete pita.