In Memory Database Remove File and FileGroup

I have a SQL Server 2016 database and I configured in-memory table. I want to drop this configuration, then I drop that table. When I want to delete that file and filegroup, I got following error. USE [InMem_Test] GO ALTER DATABASE [InMem_Test] REMOVE FILE [InMemFile] GO ALTER DATABASE [InMem_Test] REMOVE FILEGROUP [InMemFileGroup] GO > Msg 41802, Level 16, State 1, Line 3 Cannot drop the last > memory-optimized container 'InMemFile'. Msg 5042, Level 16, State 11, > Line 5 The filegroup 'InMemFileGroup' cannot be removed because it is > not empty.