Actually object_id in both views sys.dm_db_index_usage_stats and sys.indexes refers to the index's table.

Check MSDN for column description of each views.

So it's fine. 😉