All other things being equal (fill factor, etc.) then yes it could. If you have inserts over time, then you likely have unused space in the index. Doing a reorganize or rebuild physically reorders the pages in the leaf level. It also eliminates unused space on each page (subject to the fill factor, of course) thereby compacting them.

One word of caution, though, is if you use sort_in_temp_db = off, then the space used in your operation could cause the database to grow.

You're right in that any space gained will only be free space inside the database and will not be yielded back to the operating system.