That is in no way a stupid question, one that's stumped me too.
Although [SQL Server 2008 Internals] by Kalen Delaney et. al. mentions that SQL Server will store (MAX) data types just like regular LOBs like text, ntext & image, that's bit of an oversimplification.
One comment on your test - REPLICATE('a', 10000) will only result in a 8000 character long value. As 'a' is implicitly typed as a varchar with a max value of 8000. REPLICATE(CAST('a' AS varchar(MAX)), 10000) will give the 10.000 character long value as expected. It doesn't change the result of the test though.
The overflow pointer uses 24 bytes by default, but it may take up more space. Each page slot referenced takes up 12 bytes so you may see pointers of size 24, 36, 48, etc. At some point SQL Server decides to store a single reference to a text tree page instead. The largest pointer I've been able to reproduce is 72 bytes, after which it starts using 24 byte pointers to text_tree pages instead.
So when does it store a regular 16 byte LOB pointer? Only when you set the "large value types out of row" table setting to ON:
sp_tableoption N'MyTable', 'large value types out of row', 'ON'
At this point, no matter the size of the value it will be stored as a LOB value using a 16 byte LOB pointer.