I don't think this is the case. Public should have very few rights, and I'd be surprised at this. Public has few rights, and I wouldn't expect this to be the case. This should require ALTER ANY LINKED SERVER permission in 2005.

http://msdn.microsoft.com/en-us/library/ms190479%28v=sql.90%29.aspx

If I check my 2005 instance, I get permission denied for a new login with just public