Windows doesn't see 4GB. If you have a 32bit OS with 4GB of RAM, Windows can't see it all.
I saw this very interesting post from Hilton Locke at Microsoft on the issue. Apparently it's mostly hardware, depending on what you have, and an architectural decision that there are memory mapped IO reservations.
I'm not sure I understand it all, but it's an interesting post.