David, Im surprised you're so late joining the agile movement! But pleased nonetheless!

I'm a huge fan of Scrum. XP is interesting, but is seriously rigid. By comparison Scrum is a very analog approach that can be sized to fit your situation. For example, they suggest 30 days sprints and I've had success shortening them to as little as 30 days.

I agree that its counter intuitive not to plan for the future, but your analogy is flawed in my opinion. Software isnt the same as house building, the cost to refactor can be (not always) substantially cheaper than having to tear down a room to upgrade the foundation. In practice experience should play a role in what you decide to deliver, but I've learned to err on the side of leaving something out rather than risk putting one more thing in that I might not need. Too many projects fail because of sheer scope, take out all the extras and build something, then iterate.

I've seen a lot of people dismiss agile as the 'flavor of the month' and I think they are missing out. Like any tool/process it can be used badly, but used well it maps nicely to how the real world works. Most of us need to see the software to figure out it works the way we need it to, we need to use it to learn what would make it better, and we all need to get in the same room to talk about it.