I believe you are correct when you say, "To a large extent I believe that we really don't understand exactly how we should build software, and highly underestimate the complexity." I see several parts to this problem:

1. The "how" part: Since the late 1960s, there has been a movement toward the idea of software engineering. This was an attempt to apply the discipline of engineering to the writing of software. Although this made great strides, it has ultimately fallen short because the user of the software is often not a machine.

2. The "complexity" part: Many, if not most, of the simple tasks have already been computerized. Think of calculators, cell phones and such. Businesses are now attempting to solve very complex planning and logistical problems. They do not yet understand what type of solution they want, nor how to implement it as a "people process." The business management types are trying to define what results they want as they look at the results of the latest computer output. This leads to one of two cases: the software is never good enough or the software is never done.

We, as software developers, lose. We are blamed for these shortcomings (management NEVER blames themselves). We need to develop better tools for ourselves which allow us to better understand and model what the business management types want. We also need to develop better tools for rapidly providing ad hoc results to show for the next cycle of goal revisions.

Adding to our dilemma, our budgets are constrained. We cannot buy or develop the tools we really need. Sales, marketing, legal and the executive offices get what they need. Production, IT, accounting and such have to make do. In order for this "software" problem to be solved, business management types must understand and appreciate the importance of all parts of the organization as important components in solving the problems of running a modern business.