In the US colonial days, over 90% of the labor was involved in food production. Which means that less than 10% of the labor potential of people was available for other uses (including housing, clothing, transportation, etc). Now that number is reversed, so little of our national labor output is needed for food production that it can provide so many other goods and services. We are all very much richer.

When it comes down to it, you buy with the fruits of your week's labor the combined output a week of other people's labor. Productivity is the only process that actually raises the wealth of society as a whole.