Great points Steve. I have been amazed at the rapid expanse of storage space as well. We used to be proud of a sector by sector disk backup that did 5MB in 5 minutes. Then one client said that they could not afford the disk space for SQL backups. I went to a local retailer and had an ad printed for 1TB external drives.

As to files there is a system that digests transaction files like table scrap down a garbage grinder. Each processed file is placed in an archive folder. Daily those get zipped up one zip per day. We can go back and look at every transaction since the system went on-line.

We had figured out that the NTFS in XP and Windows 7 is a database. Avoiding the "Hammer and Nail" thinking allowed us to produce systems that scale in both directions.

I attended a technology demo at a large freight company whose headquarters were local to me. They were scanning in every document that came into the company and storing the images on very large optical disks. The documents were run through OCR off-line and that data was compared to the on-line systems. Impressive. It was almost two decades back by now.

"Don't worry. We will never have more than 32767 customers." The old rules for area codes in the US was that the middle digit had to be a 1 or 0 and that no area code could cross a state line. Scale considerations will come to bite you sooner or later.

The tough part is figuring out what to keep on-line and what can go off-line. We tend to want to keep everything. It takes making the tough decisions as to where to draw that line.