Excellent article. This is similar to something i have previously developed whilst working for a company which processes 10 - 1000's of data files daily.

having the information in the database made it so much easier to process and create new imports.

the one thing we didn't have was detailed info from our suppliers, we had to work this out and make loads of assumptions too.

we also had the added problem of managing excel files along with text format files.