John Rowan (10/6/2015)
You still have the Header and Trailer rows to contend with. A flat file source will allow you to skip the header, but not the trailer and it sounds like you need both to flow through to the destination files anyways.This calls for a Script source. I would recommend writing a script to process the file. You can contend with header/trailer and the primary/detail rows easier here than in a standard flat file source. You could use the script to add a column to the output that you can then key off of in a conditional split or you can write the rows to different outputs depending on the data in the primary row. Do you need to create a file for each State? If so, I'd go with adding a state column to the script output so you don't have to create 50 output buffers for the script.
You should be able to do with without a script:
1. Split out the header, trailer and detail records into separate flows
2. Add a key column (1 for header, 2 for detail, 3 for trailer)
3. do other stuff (e.g. select just the detail rows you want)
4. merge the three streams on the derived key column, specifying that the three dataflows are already sorted
Gerald Britton, Pluralsight courses