• fazalkhansb (7/15/2011)


    Greate article and the disscussion about historical data, here I am going to say something and need your help.

    I would also add effectiveStartDate and effectiveEndDate fields to the PayGrade_Details table so that you could see that Charlie Fixit spent 3 years at paygrade 12 and then got bumped up to paygrade 13 and when it happened. Then I would just grab the paygrade record whose start date/end date falls within the proper date range that I need...I'm not sure if that is the correct way to do it or what NF that is but that's the way I've been doing it

    I am also doing this fromdate/todate technique to store historical data, but I am not sure that this is the good one to do this... In this particular way we need to keep NULL value or any predefined default value into 'todate' column untill the row is treated as the current... please advice any other way to mange the historical data...

    I could also use a historical or archive paygrade table and move the old records into that table when they are changed...but I'm not sure which way is the recognized "proper" way to do it.

    I haven't try it till now, I think it could be better than first one. because there is no NULL value stored here......

    Thanks

    As I said in an earlier message, this is an area where my experience is limited. You can choose to distinguish historical and current data either by the historical data in a separate table (which complicates the introduction of a new value - the old one has to be moved) or by not recording the end date at all because it is redundant - the previous value's reign ends when the next value begins (but then you have to be wary of performance - watch out for "triangular joins") or by using a special value (or NULL, but that's usually not a good idea) for the end date to distinguish current data (that may be the best option - but don't use a date anywhere in the feasible lifetime of your database as that special value). Look at Esteban Zimányi's 2006 paper "Temporal Aggregates and Temporal Universal Quanti?cation in Standard SQL" (available on pages 16 to 25 of this pdf doc, or get hold of the book "Temporal Data and the Relational Model" by Date, Darwen and Lorentzos if you are heavily into data warehousing with history requirements. For a brief discussion the normal form level of some table types that may be useful there is this but be warned it's aimed at showing that their chosen table forms are all in 6NF, rather than showing how to do anything useful with them; but if you like it, or the tables it describes, you may want to read up on anchor modelling more generally.

    Tom