• Having a local cache is sensible. The caveat to storing data locally is to check the terms and conditions for whatever the data is.

    There's a lot of information out there in the public domain but some of it runs into legal grey areas. Effectively setting up a local cache can violate Ts&Cs.

    Where legally valid a weekly refresh sounds a good policy. Again, there needs to be full disclosure of what you are doing. If you refresh on a Monday and the source data changes on a Tuesday then the worst case is 6 days inaccurate data. The impact of incorrect data has to be understood. In the majority of cases it won't matter but if that data formed part of the risk factors for insurance then the customer would be under the impression that they were insured when in fact they were not.

    I would also not limit my thinking to what could be expressed in SQL. If you have clear rules what you have is a spec against which tests can be writtenin what ever tier of the application is appropriate