• darren_ferris (3/19/2014)


    Thanks for your reply Grant.

    Once a simulation has completed the user would then typically view the results (or a subset of the results) via our application. They might also want to delete all of that simulation's results which also needs to be pretty quick.

    Thanks again,

    Darren

    1) Writing the code to process dynamically named tables can be pretty tedious. But for each sproc or piece of code you wish to run once you write it you are largely done with it forever. And when you do need to do a refactor, that is where your code comments come into play. 🙂 I have done numerous systems like this, especially for large open-ended search scenarios and assuming you have rigorous naming conventions it really isn't that bad or tough. There are quite a bit of benefits here in that if necessary you can spin up new databases as required (even for every run believe it or not). That can be critical since you have SQL Express customers and the size limit there (10GB) can come into play with lots of data. Other benefits include the ability to spread data on different file groups, eliminating of most of the latch/lock issues you may see with the single table approach, ability to drop or truncate tables to clear out old data, index maintenance and statistics updates are much easier to handle, etc.

    2) If you do go with a single-table approach you really do paint yourself into some difficult corners as mentioned above, but as Grant said chief among your concerns for high-throughput will be your clustering key. Since you are on SQL 2012 I highly recommend 2 or more SEQUENCE objects. Each number generated will be something important, such as test number, run number, machine number, row number or whatever you need to uniquely identify individual records. You should put these in the clustered index in least to most specific order. Test#, Run#, Row# for example. This will keep your inserts away from each other in the balanced binary tree that is the clustered index, mitigating latch contention at the end of the index chain. BUT this will also cause fragmentation of the index at multiple spots. But at least each run will be mostly contiguously located, which is really what you need.

    Sounds like a VERY fun project - I am envious!!!

    Best,
    Kevin G. Boles
    SQL Server Consultant
    SQL MVP 2007-2012
    TheSQLGuru on googles mail service