Performance implications due to data growth on well tuned database

  • Hi,

    Recently our company began considering an increase in the amount of content we house in our MSSQL 2000 database. One of the obvious questions that came up was what would be the performance implications - mostly on the stored procedures, which are used by our clients, and are expected to continue to return results within a few hundred milliseconds.

    So if all our stored procedures are well tuned to use indexes and if we have a lot of RAM, would that mean that we can grow without a large affect on our stored procedure performance?

    I know this is not a yes or no question, but just wanted to hear some general thoughts/opinions.

    I guess the first question is whether my assumption is correct about the MSSQL 2000 indexes. Are they always stored in memory?

    Please let me know any suggestions

    Thank you!

  • Your assumption that indexes are stored in memory is slightly incorrect. All data, whether from a table (heap or clustered index), or a non-clustered index, is stored on disk, in the form of 8KB pages. When SQL Server is processing a query, it will look for the pages required. If the page is not found in memory, it will be loaded in from disk and cached for a period of time, depending on how often the page is used, and how much memory pressure the server is under. As the non-leaf pages of indexes are frequently accessed, these will typically stay in memory.

    In a well tuned database, it will be rare that you will be performing scans of tables. You will certainly not be scanning the past 7 years if you're only interested in the last week's data. This means that the amount of data you need to store in memory is much smaller than the entire database size.

    If you do have a table scan, your 7 years of data will likely not be in memory (it may not even fit in memory!), and so the query will have to perform a lot of I/Os against the disk - the query is "running from disk". If your query is well tuned, then only the necessary pages will be read. If this is, say, 1000 pages (8 MB), then this will run quickly whether the pages are in memory, or on disk (although obviously in-memory will be an order of magnitude faster).

    Even if you have enough memory that your entire database will fit into memory, you will still want to have tuned queries. Even if everything is in memory, scans create contention (blocking) and use a lot more CPU resources.

  • Thanks Jim!

Viewing 3 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply