SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


««12

Large Number of Tables (1,090,000) Expand / Collapse
Author
Message
Posted Thursday, November 19, 2009 7:56 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Yesterday @ 11:28 PM
Points: 6,006, Visits: 2,426
SQL may be able to handle that many tables. The problem is that these are tables being generated (if i understand correctly) through one of the Microsoft CRM products (NaVision). The creation of these tables and the code-behind dynamically query the metadata - and the tables are not the most sound design (IMO). It is due to these issues that the slowness is becoming an issue.

CRM is not designed for a DBA to create the tables, procs, or indexes. CRM likes to handle all of that stuff itself - and it shouldn't.




Jason
I have given a name to my pain...

SQL RNNR

Posting Performance Based Questions - Gail Shaw
Posting Data Etiquette - Jeff Moden
Hidden RBAR - Jeff Moden
VLFs and the Tran Log - Kimberly Tripp
Post #822050
Posted Saturday, November 21, 2009 8:54 AM
SSCommitted

SSCommittedSSCommittedSSCommittedSSCommittedSSCommittedSSCommittedSSCommittedSSCommitted

Group: General Forum Members
Last Login: Today @ 7:13 AM
Points: 1,790, Visits: 2,459
Hardware is probably the only viable solution here. Lots of RAM will help to keep more pages of data in cache, but that will not get you very far if you have to access lots of those 'company' table sets regularly. CPUs will be important too. But by far the most important thing will be IO SPEED (not capacity). You need LOTS of spindles, well configured and tuned and with significant bandwidth to and from them. Or you need some SSDs.

You are at the extreme end of the scale here - much like a client I have who has over 7000 databases on one server. There are things you need to do right to have any chance of success. I HIGHLY recommend you get a very good performance consultant in place to give your system a performance review and help get your system configured for optimal operations.

Normally when I make this recommendation to forum posters (which I do quite a bit actually since I think it is the best thing that numerous entities out there can do) I make a generic statement saying "get someone". In this case I will make an exception and say "you should consider using me" because I bring to bear the level of knowledge and experience you really need for such a configuration. I also am fortunate enough to be able to cherry pick the clients I take and your situation seems 'interesting' to me.

In any case, best of luck with your system!!



Best,

TheSQLGuru
SQL Server MVP
Post #822896
Posted Monday, November 23, 2009 9:51 AM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Wednesday, March 17, 2010 2:28 AM
Points: 70, Visits: 224
Thanks for the all your help it is all appreciated
Post #823354
Posted Monday, November 23, 2009 12:03 PM
SSC-Addicted

SSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-Addicted

Group: General Forum Members
Last Login: Wednesday, March 10, 2010 1:52 PM
Points: 469, Visits: 89
Depending on how much data you need to be available for "every day work" you might wanna consider "archiving" some of data since you must have a lot of data that is not even used. 5000 companies /2 every day=2500 days worth of data in same place ...doubt that they need that much data to be online all the time. At the most they might need some "summary data to be available" rest can go to archive.

Regards,
Vasc



Kindest Regards,

Vasc
Post #823442
Posted Monday, November 23, 2009 11:15 PM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Yesterday @ 9:01 PM
Points: 20,164, Visits: 13,702
Actually, I'd run like hell.

--Jeff Moden
"RBAR is pronounced "ree-bar" and is a "Modenism" for "Row-By-Agonizing-Row".

"Data isn't the only thing that's supposed to have Integrity."

First step towards the paradigm shift of writing Set Based code:
Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

For better, quicker answers on T-SQL questions, click on the following...
http://www.sqlservercentral.com/articles/Best+Practices/61537/

For better answers on performance questions, click on the following...
http://www.sqlservercentral.com/articles/SQLServerCentral/66909/
Post #823680
« Prev Topic | Next Topic »

««12

Permissions Expand / Collapse