SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Scaling Out the Distribution Database


Scaling Out the Distribution Database

Author
Message
David.Poole
David.Poole
SSCertifiable
SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)

Group: General Forum Members
Points: 7847 Visits: 3290
Comments posted to this topic are about the item Scaling Out the Distribution Database

LinkedIn Profile

Newbie on www.simple-talk.com
Paul White
Paul White
SSCoach
SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)SSCoach (16K reputation)

Group: General Forum Members
Points: 16116 Visits: 11355
Excellent article, David.



Paul White
SQLPerformance.com
SQLblog.com
@SQL_Kiwi
John.Sansom
John.Sansom
SSC Eights!
SSC Eights! (827 reputation)SSC Eights! (827 reputation)SSC Eights! (827 reputation)SSC Eights! (827 reputation)SSC Eights! (827 reputation)SSC Eights! (827 reputation)SSC Eights! (827 reputation)SSC Eights! (827 reputation)

Group: General Forum Members
Points: 827 Visits: 1558
Great article David!

I particularly enjoyed reading about Adding a New Distributor to an existing Replication Topology.

I would be interested to know your thoughts on implementing this but for a high availability scenario.

So for example there could be One Publisher (a Read/Write server for an application) and say 8 Subscribers (Read Only Databases for an application) i.e. a typical search pool scenario. Let’s say the Distributor could not be taken down for more than a few seconds as the Publisher and Subscribers must remain within a reasonable sync time i.e. seconds.

In such a scenario I am thinking that a solution could be to create an identical Publication, that points to a new Distributor and to move the Search Pool servers in and out of rotation one at a time. In other words, drop one subscription at a time and re-create using the new Publication (that references the new Distributor). This way the application remains operational whilst the migration is being implemented.

My major concern would of course be ensuring the consistency of the data during throughout the migration process.

I would be interested to know your thoughts and whether you have worked on a scenario such as this.


John Sansom (@sqlBrit) | www.johnsansom.com
Kendal Van Dyke
Kendal Van Dyke
Say Hey Kid
Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)

Group: General Forum Members
Points: 698 Visits: 983
David,
Can you elaborate on the following statement you made:
the number of records in the MSRepl_transactions table is likely to be very high and once it has gone beyond a certain size

and provide guidance on what that "certain size" is and how someone knows if they've exceeded it?

Also, do you have any suggestions for other steps that someone should take to optimize their existing distribution database before deciding that it's time to create a new one?

Kendal Van Dyke
@SQLDBA

Kendal Van Dyke
http://kendalvandyke.blogspot.com/
David.Poole
David.Poole
SSCertifiable
SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)

Group: General Forum Members
Points: 7847 Visits: 3290
It is very hard to say what "a certain size" is because it depends on the hardware and configuration of the machine that is acting as a distributor and also the amount of data running through the distributor.

We have some databases that replicate quite narrow tables and these are good for tens, if not hundreds of millions of records. Others have wide tables and the distributors struggle if the number of replicated transactions goes into the low millions.

LinkedIn Profile

Newbie on www.simple-talk.com
Kendal Van Dyke
Kendal Van Dyke
Say Hey Kid
Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)Say Hey Kid (698 reputation)

Group: General Forum Members
Points: 698 Visits: 983
I probably asked the question poorly, so let me take another approach...let's say I've been tasked with managing a replication environment and I notice that things are starting to get slower and slower. I've read your article and start to wonder if I should add a second disribution database, but doing so requires setting up a maintenance window and a signficant amount of work on my part. What kinds of things should I look at to get a better feel for if adding a second distribution database is really the right course of action?

Kendal Van Dyke
http://kendalvandyke.blogspot.com/
RML51
RML51
UDP Broadcaster
UDP Broadcaster (1.5K reputation)UDP Broadcaster (1.5K reputation)UDP Broadcaster (1.5K reputation)UDP Broadcaster (1.5K reputation)UDP Broadcaster (1.5K reputation)UDP Broadcaster (1.5K reputation)UDP Broadcaster (1.5K reputation)UDP Broadcaster (1.5K reputation)

Group: General Forum Members
Points: 1465 Visits: 1612
Good article!

I see that the Distributor Properties for the Transaction retention value is 72 hours, and the History retention value is 48 hours. These are default values. I'm curious if anyone changes these values, and if so why? Also, what are the consequences if these values are changed?



Brannon Weigel
Brannon Weigel
SSC-Enthusiastic
SSC-Enthusiastic (115 reputation)SSC-Enthusiastic (115 reputation)SSC-Enthusiastic (115 reputation)SSC-Enthusiastic (115 reputation)SSC-Enthusiastic (115 reputation)SSC-Enthusiastic (115 reputation)SSC-Enthusiastic (115 reputation)SSC-Enthusiastic (115 reputation)

Group: General Forum Members
Points: 115 Visits: 295
Great Article!

I have all my published databases on one SQL Instance. I have a stand-alone Distributor server which gets beat up every so often. It would be nice if it were possible to configure one distributor database per published database. Or even one distributor database per publication, if one really wanted to do that. Hopefully, there is a good reason for having only one distributor database per published instance! But, I'm hoping that changes! Smile

Brannon Weigel



Toby White
Toby White
Right there with Babe
Right there with Babe (765 reputation)Right there with Babe (765 reputation)Right there with Babe (765 reputation)Right there with Babe (765 reputation)Right there with Babe (765 reputation)Right there with Babe (765 reputation)Right there with Babe (765 reputation)Right there with Babe (765 reputation)

Group: General Forum Members
Points: 765 Visits: 639
Good article. You did a good job of describing how to add distribution databases, which is something that I had never considered before.

However, I am curious about the factors behind your recommendation. I would expect a bottleneck to occur on the CPU or I/O, but not the database itself. You referenced the MSRepl_transactions table getting too large, but I don't see that as being a problem because the clustered index is on (publisher_database_id, xact_seqno) which is int and varbinary(16). Would the concern be seeks because the index level would be deeper with more records? What advantage would there be assuming one disk array and I/O is not the bottleneck?

I think you did great in the how to, but I am wondering about the why.

Regards,

Toby
David.Poole
David.Poole
SSCertifiable
SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)SSCertifiable (7.8K reputation)

Group: General Forum Members
Points: 7847 Visits: 3290
On one particular publication we changed the retention period to 3 hours because if something goes wrong with the subscriber then the transactions start building up in the distributor and if that comes under stress the publisher log files start to bloat until disk space on the production box is threatened.

It's worth mentioning that we do have to DBCC DBREINDEX or ALTER INDEX REBUILD regularly.

The distribution databases are counted as system databases but they behave like user databases.

LinkedIn Profile

Newbie on www.simple-talk.com
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search