September 25, 2013 at 11:41 am
I have a multi instance sql server 2008 cluster running on windows server 2003. On the node that owns the quorum disk I keep getting messages that the quorum disk is running low on disk space. My question is whats cauing the quorum disk to fill up and how can I prevent the quorum disk form getting full
Thanks
October 4, 2013 at 3:56 pm
Check all the folders exists on Qouram Disk for space. Normally the size of Qouram should not be more then few MB's if there are not much of logs. You can check old logs and clear it in case that is not required. But i doubt, logs will cause space issue.
October 4, 2013 at 4:39 pm
Thanks for your reply, however the strange part is when I view the quorum disk through windows explorer, I cannot see what is filling up the disk. I could try increasing the space available to the quorum drive but that's just a fix not a root cause solution. The quorum disk is on a SAN and the cluster nodes are virtualized, so even increasing disk space would need an outage but so far I haven't had any luck finding out what's steadily eating up disk space.
October 5, 2013 at 3:11 pm
I have a similar problem that turns out to be some rather inefficient code that truncates several very large tables and reloads ALL of the data from an AS 400 every bloody night. Folks are in the process of repairing the code for performance and I'm in the process of moving the tables to a dedicated database in the SIMPLE recovery mode and to a separate set of disks not on the SAN.
My recommendation would be to run a server side trace for anything that takes too long or writes too much data. It's something that should be done once in a while even if it's not affecting quorum size.
--Jeff Moden
Change is inevitable... Change for the better is not.
October 7, 2013 at 5:52 am
Thanks for your reply Jeff, but Im unable to make the corelation between inefficient code and the quorum disk running low on disk space.
Would you be ableto elaborate on the connection between the two ?
October 7, 2013 at 5:58 am
Jeff Moden (10/5/2013)
I have a similar problem that turns out to be some rather inefficient code that truncates several very large tables and reloads ALL of the data from an AS 400 every bloody night. Folks are in the process of repairing the code for performance and I'm in the process of moving the tables to a dedicated database in the SIMPLE recovery mode and to a separate set of disks not on the SAN.
Why would that affect the cluster's quorum drive?
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
October 7, 2013 at 1:00 pm
Jeff, I am wondering doing anything in your DB, changing Database recovery model or performance issue on DB has anything to do with Quorum Drive...Quorum Drive is Database for your cluster.. So think from the angle that why your Quorum Drive is filling up fast, have you placed any thing else there or Quorum server log is filling it up.
October 7, 2013 at 3:42 pm
@Dhananjay,
From Windows explorer the only directories I see are the MSCS and MSDTC, there is nothing else that I can see that could be steadily eating up the disk space on the quorum drive.
October 7, 2013 at 4:15 pm
GilaMonster (10/7/2013)
Jeff Moden (10/5/2013)
I have a similar problem that turns out to be some rather inefficient code that truncates several very large tables and reloads ALL of the data from an AS 400 every bloody night. Folks are in the process of repairing the code for performance and I'm in the process of moving the tables to a dedicated database in the SIMPLE recovery mode and to a separate set of disks not on the SAN.Why would that affect the cluster's quorum drive?
I didn't have anything to do with this setup so I don't have a sufficient understanding of the details but our NetOps guys are apparently using the quorum drive to transfer data to our warm DR site. If the pipe between here and the DR site can't keep up, then the data on the quorum drive just gets larger and larger. There was some tipping point where if it got too far behind, it would go into a rather dedicated transfer mode and that just crushed the server when it happened. I use past tense because they supposedly fixed the problem but I'm not sure what they did to fix it other than getting a better pipe between us and the DR site.
--Jeff Moden
Change is inevitable... Change for the better is not.
October 7, 2013 at 4:24 pm
Jeff Moden (10/7/2013)
GilaMonster (10/7/2013)
Jeff Moden (10/5/2013)
I have a similar problem that turns out to be some rather inefficient code that truncates several very large tables and reloads ALL of the data from an AS 400 every bloody night. Folks are in the process of repairing the code for performance and I'm in the process of moving the tables to a dedicated database in the SIMPLE recovery mode and to a separate set of disks not on the SAN.Why would that affect the cluster's quorum drive?
Because everything that changes int these tables ends up going to the quorum drive.
If it does someone has badly configured that cluster. There should be nothing from SQL Server on the quorum drive. I don't even know if the quorum drive can be marked as a dependency of SQL, it should be off limits to everything but the cluster management itself.
All that should be on the quorum drive are logs and a few files put down by the cluster installation, it should never be used for anything else.
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
October 7, 2013 at 4:25 pm
Gerard Silveira (10/7/2013)
@Dhananjay,From Windows explorer the only directories I see are the MSCS and MSDTC, there is nothing else that I can see that could be steadily eating up the disk space on the quorum drive.
Do you have explorer set to show hidden and system files?
What are the sizes of those two directories?
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
October 7, 2013 at 4:31 pm
GilaMonster (10/7/2013)
Jeff Moden (10/7/2013)
GilaMonster (10/7/2013)
Jeff Moden (10/5/2013)
I have a similar problem that turns out to be some rather inefficient code that truncates several very large tables and reloads ALL of the data from an AS 400 every bloody night. Folks are in the process of repairing the code for performance and I'm in the process of moving the tables to a dedicated database in the SIMPLE recovery mode and to a separate set of disks not on the SAN.Why would that affect the cluster's quorum drive?
Because everything that changes int these tables ends up going to the quorum drive.
If it does someone has badly configured that cluster. There should be nothing from SQL Server on the quorum drive. I don't even know if the quorum drive can be marked as a dependency of SQL, it should be off limits to everything but the cluster management itself.
All that should be on the quorum drive are logs and a few files put down by the cluster installation, it should never be used for anything else.
Sorry... I hit post before I finished. I didn't have anything to do with this setup so I don't have a sufficient understanding of the details but our NetOps guys are apparently using the quorum drive to transfer data to our warm DR site. If the pipe between here and the DR site can't keep up, then the data on the quorum drive just gets larger and larger. From what I've been told, it's not actually SQL Server that's feeding the quorum drive. The quorum drive is only responding to what is changing on the SAN drives. What's changing on the SAN drives is an absolute shedload of data during the nightly runs. There was some tipping point where if it got too far behind, it would go into a rather dedicated transfer mode and that just crushed the server when it happened. I use past tense because they supposedly fixed the problem but I'm not sure what they did to fix it other than getting a better pipe between us and the DR site.
--Jeff Moden
Change is inevitable... Change for the better is not.
October 7, 2013 at 4:46 pm
Jeff Moden (10/7/2013)
our NetOps guys are apparently using the quorum drive to transfer data to our warm DR site.
Oh my, that is horrid cluster design. Nothing but nothing should ever go onto the quorum.
The quorum drive is only responding to what is changing on the SAN drives.
Shouldn't be doing that. There should be no impact on the quorum from the other drives. It's not a staging drive for other cluster drives, all it should be is there to 'arbitrate' ownership of the cluster
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
October 7, 2013 at 4:56 pm
Completely agree with GilaMonster.. DB has nothing to do with Quorum Drive... Not dependent too... Nothing exists on Quorum disk except few files. You need to check all hidden files and directory. If you can paste that here (snapshot), i will able to tell you the dependent files for cluster. Rest all you can remove..
October 7, 2013 at 6:16 pm
Here is the directory listing
The Quorum drive has 1GB space assigned to it
Volume in drive Q is SQLRDM02-Quorum00
Volume Serial Number is
Directory of Q:
10/07/2013 08:11 PM <DIR> MSCS
06/23/2009 03:50 PM <DIR> MSDtc
09/09/2012 09:06 AM <DIR> RECYCLER
05/31/2012 09:19 AM <DIR> System Volume Information
0 File(s) 0 bytes
Directory of Q:\MSCS
10/07/2013 08:11 PM <DIR> .
10/07/2013 08:11 PM <DIR> ..
07/07/2009 12:48 PM <DIR> 1b82aa9a-0428-400a-ba2f-98db2cdfe3a9
07/07/2009 07:30 PM <DIR> 36e30fb5-1862-490e-895d-c705e95661be
07/07/2009 12:38 PM <DIR> 54f37c4e-2a4c-4514-bd5f-045513b78133
07/07/2009 07:52 PM <DIR> 67bd7082-e6e2-4d39-8540-1c9597b68743
09/10/2010 10:09 AM 57,344 chk2E3F.tmp
10/07/2013 07:34 PM 57,344 chk5BE0.tmp
07/07/2009 07:12 PM <DIR> f883705e-1b80-4f8a-b925-74bf622b7567
06/23/2009 03:50 PM <DIR> f9670a20-d05a-40df-a6f2-f1e6e21022da
10/07/2013 08:11 PM <DIR> MSDtc
10/07/2013 07:34 PM 32,768 quolog.log
3 File(s) 147,456 bytes
Directory of Q:\MSCS\1b82aa9a-0428-400a-ba2f-98db2cdfe3a9
07/07/2009 12:48 PM <DIR> .
07/07/2009 12:48 PM <DIR> ..
07/07/2009 12:48 PM 764 00000001.CPR
1 File(s) 764 bytes
Directory of Q:\MSCS\36e30fb5-1862-490e-895d-c705e95661be
07/07/2009 07:30 PM <DIR> .
07/07/2009 07:30 PM <DIR> ..
07/07/2009 07:30 PM 764 00000001.CPR
07/07/2009 07:30 PM 8,192 00000001.CPT
10/07/2013 08:11 PM 12,288 00000002.CPT
07/07/2009 07:30 PM 8,192 00000003.CPT
07/07/2009 07:30 PM 8,192 00000004.CPT
07/07/2009 07:30 PM 8,192 00000005.CPT
07/07/2009 07:30 PM 8,192 00000006.CPT
09/25/2013 09:15 AM 12,288 00000007.CPT
8 File(s) 66,300 bytes
Directory of Q:\MSCS\54f37c4e-2a4c-4514-bd5f-045513b78133
07/07/2009 12:38 PM <DIR> .
07/07/2009 12:38 PM <DIR> ..
07/07/2009 12:38 PM 764 00000001.CPR
1 File(s) 764 bytes
Directory of Q:\MSCS\67bd7082-e6e2-4d39-8540-1c9597b68743
07/07/2009 07:52 PM <DIR> .
07/07/2009 07:52 PM <DIR> ..
07/07/2009 07:51 PM 764 00000001.CPR
07/07/2009 07:51 PM 8,192 00000001.CPT
10/07/2013 08:12 PM 12,288 00000002.CPT
07/07/2009 07:51 PM 8,192 00000003.CPT
07/07/2009 07:51 PM 8,192 00000004.CPT
07/07/2009 07:51 PM 8,192 00000005.CPT
07/07/2009 07:51 PM 8,192 00000006.CPT
09/17/2012 01:37 PM 12,288 00000007.CPT
8 File(s) 66,300 bytes
Directory of Q:\MSCS\f883705e-1b80-4f8a-b925-74bf622b7567
07/07/2009 07:12 PM <DIR> .
07/07/2009 07:12 PM <DIR> ..
07/07/2009 07:12 PM 764 00000001.CPR
1 File(s) 764 bytes
Directory of Q:\MSCS\f9670a20-d05a-40df-a6f2-f1e6e21022da
06/23/2009 03:50 PM <DIR> .
06/23/2009 03:50 PM <DIR> ..
06/23/2009 03:50 PM 764 00000001.CPR
1 File(s) 764 bytes
Directory of Q:\MSCS\MSDtc
10/07/2013 08:11 PM <DIR> .
10/07/2013 08:11 PM <DIR> ..
0 File(s) 0 bytes
Directory of Q:\MSDtc
06/23/2009 03:50 PM <DIR> .
06/23/2009 03:50 PM <DIR> ..
06/23/2009 03:50 PM 4,194,304 MSDTC.LOG
1 File(s) 4,194,304 bytes
Directory of Q:\RECYCLER
09/09/2012 09:06 AM <DIR> .
09/09/2012 09:06 AM <DIR> ..
09/09/2012 09:06 AM <DIR> S-1-5-21-1394358491-809983319-1777090905-4540
10/13/2010 01:15 PM <DIR> S-1-5-21-1394358491-809983319-1777090905-4632
09/06/2013 01:02 PM <DIR> S-1-5-21-1394358491-809983319-1777090905-500
0 File(s) 0 bytes
Directory of Q:\RECYCLER\S-1-5-21-1394358491-809983319-1777090905-4540
09/09/2012 09:06 AM <DIR> .
09/09/2012 09:06 AM <DIR> ..
09/09/2012 09:06 AM 65 desktop.ini
09/14/2012 01:21 AM 20 INFO2
2 File(s) 85 bytes
Directory of Q:\RECYCLER\S-1-5-21-1394358491-809983319-1777090905-4632
10/13/2010 01:15 PM <DIR> .
10/13/2010 01:15 PM <DIR> ..
10/13/2010 01:15 PM 65 desktop.ini
10/13/2010 06:01 PM 20 INFO2
2 File(s) 85 bytes
Directory of Q:\RECYCLER\S-1-5-21-1394358491-809983319-1777090905-500
09/06/2013 01:02 PM <DIR> .
09/06/2013 01:02 PM <DIR> ..
09/06/2013 01:02 PM 65 desktop.ini
09/27/2013 07:37 AM 20 INFO2
2 File(s) 85 bytes
Total Files Listed:
30 File(s) 4,477,671 bytes
40 Dir(s) 52,877,312 bytes free
Viewing 15 posts - 1 through 15 (of 21 total)
You must be logged in to reply to this topic. Login to reply