April 27, 2011 at 7:52 am
Good stuff , sounds like you are on the right path.
April 27, 2011 at 9:30 am
OK it would appear the differential backups (weeknights) and the full backups (weekends) are failing on one particular database, although it doesn't give much away as there are no error messages.
Here is an extract from the SQLiDA log (commvault iData agent)
7416 1de4 04/27 01:17:47 232668 ID=Allocate Buffer SPEED, Bytes Read = 8592633000, Total time = 39.718761, Average = 206.314928, Samples = 136391
7416 1de4 04/27 01:31:38 232668 CCvSQLVDI::GetBkpData() - Vdi Timeout retry count = 1
7416 1f58 04/27 01:32:26 232668 SqlDataTransferHandler::startTransfer() - Closed all the data transfer handles
7416 1f58 04/27 01:32:26 232668 SqlDataTransferHandler::startTransfer() - Total Bytes Transferred = 8649572352
7416 1f58 04/27 01:32:26 232668 CCvSQLVDI::Finalize() - Calling Finalize
7416 1f58 04/27 01:32:26 232668 CCvSQLVDI::Finalize() - Before WaitForSingleObject Calling Finalize
It will happily sit on the last line of this log as far as I can make out indefinitely... However I have been restarting the SQL service which then produces the next few lines in the log:
7416 a60 04/27 09:05:23 232668 CCvSQLVDIBase::RunQuery() - Error: A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - The specified network name is no longer available.)
7416 1f58 04/27 09:05:23 232668 CCvSQLVDI::Finalize() - After WaitForSingleObject Calling Finalize [0]
7416 1f58 04/27 09:05:23 232668 CSQLBackup::doVDIBackup() - Error encountered during backup. Error: A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - The specified network name is no longer available.)
7416 1f58 04/27 09:05:24 232668 CSQLBackup::logDatabaseBackupStatus() - Successfully logged the database [MerHybis1_SITE_Pair] into backup.out
7416 1f58 04/27 09:05:24 232668 CSQLBackup::doDbSubclientBackup() - Failed to backup the database [MerHybis1_SITE_Pair]
7416 1f58 04/27 09:05:24 232668 CSQLBackup::doDbSubclientBackup() - starting backup of db = [WSS_Content_26c18a33-3e74-4642-b98e-57514e10aca2]
As a temporary measure I have moved the apparently offending database into a different backup schedule and will monitor its behaviour there where it hopefully won't upset the main jobs.
I'm hoping a full backup of that particular database may solve things, but am doubtful.
Does this look like a SQL problem or a Commvault problem? My next step would of course be to take a SSMS backup of the database to see what happens.
Cheers,
Conrad
April 28, 2011 at 4:57 am
Perhaps the backup was a red herring!
It turns out a DBCC CHECKDB was occurring on the database that was struggling to perform.
I've nuked the maintenance plan and replaced it with a set of more suitable plans and will monitor their timings.
Strange this should only come to light after a server upgrade, perhaps the timing has been affected (i.e. its checking DBs quicker so that one is being checked earlier now).
Cheers,
Conrad
April 28, 2011 at 3:01 pm
Conrad,
Make sure your VM environment is set up properly for SQL Server. We had some huge performance issues that sound very similar to yours when we virtualized a particular server. Turns out the IT guy in charge of setting up the VM environment didn't follow SQL Server VM best practices (isolated vs. shared resources) and we had a huge memory/performance issue as a result.
Viewing 4 posts - 16 through 19 (of 19 total)
You must be logged in to reply to this topic. Login to reply