VSS / SQLWriter sequence of warnings and error (... subset of the volumes ...)

  • We have (only) one SQL2019 clustered instance reporting this sequence of error and warnings:

    Microsoft SQL Server 2019 (RTM-CU16) (KB5011644) - 15.0.4223.1 (X64) Apr 11 2022 16:24:07

    Copyright (C) 2019 Microsoft Corporation

    Standard Edition (64-bit) on Windows Server 2019 Standard 10.0 <X64> (Build 17763: ) (Hypervisor)

    01 Eventviewer WARNING

    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

    - <System>

    <Provider Name="VSS" />

    <EventID Qualifiers="0">8229</EventID>

    <Level>3</Level>

    <Task>0</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2023-02-01T03:32:03.666289100Z" />

    <EventRecordID>59328</EventRecordID>

    <Channel>Application</Channel>

    <Computer>-ClusterNode-.-mydomain-.com</Computer>

    <Security />

    </System>

    - <EventData>

    <Data>0x00000000, The operation completed successfully.</Data>

    <Data>Operation: PrepareForSnapshot Event Context: Execution Context: Writer Writer Class Id: {1072ae1c-e5a7-4ea1-9e4a-6f7964656570} Writer Name: Cluster Shared Volume VSS Writer Writer Instance ID: {9b4f1d55-730a-47a5-b2be-1df7a7a7bd70} Command Line: C:\Windows\Cluster\clussvc.exe -s Process ID: 4360</Data>

    <Binary>2...</Binary>

    </EventData>

    </Event>

    02 Eventviewer ERROR

    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

    - <System>

    <Provider Name="SQLWRITER" />

    <EventID Qualifiers="0">24579</EventID>

    <Level>2</Level>

    <Task>0</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2023-02-01T03:32:03.760047600Z" />

    <EventRecordID>59329</EventRecordID>

    <Channel>Application</Channel>

    <Computer>-ClusterNode-.-mydomain-.com</Computer>

    <Security />

    </System>

    - <EventData>

    <Data>-MyDb-</Data>

    <Data>-SQLInstance-</Data>

    <Binary>53514...</Binary>

    </EventData>

    </Event>

    03 Eventviewer WARNING

    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

    - <System>

    <Provider Name="VSS" />

    <EventID Qualifiers="0">8229</EventID>

    <Level>3</Level>

    <Task>0</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2023-02-01T03:32:03.760047600Z" />

    <EventRecordID>59330</EventRecordID>

    <Channel>Application</Channel>

    <Computer>-ClusterNode-.-mydomain-.com</Computer>

    <Security />

    </System>

    - <EventData>

    <Data>0x800423f0, The shadow-copy set only contains only a subset of the volumes needed to correctly backup the selected components of the writer.</Data>

    <Data>Operation: PrepareForSnapshot Event Context: Execution Context: Writer Writer Class Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a} Writer Name: SqlServerWriter Writer Instance Name: Microsoft SQL Server 2019:SQLWriter Writer Instance ID: {66545d18-b7e5-4c94-938a-8adc68da46ab} Command Line: "C:\Program Files\Microsoft SQL Server\90\Shared\sqlwriter.exe" Process ID: 3592</Data>

    <Binary>2...</Binary>

    </EventData>

    </Event>

    04 Eventviewer WARNING

    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

    - <System>

    <Provider Name="VSS" />

    <EventID Qualifiers="0">8229</EventID>

    <Level>3</Level>

    <Task>0</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2023-02-01T03:32:16.372471500Z" />

    <EventRecordID>59331</EventRecordID>

    <Channel>Application</Channel>

    <Computer>-ClusterNode-.-mydomain-.com</Computer>

    <Security />

    </System>

    - <EventData>

    <Data>0x800423f2, The writer's timeout expired between the Freeze and Thaw events.</Data>

    <Data>Operation: BackupShutdown Event Context: Execution Context: Writer Writer Class Id: {1072ae1c-e5a7-4ea1-9e4a-6f7964656570} Writer Name: Cluster Shared Volume VSS Writer Writer Instance ID: {9b4f1d55-730a-47a5-b2be-1df7a7a7bd70} Command Line: C:\Windows\Cluster\clussvc.exe -s Process ID: 4360</Data>

    <Binary>...</Binary>

    </EventData>

    </Event>

    We always notice this sequence in the cluster node eventviewer application logs.

    This SQLClustered instance uses 3 LUNs of which 2 are mountpoints of the drive allocated of the first lun.

    Can anyone explain:

    - what this error is all about?

    - does this jeopardise my database, my backups or my DRP ?

    It only throws this error for this particular database and not for the other database on that SQLInstance !!

    Johan

    Learn to play, play to learn !

    Dont drive faster than your guardian angel can fly ...
    but keeping both feet on the ground wont get you anywhere :w00t:

    - How to post Performance Problems
    - How to post data/code to get the best help[/url]

    - How to prevent a sore throat after hours of presenting ppt

    press F1 for solution, press shift+F1 for urgent solution 😀

    Need a bit of Powershell? How about this

    Who am I ? Sometimes this is me but most of the time this is me

  • For the SQL Server Writer piece, change the default logging option to VERBOSE

    C:\Program Files\Microsoft SQL Server\90\Shared\SqlWriterConfig.ini

    Change "TraceLevel=DEFAULT" to "TraceLevel=VERBOSE"

    Save the file, then restart the SQLServerWriter service.

    This will give you more logging in the file

    C:\Program Files\Microsoft SQL Server\90\Shared\SqlWritterLogger.txt file so you can see what is happening.

     

    In an elevated CMD prompt run

    VSSADMIN LIST WRITERS

    Verify the status of the writers, anything in a state other than stable you will need to either reboot the machine or restart that particular service.

    Once everything is stable perform a VSS backup manually

     

    From an elevated CMD prompt run

    DISKSHADOW /l <Path to somewhere to store the outputs>

    Once you are in the diskshadow exe then run

    set context persistent

    set verbose on

    add volume C:

    add volume D:

    repeat for all your disks

    writer verify SqlServerWriter

    writer verify <any other writers you want to add>

    begin backup

    create

    wait for the VSS to do its thing then

    end backup

     

    You should then be in a good position with the SqlWritterLogger.txt and the diskshadow output to see what is failing and where.

  • As this is the only clustered instance which produces these warnings and errors, we suspect SentinelOne - which we are testing on one of the nodes.

    We moved the instance to the other node and will wait for 24h to see if we get these warnings and error overthere too.

    Johan

    Learn to play, play to learn !

    Dont drive faster than your guardian angel can fly ...
    but keeping both feet on the ground wont get you anywhere :w00t:

    - How to post Performance Problems
    - How to post data/code to get the best help[/url]

    - How to prevent a sore throat after hours of presenting ppt

    press F1 for solution, press shift+F1 for urgent solution 😀

    Need a bit of Powershell? How about this

    Who am I ? Sometimes this is me but most of the time this is me

  • If it is SentinalOne I would recommend opening a case with MSFT, see if they can reproduce it as I know they update documentation based on 3rd party components which can interfere with normal operations, specifically in the AV and Cyber Security pieces, like McAfee, Sophos, CrowdStrike etc.

    https://learn.microsoft.com/en-us/troubleshoot/sql/database-engine/performance/performance-consistency-issues-filter-drivers-modules

    Maybe they can find a way to mitigate the issues in a CU or advise the needed workarounds.

Viewing 4 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic. Login to reply