chicken before the egg (block before disk queue0

  • have 2008 r2 server have had some disk issues before. Today went off the charts. have Idera DM which usually gives me the info to help, but now may have chicken before the egg. usually will see some disk issues, and then the blocking follows. But this is on all drives, (tempdb, index, data, log) So I am leaning towards disk may be the culprit. Anyone want to change my mind?

  • tcronin 95651 (6/1/2015)


    have 2008 r2 server have had some disk issues before. Today went off the charts. have Idera DM which usually gives me the info to help, but now may have chicken before the egg. usually will see some disk issues, and then the blocking follows. But this is on all drives, (tempdb, index, data, log) So I am leaning towards disk may be the culprit. Anyone want to change my mind?

    Quick question, what are the drive configurations (physical drives, partitions etc.)?

    😎

  • data, index, log and tempdb all on separate disks on new EMC SAN

  • tcronin 95651 (6/1/2015)


    data, index, log and tempdb all on separate disks on new EMC SAN

    More questions: Is the server a VM? Anything in the SAN logs?

    😎

    I've seen similar issues on VM's which were between the host and the guest, as soon as the IO latency spiked, blocking increased even if no blocking had been observed on the systems before.

  • No physical, not really anything from the SAN but after 15 years of dealing with them realizing like behind the curtain in the wizard of oz world

  • tcronin 95651 (6/1/2015)


    No physical, not really anything from the SAN but after 15 years of dealing with them realizing like behind the curtain in the wizard of oz world

    Wasn't there something like Performance Manager in the ControlCenter, nothing there either?

    My thought is that when all drives are on the same IO subsystem and all change their behaviour at the same time the culprit could be anything in the IO path, a switch, interface/nic or the storage system, any of these components can fail and one shouldn't rule anything out, even if it's still shrink wrapped.

    What about IO Wait Statistic?

    😎

  • yeah idera does show wait stats spiking, of course also doing a exchange migration, just trying to see if relationship between blocking and drive issue

  • tcronin 95651 (6/1/2015)


    yeah idera does show wait stats spiking, of course also doing a exchange migration, just trying to see if relationship between blocking and drive issue

    Are you anywhere close to saturating the capacity of the SAN?

    😎

  • that is kind of the issue, dependent on SAN people so don't want to make mad, however, would like to see after Exchange migration is done

  • tcronin 95651 (6/1/2015)


    that is kind of the issue, dependent on SAN people so don't want to make mad, however, would like to see after Exchange migration is done

    The migration might be saturating more the writes than the reads, unless you'r on RAID5 or worse I wouldn't expect that to be a problem though. Mind you (and sorry if I'm stating the obvious), for the reads the RAID type doesn't really matter, but for the writes the disk IOPS are 2 X the IOPS for RAID10, 4 X for RAID5 and 6 X for RAID6, hence a small migration can literally mean an invasion.

    😎

  • Yes that is helpful but have to place nice in the sandbox, they will be done this weekend

Viewing 11 posts - 1 through 10 (of 10 total)

You must be logged in to reply to this topic. Login to reply