October 26, 2017 at 9:06 pm
Comments posted to this topic are about the item Physical or Virtual Storage
October 27, 2017 at 12:55 am
Hi Steve,
when setting up new servers I do care about that. Recently we set up a new production server for our ERP system's database and I read carefully through the recommendations of our SAN manufacturer. We're using Dell EqualLogic and fortunately they provide a document which says in the subtitle: 'Best practices and configuration guidelines for deploying SQL Server with EqualLogic storage arrays'. And really that document is not bad.
Then I told the storage administrators what they should pay attention to when setting up that thing.
But after then it's up to the storage administrators to ensure it is running whithout problems.
Best Regards
Stefan
October 27, 2017 at 1:32 am
I think the DB world has followed a path rather similar to cars. In the early days you needed to know how to maintain the car, service it and fix various bits and bobs. Servicing was frequent, every 3,000 miles and in some cases, oil changes every 1,500.
Slowly the world changed so that 6,000 mile services became the norm, things that used to break were now pretty reliable and it was pretty easy to find someone to do the maintenance when required.
Now cars just work. My humble Ford needs a service every 12,000 miles, all I have to do is stick fuel and windscreen washer fluid in it and put air in the tyres. The car has become a tool that works for me, not something that requires me to work on it.
Some people still like to tinker under the bonnet/hood but its for pleasure rather than necessity.
I like to know the fine grained details of what my DBs are running on but it isn't necessary for me to know, particularly as my companies DBs are increasingly cloud based. DBaaS. There is very little of the oily bits exposed, it's pretty much all handled as part of the service. The DB has become a tool that works for me.
Tuning consists of good schema design rather than exotic hardware approaches. I think that is actually a step forward as it becomes a common focus within a development team.
October 27, 2017 at 2:23 am
I have been wondering about how a SQL instance should be set up properly on a VM.
With physical drives you would put your data and log files on separate drives, but how do you know that your virtual drives aren't actually running on the same physical drive? SAN servers probably wouldn't have this issue but on a normal VM server sitting in a server room, can this lead to problems and what is the best way to manage this?
October 27, 2017 at 6:25 am
I think you need to have a good overview of the physical, since not all situations will have an enlightened SAN admin nor a SAN. We had to tell several of our clients how to configure their SANs not only for SQL Server, but for other apps. Also you may still deal with small remote locations that have a few servers in closet for storage, no SAN, but several physical drives.
You still need to know basics, otherwise eventually you will be bitten or bamboozled.
October 27, 2017 at 6:40 am
David.Poole - Friday, October 27, 2017 1:32 AMI think the DB world has followed a path rather similar to cars. In the early days you needed to know how to maintain the car, service it and fix various bits and bobs. Servicing was frequent, every 3,000 miles and in some cases, oil changes every 1,500.
Slowly the world changed so that 6,000 mile services became the norm, things that used to break were now pretty reliable and it was pretty easy to find someone to do the maintenance when required.
Now cars just work. My humble Ford needs a service every 12,000 miles, all I have to do is stick fuel and windscreen washer fluid in it and put air in the tyres. The car has become a tool that works for me, not something that requires me to work on it.
Some people still like to tinker under the bonnet/hood but its for pleasure rather than necessity.I like to know the fine grained details of what my DBs are running on but it isn't necessary for me to know, particularly as my companies DBs are increasingly cloud based. DBaaS. There is very little of the oily bits exposed, it's pretty much all handled as part of the service. The DB has become a tool that works for me.
Tuning consists of good schema design rather than exotic hardware approaches. I think that is actually a step forward as it becomes a common focus within a development team.
My Ford Escape spent more time in the shop than on the road, because it's computers repeated crashed while being driven on the highway. Apparently the electronics and wire harness are so complex, that they couldn't diagnose or fix the issue, so we ended up getting a totally different brand of automobile. And recently they have had a recall because they couldn't figure out how to make a proper door latch...
I think that DBAs still need to know the basics of hardware, sys admin, programming and networking. But then again, I still deal with non-cloud systems and real world hardware including remote hardware sitting in remote sites.
October 27, 2017 at 7:13 am
Paulo de Jesus - Friday, October 27, 2017 2:23 AMI have been wondering about how a SQL instance should be set up properly on a VM.
With physical drives you would put your data and log files on separate drives, but how do you know that your virtual drives aren't actually running on the same physical drive? SAN servers probably wouldn't have this issue but on a normal VM server sitting in a server room, can this lead to problems and what is the best way to manage this?
I remember that the VMware guys said to make your virtual machine mirror your physical machine as far as spindles were concerned when it came to SQL Server. If you wanted to have your log files on a separate set of spindles, put your virtual disk that will host them on that separate set of spindles. Is this no longer a thing?
October 27, 2017 at 7:25 am
I think this mostly depends on the size of the organization. If your IT department is 5 people like mine, you probably don't have a SAN admin and you need to make sure whoever is setting up the disks knows what he/she is doing. On the other hand in my previous job in a software company, the DBA's didn't manage any of the hardware, just the software.
October 27, 2017 at 7:38 am
Because of my experience across various areas of IT, I do understand RAID, and enough about storage that for the most part I know how all of my databases are configured. However storage was taken over by a new person a few years back, and their opinion is "you don't need to know, it doesn't matter". It is quite frustrating working with people who believe they can configure their layer without regard to best practices of the layers that depend on them. Yet when something goes wrong, fingers get pointed elsewhere.
Dave
October 27, 2017 at 9:16 am
Hi Steve,
In my case, I started to work with db servers using Informix. It was used by our erp system in a hp autoraid. It worked very fine and in those times RAID 5 was the most preffered configuration besides RAID 0+1. Also in those times, to ensure the fastest search of data, the more disk plates spining and the more disk heads reading the data, it was the best recomendation.
I remmeeber that in the Informix User Group (IUG), one of the gurus send us a document where it claims that RAID 5 was the worst configuration for DB. In this configuration, if one drive fails, the spare disk will recover the data. But if a second drive in the array fail, then the whole array is down, as well as all the data. It was a shocking moment for me due to we were at RAID 5. To short the story, we changed our configuration to raid 0, that it is the recommended configuration for DB, and distributed the data trhough the disk.
This was almost 20 years ago, and nowadays, using a SAN, I still distributed the data using different SAN disk drives in SQL and spliting index and data into different SAN disk drives. I perfectly know that a SAN disk may be composed by several physical disks and it ensures me that I have a good amount of disk heads and plates spinining and that provides me, at least on my mind, the security to get all performance for my DB.
Miguel.
October 27, 2017 at 9:17 am
djackson 22568 - Friday, October 27, 2017 7:38 AMBecause of my experience across various areas of IT, I do understand RAID, and enough about storage that for the most part I know how all of my databases are configured. However storage was taken over by a new person a few years back, and their opinion is "you don't need to know, it doesn't matter". It is quite frustrating working with people who believe they can configure their layer without regard to best practices of the layers that depend on them. Yet when something goes wrong, fingers get pointed elsewhere.
I experience the same thing with our Virtualization team. "you don't need to see or have access to the Host machines. Just let us know if you have a performance problem and we'll check it out" In the words of a great former US President, "trust but verify"!
October 27, 2017 at 9:21 am
mig28mx - Friday, October 27, 2017 9:16 AMHi Steve,
In my case, I started to work with db servers using Informix. It was used by our erp system in a hp autoraid. It worked very fine and in those times RAID 5 was the most preffered configuration besides RAID 0+1. Also in those times, to ensure the fastest search of data, the more disk plates spining and the more disk heads reading the data, it was the best recomendation.
I remmeeber that in the Informix User Group (IUG), one of the gurus send us a document where it claims that RAID 5 was the worst configuration for DB. In this configuration, if one drive fails, the spare disk will recover the data. But if a second drive in the array fail, then the whole array is down, as well as all the data. It was a shocking moment for me due to we were at RAID 5. To short the story, we changed our configuration to raid 0, that it is the recommended configuration for DB, and distributed the data trhough the disk.This was almost 20 years ago, and nowadays, using a SAN, I still distributed the data using different SAN disk drives in SQL and spliting index and data into different SAN disk drives. I perfectly know that a SAN disk may be composed by several physical disks and it ensures me that I have a good amount of disk heads and plates spinining and that provides me, at least on my mind, the security to get all performance for my DB.
Miguel.
From Wikipedia:
RAID 0 consists of striping, without mirroring or parity. The capacity of a RAID 0 volume is the sum of the capacities of the disks in the set, the same as with a spanned volume. There is no added redundancy for handling disk failures, just as with a spanned volume.
Therefore RAID 0 is NOT a good idea for databases! Microsoft has many articles on using RAID for databases. It is worthwhile to review those recommendations every one in a while.
Dave
October 27, 2017 at 10:02 am
Hi Dave,
You are right! It was my mistake. I mean, we changed our RAID 5 to RAID 1.
thanks for the correction.
Miguel.
October 27, 2017 at 11:02 am
mig28mx - Friday, October 27, 2017 9:16 AMHi Steve,
In my case, I started to work with db servers using Informix. It was used by our erp system in a hp autoraid. It worked very fine and in those times RAID 5 was the most preffered configuration besides RAID 0+1. Also in those times, to ensure the fastest search of data, the more disk plates spining and the more disk heads reading the data, it was the best recomendation.
I remmeeber that in the Informix User Group (IUG), one of the gurus send us a document where it claims that RAID 5 was the worst configuration for DB. In this configuration, if one drive fails, the spare disk will recover the data. But if a second drive in the array fail, then the whole array is down, as well as all the data. It was a shocking moment for me due to we were at RAID 5. To short the story, we changed our configuration to raid 0, that it is the recommended configuration for DB, and distributed the data trhough the disk.This was almost 20 years ago, and nowadays, using a SAN, I still distributed the data using different SAN disk drives in SQL and spliting index and data into different SAN disk drives. I perfectly know that a SAN disk may be composed by several physical disks and it ensures me that I have a good amount of disk heads and plates spinining and that provides me, at least on my mind, the security to get all performance for my DB.
Miguel.
One question, do you know if the SAN pool's you're using for different disks/LUNS are different physical sets? I've seen some SANs that pool a lot of drives, then cut up the pool, so it appears I have a m: drive and a l: drive, but it's the same physicals underneath.
October 27, 2017 at 12:08 pm
mig28mx - Friday, October 27, 2017 10:02 AMHi Dave,
You are right! It was my mistake. I mean, we changed our RAID 5 to RAID 1.
thanks for the correction.
Miguel.
I am not a RAID "expert", but I do know that MS has recommendations to use each in different circumstances, based on read and write performance. I know that a SAN does have an effect but again, I am not the best person for that. Still, I believe MS recommends RAID 1 for logs, and RAID 5 (or better) for data, due to how they perform. Logs, for example, are most often written to, and not read from as often. Data requires far more reads.
You have to decide what is best for your environment, but please be aware that it may not be best to use RAID 1 across the board. Below is a decent place to start.
https://technet.microsoft.com/en-us/library/ms190764(v=sql.105).aspx
And I found this, which I am very comfortable sharing... 🙂 Gail Shaw gives a nice summary.
https://www.sqlservercentral.com/Forums/Topic1234076-1550-1.aspx
Dave
Viewing 15 posts - 1 through 15 (of 19 total)
You must be logged in to reply to this topic. Login to reply