Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase ««123»»

Virtualization for Database Development Expand / Collapse
Author
Message
Posted Tuesday, April 08, 2008 11:26 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Tuesday, February 18, 2014 9:15 AM
Points: 44, Visits: 148
We have worked with virtual servers extensively, both in the context of cloning entire environments, and in the context of insufficient machines (the live environment is 9 servers, and we have 5 machines in the QA lab, and we're not willing to re-use dev boxes as QA servers, THAT is a nightmare for deployment)

In my experience, project managing deployments onto virtual servers is a very good way of pre-testing deployments onto live machines, mostly because you can save a base state in a way that you cannot do when uninstalling windows - in other words, you can have a clean windows, or windows + SQL, or whatever, install by just "shut down, copy, paste, startup"

We were working MS Virtual Server and Virtual PC, not VMWare, only challenge we experienced is that the servers could not attach to the USB ports, which I understand VMWare can.

Most of our development (lots and lots of sharepoint) happens on VPCs - or at least, sharepoint resides on a VPC, and Visual Studio on a local box, one place we've found performance problems.

But have a look at this post from MSDN blogs about performance:

http://www.hanselman.com/blog/VirtualPCTipsAndHardwareAssistedVirtualization.aspx
https://www.microsoft.com/windowsserversystem/virtualserver/default.aspx
http://blogs.msdn.com/virtual_pc_guy/archive/2006/03/14/550712.aspx

VPC instances can be optimized a number of ways
1. Have a processor that supports hardware virtualization. It does matter! See the diagram below:

2. As you could see, there's an option in the VPC settings that allows hardware-assisted virtualization if your processor supports it. Unfortunately, in my case even my notebook PC does not support that. My processor configuration is as follows:

According to Scott's blog, and I quote him:
If you're running Intel, refer to this table at Intel to see if your chip supports VT. All the Core Duos support VT except the ones that end in "E" like the Intel Core Duo processor T2300E .
3. In no particular order, you should check the Intel Virtualization Technology (VT) web site to find out what processor supports hardware-assisted virtualization, also known as "Virtualization Inside" (cool name). Also check out AMD virtualization technology. I do not take sides, as I have processors from both AMD and Intel.
4. My home PC has an AMD Athlon 64 X2 Dual-Core processor and it supports hardware-assisted virtualization. Whenever I am preparing my demo on a VPC image, I work from home (a valid reason to my boss that I have to work from home)
5. Memory allocation: In the above VPC instance, I had 1.6GB of memory allocated to my VPC instance. In my home PC, I allocate 2GB, or more if I do not run many applications in the background. Size does matter.
6. Do not run your VPC on the same hard disk as your host OS. Run your VPC on an external HDD that has a high RPM speed. Or if you PC has two separate physical HDDs, run and store your VPC on the HDD that is not where your host OS is installed. I am using an external Maxtor HDD with an external power source. Do not use those tiny 2.5" pocket HDD that draws power from another USB port. The power from another USB port is not enough to juice it up, besides most likely the RPM speed of the pocket HDD is low.
7. I compress the folder which contains my VPC image. The reason behind this is not just because I want to optimize my storage space, but even more importantly is the fact that I want to reduce the amount of disk reads from the HDD. It's better to let my CPU work hard rather than my HDD work hard. Everyone knows that the weakest link in terms of the performance of a PC is the HDD.
8. This is how I compress the folder, right-mouse click on the folder and choose Properties:

9. You should also compact your VPC image (I don't do this as often as I should though). Here's how:

10. If you are running your VPC on Windows Server 2003, for instance, use Microsoft Virtual Server 2005 R2 instead.
If you want to read more about VPC optimization, you may check out the following blogs:
Scott Hanselman
The Virtual PC Guy

Post #481755
Posted Tuesday, April 08, 2008 11:48 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 4:30 AM
Points: 14,802, Visits: 27,280
Great info. If they relaunch the effort I'll definately use this stuff as a reference. Thanks.

----------------------------------------------------
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood..." Theodore Roosevelt
The Scary DBA
Author of: SQL Server 2012 Query Performance Tuning
SQL Server 2008 Query Performance Tuning Distilled
and
SQL Server Execution Plans

Product Evangelist for Red Gate Software
Post #481769
Posted Tuesday, April 08, 2008 11:48 PM
SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Thursday, March 20, 2014 7:32 AM
Points: 5,193, Visits: 1,368
My company is also considering to implement virtual servers. This input will really help us to consider. Thanks Grant Fritchey for sharing your inputs. Also special thanks to Mark Stacey for his valuable comments.

:)





Post #482057
Posted Wednesday, April 09, 2008 1:32 AM
Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Tuesday, April 15, 2014 10:56 PM
Points: 1,393, Visits: 435
This looks like a bad experience. The trick with using VMs is to treat them - logically speaking - as real machines. Then the next question would be: why would you install sql server AND application on the same server? And next: why would you integrate software changes in a development environment? If having real servers, wouldn't you use an integration environment with application and database servers? This would be the right environment to keep security under control because not any developer will deploy in this environment, but the DBA will do (database) deployments and db versioning. As a note, being a DBA, I talk only about the db side of deployments, but the actual work has to be done with a designed person that deploys application changes as a team.
And an important thing to make a DBA's life easier: do not bring developer VMs under domain; you cannot and shouldn't control any AD change.
If you would need to re-create a SQL Server VM with same name as an old one then, again, treat this one as a real server, which means assign a static IP which, by the way, it's not necessary to be the same with the one for the old vm, rename the server ann then rename the sql server instance (if already installed) and finally, decommission the old VM.
There are many other things to be said both pro or against using VMs for sql servers. Hope these lines would help someone in setting up VM environments.




Post #482095
Posted Wednesday, April 09, 2008 5:41 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 4:30 AM
Points: 14,802, Visits: 27,280
michaela (4/9/2008)
This looks like a bad experience. The trick with using VMs is to treat them - logically speaking - as real machines. Then the next question would be: why would you install sql server AND application on the same server?


We didn't install in the same machine. The system we were trying to implement involved having a set of virtual servers that could be created as a group. That way, the application server, the web server, the database server, would all be in a common, working state, synchronized like it was a working production environment. Then copies of that environment, all the servers, would be "deployed" for use by various development teams.

And next: why would you integrate software changes in a development environment? If having real servers, wouldn't you use an integration environment with application and database servers? This would be the right environment to keep security under control because not any developer will deploy in this environment, but the DBA will do (database) deployments and db versioning. As a note, being a DBA, I talk only about the db side of deployments, but the actual work has to be done with a designed person that deploys application changes as a team.


The problem with saying that the developers shouldn't have an integrated environment is, what happens when they're developing code that relies on interaction from other sources? This was designed to support a Service Oriented Architecture (SOA) approach to development where each application provides a service to the others. Some apps absolutely required the other apps to be in place to work. So you're faced with one of two choices, each of which we've tried to work with before we tried this alternative. Option 1) all the developers work on the same environment, but then code changes in Service 'A' breaks Application 'B'. Option 2) Have Service 'A' deploy their code to Application 'B' servers when it's "ready" but then you see your developers constantly troubleshooting new deployments and integration inconsistencies instead of developing new code.

And an important thing to make a DBA's life easier: do not bring developer VMs under domain; you cannot and shouldn't control any AD change.
If you would need to re-create a SQL Server VM with same name as an old one then, again, treat this one as a real server, which means assign a static IP which, by the way, it's not necessary to be the same with the one for the old vm, rename the server ann then rename the sql server instance (if already installed) and finally, decommission the old VM.
There are many other things to be said both pro or against using VMs for sql servers. Hope these lines would help someone in setting up VM environments.



Ah, but then you get all these servers communicating out into your general network instead of safe behind a "fence." Suffice to say, it didn't work out, but it was a very good idea.

Also, the DBA's weren't responsible for administering all these servers. We were only part of the whole process & team. We didn't administer active directory or the application servers. We just worked with the databases & database servers as usual.


----------------------------------------------------
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood..." Theodore Roosevelt
The Scary DBA
Author of: SQL Server 2012 Query Performance Tuning
SQL Server 2008 Query Performance Tuning Distilled
and
SQL Server Execution Plans

Product Evangelist for Red Gate Software
Post #482190
Posted Wednesday, April 09, 2008 7:42 AM


SSCoach

SSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoachSSCoach

Group: General Forum Members
Last Login: Monday, April 14, 2014 1:34 PM
Points: 15,442, Visits: 9,588
Good article. I've been looking at virtualization for dev and QA, and the article and discussion are definitely going to affect how I set this up.

- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread

"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
Post #482319
Posted Friday, April 11, 2008 7:33 AM


SSC Veteran

SSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC VeteranSSC Veteran

Group: General Forum Members
Last Login: Wednesday, March 11, 2009 8:20 AM
Points: 269, Visits: 1,093
Grant - good article. The timing is ironic - last night at my local developer group, we were given presentations on virtual machines. They mostly talked about virtual PCs, so your information about the virtual servers built on what we talked about.

Also, the comments provided by Mark Stacey matched very closely to what our presenters said about setting up the virtual machines. This has gotten me thinking of how I could possibly use this technology at work and at home (and what to look out for).

Thanks,


Ian.

"If you are going through hell, keep going."
-- Winston Churchill
Post #483658
Posted Friday, April 11, 2008 7:56 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 4:30 AM
Points: 14,802, Visits: 27,280
Thanks. I'm glad it was helpful. I originally wrote the thing when it looked like we had a success. I then went back and added new information to portray the failure.

----------------------------------------------------
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood..." Theodore Roosevelt
The Scary DBA
Author of: SQL Server 2012 Query Performance Tuning
SQL Server 2008 Query Performance Tuning Distilled
and
SQL Server Execution Plans

Product Evangelist for Red Gate Software
Post #483683
Posted Sunday, April 13, 2008 1:14 AM


Keeper of the Duck

Keeper of the Duck

Group: Moderators
Last Login: Thursday, April 03, 2014 10:06 PM
Points: 6,621, Visits: 1,851
I had a long conversation with our VMware SE (he's moved on to bigger and better things within VMware now) who I had a previous working relationship with long before he got to VMware. As a result, the discussion was quite candid. Lab Manager, speaking from a directory services administrator side (my primary job) is a nightmare to keep synchronized. The idea is great in concept: you have a nice little sandbox to play in that can't affect the outside world. It does this by effectively creating a snapshot in time. Meaning all changes in production have to be performed in the LabManager environment as well. That's duplicate work. And while I'm sure scripting could have been worked out using VMware's APIs and packages, um, no.

In my current organization we treat VMs as physical servers, as another poster indicated. They are in with the "regular" servers. We do segment production and development into two forests, and there is some duplication of effort, but once a change is applied on the development side, it remains. Since Lab Manager is designed to be replicated and refreshed from a point in time over and over again, you have to look at applying changes every time you stand up the environment. So while there is duplication of effort with a separate forest, it is far less than going the Lab Manager route. In our case we use the development forest to test security models and then roll them forward to production, so we put up with the duplication of effort in order to facilitate the improvement in modeling we gain.


K. Brian Kelley, CISA, MCSE, Security+, MVP - SQL Server
Regular Columnist (Security), SQLServerCentral.com
Author of Introduction to SQL Server: Basic Skills for Any SQL Server User
| Professional Development blog | Technical Blog | LinkedIn | Twitter
Post #484141
Posted Monday, April 14, 2008 5:21 AM


SSChampion

SSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampionSSChampion

Group: General Forum Members
Last Login: Today @ 4:30 AM
Points: 14,802, Visits: 27,280
Excellent feedback. Thank you for that. From everything we were told by the vendors, we should have been dancing through the streets like we were in "On the Town." Instead it all had a serious "Reservoir Dogs" feeling to it.

We probably will be expanding more & more into virtual servers for things other than SQL Server. Instances act so much like a virtual machine and are so much easier to maintain, that's probably how we'll manage things there. For now at least. Who knows what technology changes may come down the pike.


----------------------------------------------------
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood..." Theodore Roosevelt
The Scary DBA
Author of: SQL Server 2012 Query Performance Tuning
SQL Server 2008 Query Performance Tuning Distilled
and
SQL Server Execution Plans

Product Evangelist for Red Gate Software
Post #484309
« Prev Topic | Next Topic »

Add to briefcase ««123»»

Permissions Expand / Collapse