April 28, 2016 at 10:19 pm
Comments posted to this topic are about the item Stress Testing
April 29, 2016 at 2:50 am
I have found it more common to performance test solutions rather than stress test separate components such as a SQL Server database installation.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
April 29, 2016 at 4:21 am
I recently did stress testing our new hardware, before we migrated our databases to it. I used Ditributed Replay to do the stress test (i.e. replay the workload from our old environment) and SQL Monitor to collect and compare various metrics. The conclusion was our new hardware outperforms the old hardware. It's good to know for sure your new hardware performs as expected.
I hope to find some time soon to write about it on my blog devopsdba.com
April 29, 2016 at 4:58 am
In general no, but I work in a sector where complexity is more of an issue than load right now. It's very rare for load to cause any problems - I'm hoping that doesn't mean we are over engineered!
April 29, 2016 at 5:11 am
Stress testing I've been involved with previously has always been flawed in that the target software/hardware performance is slowly and finally attained to the point the hardware is almost red-lining. Management cheer saying the hardware requirements were spot on and the stress test is then stopped. That's great, but the system will be then be red-lining for many hours a day, 365 days a year... management then wonder why capacity management are reporting urgent issues a few days after launch.
April 29, 2016 at 8:15 am
The other complicating factor is that usually the move to a new hardware platform is driven by new or upgraded software. That throws several unknowns into the mix. Since we are in a virtual environment for servers we can fairly easily tune CPU and RAM, but in a database environment, it is usually disk that is the pinch point. I have to admit that we have not used very scientific means to test servers, instead relying on a mix of vendor recommendations (which are spotty at best) and peer advice.
April 29, 2016 at 8:24 am
When I worked in the video game industry, we were building massively multiplayer online games. As you could imagine, in this scenario, you are playing an online game with a graphical rendering engine on the front end, but all the assets/data are stored in a database on the backend. Thus, stress testing how many players (users) are interacting with the product was critical. Everything you do in-game touches the database as well other underlying services. If you move two feet, pick up a item from the ground, send a message to a friend or whatever. It all taps the database. While you built it with a number in mind, you don't truly know until you schedule a official stress test, get about 1,000 users online and tell them to go crazy.
April 29, 2016 at 8:44 am
Since I work almost exclusively with third party software, I generally don't need to do any kind of stress testing since all hardware requirements come from the software publishers. When we recently upgraded to a new physical SQL Server, we matched the third party vendor's requirements for CPU. We bought significantly more RAM than suggested since RAM is cheap. We also upgraded to new SSDs instead of the spinning disks we were running previously. Since this was a scheduled hardware refresh (old server was approaching 5 years old), there wasn't any reason to prove the upgrades were needed. Out of curiosity I did a test using diskspd to see the improvement between spinning disks and SSD[/url]s. I shared it with the IT Manager, but it didn't go any further than that.
April 29, 2016 at 9:50 am
stakes (4/29/2016)
I recently did stress testing our new hardware, before we migrated our databases to it. I used Ditributed Replay to do the stress test (i.e. replay the workload from our old environment) and SQL Monitor to collect and compare various metrics. The conclusion was our new hardware outperforms the old hardware. It's good to know for sure your new hardware performs as expected.I hope to find some time soon to write about it on my blog devopsdba.com
Love to see an article here. You can submit it here and after 60 days, put it on your blog. :w00t:
April 29, 2016 at 10:19 am
xsevensinzx (4/29/2016)
When I worked in the video game industry, we were building massively multiplayer online games. As you could imagine, in this scenario, you are playing an online game with a graphical rendering engine on the front end, but all the assets/data are stored in a database on the backend. Thus, stress testing how many players (users) are interacting with the product was critical. Everything you do in-game touches the database as well other underlying services. If you move two feet, pick up a item from the ground, send a message to a friend or whatever. It all taps the database. While you built it with a number in mind, you don't truly know until you schedule a official stress test, get about 1,000 users online and tell them to go crazy.
This is what I was referring to as performance testing and I believe that it is different from hardware stress testing. Of, course I have been known to be wrong. From time to time.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
April 29, 2016 at 1:47 pm
I've used Toad for DB2 (z/OS) utility to generate test data for the mainframe so I would assume they have one for SQL Server but I don't know for sure. That's just part of the process though. If you are lucky enough to work in an enlightened IT shop where they take load testing seriously (I've worked in some that are not) then these kinds of tools are fantastic for you and the business. I would be interested in a configurable tool that helped me to baseline and generate test datasets for SQL Server.
I really, really liked Quest's Project Lucy for server to server comparisons when it came to SQL Server workloads. I think it's called 'Spotlight Essentials' now but you can upload your data to their site and it will analyze your performance metrics (if you were using Spotlight anyway) and you can see how the instance stacks up against thousands of other participating instances. It was great for shutting up the SAN administrator too and simple enough (red thingys and green thingys) that even management could understand. 🙂
There's whole methodology here. Someone should write a book about; I would buy it. I only know a little bit, but would love to learn more how others do it, particularly on large scales.
April 29, 2016 at 1:47 pm
Gary Varga (4/29/2016)
xsevensinzx (4/29/2016)
When I worked in the video game industry, we were building massively multiplayer online games. As you could imagine, in this scenario, you are playing an online game with a graphical rendering engine on the front end, but all the assets/data are stored in a database on the backend. Thus, stress testing how many players (users) are interacting with the product was critical. Everything you do in-game touches the database as well other underlying services. If you move two feet, pick up a item from the ground, send a message to a friend or whatever. It all taps the database. While you built it with a number in mind, you don't truly know until you schedule a official stress test, get about 1,000 users online and tell them to go crazy.This is what I was referring to as performance testing and I believe that it is different from hardware stress testing. Of, course I have been known to be wrong. From time to time.
Not sure I see the difference outside of directly running tests on the hardware itself. In the scenario above, all the hardware behind the scenes is supporting those 1000 or so users. That's the stress they will have in a real-life scenario if we grow. Anything you do on the hardware itself is a simulation that does not represent the factual users.
I'm not saying it's not beneficial, especially if you can't get the amount of users to stress test for you, but it's still a stress test on the hardware in both cases.
April 30, 2016 at 9:03 am
xsevensinzx (4/29/2016)
Gary Varga (4/29/2016)
xsevensinzx (4/29/2016)
When I worked in the video game industry, we were building massively multiplayer online games. As you could imagine, in this scenario, you are playing an online game with a graphical rendering engine on the front end, but all the assets/data are stored in a database on the backend. Thus, stress testing how many players (users) are interacting with the product was critical. Everything you do in-game touches the database as well other underlying services. If you move two feet, pick up a item from the ground, send a message to a friend or whatever. It all taps the database. While you built it with a number in mind, you don't truly know until you schedule a official stress test, get about 1,000 users online and tell them to go crazy.This is what I was referring to as performance testing and I believe that it is different from hardware stress testing. Of, course I have been known to be wrong. From time to time.
Not sure I see the difference outside of directly running tests on the hardware itself. In the scenario above, all the hardware behind the scenes is supporting those 1000 or so users. That's the stress they will have in a real-life scenario if we grow. Anything you do on the hardware itself is a simulation that does not represent the factual users.
I'm not saying it's not beneficial, especially if you can't get the amount of users to stress test for you, but it's still a stress test on the hardware in both cases.
My understanding is that stress testing hardware is a generalised test of performance whereas performance testing is of an overall application/system i.e. a particular profile of use.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
May 2, 2016 at 5:19 am
AWS are doing interesting stuff with lambda functions to allow stress testing of your equipment in the AWS cloud.
There are tools like Gatling explicitly to machine gun your infrastructure. One AWS engineer suggested http://log.roadtrippers.com/lambda-bees-with-frickin-laser-beams.html
I suggested that Red-Gate Data Generator might form part of a stress test suite.
Providing realistic stress testing of a system is difficult enough. At least the system has a designed interface. Stress testing components in a realistic way I feel is harder. I'd say you have to design your components to be stress testable.
I also think testing to the breaking point is vital. The system is only as strong as it's weakest point. You want to know how a system behaves as it approaches it's limit so it doesn't come as a nasty shock. Some systems decline progressively, others go down like a sack of stuff
May 2, 2016 at 7:01 am
We just overbuy.
Viewing 15 posts - 1 through 15 (of 26 total)
You must be logged in to reply to this topic. Login to reply