January 29, 2003 at 2:07 am
Silicon.Com accused DBA's of being lazy in not applying the July hot fix that would have prevented the SQL SLAMMER virus.
Would anyone care to comment?
January 29, 2003 at 4:04 am
My first comment is yes. My second is that many companies have large scale roll outs or production systems without a good recovery structure. What tends to happen is they wait fo SPs because these items are tested more in depth than hotfixes.
The only problems I have seen with that is 1) bad guys don't wait until an SP to bug the world 2) Nobody wants to be a hotfix guinnea pig so they wait and watch if a hotfix gets pulled and readded due to bug and 3) even thou SPs are tested before release many companies take there sweet time implementing them so they can test (which pretty much anyone can get on a BETA for an SP so folks do it and be ready).
Now with all that they more or less just walk around with their pants down until something strikes or they get around to oking an SP. I hate Hotfixes personally but if the threat is major I will apply, if minor such as a specific way of doing a query that I don't use I won't bother. But I prefer to be involved with the BETAs and will apply an SP I have had a chance to test the day it comes out. If no issues related I will roll to production immediately (whether or not apporval has been given if I feel it fixes a threat and causes me no issues). I was prepared although many areas of where I work were not, but our Virus protection group fortunately stopped everything at the door.
January 29, 2003 at 4:12 am
I agree with everything you said.
My issue is that the fix did not say "put this on or you will get a virus" and I am not sure that it could have said that.
Unless you are a dyed in the wool techy it is difficult to appreciate exactly what a particular vulnerability means. I had to search high and low to get the specifics on what a "buffer overrun" meant i.e. how not to write software with them in!
January 29, 2003 at 4:29 am
I will say thou, from the C++ standpoint that no matter how much you think you have things covered there is generally a bug somewhere you missed. Unfortunately large scale apps like SQL server have teams working on sections that require them to communicate and try to piece together as they go along so with all those folks intercommunication within the app can have gaps noone notices until after production. That is the premise of a bug fix.
Heck, my first C++ project which I have been on for over a year was to learn?????, develope and devliever a Server Service routing data from several Call Center switches to a event database in SQL and push to clients connected to the server. The client also is written in C++ and I never would have dreamed the possibilities of what I can do in VC++ with GUI and how much it takes to handle List views without the data flashing. I know there is lots of room to improve my system and probably a whole or two I am unaware of.
January 29, 2003 at 4:39 am
I think the point about deploying hot fixes is a good one. I also think that MS needs to make the fixes a little easier to apply, some of them you're copying/backing up files to 6 different places, easy to miss one. Contrast with the ease of applying Win hotfixes via Windows Update - plus it shows you what you have not applied.
Beyond that, how many DOS attacks will we endure before someone writes some code at the firewall level that says 'the average hits on port 1434 is 10/min, it just jumped to 1000/sec, how about I just disable that and make someone say its ok!'.
Andy
January 29, 2003 at 5:51 am
I would say not lazy and I have several reasons for this:
1) One of the problems we face every day is the fact that 3rd party vendors do not want to test on the latest service packs and hotfixes and verify their stuff still works. As a result, they say, "Not supported." Now, if you have a problem with the system, and you've gone to an "unsupported" service pack or hot fix, you are unsupported. This came out into the open in the NTBugTraq forums.
2) Microsoft fell down in their hot fix Q317448. I won't go into more on this one because it's already been covered:
http://www.ntbugtraq.com/default.asp?pid=36&sid=1&A2=ind0301&L=ntbugtraq&F=P&S=&P=5163
This issue bit at least one DBA, Bryant Byrd, who posted a reply to Brian Knight's article and naturally I emailed him for the specifics of his gory experience. The post to NTBugTraq seems to confirm what Mr. Byrd went through.
3) We lived in a false sense of security. Let's face it... it's been 2 years since Code Red. As bad as Spida AKA Spider AKA Digispid was, it only went after systems that had a blank sa password and were in mixed mode. I think one estimate was around 10-15K infected. That's a fairly small handful. Also, it required a connection to SQL Server. So it's effect was a whole lot less dramatic.
With that said, I would say we're incredibly naive.
1) Solid perimeter security would have stopped quite a few of the attacks. Why are there unpatched systems exposed to the Internet?!? Also, why did folks have multihomed systems running SQL Server with one NIC on the Internet and the other NIC in their intranet without any layer of security?!? Because they had been safe so far. Anyone who has hardened a web server knows to tighten that puppy down until it screams and weigh very seriously tradeoffs between security and functionality. SQL Server hadn't really gone through such a trial by fire. Now it has.
I need to say, however, other attack vectors came from hibernated laptops being brought in to work as well as VPN and other such types of connections. So a solid perimeter would have kept you safe unless you had one of these other situations where your folks weren't sitting behind a firewall. But a lot of larger companies do... and guess who got blasted? Yup. You got it.
2) I don't think anyone envisioned a DOS attack like this one caused. TruSecure's estimate was 120,000 sites hit, with packet loss going from less than 1 % to 18 %. A UDP packet flood is so very, very nasty as it doesn't require the TCP handshake, it's a fire and forget, and the source IP can be forged (since UDP means there's no return packets). Nasty, nasty, nasty.
K. Brian Kelley
http://www.truthsolutions.com/
Author: Start to Finish Guide to SQL Server Performance Monitoring
http://www.netimpress.com/shop/product.asp?ProductID=NI-SQL1
K. Brian Kelley
@kbriankelley
January 29, 2003 at 6:07 am
Lazy, not necessarily. I work on a contract with the Federal Government. Before we can apply a Service Pack (to anything, not just SQL) it must be approved by a board. Win2K SP2 just got approved recently. And I'm sure it's not just the Federal Government that is so overly cautious.
There are benefits and downfalls to doing major, in-depth approval testing of patches, etc. If you don't patch right away, you get chewed out or worse when you get hit by something that was fixed by a patch that was released months ago. But sometimes you patch right away and there's problems with the patch and you get chewed out or worse because of that. Somewhere there's a middle ground to being either overly cautious and testing everything to death or being trusting and patching, etc. the moment something new comes out.
-SQLBill
January 29, 2003 at 7:23 am
Four very good point were said here.
First: Apply the fix as soon as you can
Second: MS has to be aware of the deployment tasks (Yes, I did miss one file in one server and it was a mess - fortunately it was the test machine)
Third: Third party vendors some times (many times!) do not keep up the pace with known bugs and their bugs, making it difficult to run some fixes at the time we want to.
Fourth: Interesting enough, we some times get a little lazy and say "let's wait until the sp comes along to apply and play safe.
What amazed me was the fact that so many industries were surprised by the worm! I heard "Microsoft knoew it, but did not release any fix for this worm!"
Another point I want to pin point is connectivity and firewall security. We have a firewall policy to block 1433. No buts, no choice, no nothig! This kept us safe (this time of course, you never know)
Sorry if I repeat many comments already stated here, but I wanted to contribute just a little in the topic.
Good luck!
January 29, 2003 at 7:50 am
Best policy on a firewall is the default deny all and then open up ports as needed. Blocking TCP/1433 wouldn't have worked in this case. Blocking UDP/1434 was required. Still, with that said, companies suffered infections from internal systems and a perimeter system isn't going to help that.
Also, keep in mind that Microsoft was done in by SQL Slammer:
http://news.com.com/2100-1001-982305.html?tag=cd_mh
K. Brian Kelley
http://www.truthsolutions.com/
Author: Start to Finish Guide to SQL Server Performance Monitoring
http://www.netimpress.com/shop/product.asp?ProductID=NI-SQL1
K. Brian Kelley
@kbriankelley
January 29, 2003 at 7:56 am
Microsoft has a service called "Software Update Services (SUS)" at http://www.microsoft.com/windows2000/windowsupdate/sus/default.asp that can greatly ease the burden of rolling hotfixes and pathhes to a large lan or wan. This can help you keep your entire network up to date, and without having to touch each individual computer for every hotfix / sp.
Tim C.
//Will write code for food
One Windows to rule them all, One Windows to find them,
One Windows to bring them all and in the darkness bind them
In the Land of Microsoft where the Shadows lie.
Tim C //Will code for food
January 29, 2003 at 8:03 am
Thank you for the input about the MS to run hotfixes.
About port 1433 and 1434. Sorry it was a typo. I did realise what I typed after I read your commnet Brian. Thank you
January 29, 2003 at 8:11 am
SUS wouldn't have helped in the case of SQL Slammer. From the FAQ:
quote:
SUS supports updates for Windows 2000 Professional with Service Pack 2, Windows 2000 Server, and Windows XP Professional. It does not include provisions for updates to any other Microsoft products such as Microsoft Office, SQL Server™, or Exchange Server.
K. Brian Kelley
http://www.truthsolutions.com/
Author: Start to Finish Guide to SQL Server Performance Monitoring
http://www.netimpress.com/shop/product.asp?ProductID=NI-SQL1
K. Brian Kelley
@kbriankelley
January 29, 2003 at 8:21 am
Missed that one Brian, good point, then maybe sticking with SMS or another third party solution is still the answer.
SUS vs SMS (with SUS update)
http://www.microsoft.com/windows2000/windowsupdate/sus/suschoosing.asp
Enterprise Software Update Management using Systems Management Server 2.0 Software Update Services Feature Pack
Of course this solution is not free, but it does manage ALL of the updates.
Tim C.
//Will write code for food
One Windows to rule them all, One Windows to find them,
One Windows to bring them all and in the darkness bind them
In the Land of Microsoft where the Shadows lie.
Tim C //Will code for food
January 29, 2003 at 8:39 am
We're implementing UpdateExpert from St. Bernard in our environment. They do most of the work for you as far as building the packages, knowing the order, etc.!
K. Brian Kelley
http://www.truthsolutions.com/
Author: Start to Finish Guide to SQL Server Performance Monitoring
http://www.netimpress.com/shop/product.asp?ProductID=NI-SQL1
K. Brian Kelley
@kbriankelley
January 29, 2003 at 8:52 am
Looks like a nice product, with a nice price as well. SMS would be overkill if all you wanted was to use it to update your software. Have you written a review on UpdateExpert yet? I would be greatly interested to hear what you had to say about it after using it. I have implemented SMS in the past with quite some pain and very little fondness in my memory for it, but that was version 1.0, and as of yet have not run 2.0 so I can not speak for it.
St. Bernard Software (so no one else has to google it...)
http://www.stbernard.com/default.asp
Tim C.
//Will write code for food
One Windows to rule them all, One Windows to find them,
One Windows to bring them all and in the darkness bind them
In the Land of Microsoft where the Shadows lie.
Tim C //Will code for food
Viewing 15 posts - 1 through 15 (of 42 total)
You must be logged in to reply to this topic. Login to reply