April 25, 2020 at 5:38 pm
Wow... just freakin' WOW!!! I thought that continuous integrationist and deployment was supposed to prevent this type of thing.
The first paragraph (quoted below) is what is really shocking to me.
At Microsoft, 47,000 developers generate nearly 30 thousand bugs a month. These items get stored across over 100 AzureDevOps and GitHub repositories. To better label and prioritize bugs at that scale, we couldn't just apply more people to the problem. However, large volumes of semi-curated data are perfect for machine learning. Since 2001 Microsoft has collected 13 million work items and bugs. We used that data to develop a process and machine learning model that correctly distinguishes between security and non-security bugs 99 percent of the time and accurately identifies the critical, high priority security bugs, 97 percent of the time.
That last sentence is spooky... who the hell wrote the ML to find the bugs??? Hopefully it wasn't the same people that wrote the buggy code to begin with. AND has it come even close to finding all of the new ones and is it checking for regression so we don't have a disaster like the RTM and early SPs of SQL Server 2012?
This does explain a lot of the disasters that have occurred in SQL Server over the years. It also explains the need for monthly CUs because it sounds to me like they're so interested in rapid releases that they'll never "do it right the first" time. Unfortunately, it has become like a very bad drug habit... the more you do it, the more you're going to need to do it.
This is also one of the bad parts about having the internet. In the last couple of decades, more an more companies have apparently adopted the philosophy that it's ok to ship crap code instead of actually doing it right because the users can easily download the "fixes" (which might also be broken, BTW) and they have these EULA's that relieve them of all responsibility for damages that their "hey, we shipped on time again" crap code may wreak upon customer's data.
To wit, I used to worry about computer viruses a whole lot. I now worry MORE about what a monthly CU or new release is going to "F" up.
--Jeff Moden
Change is inevitable... Change for the better is not.
April 25, 2020 at 8:02 pm
You're misunderstanding CI and ML.
CI is designed to catch these things. Developers do not magically write better code with CI. This is < 1 bug/month per dev. Shocking? I'd say that's pretty good. CI is designed to evaluate the code, according to the rules/tests/Static code analysis, and do it completely. Developers should do this, but they're human, and across 47k people, each making changes and compiling most days, they'll forget things. CI is independent validation and verification of code on another machine. I'm stunned it's 30k bugs. I'd have expected 100k+/month, maybe more like 470k for 47k devs.
Likely, there are more bugs caught, but these are the security bugs, not all bugs. There are also bugs that slip through, but the unit testing and ML evolve to keep getting better at catching things.
The ML isn't written. What happens is someone looks at the code that produces various security bugs, say a buffer overrun. They find the C++ (C#) that doesn't end check a parameter or something else. They then set up some extraction for this as a factor. Think this:
IF {
some_code.substring() == [poor parameter checking]
}
print 1This gives them factors. They have lots of these, then they feed these through an ML algorithm that looks for patterns. There are real developers that evaluate code and give the algorithm some idea of what actually produces a bug and what doesn't. Over time, this trains the model (lots of repetitions here).
This is the same way people learn about buts. Imagine you saw declare @i varchar a few times. At some point, you're realize this a bug and stop doing it (or launch a pork chop at a dev). The ML is teaching the system to find this and automatically raise a flag.
Over time, they get an ML model (not an algorithm), that is good at recognizing the code that does and does not cause issues. They deploy this, scan code, and have humans verify results. Eventually, they turn the algorithm loose and let it notify developers by failing CI with these bugs highlighted.
In the real world, periodically, or continually, humans randomly sample the ML results (both hits and misses) and try to see if the model needs to be retrained. It's a boring evaluate, set new factors, rerun model against training data, re-run against sample data, review, repeat. This is what data scientists do, and it's a lot manual analysis of samples, then letting ML run against small samples, then large samples. It's, IMHO, a boring job for really, really detailed oriented numbers scientists.
The goal is to do it right the first time, but there are pressures from business to get things done, spend resources on new things, and a bunch of other real world software pressures. The quality has gone way up. It's not perfect, never will be, but things like ML here to prevent simple mistakes. The bar keeps being raised. Not where you want it, and certainly the focus on features is not something addressed here.
That is orthogonal to quality. Spending more time on a feature isn't a quality decision. Ex. String_split. MS thinks it works fine and meets the spec. Many of us think it's woefully incomplete. From a quality standpoint, string_split() works. From a customer standpoint, it doesn't
April 25, 2020 at 8:20 pm
You're misunderstanding CI and ML.
Maybe but I don't believe so. For example, you wrote...
CI is designed to catch these things. Developers do not magically write better code with CI. This is < 1 bug/month per dev. Shocking? I'd say that's pretty good. CI is designed to evaluate the code, according to the rules/tests/Static code analysis, and do it completely. Developers should do this, but they're human, and across 47k people, each making changes and compiling most days, they'll forget things. CI is independent validation and verification of code on another machine. I'm stunned it's 30k bugs. I'd have expected 100k+/month, maybe more like 470k for 47k devs.
Considering all of the errors that they say have been released to 100's of "Githubs" (to coin a phrase), I'm thinking that it's not so complete. I also guess you and I have different thresholds for what number of errors is considered to be stunning. 😀
I guess we'll have to agree to disagree on that point. I AM very happy that they apparently realize they have a problem and are trying to do something to correct the problem though.
--Jeff Moden
Change is inevitable... Change for the better is not.
April 27, 2020 at 9:50 am
Sooooo… Anyone else hear of the Cognizant ransomware attack?
People have the customer data and are calling up said customers asking for emergency VPN access so that "We can continue providing you services."
It hurts just thinking about it that people might actually have (or will) fall for that line and open up their systems to hackers because they're not paying attention.
April 27, 2020 at 2:01 pm
Hadn't heard of it, but I'm not surprised. This is a time criminals will take advantage of the stressful situation
April 27, 2020 at 6:55 pm
(Gets ready for Jeff to begin his comments about the "cloud")
So go to get started on some work this morning, getting some of the fiddly-bits straightened out in our new "cloud-ish" environment (really, it's just VMs hosted but another organization, not SAAS / PAAS / Azure-ish) only to find out...
ALL my SQL Servers are inaccessible. At all. No RDP, not SQL connections, nothing.
So, open a ticket with the cloud team to look into it.
And, during our Monday morning cloud meeting find out that "all the SQL VMs are giving black screens and we can't even log in from the consoles, can we reboot them?" Then, as of about 12:30, ONE of my 7 servers was (potentially) reachable (or at least RDP didn't barf an error in the first 30 seconds.)
So I've been dead in the water all day as far as our cloud stuff goes, just spent the afternoon doing various paperwork that needed to be done...
My worst fear, of course, will be getting an e-mail tomorrow from the cloud team along the lines of "yeah, we're sorry about this, but we're going to have to stand up new servers for the SQL servers and you're going to have to load them from scratch. Oh, and it'll be about 3 weeks before we've got them ready for you, this won't impact the timeline, right?" Keep in mind, our intended cut-over date is June 1...
April 27, 2020 at 7:54 pm
I usually get speeches like this - "we usually would have SoAndSo work on that, but he's on vacation for a month. But don't worry, we have a new resource that just joined the team, graduated last week so he should be able to help you"
-------------------------------------------------------------------------------------------------------------------------------------
Please follow Best Practices For Posting On Forums to receive quicker and higher quality responses
April 27, 2020 at 8:21 pm
(Gets ready for Jeff to begin his comments about the "cloud")
There is no umbrella big enough or strong enough to protect anyone from such a s**t storm as that. My thoughts are with you and the ghosts of servers past. 😀
I also just read an article about DBCC CHECKDB and how Azure will take your database offline if corruption is detected and the fact that you can't do a rebuild. Fun stuff, that cloud thingy is . 😀 The silver lining is that it's much more stable, reliable, and cost effective than on premise, right? 😀 😀 😀
If it makes you feel any better, we're actually getting ready for the smoke'n'mirrors in the sky ourselves (not anything near my idea). I'm going to do my damnedest to make sure it's as safe as possible but nothing can prepare anyone for what you're going through. I can only hope we never have such an issue.
--Jeff Moden
Change is inevitable... Change for the better is not.
April 27, 2020 at 8:55 pm
Heh... let the silliness continue. I just got an email saying...
KILL 99.9% OF BACTERIA & HARMFUL MICROBES
FORMS A PROTECTIVE SHIELD THAT DELAYS GROWTH OF HARMFUL MICROBES ON INTERIOR SURFACES
Reg $129.95
2 Hour Service
That 0.1% it didn't kill is going to be kickass and you've just paid $129.95 to remove all of it's competition, good and bad. 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
May 1, 2020 at 7:14 am
Heh... let the silliness continue. I just got an email saying...
KILL 99.9% OF BACTERIA & HARMFUL MICROBES
FORMS A PROTECTIVE SHIELD THAT DELAYS GROWTH OF HARMFUL MICROBES ON INTERIOR SURFACES
Reg $129.95
2 Hour Service
That 0.1% it didn't kill is going to be kickass and you've just paid $129.95 to remove all of it's competition, good and bad. 😀
There's a product over here that claims to kill 99% or so of bacteria and viruses and I've always thought it's the 1% we need to worry about that are left. If bleach won't kill it, run away.
How to post a question to get the most help http://www.sqlservercentral.com/articles/Best+Practices/61537
May 1, 2020 at 9:31 am
There's a product over here that claims to kill 99% or so of bacteria and viruses and I've always thought it's the 1% we need to worry about that are left. If bleach won't kill it, run away.
The manufacturers of that product then changed the advertising tag line to "Kills all known germs. Dead." Which always seemed a bit redundant, and also has a nice get-out clause... "well, we didn't know about that germ when we tested it."
Thomas Rushton
blog: https://thelonedba.wordpress.com
May 1, 2020 at 4:04 pm
I love the "or so". Does that mean 99.99% or 90% or maybe 50%. I mean, so isn't exactly a precise measurement
May 1, 2020 at 9:15 pm
Jeff Moden wrote:Heh... let the silliness continue. I just got an email saying...
KILL 99.9% OF BACTERIA & HARMFUL MICROBES
FORMS A PROTECTIVE SHIELD THAT DELAYS GROWTH OF HARMFUL MICROBES ON INTERIOR SURFACES
Reg $129.95
2 Hour Service
That 0.1% it didn't kill is going to be kickass and you've just paid $129.95 to remove all of it's competition, good and bad. 😀
There's a product over here that claims to kill 99% or so of bacteria and viruses and I've always thought it's the 1% we need to worry about that are left. If bleach won't kill it, run away.
You need to ingest what's called a "Minimal infective dose (MID)" to get infected by bacteria. So the theory is if you kill 99.9% of them there is not enough left to cause infection.
May 3, 2020 at 4:52 am
Neil Burton wrote:Jeff Moden wrote:Heh... let the silliness continue. I just got an email saying...
KILL 99.9% OF BACTERIA & HARMFUL MICROBES
FORMS A PROTECTIVE SHIELD THAT DELAYS GROWTH OF HARMFUL MICROBES ON INTERIOR SURFACES
Reg $129.95
2 Hour Service
That 0.1% it didn't kill is going to be kickass and you've just paid $129.95 to remove all of it's competition, good and bad. 😀
There's a product over here that claims to kill 99% or so of bacteria and viruses and I've always thought it's the 1% we need to worry about that are left. If bleach won't kill it, run away.
You need to ingest what's called a "Minimal infective dose (MID)" to get infected by bacteria. So the theory is if you kill 99.9% of them there is not enough left to cause infection.
Yep... and there's a pretty good chance that if bleach doesn't kill a bug, you won't have to ingest much for it to "get you". I understand the theory of MID but, as someone actually has in their signature line, "In theory, practice and theory are the same. In practice, they are not". 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
May 4, 2020 at 8:13 pm
stupid question time. when you guys deal with X12 spec files, is there a reasonable argument *against* loading them into tables named for loops/segments/ref numbers?
I get that they are variable files, not all loops contain all things all the time, but that's what NULL would be for....is there some sort of referential integrity that PREVENTS such a method that I'm not thinking of?
I tire of not being able to find something or having to dig through the raw data because nobody thought we would need something
-------------------------------------------------------------------------------------------------------------------------------------
Please follow Best Practices For Posting On Forums to receive quicker and higher quality responses
Viewing 15 posts - 64,816 through 64,830 (of 66,816 total)
You must be logged in to reply to this topic. Login to reply