August 1, 2012 at 7:48 am
Grant Fritchey (8/1/2012)
SQLRNNR (7/31/2012)
Grant Fritchey (7/31/2012)
I worked on a project for about three years (it was supposed to take 18 months) that still wasn't done when I left the company. They too had jumped on the premature optimization band wagon taking it to mean, as many do, that you don't have to design anything, ever. They primarily used nHibernate as the means of ignoring the database (instead of ORM, they made it into OOM, Object-to-Object-Mapping). It completely eliminated the database and DBAs from their development process...Now, 4.5 years into the project (again, it was only going to be 18 months long, eliminating the DBAs is part of what makes it so much faster to deliver...) they still haven't delivered and they're finding HUGE performance issues. Plus, here's a shocker, people actually expect to be able to run reports on the data, but since it's stored as objects, you can't easily cut across it. They are throwing money at the problem currently. They've implemented Availability Groups to create a read-only mirror of the database so that they can remove reporting from the production system, but that didn't fix the performance issues. Now, they're working on a full-blown database design and will build a load process to go from the object model to a relational model, but only for reporting. It's a nightmare. What's more, it's a nightmare that I predicted, in a detailed report, written 4.5 years ago when the project started.
Wow, I can't believe they are still pushing forward in the same manner. I remember these stories about the project some time ago and am dumbfounded that they have not changed (well except that you left them:-D )
I think in this case, the project has effectively failed a couple of times already so they're pushing ahead regardless of issues with methodology. I know that no other projects are following this approach at the company when, for a time, this was going to be the New Model.
But yeah, a big part of what made me quit was this project. When I'm standing there, telling them everything that is going to go wrong and getting ignored... Then, when they brought in a Microsoft consultant who told them what I told them... Then, when that consultant actually said to them, "Why did you bring me in for this? You have Grant sitting right across the hall." Just couldn't stand it there after that.
Ooohhh! I bet that made you popular!
Not that the consultant did anything wrong, but I bet a few people took his comment as a kick to the ego-groin.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
August 1, 2012 at 11:35 am
I guess this all goes in part of being a newbie, and those who are stubborn enough to not care about appropriate database development and technique.
I guess the part of all of this that really burns me is that change in our industry is not only inevitable, but expected, and yet - for whatever reason - those who need to be the most flexible when it comes to design are the most stringent and fixed on things that are going to wind up causing more problems in the end, than solutions.
It smacks of those who need to blame someone else for when things go wrong, and typically - the things going wrong aren't the issue being blamed, but a great cover for bad application design, and an ignoring for database development. Completely irresponsible, and a horrible way to manage any enterprise environment that needs accuracy and a steadfast design to grow with over years to come.
Why would you build a house before building the foundation that it needs to lay on top of? Any layman in that scenario would look at you and think you were crazy, and this is no different.
August 1, 2012 at 9:27 pm
There is a reason why my old university teaches you normalisation before they even let you touch SQL with a kilometre long pole. In my opinion, the only remotely forgivable reason for not using foreign keys is if your new to MySQL and don't know about the difference between MyISAM and InnoDb.
I used to call my self The Master on forums, then I entered the IT industry and realised how much I still have to learn.
August 1, 2012 at 10:34 pm
Grant Fritchey (8/1/2012)
But yeah, a big part of what made me quit was this project. When I'm standing there, telling them everything that is going to go wrong and getting ignored...
IIRC, I met some of those people when we went out for pizza a couple of summers ago. From some of the things I heard them say, I don't blame you a bit. That really had to be frustrating.
--Jeff Moden
Change is inevitable... Change for the better is not.
August 2, 2012 at 5:36 am
GSquared (8/1/2012)
Ooohhh! I bet that made you popular!Not that the consultant did anything wrong, but I bet a few people took his comment as a kick to the ego-groin.
I think that's how he meant it. But we're way off topic.
There was another group using Entity Framework in the company. But they came over to the DBA group and asked how best to make it work. We worked with them and churned out a decent little app in record time. It is possible.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
August 3, 2012 at 6:19 am
Grant Fritchey (8/2/2012)
GSquared (8/1/2012)
Ooohhh! I bet that made you popular!Not that the consultant did anything wrong, but I bet a few people took his comment as a kick to the ego-groin.
I think that's how he meant it. But we're way off topic.
There was another group using Entity Framework in the company. But they came over to the DBA group and asked how best to make it work. We worked with them and churned out a decent little app in record time. It is possible.
Yep. Some people just haven't learned yet that you can't automate judgement.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
August 3, 2012 at 8:28 am
Michael Valentine Jones (7/30/2012)
I'm really not sure exactly what you want to discuss.There are plenty of people that create denormalized databases and claim they did it for speed, and not just in e-Commerce applications.
Usually, the answer is simple: they just didn't know how to create a normalized database, so they claim they did it for performance.
I'm one of those who might create a denormalized database for performance, especially in the case of a web application. In these cases, performance could itself be a design criteria, and I would seriously consider handling consistency outside of the database layer. However, it wouldn't be because I don't know how to create a normalized database, it would be that I don't want to spend the hardware budget.
August 3, 2012 at 11:08 am
patrickmcginnis59 (8/3/2012)
I'm one of those who might create a denormalized database for performance, especially in the case of a web application. In these cases, performance could itself be a design criteria, and I would seriously consider handling consistency outside of the database layer. However, it wouldn't be because I don't know how to create a normalized database, it would be that I don't want to spend the hardware budget.
Problem then becomes the short sightedness of that configuration decision. That's probably fine for a web based app that is never going to get over a gigabyte in size, but when you all of a sudden grow into a monster, half-terabyte or larger sized database???
Be it safe to say that you will want the price of that hardware you didn't buy (way back when) by the time you reach that point.
August 3, 2012 at 11:40 am
SQL_ME_RICH (8/3/2012)
patrickmcginnis59 (8/3/2012)
I'm one of those who might create a denormalized database for performance, especially in the case of a web application. In these cases, performance could itself be a design criteria, and I would seriously consider handling consistency outside of the database layer. However, it wouldn't be because I don't know how to create a normalized database, it would be that I don't want to spend the hardware budget.
Problem then becomes the short sightedness of that configuration decision. That's probably fine for a web based app that is never going to get over a gigabyte in size, but when you all of a sudden grow into a monster, half-terabyte or larger sized database???
Be it safe to say that you will want the price of that hardware you didn't buy (way back when) by the time you reach that point.
It was poor phrasing on my part. I meant that I didn't want to spend the hardware budget per web request, not the whole computer.
August 6, 2012 at 6:26 am
patrickmcginnis59 (8/3/2012)
Michael Valentine Jones (7/30/2012)
I'm really not sure exactly what you want to discuss.There are plenty of people that create denormalized databases and claim they did it for speed, and not just in e-Commerce applications.
Usually, the answer is simple: they just didn't know how to create a normalized database, so they claim they did it for performance.
I'm one of those who might create a denormalized database for performance, especially in the case of a web application. In these cases, performance could itself be a design criteria, and I would seriously consider handling consistency outside of the database layer. However, it wouldn't be because I don't know how to create a normalized database, it would be that I don't want to spend the hardware budget.
The point of a number of prior posts is that "create a denormalized database for performance" is only a valid decision if you build both version, and test them under load for actual performance characteristics.
I'm also unclear on how normalization requires more hardware than denormalization. Normalized databases usually take less storage, less IO bandwidth, often take less RAM/cache, and frequently burn less CPU cycles, than denormalized ones.
Joins require more resources for those things that hit them, but narrower tables are faster in atomic CRUD operations.
After all, relational design and normalization were originally designed to make databases easier to deal with and faster to operate on, on primitive hardware where every byte mattered.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
August 6, 2012 at 6:32 pm
Would I be correct in saying that decreased data replication is a major reason why normalisation and foreign keys decrease storage requirements?
I used to call my self The Master on forums, then I entered the IT industry and realised how much I still have to learn.
August 7, 2012 at 6:26 am
Boltz442 (8/6/2012)
Would I be correct in saying that decreased data replication is a major reason why normalisation and foreign keys decrease storage requirements?
Yes.
Of course, the amount it's decreased depends on the data being stored.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
December 21, 2022 at 9:13 am
This was removed by the editor as SPAM
Viewing 13 posts - 16 through 27 (of 27 total)
You must be logged in to reply to this topic. Login to reply