Removing stored procedures to move to cloud

Question

Post reply

Removing stored procedures to move to cloud

Viewing 15 posts - 106 through 120 (of 192 total)

You must be logged in to reply to this topic. Login to reply

Sergiy SSC Guru Points: 110209 More actions · Answer 1

Here it is - DevOps Car:

_____________
Code for TallyGenerator

Orlando Colamatteo SSC Guru Points: 182293 More actions · Answer 2

sgmunson - Monday, May 7, 2018 6:56 AM
Orlando Colamatteo - Saturday, May 5, 2018 6:44 AM
This gave me a pretty good laugh because in real Scrum there are no project managers. That's why you have scrum meetings. And a project manager.It all depends on what the optimizing goals are for the architecture. Saddling developers with external dependencies like having to wait for a proc to be developed by someone else or even send it to someone else for revisions before it can be deployed is an impediment (in Scrum terms) and as we look for ways to help the team go faster we look to remove impediments.Logic follows:1 Procs require specialized skills or are somehow slowing the team down2 Some critical mass of decision makers decide Procs are an impediment3 Decision makers decide on an architecture that does not include Procs The proof is not in the Procs worth itâ€™s in their ability to be part of an agile development process. Have you looked into Visual Studio Database Projects or redgate ReadyRoll? Change or be phased out. Those are your options.
I can only conclude that someone forgot about the rather strong and CRITICAL need to manage the database, or that somehow, databases are things to be played with instead of properly cared for. You don't abandon stored procedures because you have developers that don't know how to write them. You either train the ones you have to be able to properly code stored procedures, hire better developers to begin with, or suffer consequences. Your choice... And while agile/scrum advocates are prone to whining about the costs of such things, I simply explain that if you're worried about costs, why did you hire such poorly skilled developers in the first place? Or how about the cost of the agile infrastructure because they continue to perceive the database as an impediment? Things tend to get real quiet whenever I bring that topic up. It's as if they just don't consider a database an engine that drives the business. The level of incompetence that kind of thinking represents is extraordinary. I basically go back to "If you think professionals are expensive, wait until you hire an amateur" (Red Adair).

How does your organization model you in terms of your role? Are you considered a developer, a DBA, something else, and is that in name/title or in practice? I am assuming you are not on a Scrum team, but if you were do you think you would end up picking up the majority of the SQL development tasks in the Sprint? I am curious if you were on a Scrum team would the team you were on be able to produce SQL code to your level of satisfaction? If someone else on your Scrum team did a SQL coding task would you be open to doing a code review for them and providing feedback within moments of them committing their code? Who would code review your code?

One other comment because I don't want us to lose the plot here. Databases do not drive businesses, customers do.

There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato

Orlando Colamatteo SSC Guru Points: 182293 More actions · Answer 3

Jeff Moden - Monday, May 7, 2018 7:34 AM
Also, one of the reasons why I don't like pure Agile environments is because Project Managers are supposedly "not needed". One of the things that people don't realize is that a good Project Manager will not only help keep the current sprints on track but they also serve the huge and greatly needed functionality of being an enabler. If you need something to get the job done, a good Project Manager will go into the mode of pounding on people that can provide the correct resources, physical or otherwise, which prevents Developers from having to suffer such distractions.

By "pure Agile" I think you mean Scrum but let me know if I guessed incorrectly. Everything you described as the responsibility of the traditional Project Manager was divvied up per the Scrum Guide between the Development Team and the ScrumMaster which is why "Project Manager" is not a role in Scrum. The ScrumMaster helps with all of what you mentioned and more and they are an integral part of the process. If you haven't ever worked with a ScrumMaster that helps with all these things and more I hope one day you experience it.

There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato

Orlando Colamatteo SSC Guru Points: 182293 More actions · Answer 4

Sergiy - Monday, May 7, 2018 7:18 PM
Sorry, Steve, it's Agile that's bad.
Agile, DevOps may seem extremely cool, but the only thing I can see in them is degradation.
They promote the idea of "everybody does everything" approach.
But we all know about the "Division of Labour" thing, which was introduced into production processes ages ago.
Division of labour is what made industries out of random manufacturers.
You cannot imagine a car designed and produced by a single person.
You cannot see houses built by a single developer/architect.
Well, there are such examples. They named "tree houses" and "hobby cars".
Things, which are not for trade, not certain quality, and quite expensive in terms of labour.
Regular production in every industry is performed by specialists, and a plumber does not have a say in electrical wiring.
IT is heading the opposite way
There is enourmous amount of belief in Full Stack Developers, who alegetly can do everything.
As a result - look around. The whole field is full of tree houses and hobby cars.
There are barely a handfull of products which can be taken a a show case. Which are not a subject of security breaches, data leaks, planned and unplanned outages, simply faulty functionality.
And qualities of newer products are almost certainly worse than of older ones. We all write on this forum, so everybody has a proof in front of their eyes.
Until the approach is changed, until jerks are removed from decision making process in the IT industry, I expect the stories like "The Technology Journey to Disaster" will continue to come, in increasing numbers.

They promote the idea of "everybody does everything" approach.

This is a common misunderstanding. I would urge you to read about cross-functional teams and think about cross-functional at both the individual and team level. Cross Functional Doesnâ€™t Mean Everyone Can Do Everything by Mike Cohn

Division of labour is what made industries out of random manufacturers.

You've offered up a Tayloristic view of the world but we're living in a post-industrial society. And we build software not cars or houses. You're humor is good though. I thought your DevOps car was funny 🙂

There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato

Orlando Colamatteo SSC Guru Points: 182293 More actions · Answer 5

Sue_H - Monday, May 7, 2018 4:12 PM
Agile is a development methodology, not a deployment methodology...

Agile, according to the Agile Manifesto which marked the beginning of the movement we know as Agile (with a capital A) software development, defines a set of four values and twelve principles. Agile is not a development methodology or a methodology of any kind for that matter.

There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato

Sergiy SSC Guru Points: 110209 More actions · Answer 6

Orlando Colamatteo - Monday, May 7, 2018 11:23 PM
Sue_H - Monday, May 7, 2018 4:12 PM
Agile is a development methodology, not a deployment methodology...
Agile, according to the Agile Manifesto which marked the beginning of the movement we know as Agile (with a capital A) software development, defines a set of four values and twelve principles. Agile is not a development methodology or a methodology of any kind for that matter.

Yeah, I like this bulk of nonsense.
"Working software over comprehensive documentation"
What's the criteria for "working software"?
Who/what defines if it's working or not?

_____________
Code for TallyGenerator

Sergiy SSC Guru Points: 110209 More actions · Answer 7

Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.

“If I had asked people what they wanted, they would have said faster horses.” ? Henry Ford

The best architectures, requirements, and designs
emerge from self-organizing teams.

100% false.
Best architectures and designs come from individuals which go against the teams.
Only mediocre architectures, requirements, and designs emerge from any kind of teams.

_____________
Code for TallyGenerator

Orlando Colamatteo SSC Guru Points: 182293 More actions · Answer 8

Jeff Moden - Monday, May 7, 2018 11:18 AM
Heh... Orlando said that they didn't want to wait on someone to write stored procedures. That strongly implies that the developers don't know how to do such things on their own which implies that they're not trained to do it... or simply don't want to do it.

Goodness no. Not what I was implying at all. I was talking about the scenario where members of a Development Team (a subset of the Scrum Team) are qualified to some degree to write SQL code and would like to be able to develop their own schema and procedures however the organization pays specialists (usually called DBAs) to do that kind of work as needed and supply it to the Development Team as an input to their development process. Since these specialists have other operational responsibilities like looking after server builds, server health, environment management, backups and restores, production support, etc. etc. etc. they are too highly paid for the organization to hire one for every development team and even if the organization had that kind of cash these specialists would be too busy to be fully dedicated to the work of the Development Team.

What I actually said:

It all depends on what the optimizing goals are for the architecture. Saddling developers with external dependencies like having to wait for a proc to be developed by someone else or even send it to someone else for revisions before it can be deployed is an impediment (in Scrum terms) and as we look for ways to help the team go faster we look to remove impediments.
Logic follows:
1 Procs require specialized skills or are somehow slowing the team down
2 Some critical mass of decision makers decide Procs are an impediment
3 Decision makers decide on an architecture that does not include Procs
The proof is not in the Procs worth itâ€™s in their ability to be part of an agile development process. Have you looked into Visual Studio Database Projects or redgate ReadyRoll? Change or be phased out. Those are your options.

What I tried to say was that anything the Development Team has to go outside of themselves to accomplish amounts to an external dependency...even a code review. A Development Team should have all the skills on it to accomplish their work. That means if the software needs a data store and they choose SQL Server they better have some people that know or SQL Server or are getting after learning SQL Server in a hurry.

Scrum does not solve your problems but it exposes them in a hurry. If your organization does not like Scrum it probably does not like transparency and openness.

There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato

Orlando Colamatteo SSC Guru Points: 182293 More actions · Answer 9

Sergiy - Tuesday, May 8, 2018 12:32 AM
Orlando Colamatteo - Monday, May 7, 2018 11:23 PM
Sue_H - Monday, May 7, 2018 4:12 PM
Agile is a development methodology, not a deployment methodology...
Agile, according to the Agile Manifesto which marked the beginning of the movement we know as Agile (with a capital A) software development, defines a set of four values and twelve principles. Agile is not a development methodology or a methodology of any kind for that matter.
Yeah, I like this bulk of nonsense.
"Working software over comprehensive documentation"
What's the criteria for "working software"?
Who/what defines if it's working or not?

Simple. The customer.

There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato

Orlando Colamatteo SSC Guru Points: 182293 More actions · Answer 10

Sergiy - Tuesday, May 8, 2018 12:51 AM
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
â€œIf I had asked people what they wanted, they would have said faster horses.â€ â€• Henry Ford

I am not seeing your point. Where does it say we are asking the customer? If you're alluding to disruptive technology and innovation then sometimes the customer does not know what they want until you show it to them (Steve Jobs) however that is the exceptional case, same as when we made the leap from horse and buggy to car.

There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato

Sergiy SSC Guru Points: 110209 More actions · Answer 11

Orlando Colamatteo - Tuesday, May 8, 2018 12:57 AM
Sergiy - Tuesday, May 8, 2018 12:32 AM
Yeah, I like this bulk of nonsense.
"Working software over comprehensive documentation"
What's the criteria for "working software"?
Who/what defines if it's working or not?
Simple. The customer.

I see.
Exposing software to customer without knowing if it's working or not.
That's the way to go!

_____________
Code for TallyGenerator

Sergiy SSC Guru Points: 110209 More actions · Answer 12

Orlando Colamatteo - Tuesday, May 8, 2018 1:04 AM
Sergiy - Tuesday, May 8, 2018 12:51 AM
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
â€œIf I had asked people what they wanted, they would have said faster horses.â€ â€• Henry Ford
I am not seeing your point. Where does it say we are asking the customer?

Here:
> to satisfy the customer
If you did not ask for customer desires you have pretty much nothing to satisfy.

And here:

> Customer collaboration over contract negotiation
> Responding to change over following a plan

The last one is the best.
Makes it absolutely impossible to create a consistent, integral product.

_____________
Code for TallyGenerator

Sue_H SSC Guru Points: 90860 More actions · Answer 13

Orlando Colamatteo - Monday, May 7, 2018 11:23 PM
Sue_H - Monday, May 7, 2018 4:12 PM
Agile is a development methodology, not a deployment methodology...
Agile, according to the Agile Manifesto which marked the beginning of the movement we know as Agile (with a capital A) software development, defines a set of four values and twelve principles. Agile is not a development methodology or a methodology of any kind for that matter.

And even on the Agile Alliance site, in explaining to beginners:
What is Agile Software Development?
Agile Software Development is an umbrella term for a set of methods and practices based on the values and principles expressed in the Agile Manifesto.

I could go on and on and on with more. But I'm sure everyone else wrong. So go pontificate some more. With that capital A.

Sue

sgmunson SSC Guru Points: 110651 More actions · Answer 14

funbi - Monday, May 7, 2018 2:33 PM
If devs see stored procs as impediments (and I don't think that they do) it would not be because devs can't write stored procs. It would be because any DB changes mean that they then have to factor in a DBA into their deployment pipeline - who would be needed to either apply the changes or set up DB permissions so that the dev team can apply them i.e. bringing a human into a process that is automated in every other way.

Heaven forbid a HUMAN gets involved to do the right thing... and the idea that anyone actually believes you can totally automate database work is so incredibly foolish that I can't even imagine going down that road at all. You pretty much make my point for me. If you had any genuinely useful experience with database work, you would know better. Clearly, you think databases are just supposed to do all the work for you. Good luck with that...

Steve (aka sgmunson) 🙂 🙂 🙂
Rent Servers for Income (picks and shovels strategy)

sgmunson SSC Guru Points: 110651 More actions · Answer 15

Sean Lange - Monday, May 7, 2018 3:07 PM
sgmunson - Monday, May 7, 2018 2:18 PM
Steve Jones - SSC Editor - Monday, May 7, 2018 9:25 AM
Jeff Moden - Monday, May 7, 2018 7:34 AM
sgmunson - Monday, May 7, 2018 6:56 AM
I can only conclude that someone forgot about the rather strong and CRITICAL need to manage the database, or that somehow, databases are things to be played with instead of properly cared for. You don't abandon stored procedures because you have developers that don't know how to write them. You either train the ones you have to be able to properly code stored procedures, hire better developers to begin with, or suffer consequences. Your choice... And while agile/scrum advocates are prone to whining about the costs of such things, I simply explain that if you're worried about costs, why did you hire such poorly skilled developers in the first place? Or how about the cost of the agile infrastructure because they continue to perceive the database as an impediment? Things tend to get real quiet whenever I bring that topic up. It's as if they just don't consider a database an engine that drives the business. The level of incompetence that kind of thinking represents is extraordinary. I basically go back to "If you think professionals are expensive, wait until you hire an amateur" (Red Adair).
Heh.... you beat me to it with the Red Adair quote. One of my favorite quotes is on of my own making... "If you want it real bad, you usually get it that way". 😀
Also, one of the reasons why I don't like pure Agile environments is because Project Managers are supposedly "not needed". One of the things that people don't realize is that a good Project Manager will not only help keep the current sprints on track but they also serve the huge and greatly needed functionality of being an enabler. If you need something to get the job done, a good Project Manager will go into the mode of pounding on people that can provide the correct resources, physical or otherwise, which prevents Developers from having to suffer such distractions.
I'll argue the other side here slightly. I don't think Orlando is saying that you shouldn't have competence, but that you don't necessarily need to slow down with process.
Smunson says that you need to train developers. Where in Orlando's post did he mention that developers shouldn't be trained? He said no project managers and everyone needs to work together. The proof is in the procs being used in the agile process, not in the procedures, the developers, or the DBAs.
Jeff dislikes the lack of PMs, who can save developers' time by getting things done. I'll say that I've worked in no shortage of environments where the PMs don't get things done, and really just push on everyone to do more work instead of smoothing the process. It's not Agile that's bad or PMs that are good, but you are highly dependent on the people doing a good job. It's really the people and most of you that bang on Agile or DevOps have worked with lots of unprofessional and competent people and you're blaming the system as much as the staff.
Whether you work in Agile or DevOps or Scrum or whatever, the system isn't usually the problem. Some systems hide problems more than others, but all will fail if you don't have your staff, and your management, supporting the system. DevOps, Agile, etc. are asking people to regularly and constantly improve. That doesn't just mean that work flows faster and faster, which is what developers start to want, and management does. Skills improve, we change based on feedback, and we self-manage in many ways.
Is that a good use of time? I'm torn. Certainly having a highly paid, talented Mr. Moden chase down the allocation of new resources when he could tune code (or teach/mentor others) might not be a good use of time. The thing I think about is that having a break and some changes in routine, some breaks, is a good way to ensure that talented people don't burn out. Finding a balance there is tough, and certainly in the DevOps space, this doesn't mean less staff. It means that all staff gets trained to handle a bit of everything. The HADR guy doesn't get the same coding responsibilities as the C# developer, but they ought to be able to help out in places and know a little. Just like the developer learns a bit about HADR, which might change how they build something.
It's almost always the people, not the system or platform or architecture.
I'm going to have to agree with Jeff. If devs see sprocs as an impediment in any way then there's a considerable lack of expertise in not just the devs, but also in the scrum masters and or project managers that go along with that nonsense. In my book, that rises above mere incompetence, to the level of negligence. Treating a database like it's basic structures are impediments is ludicrous, at best, and that's why my book says negligence. If the PM's are getting away with it, then the level of incompetence probably rises well up into management.
I would say that it almost always rises well up into management. Remember that there are very few (if any) management types that are one or two levels over a DBA or dev that have any depth of technical knowledge. In my wife's job she knew it was wrong to take all the sql out of procedures and move it to the application but she doesn't have the depth of knowledge to be able to explain to the development manager as to why. She asked me about it a week or so after they made that decision and I nearly fell out of my chair. They didn't treat or see the database as an impediment to progress, they foolishly saw procedures as the performance bottleneck. From a performance perspective their weeks long effort was an exercise in futility. But from a system and maintenance perspective it was huge leaps backwards. And they still have massive performance issues and she can't get the dev guy to listen to me or all the others out there that tell them they are going about is all wrong. They have queries that generate thousand and thousand of rows for grids that are paged. They aren't doing any server side paging or caching, just querying the same multiple thousand row query for every page of data.

Thanks for providing a real-world example of what happens when there's no fully qualified database professional around to provide guidance on such things. In that situation, it seems that even when confronted with information suggesting they went down the wrong road, they were so unwilling to admit they'd made a mistake, that they chose to double down on it instead. And people get their panties in a wad because I'm willing to tell the truth instead of lie to their faces... Ughhh...

Steve (aka sgmunson) 🙂 🙂 🙂
Rent Servers for Income (picks and shovels strategy)