December 17, 2014 at 7:25 am
Brandie Tarvin (12/17/2014)
WayneS (12/17/2014)
Jeff Moden (12/17/2014)
Koen Verbeeck (12/17/2014)
Brandie Tarvin (12/17/2014)
Brandie Tarvin (12/16/2014)
WHOO! #HappySnoopyDanceI just spent 2 & 1/2 days reworking a package and its supporting tables / stored procedures. Previous time of execution between 8 - 12 hours.
Current execution time ... < 5 minutes.
"Celebration Time, come on!"
Sigh. So I presented coworker with my changes and asked for a peer review. Partially to make sure I didn't miss anything, partially to give coworker a learning opportunity since coworker was the one who designed the package. There were 7 files / queries involved and lots of tables. The response I got when I told coworker was: "You redesigned all the queries? But only the one was giving us problems!"
Me, I'm looking at the package and seeing opportunities for cleaning up extra variables, reducing the number of containers in half by moving File System Tasks from one container to another and deleting the excess, stripping out code and variables that were built in the initial development and not used. Also, I'm seeing 7 procs where excrutiating RBAR WHILE loops are slowing down processing. Granted, 6 of those procs were not having issues because they are usually working on files that are pretty small. But that doesn't mean they won't cause a problem in the future. After all, the one problem we did have wasn't a problem when this process was first put into place.
And being fair, this package was the first major ETL process designed by my coworker. Coworker had a huge learning curve not only for SSIS but the 7+ files that were being sent from a different system. Using the code I got from the forums here, he redesigned one file into a separate package that worked effectively. The other package (with the other files) were left alone.
But still... To my mind, since I'm already fixing one part of the package, why shouldn't I proactively fix the rest of it to make sure the other 6 procs and ETL paths do NOT become a problem down the road?
Or am I just being too efficient?
I do the same.
Small bug in a T-SQL script?
Whoops, I reformatted all of the code and did some performance tuning 😀
I'm the same way. Just understand that it IS the co-worker's work and since everything was rewritten, that co-worker is feeling seriously wounded.
And that PC stuff is a skill that I haven't yet mastered. "Yeah, your stuff sucked, so I used the opportunity and my expertise to give it a tune-up."
I wouldn't say it sucked. It was a damn good first try. The process had been in production for almost 2 years before it started having issues. Then our boss told us it needed to be optimized. We were both working on other projects at the time and this fell to the bottom of the list. I just happened to get to it before coworker did.
But yes, hurt feelings I get. I've had my boss call me over to his desk because my code forgot something or did something it wasn't supposed to do.
Considering that a process used to take 8-12 hours and now takes less than 5 minutes they shouldn't really feel too hurt. Sure it a blow to the ego but look at the learning opportunity for the future. My first reaction would likely be to ask to see what you changed to make it so much faster. Then again I am not at all diplomatic and have skin thicker than an alligator. 😉
_______________________________________________________________
Need help? Help us help you.
Read the article at http://www.sqlservercentral.com/articles/Best+Practices/61537/ for best practices on asking questions.
Need to split a string? Try Jeff Modens splitter http://www.sqlservercentral.com/articles/Tally+Table/72993/.
Cross Tabs and Pivots, Part 1 – Converting Rows to Columns - http://www.sqlservercentral.com/articles/T-SQL/63681/
Cross Tabs and Pivots, Part 2 - Dynamic Cross Tabs - http://www.sqlservercentral.com/articles/Crosstab/65048/
Understanding and Using APPLY (Part 1) - http://www.sqlservercentral.com/articles/APPLY/69953/
Understanding and Using APPLY (Part 2) - http://www.sqlservercentral.com/articles/APPLY/69954/
December 17, 2014 at 7:26 am
Grant Fritchey (12/17/2014)
WayneS (12/17/2014)
Jeff Moden (12/17/2014)
Koen Verbeeck (12/17/2014)
Brandie Tarvin (12/17/2014)
Brandie Tarvin (12/16/2014)
WHOO! #HappySnoopyDanceI just spent 2 & 1/2 days reworking a package and its supporting tables / stored procedures. Previous time of execution between 8 - 12 hours.
Current execution time ... < 5 minutes.
"Celebration Time, come on!"
Sigh. So I presented coworker with my changes and asked for a peer review. Partially to make sure I didn't miss anything, partially to give coworker a learning opportunity since coworker was the one who designed the package. There were 7 files / queries involved and lots of tables. The response I got when I told coworker was: "You redesigned all the queries? But only the one was giving us problems!"
Me, I'm looking at the package and seeing opportunities for cleaning up extra variables, reducing the number of containers in half by moving File System Tasks from one container to another and deleting the excess, stripping out code and variables that were built in the initial development and not used. Also, I'm seeing 7 procs where excrutiating RBAR WHILE loops are slowing down processing. Granted, 6 of those procs were not having issues because they are usually working on files that are pretty small. But that doesn't mean they won't cause a problem in the future. After all, the one problem we did have wasn't a problem when this process was first put into place.
And being fair, this package was the first major ETL process designed by my coworker. Coworker had a huge learning curve not only for SSIS but the 7+ files that were being sent from a different system. Using the code I got from the forums here, he redesigned one file into a separate package that worked effectively. The other package (with the other files) were left alone.
But still... To my mind, since I'm already fixing one part of the package, why shouldn't I proactively fix the rest of it to make sure the other 6 procs and ETL paths do NOT become a problem down the road?
Or am I just being too efficient?
I do the same.
Small bug in a T-SQL script?
Whoops, I reformatted all of the code and did some performance tuning 😀
I'm the same way. Just understand that it IS the co-worker's work and since everything was rewritten, that co-worker is feeling seriously wounded.
And that PC stuff is a skill that I haven't yet mastered. "Yeah, your stuff sucked, so I used the opportunity and my expertise to give it a tune-up."
Heck, I'm more diplomatic than this, and that's saying something.
But I agree Brandie, you didn't do anything wrong.
+1 to not doing anything wrong, but I'm not very diplomatic either. The technical reasons for fixing this are clear. On the human side of the equation, if you don't take the opportunity to fix and use it as a teaching moment, how can you expect the junior person to learn?
December 17, 2014 at 7:30 am
Ed Wagner (12/17/2014)
Koen Verbeeck (12/17/2014)
Brandie Tarvin (12/17/2014)
Brandie Tarvin (12/16/2014)
WHOO! #HappySnoopyDanceI just spent 2 & 1/2 days reworking a package and its supporting tables / stored procedures. Previous time of execution between 8 - 12 hours.
Current execution time ... < 5 minutes.
"Celebration Time, come on!"
Sigh. So I presented coworker with my changes and asked for a peer review. Partially to make sure I didn't miss anything, partially to give coworker a learning opportunity since coworker was the one who designed the package. There were 7 files / queries involved and lots of tables. The response I got when I told coworker was: "You redesigned all the queries? But only the one was giving us problems!"
Me, I'm looking at the package and seeing opportunities for cleaning up extra variables, reducing the number of containers in half by moving File System Tasks from one container to another and deleting the excess, stripping out code and variables that were built in the initial development and not used. Also, I'm seeing 7 procs where excrutiating RBAR WHILE loops are slowing down processing. Granted, 6 of those procs were not having issues because they are usually working on files that are pretty small. But that doesn't mean they won't cause a problem in the future. After all, the one problem we did have wasn't a problem when this process was first put into place.
And being fair, this package was the first major ETL process designed by my coworker. Coworker had a huge learning curve not only for SSIS but the 7+ files that were being sent from a different system. Using the code I got from the forums here, he redesigned one file into a separate package that worked effectively. The other package (with the other files) were left alone.
But still... To my mind, since I'm already fixing one part of the package, why shouldn't I proactively fix the rest of it to make sure the other 6 procs and ETL paths do NOT become a problem down the road?
Or am I just being too efficient?
I do the same.
Small bug in a T-SQL script?
Whoops, I reformatted all of the code and did some performance tuning 😀
+1. Isn't that normal? When you have a problematic process, you should consider the whole process and tune. Making it handle larger sets more efficiently is always a good thing. So it has a small set now...that doesn't mean they won't get bigger. In general, data tends to grow over time, so why not design and code properly in the first place?
IMHO - yes. I tell my clients you have to look at the whole picture and not just one problem with tunnel vision.
Jason...AKA CirqueDeSQLeil
_______________________________________________
I have given a name to my pain...MCM SQL Server, MVP
SQL RNNR
Posting Performance Based Questions - Gail Shaw[/url]
Learn Extended Events
December 17, 2014 at 8:39 am
Jeff Moden (12/16/2014)
Writeup? I was thinking the same thing. It would have to be a "what I found" type of article rather than an "SME" type of article because I didn't know a thing about it before I did the testing.
What I found would be good. Those are good learning things. People see how to test something, look at a solution, get results, etc. as well as learning about this.
December 17, 2014 at 8:44 am
So with only 4 1/2 days of current contract I'm finally have fun trying to solve a problem.
So nice little deadline there. Was a straightforward request but now that I've asked questions and supplied data we have realised the requirements are more involved - Aren't they always.
😀
Rodders...
December 17, 2014 at 9:23 am
rodjkidd (12/17/2014)
So with only 4 1/2 days of current contract I'm finally have fun trying to solve a problem.So nice little deadline there. Was a straightforward request but now that I've asked questions and supplied data we have realised the requirements are more involved - Aren't they always.
And then, after you leave, everyone there will blame you for it not getting done correctly. Even though they should have known about the problems and the time constraints. @=)
December 17, 2014 at 9:32 am
Brandie Tarvin (12/17/2014)
rodjkidd (12/17/2014)
So with only 4 1/2 days of current contract I'm finally have fun trying to solve a problem.So nice little deadline there. Was a straightforward request but now that I've asked questions and supplied data we have realised the requirements are more involved - Aren't they always.
And then, after you leave, everyone there will blame you for it not getting done correctly. Even though they should have known about the problems and the time constraints. @=)
Ha ha Brandie quite! Come Jan 5th it will all be "Its Rodney fault" 🙂
We now have an hour long conference call booked to discuss handover and current issues. Oh how I love conference calls that last longer than 10 minutes!
To be fair, having got to the bottom of the actual question (insert own Hitch Hikers Guide joke here) it's actually a lot harder to answer than just 42 🙂 Not quite 7 million years, but longer than the 2 weeks I was given - which also included building the SSIS to do the extract, get the firewall permissions sorted out for sftp and use the "new" way of doing sftp...
Rodders...
December 17, 2014 at 12:01 pm
Ed Wagner (12/17/2014)
Grant Fritchey (12/17/2014)
WayneS (12/17/2014)
Jeff Moden (12/17/2014)
Koen Verbeeck (12/17/2014)
Brandie Tarvin (12/17/2014)
Brandie Tarvin (12/16/2014)
WHOO! #HappySnoopyDanceI just spent 2 & 1/2 days reworking a package and its supporting tables / stored procedures. Previous time of execution between 8 - 12 hours.
Current execution time ... < 5 minutes.
"Celebration Time, come on!"
Sigh. So I presented coworker with my changes and asked for a peer review. Partially to make sure I didn't miss anything, partially to give coworker a learning opportunity since coworker was the one who designed the package. There were 7 files / queries involved and lots of tables. The response I got when I told coworker was: "You redesigned all the queries? But only the one was giving us problems!"
Me, I'm looking at the package and seeing opportunities for cleaning up extra variables, reducing the number of containers in half by moving File System Tasks from one container to another and deleting the excess, stripping out code and variables that were built in the initial development and not used. Also, I'm seeing 7 procs where excrutiating RBAR WHILE loops are slowing down processing. Granted, 6 of those procs were not having issues because they are usually working on files that are pretty small. But that doesn't mean they won't cause a problem in the future. After all, the one problem we did have wasn't a problem when this process was first put into place.
And being fair, this package was the first major ETL process designed by my coworker. Coworker had a huge learning curve not only for SSIS but the 7+ files that were being sent from a different system. Using the code I got from the forums here, he redesigned one file into a separate package that worked effectively. The other package (with the other files) were left alone.
But still... To my mind, since I'm already fixing one part of the package, why shouldn't I proactively fix the rest of it to make sure the other 6 procs and ETL paths do NOT become a problem down the road?
Or am I just being too efficient?
I do the same.
Small bug in a T-SQL script?
Whoops, I reformatted all of the code and did some performance tuning 😀
I'm the same way. Just understand that it IS the co-worker's work and since everything was rewritten, that co-worker is feeling seriously wounded.
And that PC stuff is a skill that I haven't yet mastered. "Yeah, your stuff sucked, so I used the opportunity and my expertise to give it a tune-up."
Heck, I'm more diplomatic than this, and that's saying something.
But I agree Brandie, you didn't do anything wrong.
+1 to not doing anything wrong, but I'm not very diplomatic either. The technical reasons for fixing this are clear. On the human side of the equation, if you don't take the opportunity to fix and use it as a teaching moment, how can you expect the junior person to learn?
That might be the best answer (i.e. make it a teaching moment). I've had this exact scenario go seriously sideways (on the human side). Nothing like getting dragged into your manager's office to have it out as to why you "sabotaged" johnny's code.
When you find myself rewriting the whole thing, it might be worth stopping what you're doing, going to find these folks and pull them into the conversation DURING the rewrite. Bonus points if you can pull them aside and do it somewhat discreetly (not in front of an audience). Triple bonus points if you can guide them to make the fixes themselves (again - a huge "face-saving" tactic). If it doesn't go well - you can always escalate the issue and get air cover before you replace their work.
----------------------------------------------------------------------------------
Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?
December 18, 2014 at 12:35 am
... Mark one off, 72 days on the calendar to go. 72 days on the calendar to go, 72 days to go, ...
December 18, 2014 at 4:52 am
So, I'm curious. Anyone have any major projects that people want before the end of the year?
We have a business user that keeps asking for X fix, which I already have coded actually. But our SDLC requires the QA team to do thorough testing and none of them have the bandwidth until mid-January, which makes this a February release. None of which stops the BU from asking me if I can put the code into production NOW PLEASE.
What's worse is that he should know better given what impatience has done to his processes before. But some lessons just don't seem to stick.
December 18, 2014 at 4:55 am
Brandie Tarvin (12/18/2014)
So, I'm curious. Anyone have any major projects that people want before the end of the year?We have a business user that keeps asking for X fix, which I already have coded actually. But our SDLC requires the QA team to do thorough testing and none of them have the bandwidth until mid-January, which makes this a February release. None of which stops the BU from asking me if I can put the code into production NOW PLEASE.
What's worse is that he should know better given what impatience has done to his processes before. But some lessons just don't seem to stick.
Did have some. One has now been postponed until Feb 1st. Mainly due to the responses of "Are you Mad! A Jan 1st go live date!!"
The other requires a DBA (thankfully an Oracle one) to login at midnight 31st December to apply changes so everything is nice and shiny for the Jan 1st.
Rodders...
December 18, 2014 at 5:22 am
Brandie Tarvin (12/18/2014)
So, I'm curious. Anyone have any major projects that people want before the end of the year?We have a business user that keeps asking for X fix, which I already have coded actually. But our SDLC requires the QA team to do thorough testing and none of them have the bandwidth until mid-January, which makes this a February release. None of which stops the BU from asking me if I can put the code into production NOW PLEASE.
What's worse is that he should know better given what impatience has done to his processes before. But some lessons just don't seem to stick.
We have production release today and I'm leaving for holiday in an hour 🙂
December 18, 2014 at 5:31 am
Brandie Tarvin (12/18/2014)
So, I'm curious. Anyone have any major projects that people want before the end of the year?
Yes. The complete reverse engineer and rewrite of a crop modelling system, wanted by 6th Jan. Entertainment for next week = 1600 line long module (front end, not SQL) with no comments, erratic indentation and no naming standard.
Ah well, disappointment is good for people.
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
December 18, 2014 at 5:53 am
GilaMonster (12/18/2014)
Entertainment for next week = 1600 line long module (front end, not SQL) with no comments, erratic indentation and no naming standard.
Oh! Joy 🙂
Sounds like one of my old programs 😛
Far away is close at hand in the images of elsewhere.
Anon.
December 18, 2014 at 6:13 am
David Burrows (12/18/2014)
GilaMonster (12/18/2014)
Entertainment for next week = 1600 line long module (front end, not SQL) with no comments, erratic indentation and no naming standard.Oh! Joy 🙂
Sounds like one of my old programs 😛
Phew, I thought for a second I wrote it!
How to post a question to get the most help http://www.sqlservercentral.com/articles/Best+Practices/61537
Viewing 15 posts - 46,636 through 46,650 (of 66,549 total)
You must be logged in to reply to this topic. Login to reply