January 22, 2011 at 7:55 am
Within a development team I'm in the minority as the SQL Server BI/Database architect. Most of the others are hard-core C#/.NET developers.
We're currently working on a data warehouse solution and I've tried to demonstrate the benefits of SSIS 2008 by getting a 7x performance improvement on some of our current data warehouse loaders.
However, I'm getting violent resistance to the use of the BIDS plugin to VS2008 by the other developers on the grounds of:
Usability:
- Copy and paste functions. The position of the paste is random.
- Containment hierachy is othen lost when the parent container is resized.
- VB expression boxes too small
Coding encapsulation:
- Reuse is via cut and paste hard to make reusable components without writing SSIS pipeline extension classes and adding them to the toolbox
Development:
- Impossible to find refences to things eg show all components that reference a variable
- Unable to debug Script component within a dataflow task
- Change control and see the coding differences within a package (and the C# code in script task/components)
- Merging knightmare on development and supoort change control branching
Has anybody else come across issues like these and how you overcame BIDS?
(I have some sympathy since BIDS is a poor-relation when compared to the functionality in VS2008)
January 23, 2011 at 8:09 pm
I would have to agree with your developers about these issues. They are problems that I have but with no workaround/solution.
Perhaps you could consider stepping back and looking at the requirements you have for an ETL tool (which could be a C# custom written application or its equivalent SSIS packages). These requirements probably include things like performance, maintainability etc. Try to quantify/weight each of these and see our each solution stacks up against each other.
And there are some tools out there that do some level of MS BI solutions (BI Documentor is one of them). They may help with some of the issues (but not all of them).
January 26, 2011 at 2:23 pm
If you feel that the usability of SSIS BIDS is an issue then please vote for the enhancements within connect.microsoft.com
Feedback on SQL Server (public forum)
Topic: Usability of SSIS BIDS deters people from using SSIS
Then may be something might be done.
January 27, 2011 at 2:06 pm
jm99 (1/22/2011)
Within a development team I'm in the minority as the SQL Server BI/Database architect. Most of the others are hard-core C#/.NET developers.We're currently working on a data warehouse solution and I've tried to demonstrate the benefits of SSIS 2008 by getting a 7x performance improvement on some of our current data warehouse loaders.
However, I'm getting violent resistance to the use of the BIDS plugin to VS2008 by the other developers on the grounds of:
Usability:
- Copy and paste functions. The position of the paste is random.
- Containment hierachy is othen lost when the parent container is resized.
- VB expression boxes too small
Coding encapsulation:
- Reuse is via cut and paste hard to make reusable components without writing SSIS pipeline extension classes and adding them to the toolbox
Development:
- Impossible to find refences to things eg show all components that reference a variable
- Unable to debug Script component within a dataflow task
- Change control and see the coding differences within a package (and the C# code in script task/components)
- Merging knightmare on development and supoort change control branching
Has anybody else come across issues like these and how you overcame BIDS?
(I have some sympathy since BIDS is a poor-relation when compared to the functionality in VS2008)
These are indeed known issues. But as they are hard-code developers, they are looking at BIDS and SSIS as an application development tool.
It is not. It is an Enterprise ETL tool. It requires another way of thinking and working.
The reasons why some of the traditional "team coding stuff" doesn't work in SSIS, is because it is actually an XML file, but the slightest change in layout can give great changes in the XML, making it hard to merge.
There is still a lot to improve in SSIS in terms of usability, but the advantages usually outweigh those minor setbacks:
* graphical development
* streamlined pipeline to move data extremely fast
* integrated logging, package configurations and event handling
* extremely extensible with .NET (although IMO, this is more of a last resort, not a standard practice)
* source control is possible, just not with merge 🙂 (more of a check in - check out schema)
Remember, it is a graphical tool (aka small building blocks connected together), not a coding tool. They are used to working with a hammer, so they see every problem as a nail.
Tell your colleagues I will take them on :-D:
give both of us a real ETL requirement, and I'll bet I can create the package faster in SSIS than they can code a solution with all the same functionalities.
And I'm pretty confident the package will run faster too 🙂
Need an answer? No, you need a question
My blog at https://sqlkover.com.
MCSE Business Intelligence - Microsoft Data Platform MVP
January 27, 2011 at 3:34 pm
I'd take that challenge too. Even a simple truncate and load could be generated in a couple minutes without rushing in SSIS, I don't think that you could do it that quick in .net and even if you could it would be incredibly rushed.
CEWII
February 5, 2011 at 9:36 am
jm99 -your development team is not wrong in expressing 'Volient Resistance'. They are not the only ones. You only have to google "hate" and "SSIS" and then loose the rest of your day (dare I say week?) trying to read through the litany of disasters relating to BIDS/SSIS as it fails in most of its goals to be an effective enterprise develoment/business tool. There is even an I hate SSIS Facebook group... I empathise with their pain, but you shouldn't be surprised at the 'violent resistance'. I experienced the same pressures as you but have managed to overcome them mostly over time.
You say you have demonstrated the benefits of a 7x perf improvement but you face a challenge of successfully convincing a hostile team if you seek to win over their confidence in BIDS/SSIS if you solely concentrate on performance as your argument. That is not enough when one considers how poor the human factors are when trying to use this tool.
Yes indeed big improvements can be made using SSIS if the correct training, patience, design, strategy and 'nurturing' from you - their architect. There are many factors in developing data warehouse solutions which outweigh pure performance as the sole argument -Lets consider: usability, discoverabilty, resilience to change (especially to db schema changes) , refactoring and maintenance, debugging to mention just a few.
The tool is not well suited to some modern dev techniques, most notaby the ability to fragment the dtsx package within BIDS and poor support for change control and comparison of changes. This itself is almost a deal breaker for some dev teams.
Congrats on raising the feedback on connect.microsoft.com I will be voting for improvement in all these area as SSIS.
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply