This is the third article of our series discussing SQL Server 2005 Integration Services (SSIS), which provides Extraction, Transformation, and Loading features, replacing Data Transformation Services (DTS) available in SQL Server 2000 and 7.0. We have already presented an overview of the basic concepts necessary to design and implement SSIS-based projects. We have also stepped through the creation of one such project using Business Intelligence Development Studio. Our sample SSIS package, described in our previous article, delivered the basic functionality we needed (running an external process and loading the outcome of its execution stored temporarily in a text file into a database), however, it had one major shortcoming - a lack of support for reusability.
In the first article of this series, we presented the basic concepts relevant to understanding SQL Server 2005 Integration Services, which is the primary mechanism for Extraction, Transformation, and Loading (ETL) functionality available in SQL Server 2005. We also provided a quick overview of tools that allow you to manage such activities as DTS package design, development, and storage, as well as interactive and scheduled execution. In order to gain better familiarity with each of these activities, we will look at them from a more practical perspective, using a number of fairly straightforward examples.
One of the SQL Server 2005 Beta 2 features that has undergone considerable architectural, functional, and interface changes, compared with its earlier versions, deals with extraction, transformation, and loading (also known as simply ETL) of data. (In other words, allowing you to move data between a variety of stores and to modify it in the process, according to your arbitrary requirements). While in SQL Server 7.0 and 2000, such capabilities have been provided by Data Transformation Services (DTS in short), Microsoft decided that the degree of modifications introduced in the new version justified re-branding it, hence its rearchitectured and significantly improved implementation has been introduced under the new name of SQL Server Integration Services (SSIS).
The Data Trasformation Services are a powerful tool, and sometime its features are so useful that you’d like to invoke a DTS package not only from SQL Server but from an external program.
To do this you have several choices: you can use the DTSRun.exe tool or you can do it leveraging the SQL-DMO features.
Unfortunately if you’re developing a web application (ASP, ASP.Net or whatever you use) none of them seems to be the right choice: too much problems, too much effort and a very modest results. In addition none of these solutions can be called asynchronously: if you just need to implement a “fire-and-forget” technique, you just cannot do that!
In Project REAL we are using large volumes of real data and scenarios from real companies to implement business intelligence systems using early releases of Microsoft SQL Server 2005. In the process, best practices are being developed and potential problems uncovered. This article reports some of the lessons learned while working on the data extraction, transformation, and loading (ETL) portion of the first phase of Project REAL.
SQL Server 2005 will truly introduce a new paradigm in how developers and DBAs work with SQL Server. One area in which dramatic changes have taken place is DTS, which is now renamed to SQL Server Integration Services or SSIS. Expert author Jamie Thomson brings us a look at how the fundamentals of workflow change from DTS to SSIS.