Tim Mitchell is a business intelligence consultant, author, trainer, and SQL Server MVP with over a decade of data management experience. He is a partner with Linchpin People and is the principal of Tyleris Data Solutions.
Tim has spoken at international and local events including the SQL PASS Summit, SQLBits, SQL Connections, along with dozens of tech fests, code camps, and SQL Saturday events. He is coauthor of the book SSIS Design Patterns, and is a contributing author on MVP Deep Dives 2.
During data load or transformation processes, capturing a distinct row number for incoming data can be beneficial for the ETL process itself, as well as for use in the destination database. Having an arbitrary, incrementing row number assigned to each row can help to determine the order in which the… Read more
Now that the details have been finalized, I can announce that a new SSIS class coming up this fall. My Linchpin People cohort Andy Leonard and I will be delivering our Advanced SSIS training class in the Washington, DC area in December. This class is four days in length, ideally… Read more
Temporary Post Used For Theme Detection (d2df4373-a605-4b61-86ee-a1afe06d7e22 – 3bfe001a-32de-4114-a6b4-4005b770f6d7)
This is a temporary post that was not deleted. Please delete this manually. (af8cc091-a628-437f-8918-348486ad1d80 – 3bfe001a-32de-4114-a6b4-4005b770f6d7)
Through the course of my 8-someodd years of building and fixing ETL processes, I’ve had the opportunity to see a lot of ETL code. Some of that code was really good, well-thought-out and carefully executed. Other load processes were – well, let’s just say that they provide plenty of consulting… Read more
If you’ve read my blog or attended one of my presentations before, you almost certainly know that I’m a huge fan of Biml. If you’re building or maintaining SSIS packages (or you’re supervising those that do), you owe it to yourself to learn about Biml. I don’t make a lot… Read more
If you’re reading this, we have done business together, either directly or indirectly. You might be a retail website from which I purchase products or services. You could be a social media empire on which I share blog posts, life events, and… Read more
In my most recent post in this series, I talked about how to use the SSIS object variable as an ADO recordset as a source in a data flow. By loading the result set of a query into this variable, the contents of the variable can be read by an… Read more
Everybody wants to be liked, to some extent. Being disliked by others feels like a deficiency, a shortcoming in oneself that must be fixed. It feels like a failure that you’ve let someone down, or even worse, it can make you feel like you’re somehow broken or deficient.
I would… Read more
Object variables in SSIS are incredibly versatile, allowing the storage of almost any type of data (even .NET objects). In my last post on this topic, I demonstrated how an SSIS object variable containing a .NET DataSet object could be used by the for each loop container as an… Read more
Some time back I wrote about how to use the script component to parse out ragged data files in SSIS. In this post, I’ll continue the discussion to describe how to handle mixed format data files – specifically, those with several different record types in each file.
In a… Read more
When I think about Baltimore, I think about Edgar Allan Poe. Baltimore was Poe’s one-time home, and in fact there is a museum located in a home in which he once lived. I’ve been a Poe fan since high school, and am looking forward to seeing some of the… Read more
I’ll be honest: I used to be afraid of being the person described above. I didn’t want to be known as someone whose ideas didn’t work. And to that end, I was successful: most of what I tried was successful. The… Read more
The answer: It Depends.
One of the more common problems I encounter when managing data quality, especially in an ETL process, is the handling of null values, blanks, and zeroes. When I put on my preaching shoes to talk about bad data, this is one the areas I have to… Read more
This week, PASS announced the Best of PASS Summit 2014, a collection of the 10 highest-rated sessions from last fall’s PASS Summit in Seattle. I was pleasantly surprised to learn that my session, “Building Bullet-Resistant SSIS Packages”, was among these. All of these sessions (including lots of other content)… Read more
Earlier today, the fine folks at the F. Oobar Corporation released a revolutionary product: a universal data integration utility. This software component, known as the Baseline Ongoing Generic Utility for Synergy, will run on any platform and can convert data to and from almost any format automatically. It also reads… Read more
Yesterday I read an article entitled “Why I Don’t Want to Have Coffee With You”, in which the author writes that he doesn’t have the time or the desire to simply “have coffee”. While I empathize with some of the author’s justifications for his position, I was disappointed… Read more
How many times have you said to yourself, “Someone should build an application that does [x]…”, or “Wouldn’t it be easy to add automation to [y]”, or “It would be a lot of fun to work on a project to build [z]”? For me, this has happened a lot, and… Read more
There is a flat file processing issue I’ve run into a number of times over the years, and it’s come up again several times recently. The issue relates to the line terminators used in data files. Occasionally, changes to the systems generating these data files, or perhaps even manual edits,… Read more
Wrangling large or complex Excel workbooks in SSIS can be a challenge. From managing data types (more about that in this post by Koen Verbeeck) to addressing multiple worksheets in a single document, configuring SSIS to properly read from or write to Excel documents is tedious at best. While there… Read more
I’m not a great skier. It would probably be a stretch to say that I’m a good skier. Still, I enjoy doing it, and I want to (and can) get better at it. Since I live in… Read more