Tim Mitchell

Moving On From PASS. What’s Next?

• by Tim Mitchell

• Tim Mitchell

The COVID-19 global pandemic was by far the biggest influencer of 2020. This virus has cost over 1.8 million lives worldwide, and has upended normal life and commerce for...

2021-01-12

29 reads

Connecting to SharePoint Lists with SSIS

• by Tim Mitchell

• Tim Mitchell

SharePoint lists are popular and simple tools for storing modestly-sized discrete sets of data. SQL Server Integration Services (SSIS) includes a source component to allow reading data from SharePoint...

2021-01-15 (first published: 2021-01-09)

378 reads

ETL Antipattern: Ignoring the “Why?”

• by Tim Mitchell

• Tim Mitchell

In this final post in my ETL Antipatterns series, I’ll talk about a common big-picture mistake when building ETL processes: ignoring the fundamental question, “why are we doing this?”...

2020-12-22

ETL Antipattern: Ignore the Logging

• by Tim Mitchell

• Tim Mitchell

In my last ETL Antipatterns post, I wrote about the unexciting but very necessary work of documenting ETL processes. The logging of ETL operations is just as (un)captivating as...

2020-12-21

1 reads

ETL Antipattern: Skipping The Documentation

• by Tim Mitchell

• Tim Mitchell

Documentation is an asset that is both loathed and loved. Creating technical and business documentation is often looked upon as a tedious chore, something that really ought to be...

2020-12-20

ETL Antipattern: Load Processes that Don’t Scale

• by Tim Mitchell

• Tim Mitchell

One of the most significant design considerations in ETL process development is the volume of data to be processed. Most ETL processes have time constraints that require them to...

2020-12-19

230 reads

ETL Antipattern: Failure to Test and Validate

• by Tim Mitchell

• Tim Mitchell

“If it compiles, it works.” – An unemployed developer Building ETL processes is quite easy. Building ETL processes that deliver accurate results as quickly as possible is substantially more...

2020-12-18

1 reads

ETL Antipattern: Failing to Treat ETL Logic as Source Code

• by Tim Mitchell

• Tim Mitchell

In most data projects, building the extract-transform-load (ETL) logic takes a significant amount of time. Enterprise ETL processes must do several things well: retrieve enough data to satisfy the...

2020-12-17

2 reads

ETL Antipattern: No Error Handling Logic

• by Tim Mitchell

• Tim Mitchell

I usually avoid talking about technology in absolutes, but here’s one that I can share without reservation: On a long enough timeline, every single ETL process will eventually fail....

2020-12-28 (first published: 2020-12-16)

426 reads

ETL Antipattern: Lazy Metadata

• by Tim Mitchell

• Tim Mitchell

If data is a train, then metadata is the track on which it travels. A good metadata definition in ETL processes will help to ensure that the flow of...

2020-12-15

4 reads