ETL Antipattern: Ignore the Logging
In my last ETL Antipatterns post, I wrote about the unexciting but very necessary work of documenting ETL processes. The logging of ETL operations is just as (un)captivating as...
2020-12-21
6 reads
Tim Mitchell
- Interests: SQL Server, Data Warehousing, ETL, Data Architecture, Python, Dbt
- Website: https://www.TimMitchell.net
- Bluesky: tmitch.net
ETL Antipattern: Skipping The Documentation
Documentation is an asset that is both loathed and loved. Creating technical and business documentation is often looked upon as a tedious chore, something that really ought to be...
2020-12-20
14 reads
ETL Antipattern: Skipping The Documentation
Documentation is an asset that is both loathed and loved. Creating technical and business documentation is often looked upon as a tedious chore, something that really ought to be...
2020-12-20
6 reads
ETL Antipattern: Load Processes that Don’t Scale
One of the most significant design considerations in ETL process development is the volume of data to be processed. Most ETL processes have time constraints that require them to...
2020-12-19
261 reads
ETL Antipattern: Load Processes that Don’t Scale
One of the most significant design considerations in ETL process development is the volume of data to be processed. Most ETL processes have time constraints that require them to...
2020-12-19
6 reads
ETL Antipattern: Failure to Test and Validate
“If it compiles, it works.” – An unemployed developer Building ETL processes is quite easy. Building ETL processes that deliver accurate results as quickly as possible is substantially more...
2020-12-18
15 reads
ETL Antipattern: Failure to Test and Validate
“If it compiles, it works.” – An unemployed developer Building ETL processes is quite easy. Building ETL processes that deliver accurate results as quickly as possible is substantially more...
2020-12-18
7 reads
ETL Antipattern: Failing to Treat ETL Logic as Source Code
In most data projects, building the extract-transform-load (ETL) logic takes a significant amount of time. Enterprise ETL processes must do several things well: retrieve enough data to satisfy the...
2020-12-17
21 reads
ETL Antipattern: Failing to Treat ETL Logic as Source Code
In most data projects, building the extract-transform-load (ETL) logic takes a significant amount of time. Enterprise ETL processes must do several things well: retrieve enough data to satisfy the...
2020-12-17
11 reads
ETL Antipattern: No Error Handling Logic
I usually avoid talking about technology in absolutes, but here’s one that I can share without reservation: On a long enough timeline, every single ETL process will eventually fail....
2020-12-16
15 reads
Blogs
How to Connect to SQL Server When Nothing Else Works – DAC
By gbargsley
It's 2 AM. Your phone is going off. Users can't connect to the application,...
Get a Range of Sequence Values: #SQLNewBlogger
By Steve Jones
I discovered a procedure recently that I wasn’t aware of: sp_sequence_get_range. This post looks...
The Agent Era: When “How do I…?” Replaces “Where do I click?”
By Arun Sirpal
After a year away getting to grips with AI and its application across the...
Forums
Hiring: Financial SQL Systems Administrator
Fisher Phillips is looking for a Financial Systems Administrator to help support and improve our financial...
Job Opening for ERP Systems Administrator - Hybrid
Employee owned company, been around for over 50 years. Hybrid opportunity, looking folks in Pacific...
data compare between 2 tables with same structure for any changes after etl run
i have huge table with lot of data and is also wide. i took...
Question of the Day
Creating a JSON Document II
I want to create a JSON document that contains data from this table:
TeamID TeamNameCity YearEstablished 1 Cowboys Dallas 1960 2 Eagles Philadelphia 1933 3 Packers Green Bay 1919 4 Chiefs Kansas City 1960If I run this code, what document(s) is/are returned?
SELECT json_objectagg( n.city : n.TeamName) FROM dbo.NFLTeams;See possible answers