Provisioning ADF - Level 2 of the Stairway to Azure Data Factory
In this next level of the Stairway to Azure Data Factory, learn how to build your first ADF environment.
2020-01-15 (first published: 2019-09-04)
4,184 reads
ETL (Extract Transform and Load)
Data Transfer Strategies between MongoDB and SQL Server
There are a variety of days to move data between MongoDB and SQL Server. This article covers some of your options and gives you ideas on which method might work best for you.
2019-07-22
13,655 reads
Preventing Invalid Data Early In An ETL Flow
In this post, Tim Smith will look at some of the considerations for both data rules and logic to prevent invalid data early in the ETRL process.
2018-05-02
3,324 reads
3 Solutions For Auditing Long SQL Server Extracts or Loads
In this post, Tim Smith looks at the different options you can use to audit your SQL Server extracts and loads during the ETL process.
2017-07-04
3,755 reads
Bash for ETL pre-processing
People were doing ETL long before ETL packages were invented. Some of those facilities still have worth today
2016-08-15
5,606 reads
Simplify SQL Server ETL Processes with T-SQL Stored Procedures
Jeremy Kadlec introduces the Enhanced Data Integration Suite (EDIS), a SQL Server T-SQL stored procedure based solution to automate, manage, and audit ETL processes so you no longer have to manually build SSIS Packages.
2016-02-03
6,164 reads
How to get ETL Horribly Wrong
ETL ( Extract, transform, load) doesn't have to be like a spell on hell. To make a success of ETL systems, you need the freedom and ability to make graceful U-turns when you detect a mistake in architecture or configuration: to fix the root problem rather than to merely tackle the symptoms. Feodor lists the eight most common root causes of failure in ETL systems, and how to fix them.
2015-01-30
13,243 reads
Microsoft's Log Parser Utility: Swell ETL
For loading text, CSV or XML files into SQL Server, the Log Parser utility, with its amazing SQL engine, is likely to be the obvious choice. Although initially developed purely for converting IIS logs, the Log Parser can turn its hand to a range of formats including even event logs or the Windows registry.
2013-11-27
3,565 reads
Microsoft's Log Parser Utility: Swell ETL
For loading text, CSV or XML files into SQL Server, the Log Parser utility, with its amazing SQL engine, is likely to be the obvious choice. Although initially developed purely for converting IIS logs, the Log Parser can turn its hand to a range of formats including even event logs or the Windows registry.
2013-11-20
2,923 reads
The ETL from Hell - Diagnosing Batch System Performance Issues
Too often, the batch systems that underlie a lot of database processing just grow without conscious design. When runs start to extend beyond their allotted time, and tuning no longer solves the problem, it is often discovered that batches are run in series, with draconian error handling. It is time to impose some rational design, and Nigel is a seasoned healer of batch processes.
2013-10-30
3,753 reads
Blogs
Friday Basics: the CIA Triad
By Brian Kelley
In information security (INFOSEC), there several foundational concepts and principles. One of the ones...
A New Word: the standard blues
By Steve Jones
the standard blues– n. the dispiriting awareness that the twists and turns of your...
How Redgate Flyway Can Boost Your DevOps Journey
By Chris Yates
A brief introduction to the tool and its advantages for database migrations DevOps is...
Forums
client_app_name is empty in Extended Events output but present in sp_who2
I'm tracing activity on one database and would like to include the client_app_name in...
How to compare data in customer table with other customers to find related cust
select Custno, Addr1, City, Res_Phone, Bus_Phone, Fax_Phone, Marine_Phone, Pager_Phone, Other_Phone, email1, email2 from customer...
process records in loop
I'm only processing 50,000 records not everything from the Table where there are 250,00...
Question of the Day
The Marked Transaction
I want to mark a transaction in the log as a recovery point. How do I do this in my code if I use the transaction, myTran?
See possible answers