How Effective Is Your Data Load Monitoring in SQL and Python?
Learn how you can use monitor your data warehouse load process with Python code and a structured process.
2023-10-06 (first published: 2023-09-20)
2,016 reads
Extract, Transform, Load (ETL)
The Art of Data Ingestion - Part 1
Data movement is a fundamental piece of a data engineer’s duties, and recently I’ve been thinking about the art of data movement. What are some of the most important pieces that a data engineer needs to think about when confronted with data ingestion? There is of course data exporting as well, and in that case, […]
2022-11-11
6,077 reads
2021-04-16
552 reads
Function to find non-printable characters
The function is used to find non-printable ASCII characters in an input string.
2009-05-21 (first published: 2009-04-24)
2,884 reads
Kimball ETL Part 1 – Data Profiling via SSIS Data Flow
A new series sponsored by Actuality Business Intelligence on data warehousing. In part 1, the data flow in SSIS packages are used to profile the source data and determine how it should be handled in the process.
2008-07-23
9,922 reads
ETL for Free-Form Data
This article demonstrates how to extract, transform and load free-form data into a structured form like a staging table by way of XML conversions.
2008-03-20
5,575 reads
2008-01-22
2,966 reads
2007-12-10
3,004 reads
Importing Text-based data: Workbench
Robyn and Phil return with some fresh ideas about how to import text files into SQL Server, without resorting to DTS or SSIS scripting. They go on to show how much can be done in TSQL
2007-11-19
4,235 reads
Parallel Processing of Large Volume ETL Jobs
ETL processing, generally involves copying/moving, transforming, cleaning the records/transactions from one or multiple sources. Most of the batch processing or warehousing projects involve such data processing in millions on daily/weekly basis. Typically, there is a Staging area and production area. Records are cleaned, transformed, filtered and verified from staging to production area. This demands SQL Set theory based queries, parallel processing with multiple processors/CPU. The article focuses on need of SQL Set theory approach and parallel processing while processing large volume of ETL records using programming approach.
2007-11-08
7,104 reads
Blogs
Why your data still can’t answer a simple question
Every organization I talk to has the same problem dressed up in different clothes....
T-SQL Tuesday #197 Invitation – An impactful session or two from a conference
By DataOnWheels
I am delighted to host this month’s T-SQL Tuesday invitation. If you are new...
Did You Really Name That Default?
By alevyinroc
Ten years (and a couple jobs) ago, I wrote about naming default constraints to...
Forums
The day-to-day pressures of a DBA team, and how we can work smarter with automation and AI
Comments posted to this topic are about the item The day-to-day pressures of a...
Using OPENJSON
Comments posted to this topic are about the item Using OPENJSON
Data Modeling with dbt for Visual Code: The Fabric Modern Data Platform
Comments posted to this topic are about the item Data Modeling with dbt for...
Question of the Day
Using OPENJSON
I have some data in a table that looks like this:
BeerID BeerName brewer beerdescription 1 Becks Interbrew Beck's is a German-style pilsner beer 2 Fat Tire New Belgium Toasty malt, gentle sweetness, flash of fresh hop bitterness. 3 Mac n Jacks Mac & Jack's Brewery This beer erupts with a floral, hoppy taste 4 Alaskan Amber Alaskan Brewing Alaskan Brewing Amber Ale is an "alt" style beer 8 Kirin Kirin Brewing Kirin Ichiban is a Lager-type beerIf I run this, what is returned?
select t1.key
from openjson((select t.* FROM Beer AS t for json path)) t1 See possible answers