Passing status messages and results back from Databricks to ADF
When we use ADF to call Databricks we can pass parameters, nice. When we finish running the Databricks notebook we often want to return something back to ADF so...
2020-02-05
22 reads
Ed Elliott's Sql Developer Blog
Spark Delta Lake, Updates, Deletes and Time Travel
When you use delta lake there are a couple of interesting things to note based around the fact that the data is stored in parquet files which are read-only...
2020-01-20
26 reads
Spark Delta Lake, Updates, Deletes and Time Travel
When you use delta lake there are a couple of interesting things to note based around the fact that the data is stored in parquet files which are read-only...
2020-01-20
10 reads
Validating upstream data quality in ETL processes, SQL edition
It is a non-null constraint, not a non-ish-null constraint You are writing an ETL process, part of this process you need to import a semi-structured file (think CSV, JSON,...
2019-10-28
182 reads
Validating upstream data quality in ETL processes, SQL edition
It is a non-null constraint, not a non-ish-null constraint You are writing an ETL process, part of this process you need to import a semi-structured file (think CSV, JSON,...
2019-10-28
15 reads
SQLCover 0.5 - Fixes, smaller features and an exciting surprise
It has been a little while but I have updated SQLCover to include a number of fixes and small features, the majority of which are improvements to the html...
2019-10-30 (first published: 2019-10-16)
421 reads
SQLCover 0.5 - Fixes, smaller features and an exciting surprise
It has been a little while but I have updated SQLCover to include a number of fixes and small features, the majority of which are improvements to the html...
2019-10-16
14 reads
How to test ETL Processes in production
This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...
2019-10-02
143 reads
How to test ETL Processes in production
This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...
2019-10-02
14 reads
How to connect spark to ms sql server without '[Error] [JvmBridge] java.sql.SQLException: No suitable driver'?
“[Error] [JvmBridge] java.sql.SQLException: No suitable driver” - unable to connect spark to Microsoft SQL Server.
In spark when you want to connect to a database you use Read() passing in...
2019-10-01
433 reads
Blogs
Announcements from the Microsoft Fabric Community Conference — Barcelona 2026
By James Serra
Microsoft shared a broad set of Fabric, Power BI, and SQL updates at FabCon...
Au Revoir for now
By Steve Jones
No, I’m not quitting or retiring. Just going on vacation, but I leave tonight...
Forums
Let's Talk Certifications
Comments posted to this topic are about the item Let's Talk Certifications, which is...
Force join by longest possible choice
Title pretty much says it. I want to join to a table that has...
Building a SQL Server Fleet Audit That Leadership Actually Reads
Comments posted to this topic are about the item Building a SQL Server Fleet...
Question of the Day
RegEx Functions IV
I have this data in a table in a SQL Server 2025 database:
EmailAddressID EmailAddress 7 dylan0@ADVENTURE-WORKS.COM 8 Diane1@ADVENTURE-WORKS.COMIf I run this query, which row(s) are returned?
SELECT top 10 * FROM person.EmailAddress WHERE REGEXP_LIKE(EmailAddress, '^D', 'i') AND BusinessEntityID IN (7,8)See possible answers