Passing status messages and results back from Databricks to ADF
When we use ADF to call Databricks we can pass parameters, nice. When we finish running the Databricks notebook we often want to return something back to ADF so...
2020-02-05
13 reads
Ed Elliott's Sql Developer Blog
Spark Delta Lake, Updates, Deletes and Time Travel
When you use delta lake there are a couple of interesting things to note based around the fact that the data is stored in parquet files which are read-only...
2020-01-20
19 reads
Spark Delta Lake, Updates, Deletes and Time Travel
When you use delta lake there are a couple of interesting things to note based around the fact that the data is stored in parquet files which are read-only...
2020-01-20
3 reads
Validating upstream data quality in ETL processes, SQL edition
It is a non-null constraint, not a non-ish-null constraint You are writing an ETL process, part of this process you need to import a semi-structured file (think CSV, JSON,...
2019-10-28
177 reads
Validating upstream data quality in ETL processes, SQL edition
It is a non-null constraint, not a non-ish-null constraint You are writing an ETL process, part of this process you need to import a semi-structured file (think CSV, JSON,...
2019-10-28
8 reads
SQLCover 0.5 - Fixes, smaller features and an exciting surprise
It has been a little while but I have updated SQLCover to include a number of fixes and small features, the majority of which are improvements to the html...
2019-10-30 (first published: 2019-10-16)
411 reads
SQLCover 0.5 - Fixes, smaller features and an exciting surprise
It has been a little while but I have updated SQLCover to include a number of fixes and small features, the majority of which are improvements to the html...
2019-10-16
12 reads
How to test ETL Processes in production
This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...
2019-10-02
132 reads
How to test ETL Processes in production
This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...
2019-10-02
8 reads
How to connect spark to ms sql server without '[Error] [JvmBridge] java.sql.SQLException: No suitable driver'?
“[Error] [JvmBridge] java.sql.SQLException: No suitable driver” - unable to connect spark to Microsoft SQL Server.
In spark when you want to connect to a database you use Read() passing in...
2019-10-01
426 reads
Blogs
No Shortcuts for the SQLCMD Batch Terminator: #SQLNewBlogger
By Steve Jones
I was messing around with SQLCMD and I realized something I hadn’t known. I’ve...
Where Is My SQL Agent? Running Scheduled Jobs Against Azure SQL Database
By gbargsley
One of the first things I review when I inherit a new SQL Server...
AgentDBA vs Critical SQL Server
By Arun Sirpal
It’s 07:43. Someone’s already left a message. “Something’s wrong with the DB server.” You...
Forums
Increment a number in a SQL Query based on a value
I have an issue where I have a Bill of Material list of items...
BCA KCU THAMRIN | Tlp/Wa:0817866887
Tlp/Wa_Cs:0817-866-887. Menara BCA, Grand Indonesia, Jl. M.H. Thamrin No.1, RT.1/RW.5, Menteng, Kec. Menteng, Kota...
Alamat kontak resmi BCA KCP Tebet Telp:0818751777
WhatsApp:0818-751-777 Gedung Gajah Tebet, Jl. Dr. Saharjo No.Raya 111 Unit N & O, RT.1/RW.1,...
Question of the Day
Detecting Characters
I have a SQL Server 2022 English default installation on a server. I want to detect if there are any upper case characters in rows and I have this code:
SELECT CustomerNameID,
CustomerName
FROM dbo.CustomerName
WHERE CustomerName = LOWER(CustomerName)
Here is the sample data I am testing with:
CustomerNameID CustomerName 1 John Smith 2 Sarah Johnson 3 MICHAEL WILLIAMS 4 JENNIFER BROWN 5 david jones 6 emily davis 7 Robert Miller 8 LISA WILSON 9 christopher moore 10 Amanda TaylorHow many rows are returned? See possible answers