Delta Lake over Spark Connect
All Spark Connect Posts
I have just finished an update for the spark connect dotnet lib that contains the DeltaTable implementation so that we can now use .NET to maintain...
2025-06-20
11 reads
All Spark Connect Posts
I have just finished an update for the spark connect dotnet lib that contains the DeltaTable implementation so that we can now use .NET to maintain...
2025-06-20
11 reads
I recently encountered an interesting issue with ADF where the publish feature suddenly attempted to republish every single object, claiming they were new, despite having incrementally published changed objects...
2025-07-02 (first published: 2025-06-15)
180 reads
All Spark Connect Posts
Introduction There have been quite a few changes in the last couple of months and I just wanted to give a quick update on the current...
2024-11-16
16 reads
All Spark Connect Posts
I recently published the latest version of the Spark Connect Dotnet library which includes support for the new Variant data type in Apache Spark 4.0 here....
2024-10-12
15 reads
All Spark Connect Posts
Code
What to do? At some point we will want to do something with delta lake and so I wanted to explore the options. Before we do...
2024-02-17
3 reads
All Spark Connect Posts
Code
Goal of this post The goal of this post is to look at creating a SparkSession and a DataFrame that will wrap the Range relation and...
2024-02-10
35 reads
All Spark Connect Posts
Code
Goal of this post I wanted to explore what the Spark Connect API looked like from other languages, I am not a php developer - I...
2024-02-10
85 reads
All Spark Connect Posts
Code
Goal of this post So there are two goals of this post, the first is to take a look at Apache Arrow and how we can...
2024-02-05
71 reads
All Spark Connect Posts
Code
Goal of this post In this post we will continue looking at the gRPC API and the AnalyzePlan method which takes a plan and analyzes it....
2024-02-01
103 reads
All Spark Connect Posts
Code
Goal of this post In the first two posts, we looked at how to run some Spark code, firstly against a local Spark Connect server and...
2024-01-26
283 reads
By Brian Kelley
If you're not having success in convincing your organization to send you to this...
By Rohit Garg
When deploying SQL Server in enterprise environments, choosing the right service account model is...
By Brian Kelley
There are always bad actors who will seek to get access to and, likely,...
Comments posted to this topic are about the item Determining the Updated Columns
Comments posted to this topic are about the item Carrots and Sticks
I am trying to create and Update query based on a like join. Is...
I am creating a trigger in SQL Server 2022 and want to detect a change in a table. Here is my table DDL:
CREATE TABLE CustomerLarge ( CustomerID INT NOT NULL IDENTITY(1,1) CONSTRAINT CustomerLargePK PRIMARY KEY CLUSTERED , CustomerName VARCHAR(20) , CustomerContactFirstName VARCHAR(40) , CustomerContactLastName VARCHAR(40) , Address VARCHAR(20) , Address2 VARCHAR(20) , City VARCHAR(20) , CountryCode CHAR(3) , Postal VARCHAR(20) , creditlimit INT , discount numeric(4,2) , lastorderdate DATETIME , lastorderamount NUMERIC(10,2) , lastordercontact VARCHAR(20) , created DATETIME , modified DATETIME , modifiedby VARCHAR(20) , statusid INT , active BIT , customersize INT , primarysalesid INT) GOIf I want to detect that the creditlimit was updated, what IF statements should I use? See possible answers