Delta Lake over Spark Connect
All Spark Connect Posts
I have just finished an update for the spark connect dotnet lib that contains the DeltaTable implementation so that we can now use .NET to maintain...
2025-06-20
15 reads
Ed Elliott's Sql Developer Blog
ADF: Publish suddenly includes everything where it used to be incremental changes since the last publish
I recently encountered an interesting issue with ADF where the publish feature suddenly attempted to republish every single object, claiming they were new, despite having incrementally published changed objects...
2025-07-02 (first published: 2025-06-15)
192 reads
Spark Connect Dotnet November 2024 Where are we?
All Spark Connect Posts
Introduction There have been quite a few changes in the last couple of months and I just wanted to give a quick update on the current...
2024-11-16
46 reads
Spark Connect Dotnet Variant Data Type
All Spark Connect Posts
I recently published the latest version of the Spark Connect Dotnet library which includes support for the new Variant data type in Apache Spark 4.0 here....
2024-10-12
16 reads
Delta Lake from .NET in a Spark Connect gRPC world
All Spark Connect Posts
Code
What to do? At some point we will want to do something with delta lake and so I wanted to explore the options. Before we do...
2024-02-17
5 reads
Implementing functions and more fun in Spark Connect using gRPC and .NET
All Spark Connect Posts
Code
Goal of this post The goal of this post is to look at creating a SparkSession and a DataFrame that will wrap the Range relation and...
2024-02-10
36 reads
Apache Spark from PHP - it is not just a .NET thing
All Spark Connect Posts
Code
Goal of this post I wanted to explore what the Spark Connect API looked like from other languages, I am not a php developer - I...
2024-02-10
95 reads
Moving towards the DataFrame API using the Spark Connect gRPC API in .NET
All Spark Connect Posts
Code
Goal of this post So there are two goals of this post, the first is to take a look at Apache Arrow and how we can...
2024-02-05
74 reads
Exploring the Spark Connect gRPC API more
All Spark Connect Posts
Code
Goal of this post In this post we will continue looking at the gRPC API and the AnalyzePlan method which takes a plan and analyzes it....
2024-02-01
108 reads
Exploring the Spark Connect gRPC API
All Spark Connect Posts
Code
Goal of this post In the first two posts, we looked at how to run some Spark code, firstly against a local Spark Connect server and...
2024-01-26
307 reads
Blogs
Inspecting TSQL Abstract Syntax Trees
By Zikato
TSQL ScriptDOM is a useful library for parsing scripts into an abstract syntax tree....
Becoming a creator, my website experience
By DataOnWheels
Over the past few months, I have debated starting a new blog to discuss...
T-SQL Tuesday #192 Invitation: SQL Server 2025 Excitement
By Steve Jones
It’s that time of the month again, and once again, I’m late and I’m...
Forums
Do You Really Need HA?
Comments posted to this topic are about the item Do You Really Need HA?
The New Database CheckDB
Comments posted to this topic are about the item The New Database CheckDB
MySQL’s BLACKHOLE Storage Engine: The Odd Feature I Wish SQL Server Shipped With
Comments posted to this topic are about the item MySQL’s BLACKHOLE Storage Engine: The...
Question of the Day
The New Database CheckDB
I run this code:
create database experiment
go
use Experiment
go
select DATABASEPROPERTYEX('Experiment', 'LastGoodCheckDbTime')
What is returned? See possible answers