Apache Spark from PHP - it is not just a .NET thing
All Spark Connect Posts
Code
Goal of this post I wanted to explore what the Spark Connect API looked like from other languages, I am not a php developer - I...
2024-02-10
13 reads
All Spark Connect Posts
Code
Goal of this post I wanted to explore what the Spark Connect API looked like from other languages, I am not a php developer - I...
2024-02-10
13 reads
All Spark Connect Posts
Code
Goal of this post The goal of this post is to look at creating a SparkSession and a DataFrame that will wrap the Range relation and...
2024-02-10
15 reads
All Spark Connect Posts
Code
Goal of this post So there are two goals of this post, the first is to take a look at Apache Arrow and how we can...
2024-02-05
30 reads
All Spark Connect Posts
Code
Goal of this post In this post we will continue looking at the gRPC API and the AnalyzePlan method which takes a plan and analyzes it....
2024-02-01
26 reads
Introductory Ramble Spark Connect In July 2022, at the Data and AI summit, Apache Spark announced “Spark Connect,” which was a way of connecting to Apache Spark using the...
2024-01-31 (first published: 2024-01-17)
319 reads
All Spark Connect Posts
Code
Goal of this post In the first two posts, we looked at how to run some Spark code, firstly against a local Spark Connect server and...
2024-01-26
84 reads
All Spark Connect Posts
Goal of this post This post aims to show how we can create a .NET application, deploy it to Databricks, and then run a Databricks job...
2024-01-24 (first published: 2024-01-23)
21 reads
Spark Connect References https://spark.apache.org/docs/latest/spark-connect-overview.html
https://www.databricks.com/blog/2022/07/07/introducing-spark-connect-the-power-of-apache-spark-everywhere.html
Reference Implementations Rust
GOLang
Source gRPC proto files https://github.com/apache/spark/tree/master/connector/connect/common/src/main/protobuf/spark/connect
Useful Videos If this doesn’t get you excited then nothing will it is, of course, suitable for work, it is...
2024-01-01
11 reads
I made a mistake recently when I was creating an ADF pipeline, annoyingly I made loads of changes and then clicked the debug button, when I pressed debug the...
2021-03-17 (first published: 2021-03-12)
760 reads
I made a mistake recently when I was creating an ADF pipeline, annoyingly I made loads of changes and then clicked the debug button, when I pressed debug the...
2021-03-12
7 reads
By Will Assaf
I recently reviewed, worked on, and added a similar example to the DATETIMEOFFSET Microsoft...
Database administrators (DBAs) are the backbone of data-driven organizations. If you're looking to break...
By Brian Kelley
Another security fundamentals topic is authentication versus authorization. For those who have a clear...
Comments posted to this topic are about the item Ingest and Retrieve data from...
Comments posted to this topic are about the item The DBAccountant
I have a SQL Server 2019 Enterprise Edition on CU 25. It has in-memory...
I run this command in SQL Server 2022. What is the extension of the backup file?
BACKUP DATABASE HerdofTwo TO DISK = 'HerdOfTwo_20240501'See possible answers