Synapse Analytics Spark Example 1 - Group By
I have been playing around with the new Azure Synapse Analytics, and I realised that this is an excellent opportunity for people to move to Apache Spark. Synapse Analytics...
2021-01-18
47 reads
Ed Elliott
Synapse Analytics Spark Example 2 - ROW_NUMBER()
The next example is how to do a ROW_NUMBER(), my favourite window function.
T-SQL SELECT *, ROW_NUMBER() OVER(ORDER BY dateTime) as RowNumber FROM chicago.safety_data Spark SQL SELECT *, ROW_NUMBER() OVER(ORDER...
2021-01-18
422 reads
How to get data in a DataFrame via .NET for Apache Spark
When I first started working with Apache Spark, one of the things I struggled with was that I would have some variable or data in my code that I...
2021-01-17
46 reads
How to get data in a DataFrame via .NET for Apache Spark
When I first started working with Apache Spark, one of the things I struggled with was that I would have some variable or data in my code that I...
2021-01-17
7 reads
Git in 10 commands to do 99% of gitting and keeping out of trouble
Git is hard, probably harder than it needs to be but I have been using it for about 5 years and have a workflow that works for me. I...
2021-01-12
4 reads
TF-IDF in .NET for Apache Spark Using Spark ML v2
Spark ML in .NET for Apache Spark Apache Spark has had a machine learning API for quite some time and this has been partially implemented in .NET for Apache...
2020-12-19
75 reads
TF-IDF in .NET for Apache Spark Using Spark ML v2
Spark ML in .NET for Apache Spark Apache Spark has had a machine learning API for quite some time and this has been partially implemented in .NET for Apache...
2020-12-19
8 reads
Blog .NET for Apache Spark UDFs Missing Shared State
The Problem When you use a UDF in .NET for Apache Spark, something like this code:
class Program { static void Main(string args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select");...
2020-10-29
102 reads
Blog .NET for Apache Spark UDFs Missing Shared State
The Problem When you use a UDF in .NET for Apache Spark, something like this code:
class Program { static void Main(string args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select");...
2020-10-29
5 reads
Spark .NET - Spark ML TF-IDF
Spark ML in .NET for Apache Spark Spark is awesome, .NET is awesome, machine learning (ML) is awesome, so what could be better than using .NET to write ML...
2020-07-07
43 reads
Blogs
The Benefits of Teaching
By Brian Kelley
Even preparing for a class or seminar with set materials takes a lot of...
Power BI, Excel, OneLake – Dreams Do Come True!
By DataOnWheels
I can’t believe it’s finally here! A way to have Excel live in OneDrive...
The Book of Redgate: What Our Staff Says
By Steve Jones
This image is from 2010, and it goes along with my last post of...
Forums
Unlocking Data Transformation: My journey with dbt (Data Build Tool) on SQL Server
Comments posted to this topic are about the item Unlocking Data Transformation: My journey...
Putting the Player with the Number
Comments posted to this topic are about the item Putting the Player with the...
Your Security Checkup
Comments posted to this topic are about the item Your Security Checkup, which is...
Question of the Day
Putting the Player with the Number
In SQL Server 2025, what does this return?
DECLARE @player varchar(20) = 'Bo Nix', @num VARCHAR = '10' SELECT @player || @numSee possible answers