The Problem When you use a UDF in .NET for Apache Spark, something like this code:
class Program { static void Main(string args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select");...
2020-10-29
98 reads
The Problem When you use a UDF in .NET for Apache Spark, something like this code:
class Program { static void Main(string args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select");...
2020-10-29
2 reads
Spark ML in .NET for Apache Spark Spark is awesome, .NET is awesome, machine learning (ML) is awesome, so what could be better than using .NET to write ML...
2020-07-07
35 reads
Spark ML in .NET for Apache Spark Spark is awesome, .NET is awesome, machine learning (ML) is awesome, so what could be better than using .NET to write ML...
2020-07-07
5 reads
Last Updated: 2020-10-18
NOTE: What you read here was before .NET for Apache Spark 1.0 which includes everything we need to do this purely in .NET - in this post...
2020-07-07
16 reads
Databricks is fantastic, but there is a small issue with how people use it. The problem is that Databricks is all things to all people. Data scientists and data...
2020-06-03
190 reads
Databricks is fantastic, but there is a small issue with how people use it. The problem is that Databricks is all things to all people. Data scientists and data...
2020-06-03
1 reads
Every ETL pipeline is only ever as reliable as the data that the upstream system provides. It is inevitable that assumptions you make about the data you are provided...
2020-03-06
12 reads
Every ETL pipeline is only ever as reliable as the data that the upstream system provides. It is inevitable that assumptions you make about the data you are provided...
2020-03-06
2 reads
2020-02-21
3 reads
In the modern data-centric landscape, SQL stands out as the dominant tool. It's essential...
By Steve Jones
Thanks to everyone that came to my talks. Slides are below. Best Practices for...
Hi, We are using database snapshot together with changetracking to keep a datawarehouse up...
Hi everyone, For several weeks now we have issues with an SSIS package connected...
Comments posted to this topic are about the item The Load of Real Time...
In the current (2024) version of the Data Migration Assistant, which on-premises targets are available?
See possible answers