How to get data in a DataFrame via .NET for Apache Spark
When I first started working with Apache Spark, one of the things I struggled with was that I would have some variable or data in my code that I...
2021-01-17
2 reads
Ed Elliott
Git in 10 commands to do 99% of gitting and keeping out of trouble
Git is hard, probably harder than it needs to be but I have been using it for about 5 years and have a workflow that works for me. I...
2021-01-12
3 reads
TF-IDF in .NET for Apache Spark Using Spark ML v2
Spark ML in .NET for Apache Spark Apache Spark has had a machine learning API for quite some time and this has been partially implemented in .NET for Apache...
2020-12-19
75 reads
TF-IDF in .NET for Apache Spark Using Spark ML v2
Spark ML in .NET for Apache Spark Apache Spark has had a machine learning API for quite some time and this has been partially implemented in .NET for Apache...
2020-12-19
3 reads
Blog .NET for Apache Spark UDFs Missing Shared State
The Problem When you use a UDF in .NET for Apache Spark, something like this code:
class Program { static void Main(string args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select");...
2020-10-29
98 reads
Blog .NET for Apache Spark UDFs Missing Shared State
The Problem When you use a UDF in .NET for Apache Spark, something like this code:
class Program { static void Main(string args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select");...
2020-10-29
2 reads
Spark .NET - Spark ML TF-IDF
Spark ML in .NET for Apache Spark Spark is awesome, .NET is awesome, machine learning (ML) is awesome, so what could be better than using .NET to write ML...
2020-07-07
40 reads
TF-IDF in .NET for Apache Spark Using Spark ML
Spark ML in .NET for Apache Spark Spark is awesome, .NET is awesome, machine learning (ML) is awesome, so what could be better than using .NET to write ML...
2020-07-07
8 reads
TF-IDF in .NET for Apache Spark Using Spark ML
Last Updated: 2020-10-18
NOTE: What you read here was before .NET for Apache Spark 1.0 which includes everything we need to do this purely in .NET - in this post...
2020-07-07
17 reads
Approaches to running Databricks ETL code from Azure ADF
Databricks is fantastic, but there is a small issue with how people use it. The problem is that Databricks is all things to all people. Data scientists and data...
2020-06-03
192 reads
Blogs
The trade-offs associated with low-code solutions
Low-code solutions often accelerate development and make tasks accessible to people who can’t or...
Monday Monitor Tips: Looking Back in Time
By Steve Jones
Often we find out about a problem reported by a customer after the incident...
Can You Tell Why a Plan Didn’t Go Parallel?
Let’s face it, most of the time, you probably don’t want your SQL Server...
Forums
Is there a way to determine an Azure Data Factory Trigger's "next run time"?
Many job schedulers (e.g., SQL Agent, Airflow, Azure Logic Apps) can report when a...
The Table Backup
Comments posted to this topic are about the item The Table Backup
Dynamic T-SQL Script Parameterization Using Python
Comments posted to this topic are about the item Dynamic T-SQL Script Parameterization Using...
Question of the Day
The Table Backup
I run this code in SQL Server 2022. What happens?
BACKUP TABLE beercount TO DISK = 'beercount.bak'See possible answers