spark-dotnet how to manually break a file into rows and columns
I found this question on stack overflow that went something like this:
“I have a file that includes line endings in the wrong place and I need to parse the...
2019-08-06
1 reads
Ed Elliott's Sql Developer Blog
java.lang.ClassNotFoundException: org.apache.spark.deploy.DotnetRunner with 0.4.0 of spark-dotnet
There was a breaking change with version 0.4.0 that changed the name of the class that is used to load the dotnet driver in Apache Spark.
To fix the issue...
2019-08-02
230 reads
java.lang.ClassNotFoundException: org.apache.spark.deploy.DotnetRunner with 0.4.0 of spark-dotnet
There was a breaking change with version 0.4.0 that changed the name of the class that is used to load the dotnet driver in Apache Spark.
To fix the issue...
2019-08-02
5 reads
How do we test ETL pipelines? Part one unit tests
Why do we bother testing? Testing isn’t an easy thing to define, we all know we should do it, when something goes wrong in production people shout and ask...
2019-07-25
44 reads
How do we test ETL pipelines? Part one unit tests
Why do we bother testing? Testing isn’t an easy thing to define, we all know we should do it, when something goes wrong in production people shout and ask...
2019-07-25
2 reads
spark-dotnet examples - reading and writing csv files
How do you read and write CSV files using the dotnet driver for Apache Spark?
I have a runnable example here:
https://github.com/GoEddie/dotnet-spark-examples
Specifcally:
https://github.com/GoEddie/dotnet-spark-examples/tree/master/examples/split-csv
Let’s take a walkthrough of the demo:
Console.WriteLine("Hello Spark!"); var spark...
2019-07-15
33 reads
spark-dotnet examples - reading and writing csv files
How do you read and write CSV files using the dotnet driver for Apache Spark?
I have a runnable example here:
https://github.com/GoEddie/dotnet-spark-examples
Specifcally:
https://github.com/GoEddie/dotnet-spark-examples/tree/master/examples/split-csv
Let's take a walkthrough of the demo:
Console.WriteLine("Hello Spark!"); var spark...
2019-07-15
4 reads
spark-dotnet how does user .net code run spark code in a java vm
Apache Spark is written in scala, scala compiles to Java and runs inside a Java virtual machine. The spark-dotnet driver runs dotnet code and calls spark functionality, so how...
2019-07-14
7 reads
spark-dotnet how does user .net code run spark code in a java vm
Apache Spark is written in scala, scala compiles to Java and runs inside a Java virtual machine. The spark-dotnet driver runs dotnet code and calls spark functionality, so how...
2019-07-14
5 reads
Spark and dotnet in a single docker container
I really like the new dotnet driver for Spark because I think it makes spark more accesable to devs who might not know pythpn or scala.
If you want to...
2019-07-11
14 reads
Blogs
Running SQL Server on KubeVirt – Getting Started
With all the changes that have happened with VMware since the Broadcom acquisition I...
Why is shrinking IT budgets a good thing?
Each year around this time, companies enter the familiar ritual of budgeting. For many,...
Reverse Engineering a Physical Model Diagram
By Steve Jones
I recently wrote about a logical diagram with Redgate Data Modeler. That was interesting,...
Forums
Best analysis approach to reduce data type length in multiple tables
Hello SSC, I hope everyone is having a happy and safe holiday season! I...
Do you separate data/log files and TempDB files on different disks using AWS EC2
Hello, I am receiving multiple direction on deploying SQL Servers in AWS EC2. ...
Service account being in local admin in AWS EC2 environment?
Hello, I am working with a company on their database administration. I am trying...
Question of the Day
A New Operator
In SQL Server 2025, what does this code return?
SELECT '1' || '0'See possible answers