How do we test ETL pipelines? Part one unit tests
Why do we bother testing? Testing isn’t an easy thing to define, we all know we should do it, when something goes wrong in production people shout and ask...
2019-07-25
44 reads
Ed Elliott
How do we test ETL pipelines? Part one unit tests
Why do we bother testing? Testing isn’t an easy thing to define, we all know we should do it, when something goes wrong in production people shout and ask...
2019-07-25
spark-dotnet examples - reading and writing csv files
How do you read and write CSV files using the dotnet driver for Apache Spark?
I have a runnable example here:
https://github.com/GoEddie/dotnet-spark-examples
Specifcally:
https://github.com/GoEddie/dotnet-spark-examples/tree/master/examples/split-csv
Let’s take a walkthrough of the demo:
Console.WriteLine("Hello Spark!"); var spark...
2019-07-15
32 reads
spark-dotnet examples - reading and writing csv files
How do you read and write CSV files using the dotnet driver for Apache Spark?
I have a runnable example here:
https://github.com/GoEddie/dotnet-spark-examples
Specifcally:
https://github.com/GoEddie/dotnet-spark-examples/tree/master/examples/split-csv
Let's take a walkthrough of the demo:
Console.WriteLine("Hello Spark!"); var spark...
2019-07-15
2 reads
spark-dotnet how does user .net code run spark code in a java vm
Apache Spark is written in scala, scala compiles to Java and runs inside a Java virtual machine. The spark-dotnet driver runs dotnet code and calls spark functionality, so how...
2019-07-14
7 reads
spark-dotnet how does user .net code run spark code in a java vm
Apache Spark is written in scala, scala compiles to Java and runs inside a Java virtual machine. The spark-dotnet driver runs dotnet code and calls spark functionality, so how...
2019-07-14
3 reads
Spark and dotnet in a single docker container
I really like the new dotnet driver for Spark because I think it makes spark more accesable to devs who might not know pythpn or scala.
If you want to...
2019-07-11
14 reads
Spark and dotnet in a single docker container
I really like the new dotnet driver for Spark because I think it makes spark more accesable to devs who might not know pythpn or scala.
If you want to...
2019-07-11
Where do you do your database development? Hopefully not production
Here are three scenarios, if you work with SQL Server either as a provider of database environments (DBA) or a consumer of database environments for your application (developer) then...
2019-07-05
3 reads
Where do you do your database development? Hopefully not production
Here are three scenarios, if you work with SQL Server either as a provider of database environments (DBA) or a consumer of database environments for your application (developer) then...
2019-07-05
1 reads
Blogs
A New Word: Lookaback
By Steve Jones
lookaback– n. the chock of meeting back up with someone and learning that your...
Azure Vulnerability Assessments
Regarding managing databases in the cloud, security should never be an afterthought. That’s where...
Speaking at the NYC Lunch and Learn–May 23
By Steve Jones
In a couple of weeks, I’ll be in New York City for the Redgate...
Forums
Deleting a Database
Comments posted to this topic are about the item Deleting a Database
Data Analyst AI tool
Comments posted to this topic are about the item Data Analyst AI tool
Removing Log Shipping
Comments posted to this topic are about the item Removing Log Shipping
Question of the Day
Removing Log Shipping
If I want to delete a log shipping secondary server where only one database is used to restore the logs, what should I do?
See possible answers