It is a non-null constraint, not a non-ish-null constraint You are writing an ETL process, part of this process you need to import a semi-structured file (think CSV, JSON,...
2019-10-28
133 reads
It has been a little while but I have updated SQLCover to include a number of fixes and small features, the majority of which are improvements to the html...
2019-10-30 (first published: 2019-10-16)
352 reads
This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...
2019-10-02
6 reads
“[Error] [JvmBridge] java.sql.SQLException: No suitable driver” - unable to connect spark to Microsoft SQL Server.
In spark when you want to connect to a database you use Read() passing in...
2019-10-01
1 reads
In the first two parts of this series (https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ and https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/), I talked about how to unit test your business logic and integration test your ETL infrastructure code. Having...
2019-09-30
2 reads
I finally got around to updating the tSQLt test adapter for visual studio, you can download it from:
https://marketplace.visualstudio.com/items?itemName=vs-publisher-263684.GoEddietSQLt2019
or the search in visual studio extensions thingy finds it as well.
For...
2019-09-02
16 reads
Steps needed Getting Apache Spark running on windows involves:
Installing a JRE 8 (Java 1.8/OpenJDK 8) Downloading and extracting SPARK and setting SPARK_HOME Downloading winutils.exe and setting HADOOP_HOME If...
2019-08-29
11 reads
When you run an application using spark-dotnet, to launch the application you need to use spark-submit to start a java virtual machine which starts the spark-dotnet driver which then...
2019-08-20
10 reads
ETL Testing Part 2 - Operational Data Testing
This is the second part of a series on ETL testing, the first part explained about unit testing, and in this part,...
2019-08-13
186 reads
I found this question on stack overflow that went something like this:
“I have a file that includes line endings in the wrong place and I need to parse the...
2019-08-06
5 reads
By Rayis Imayev
(2019-Dec-15) While working on data integration projects and using Azure Data Factory as your main...
By Tim Mitchell
I’m a creature of habit. With the software tools that I use, I like...
By Gerard
Don’t know about you, but one of my least favourite data pipeline errors is...
Hi All, In server 2012 and 2016 , SQl error logs show this error...
Hi Everyone, I have a database (~50GB in size) with content that is constantly...
Comments posted to this topic are about the item Updating Statistics
Ask SSC