It is a non-null constraint, not a non-ish-null constraint You are writing an ETL process, part of this process you need to import a semi-structured file (think CSV, JSON,...
2019-10-28
137 reads
It has been a little while but I have updated SQLCover to include a number of fixes and small features, the majority of which are improvements to the html...
2019-10-30 (first published: 2019-10-16)
365 reads
This is the final part in the four-part series into testing ETL pipelines, how exciting!
Part 1 - Unit Testing https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ Part 2 - Integration Testing https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/ Part 3...
2019-10-02
9 reads
“[Error] [JvmBridge] java.sql.SQLException: No suitable driver” - unable to connect spark to Microsoft SQL Server.
In spark when you want to connect to a database you use Read() passing in...
2019-10-01
6 reads
In the first two parts of this series (https://the.agilesql.club/2019/07/how-do-we-test-etl-pipelines-part-one-unit-tests/ and https://the.agilesql.club/2019/08/how-do-we-prove-our-etl-processes-are-correct-how-do-we-make-sure-upstream-changes-dont-break-our-processes-and-break-our-beautiful-data/), I talked about how to unit test your business logic and integration test your ETL infrastructure code. Having...
2019-09-30
2 reads
I finally got around to updating the tSQLt test adapter for visual studio, you can download it from:
https://marketplace.visualstudio.com/items?itemName=vs-publisher-263684.GoEddietSQLt2019
or the search in visual studio extensions thingy finds it as well.
For...
2019-09-02
19 reads
Steps needed Getting Apache Spark running on windows involves:
Installing a JRE 8 (Java 1.8/OpenJDK 8) Downloading and extracting SPARK and setting SPARK_HOME Downloading winutils.exe and setting HADOOP_HOME If...
2019-08-29
13 reads
When you run an application using spark-dotnet, to launch the application you need to use spark-submit to start a java virtual machine which starts the spark-dotnet driver which then...
2019-08-20
10 reads
ETL Testing Part 2 - Operational Data Testing
This is the second part of a series on ETL testing, the first part explained about unit testing, and in this part,...
2019-08-13
188 reads
I found this question on stack overflow that went something like this:
“I have a file that includes line endings in the wrong place and I need to parse the...
2019-08-06
5 reads
By Steve Jones
According to this post, I rate 2019 as a Failure. Of the books I...
By Kamil
Introduction Tomaž Kastrun is BI & DEV developer, data analyst & data science. With...
By Steve Jones
I love SQL Prompt, and I’m regularly impressed by the enhancements our teams continue...
Comments posted to this topic are about the item Automatic Redaction of PII
I have a table with a varchar field. It contains a number of characters...
I'm creating a table to store serialized shopping cart data associated with a user...
Ask SSC