Implementing ETL Logging on Lakehouse using Delta Lake's Time Travel capability
In this article I demonstrate how to use Delta Lake's time travel functionality for ETL logging along with steps to take to implement.
2022-07-01
Azure Data Lake
Dynamically Add a Timestamp To Files in Azure Data Factory
This article will describe how to add your local timestamp at the end of the each file in Azure Data Factory (ADF). In general, ADF gets a UTC timestamp, so we need to convert the timestamp from UTC to EST, since our local time zone is EST. For example, if the input Source file name […]
2021-04-22
13,087 reads
Processing Data Using Azure Data Lake, Azure Data Analytics, and Microsoft’s Big Data Language U-SQL
Data analytics has become one of the powerful domains in the world of data science. An enormous amount of data is being generated by each organization in every sector. Computer science has found solutions to store and process this data in a smart way through a distributed file system. One such example is Azure Data Lake. It uses the Hadoop Distributed File System, and to perform analytics on this data, Azure Data Lake storage is integrated with Azure Data Analytics Service and HDInsight. In this article, Suhas Pande will explain how to store data using Azure Data Lake and how to perform data analysis on it using U-SQL, a big data SQL and C# language.
2019-04-29
Processing Data Using Azure Data Lake, Azure Data Analytics, and Microsoft’s Big Data Language U-SQL
Data analytics has become one of the powerful domains in the world of data science. An enormous amount of data is being generated by each organization in every sector....
2019-04-09
Guide to Quickly Creating an Azure Data Lake Storage
Read a step by step guide to setting up an Azure Data Lake Storage account.
2021-02-05 (first published: 2019-03-19)
4,804 reads
Azure Data Lake Analytics using U-SQL Queries
This article will help with gaining confidence and familiarity with Microsoft Azure's Data Lake Analytics offering to process large datasets quickly while demonstrating the potential and capabilities of U-SQL to aggregate and process big data files.
2019-02-20
2,259 reads
Update an Azure SQL Data Mart with ADLS files
In this tip we learn how to update the Azure SQL Data Mart in an automated way.
2019-01-01
1,758 reads
Overview of Azure Data Lake
Azure Data Lake stores petabytes of data and analyzes trillions of objects in one place with no constraints. Data Lake Store can store any type of data including massive data like high-resolution video, medical data, and data from a wide variety of industries. Data Lake Store scales throughput to support any size of analytic workload with low latency. Read on to learn more.
2017-10-20
3,483 reads
Stairway to U-SQL Level 4: U-SQL and Data Lake Tooling with Visual Studio
A guide to using Visual Studio to create U-SQL projects and scripts.
2016-07-20
2,907 reads
Blogs
Scary Scalar Functions - Part Two: Performance
By Zikato
Scary Scalar Functions series overview Part One: Parallelism Part Two: Performance Foreword In the second part of this...
Goal Progress for June 2022
By Steve Jones
I set goals at the beginning of the year, and I’m tracking my progress...
Using Sessionize for Your User Group – Part 2
Now that your CFS has been setup it’s time to get the word out...
Forums
The July Fourth Anniversary
Comments posted to this topic are about the item The July Fourth Anniversary
Independence Day 2022
Comments posted to this topic are about the item Independence Day 2022
Employee Hierarchy Recursive CTE Different Top Level Directors Overtime
Hi, I have looked at some examples of Employee > Manager Hierarchy using Recursive...
Ask SSC