System Variables in ADF

How to Recover a Corrupted Azure Data Factory Integration Runtime

I would like to share my recent experience with Azure Data Factory (ADF) where AutoResolveIntegrationRuntime become corrupted and how did I recover it. I still don't know how the Integration Runtime (IR) was corrupted. However, if it happens, then this article will help you to solve the issue. Problem In general, the ADF AutoResolveIntegrationRuntime should […]

Incremental Data Loading Using ADF and Change Tracking

This article looks at moving data in three related tables to Azure SQL Database using the change tracking feature in SQL Server to determine which rows have been changed.

Deleting a Folder in Azure Data Factory

Working with the Delete Activity in Azure Data Factory

Introduction We can use the delete activity in Azure Data Factory to delete files from both on-premises and cloud storage. In this article, we will discuss the delete activity with the various options available for the file deletion. I will show the following operations with the Delete Activity: Delete files from a folder. Delete contents […]

How to Flatten JSON in Azure Data Factory?

When you work with ETL and the source file is JSON, many documents may get nested attributes in the JSON file. Your requirements will often dictate that you flatten those nested attributes. There are many ways you can flatten the JSON hierarchy, however; I am going to share my experiences with Azure Data Factory (ADF) […]

Azure Data Factory Incremental Pipeline from Azure SQL Database to Databricks

In this tip find out how to use Azure Data factory to build an incremental pipeline from Azure SQL Database to Databricks.

Working with Get Metadata Activity in Azure Data Factory

Learn how to gather metadata information about the files being processed in your ADF pipeline.

Merge Multiple Files in Azure Data Factory

A short tutorial on how to combine multiple files together in an Azure Data Factory Copy Data activity.

Why Power Query as an Activity in Azure Data Factory and SSIS?

There is a Power Query activity in SSIS and Azure Data Factory, which can be more useful than other tasks in some situations.