How to use a Script Activity in ADF as a Lookup
In Azure Data Factory (ADF, but also Synapse Pipelines and Fabric Pipelines), you have a typical scenario where you retrieve some metadata from a database and then loop over...
2024-06-27
In Azure Data Factory (ADF, but also Synapse Pipelines and Fabric Pipelines), you have a typical scenario where you retrieve some metadata from a database and then loop over...
2024-06-27
I wrote a piece of SQL that had some new T-SQL syntax in it: IS DISTINCT FROM. This predicate checks if two expressions are equal taking NULL into account....
2024-06-24
I’m very excited to announce I’ve been selected as a speaker for dataMinds Connect 2024, Belgium’s (and maybe/probably/most likely mainland Europe’s) best and greatest Microsoft data platform conference. My...
2024-06-17
When you develop data pipelines in Microsoft Fabric (the Azure Data Factory equivalent in Fabric, not to be confused with deployment pipelines), you will most likely have some activities...
2024-06-14 (first published: 2024-06-13)
I have a metadata-driven ELT framework that heavily relies on dynamic SQL to generate SQL statements that load data from views into a respective fact or dimension. Such a...
2024-06-08
I came across a bit of a weird error when I was trying to connect to a PostGreSQL database (hosted as an Azure DB for PostGreSQL), where “xx.xx.xx.xxx” is...
2024-06-08 (first published: 2024-06-07)
This book was recommended by some of my colleagues, so I decided to give it a try. After finishing the book, I was pleased that I did. As the...
2024-06-07 (first published: 2024-06-05)
In the introduction of this blog post series, I explained the use case: extracting data from the Planview Portfolios REST API using Azure Data Factory. Any tool that can...
2024-05-26
In the introduction of this blog post series, I explained the use case: extracting data from the Planview Portfolios RESET API using Azure Data Factory. Any tool that can...
2024-05-26
1 reads
In the introduction, I explained the setup of the use case and why we need to use the OData feed: to get the list of Project IDs, because the...
2024-05-26
6 reads
Are you ready to take your SQL skills from theoretical to tactical? If you've...
By Steve Jones
Note: I DO NOT recommend this. Any changes to a pipeline should be in...
By Steve Jones
symptomania – n. the fantasy that there’s some elaborate diagnosis out there that neatly...
Comments posted to this topic are about the item New article
Comments posted to this topic are about the item Monitoring Azure Blob Storage
Azure Data Lake Storage Gen 2 is built on ...?
See possible answers