Posts

Custom deployment parameters in Azure Data Factory - Forget Me Not

(2020-Sep-26) Last week one of my Azure Data Factory (ADF) deployment pipelines failed with an error that it couldn’t find some of the deployment parameters that I try to override, and I wondered what might have caused the issue. Usually, after configuring and testing all the deployment steps, there are very few things that can topple a normal and successful deployment process of your data factory, and it’s got to be big.My first step was to remove parameters overriding instances from a deployment pipeline. There is no need to reference and override a deployment pipeline parameter if it hadn’t come out of the process of creating my ADF Azure Resource Manager (ARM) template. That quick fix worked and my ADF deployment process successfully continued and new code was deployed to a testing environment. Photo by Karolina Grabowska from Pexels However, when I checked one of the web-activity tasks in my testing Data Factory after the deployment was finished, that activity still had a URL refer…

Passing array of arrays as a parameter in Azure Data Factory and the role of an @item element in this

Setting default values for Array parameters/variables in Azure Data Factory

Transforming JSON data with the help of Azure Data Factory - Part 5 - Comparing it with Databricks

Collecting custom parameters/variables for Azure Data Factory deployment

Send a Database Query Result as Email via Logic App in HTML format using Azure Data Factory

Transforming JSON data with the help of Azure Data Factory - Part 4 - Passing Arrays around

Replicating SQL IN operator in Azure Data Factory

Stacking Up Datasets in Azure Data Factory

Using Azure Functions in Azure Data Factory

Transforming JSON data with the help of Azure Data Factory - Part 3

Extracting data from Common Data Services (Microsoft Dynamics 365) using Azure Data Factory

Transforming JSON to CSV with the help of Azure Data Factory - Part 2 (Wrangling data flows)

Transforming JSON to CSV with the help of Flatten task in Azure Data Factory

Using Azure Data Factory Mapping Data Flows to populate Data Vault - PASS Video

Azure Databricks Performance Notes

Continuous integration and delivery (CI/CD) in Azure Data Factory using DevOps and GitHub - Part 2

Notifications in Azure Data Factory: Email, Internal Alerts, Log Analytics Alerts

Wrangling data flows in Azure Data Factory: Calculating distance between two Geo-points

Event-driven architecture (EDA) with Azure Data Factory - Triggers made easy

Continuous integration and delivery (CI/CD) in Azure Data Factory using DevOps and GitHub - PASS Video

Switch activity in Azure Data Factory: Container with many IFs