Posts

Data Masking with Azure Databricks

“ Alice : Would you tell me, please, which way I ought to go from here? The Cheshire Cat : That depends a good deal on where you want to get to.” — Lewis Carroll, Alice’s Adventures in Wonderland (2025-Jan-13)  Working on the data masking project has been a long journey; writing about it will take some time. I’m fine if you’re not a big fan of long reads and prefer scrolling, skipping, or engaging in shorter, bite-sized learning experiences. I will keep this writing for myself as a chronicle of my journey along a path that may seem overwhelming. However, the experience throughout this journey and the joy of reaching the final destination will be hard to forget. When I think of data masking, or the term "masking" in particular — the intention to “hide” something from someone comes to mind. I picture a man in a mask, presumably to conceal his identity, or an image blurred beyond recognition. In the world of computer data, masking represents the process of obfuscating, redacting...

What I learned volunteering at a children's summer camp

Extracting PostgreSQL database metadata for presentation in Excel format

Don't Trust the Defaults

Handling physical deletes from the source and continue populating your analytical data store

Azure DevOps: Enterprise Power BI report deployment with connections to Shared datasets

Adding microseconds to a timestamp in Azure Data Factory

Using Azure Data Factory to read and process REST API datasets

Metadata-driven pipelines in Azure Data Factory | Part 4 - Analytical Processing

Thinking about data points and more …

Azure DevOps: Deploying Power BI reports with a parameterized gateway-based Data Source

Azure DevOps: Merging code to Main branch from a specific branch only

Including reference data into a database deployment process or passing lookup tables to Production

Metadata-driven pipelines in Azure Data Factory | Part 3 - Column Metadata

Populating PostgreSQL JSONB column using Azure Data Factory Data Flow

Metadata-driven pipelines in Azure Data Factory | Part 2 - Feed Configuration

Metadata-driven pipelines in Azure Data Factory | Part 1 - Data Copy

Fixing a giant or running a SQL Server project deployment with Temporal tables

Trusting the Fine Print or connecting Azure Data Factory with Salesforce

Can I create a CI/CD pipeline to deploy Python Function to Azure Function App using Windows self-hosted Azure DevOps agent?

“Could not find the modules: 'Az.Accounts' with Version: ''” error message and a story to remember

Fail activity in Azure Data Factory and Why would I want to Fail

Error [IM002] [Microsoft][ODBC Driver Manager] "Data source name not found and no default driver specified" and who do you trust?

Lambda Architecture in data systems and possible meaning of this name

Data Modeling at your fingertips

New Azure Data Factory home page, what's in it for me?

Executing Azure Data Factory pipelines by Power App / Automate Flow

Azure Function Drain mode

Blog post about Copying Azure files to SharePoint, comparing Logic App and Power Automate Flow for this, and Trust No One

Creating a generic (template) pipeline in Azure Data Factory to send email messages with attached files from Azure Storage account