Skip to main content

Data Adventures

My personal journey in this intricate world of data and continuous efforts to make it more structured and well understood - Rayis Imayev.

Search This Blog

Posts

Paginated Reporting at Scale: Choosing the Right Microsoft Platform

Posted by Rayis Imayev on November 08, 2026

"I've seen things you people wouldn't believe..." — Roy Batty, Nexus-6 Replicant, Blade Runner (1982) (2026-Nov-08) Yep, that's me! On September 26th this year, I was singing with my Welsh choir at our local Celtic Festival. Then, just six hours later, I was giving a technical talk at the Toronto Day of Data event about using Power BI Report Server for paginated reports. You're probably wondering how I ended up in this situation. Well, it’s a bit of a long story. Let me take you back to where it all began... 🙂

Get link
Facebook
X
Pinterest
Email
Other Apps

Chatting with Power BI: Who Benefits from This Conversation?

Posted by Rayis Imayev on November 24, 2025

Get link
Facebook
X
Pinterest
Email
Other Apps

Down the Rabbit Hole: Dealing with Ad-Hoc Data Requests

Posted by Rayis Imayev on October 13, 2025

Get link
Facebook
X
Pinterest
Email
Other Apps

What a Summer Camp Kitchen Still Teaches Me

Posted by Rayis Imayev on August 04, 2025

Get link
Facebook
X
Pinterest
Email
Other Apps

Stepping into Event Streaming with Microsoft Fabric

Posted by Rayis Imayev on July 06, 2025

Get link
Facebook
X
Pinterest
Email
Other Apps

Salesforce CDC Data Integration

Posted by Rayis Imayev on June 30, 2025

Get link
Facebook
X
Pinterest
Email
Other Apps

Logging in Azure Data Factory data flows

Posted by Rayis Imayev on June 15, 2025

Get link
Facebook
X
Pinterest
Email
Other Apps

Dealing with Changing Data Formats: Schema Drift in Azure Data Factory

Posted by Rayis Imayev on February 12, 2025

Get link
Facebook
X
Pinterest
Email
Other Apps

Data Masking with Azure Databricks

Posted by Rayis Imayev on January 13, 2025

Get link
Facebook
X
Pinterest
Email
Other Apps

What I learned volunteering at a children's summer camp

Posted by Rayis Imayev on September 13, 2024

Get link
Facebook
X
Pinterest
Email
Other Apps

Extracting PostgreSQL database metadata for presentation in Excel format

Posted by Rayis Imayev on March 16, 2024

Get link
Facebook
X
Pinterest
Email
Other Apps

Don't Trust the Defaults

Posted by Rayis Imayev on March 08, 2024

Get link
Facebook
X
Pinterest
Email
Other Apps

Handling physical deletes from the source and continue populating your analytical data store

Posted by Rayis Imayev on October 15, 2023

Get link
Facebook
X
Pinterest
Email
Other Apps

Azure DevOps: Enterprise Power BI report deployment with connections to Shared datasets

Posted by Rayis Imayev on July 15, 2023

Get link
Facebook
X
Pinterest
Email
Other Apps

Adding microseconds to a timestamp in Azure Data Factory

Posted by Rayis Imayev on April 25, 2023

Get link
Facebook
X
Pinterest
Email
Other Apps

Using Azure Data Factory to read and process REST API datasets

Posted by Rayis Imayev on April 10, 2023

Get link
Facebook
X
Pinterest
Email
Other Apps

Metadata-driven pipelines in Azure Data Factory | Part 4 - Analytical Processing

Posted by Rayis Imayev on February 20, 2023

Get link
Facebook
X
Pinterest
Email
Other Apps

Thinking about data points and more …

Posted by Rayis Imayev on August 28, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Azure DevOps: Deploying Power BI reports with a parameterized gateway-based Data Source

Posted by Rayis Imayev on August 08, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Azure DevOps: Merging code to Main branch from a specific branch only

Posted by Rayis Imayev on July 25, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Including reference data into a database deployment process or passing lookup tables to Production

Posted by Rayis Imayev on June 20, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Metadata-driven pipelines in Azure Data Factory | Part 3 - Column Metadata

Posted by Rayis Imayev on June 06, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Populating PostgreSQL JSONB column using Azure Data Factory Data Flow

Posted by Rayis Imayev on May 31, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Metadata-driven pipelines in Azure Data Factory | Part 2 - Feed Configuration

Posted by Rayis Imayev on May 08, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Metadata-driven pipelines in Azure Data Factory | Part 1 - Data Copy

Posted by Rayis Imayev on March 04, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Fixing a giant or running a SQL Server project deployment with Temporal tables

Posted by Rayis Imayev on January 27, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Trusting the Fine Print or connecting Azure Data Factory with Salesforce

Posted by Rayis Imayev on January 19, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

Can I create a CI/CD pipeline to deploy Python Function to Azure Function App using Windows self-hosted Azure DevOps agent?

Posted by Rayis Imayev on January 10, 2022

Get link
Facebook
X
Pinterest
Email
Other Apps

“Could not find the modules: 'Az.Accounts' with Version: ''” error message and a story to remember

Posted by Rayis Imayev on December 20, 2021

Get link
Facebook
X
Pinterest
Email
Other Apps

Fail activity in Azure Data Factory and Why would I want to Fail

Posted by Rayis Imayev on November 30, 2021

Get link
Facebook
X
Pinterest
Email
Other Apps

Error [IM002] [Microsoft][ODBC Driver Manager] "Data source name not found and no default driver specified" and who do you trust?

Posted by Rayis Imayev on October 31, 2021

Get link
Facebook
X
Pinterest
Email
Other Apps

Lambda Architecture in data systems and possible meaning of this name

Posted by Rayis Imayev on September 21, 2021

Get link
Facebook
X
Pinterest
Email
Other Apps

Labels

Azure Data Factory Azure Power BI Data ADF Pipeline JSON Maps Variables Array

Azure Logic App Data Modeling DevOps CI/CD Excel Metadata Open Data Parameter Power Query SQL Server Visualization continuous integration ADF. Azure Data Factory Azure Function Data Factory Mapping Data Flows continuous delivery Analytics Durable Function Email Notification deployment ArcGIS ArgGIS Map Data Vault Database Databricks Fabric Flatten GIS GitHub Latitude Longitude PostgreSQL QGIS SQL Salesforce Shape Map Trigger API Attachment Azure Data Lake Store Azure DevOps Big Data Bing Map Blob CDC CSV Custom Shapefile Data Flow ESRI Error File Management Load Mapbox PASS Power Automate Flow Python REST API SQL Azure SQL Azure Database SSMS SSRS T-SQL Template WGS 84 Webhook Wrangling data flows XML 200 202 3D Map ADLS AI ARM Template Alerts Alter ow Animation Apache Spark Append Variable Architect Architecture Azure Data Studio Azure Databricks Azure Monitor Azure Service Bus Azure Storage Bayeux Blob Storage Container Build CDS CRM Calendar Change Data Capture Cloud Column Column mapping CometD Common Data Services Condition Copilot Custom Maps Custom Solution Data Architecture Data Copy Data Lake Data Source Data Systems Data warehouse DataForACause Database Design Default Value Delete Delete Activity Distance Drain mode Dual axes EPSG 4326 ER Studio Events Eventstream Execution Explode Extension Fail Filled Map Filter Flow map Function Function App Gallery Template Geometry Git Global Fishing Globe Map HDFS HTML HTTP Heatmap IF IF activity ISO Icon Map Ingestion Integration JSONB Lambda Architecture Latency Learning Log Analytics Logging Logic Operation Long polling Look Data M language Macro Maintenance Masking Merge Metrics Microsoft Microsoft Dynamics 365 Migration Monitor Narrative Science Network ODBC Oracle Paginated Report Parameterized Parent Child Nodes Parse Performance Personal Story Polling Polygon Post Deployment Power App Power BI Report Server Power Map PowerShell Predictive Analytics Primary Secondary axis Pulse Chart Queue Reference Data Reference Layer Release Resource Manager SQL Saturday SQL Server Project SSAS Scale Controller Schema Drift Send Message SharePoint Shared Dataset Solution Template Source Control Spark Spatial Data Spatial Functions Special character Statistics Status Code Stored Procedure Streamin Data Streaming Support Switch Activity System Variables Table Tabular Template Gallery Temporal table Throughput Time Time Slider TopoJSON Twitter UX Unlearning Upgrade Usability VS Studio Validation Variable Version Web Week arm-template-parameters-definition.json career custom parameter dbdiagram dbeaver er studio data architect erwin data modeler long running process microseconds oracle sql developer data modeler scale in scale out self-hosted agent timestamp

Show more Show less

Rayis Imayev

Follow @RayisImayev

Archive

2026 1
- November 1
  - Paginated Reporting at Scale: Choosing the Right M...

2025 8
- November 1
- October 1
- August 1
- July 1
- June 2
- February 1
- January 1
2024 3
- September 1
- March 2
2023 5
- October 1
- July 1
- April 2
- February 1
2022 11
- August 2
- July 1
- June 2
- May 2
- March 1
- January 3
2021 20
- December 1
- November 1
- October 1
- September 1
- July 2
- June 2
- May 3
- April 1
- March 2
- February 3
- January 3
2020 21
- December 2
- October 2
- September 2
- August 1
- July 1
- June 2
- May 2
- April 3
- March 4
- February 1
- January 1
2019 19
- December 1
- November 1
- October 3
- June 1
- May 1
- April 2
- March 3
- February 4
- January 3
2018 16
- December 1
- November 2
- October 2
- August 1
- July 3
- May 2
- March 1
- February 2
- January 2
2017 12
- December 1
- November 1
- October 1
- September 1
- August 1
- July 1
- June 1
- May 1
- April 1
- March 1
- February 1
- January 1
2016 9
- November 1
- October 1
- September 1
- August 1
- July 1
- June 1
- May 1
- April 1
- February 1
2015 10
- December 1
- November 1
- October 1
- August 1
- July 2
- June 1
- May 1
- April 1
- March 1
2014 11
- December 1
- November 1
- October 2
- August 1
- July 3
- April 1
- March 1
- February 1

Show more Show less

Labels

2001 2021 3D Map1 ADF14 ADF. Azure Data Factory5 ADLS1 AI1 Alerts1 Alter ow1 Analytics4

Animation1 Apache Spark1 API2 Append Variable1 ArcGIS3 Architect1 Architecture1 ArgGIS Map3 ARM Template1 arm-template-parameters-definition.json1 Array7 Attachment2 Azure25 Azure Data Factory51 Azure Data Lake Store2 Azure Data Studio1 Azure Databricks1 Azure DevOps2 Azure Function5 Azure Logic App7 Azure Monitor1 Azure Service Bus1 Azure Storage1 Bayeux1 Big Data2 Bing Map2 Blob2 Blob Storage Container1 Build1 Calendar1 career1 CDC2 CDS1 Change Data Capture1 CI/CD6 Cloud1 Column1 Column mapping1 CometD1 Common Data Services1 Condition1 continuous delivery5 continuous integration6 Copilot1 CRM1 CSV2 Custom Maps1 custom parameter1 Custom Shapefile2 Custom Solution1 Data16 Data Architecture1 Data Copy1 Data Factory5 Data Flow2 Data Lake1 Data Modeling7 Data Source1 Data Systems1 Data Vault3 Data warehouse1 Database3 Database Design1 Databricks3 DataForACause1 dbdiagram1 dbeaver1 Default Value1 Delete1 Delete Activity1 deployment4 DevOps7 Distance1 Drain mode1 Dual axes1 Durable Function4 Email Notification4 EPSG 43261 ER Studio1 er studio data architect1 Error2 erwin data modeler1 ESRI2 Events1 Eventstream1 Excel6 Execution1 Explode1 Extension1 Fabric3 Fail1 File Management2 Filled Map1 Filter1 Flatten3 Flow map1 Function1 Function App1 Gallery Template1 Geometry1 GIS3 Git1 GitHub3 Global Fishing1 Globe Map1 HDFS1 Heatmap1 HTML1 HTTP1 Icon Map1 IF1 IF activity1 Ingestion1 Integration1 ISO1 JSON9 JSONB1 Lambda Architecture1 Latency1 Latitude3 Learning1 Load2 Log Analytics1 Logging1 Logic Operation1 Long polling1 long running process1 Longitude3 Look Data1 M language1 Macro1 Maintenance1 Mapbox2 Mapping Data Flows5 Maps8 Masking1 Merge1 Metadata6 Metrics1 microseconds1 Microsoft1 Microsoft Dynamics 3651 Migration1 Monitor1 Narrative Science1 Network1 ODBC1 Open Data6 Oracle1 oracle sql developer data modeler1 Paginated Report1 Parameter6 Parameterized1 Parent Child Nodes1 Parse1 PASS2 Performance1 Personal Story1 Pipeline11 Polling1 Polygon1 Post Deployment1 PostgreSQL3 Power App1 Power Automate Flow2 Power BI25 Power BI Report Server1 Power Map1 Power Query6 PowerShell1 Predictive Analytics1 Primary Secondary axis1 Pulse Chart1 Python2 QGIS3 Queue1 Reference Data1 Reference Layer1 Release1 Resource Manager1 REST API2 Salesforce3 Scale Controller1 scale in1 scale out1 Schema Drift1 self-hosted agent1 Send Message1 Shape Map3 Shared Dataset1 SharePoint1 Solution Template1 Source Control1 Spark1 Spatial Data1 Spatial Functions1 Special character1 SQL3 SQL Azure2 SQL Azure Database2 SQL Saturday1 SQL Server6 SQL Server Project1 SSAS1 SSMS2 SSRS2 Statistics1 Status Code1 Stored Procedure1 Streamin Data1 Streaming1 Support1 Switch Activity1 System Variables1 T-SQL2 Table1 Tabular1 Template2 Template Gallery1 Temporal table1 Throughput1 Time1 Time Slider1 timestamp1 TopoJSON1 Trigger3 Twitter1 Unlearning1 Upgrade1 Usability1 UX1 Validation1 Variable1 Variables8 Version1 Visualization6 VS Studio1 Web1 Webhook2 Week1 WGS 842 Wrangling data flows2 XML2

Show more Show less

Popular Posts

Continuous integration and delivery (CI/CD) in Azure Data Factory using DevOps and GitHub

(2019-Feb- 18) With Azure Data Factory (ADF) continuous integration, you help your team to collaborate and develop data transformation solutions within the same data factory workspace and maintain your combined development efforts in a central code repository. Continuous delivery helps to build and deploy your ADF solution for testing and release purposes. Basically, the CI/CD process helps to establish a good software development practice and aims to build a healthy relationship between development, quality assurance, and other supporting teams. Update 2020-Mar-15: Part 2 of this blog post published in 2020-Jan-28: http://datanrg.blogspot.com/2020/01/continuous-integration-and-delivery.html Update 2019-Jun- 24: Video recording of my webinar session on Continuous Integration and Delivery (CI/CD) in Azure Data Factory at a recent PASS Cloud Virtual Group meeting.

Executing Azure Data Factory pipelines by Power App / Automate Flow

(2021-Jun-30) One of my university professors liked to tell us a quote, “ The Sleep of Reason Produces Monsters ”, in a way to help us, his students, to stay active in our thinking process. I’m not sure if Francisco Goya , had a similar aspiration when he was creating his artwork with the same name. So, let me explain my reasons to create a solution to trigger Azure Data Factory (ADF) pipelines from a Power App and why it shouldn't be considered as a monster :-)

Append Variable activity in Azure Data Factory: Story of combining things together

(2018-Oct-29) There are only a few sentences in the official Microsoft web page that describe newly introduced activity task ( Append Variable) to add a value to an existing array variable defined in Azure Data Factory - Append Variable Activity in Azure Data Factory But it significantly improves your ability to control a workflow of the data transformation activities of your Data Factory pipeline.

System Variables in Azure Data Factory: Your Everyday Toolbox

(2018-Nov-20) After working and testing the functionality of variables within Azure Data Factory pipelines, I realized that it's worth to explore existing system variables. That's basically could be my toolbox to collect and store control flow metrics of my pipelines.

Transforming JSON to CSV with the help of Flatten task in Azure Data Factory

(2020-Mar- 19) Recently, Microsoft introduced a new Flatten task to the existing set of powerful transformations available in the Azure Data Factory (ADF) Mapping Data Flows - https://docs.microsoft.com/en-us/azure/data-factory/data-flow-flatten . What this new task does it helps to transform/transpose/flatten your JSON structure into a denormalized flatten datasets that you can upload into a new or existing flat database table. 2020-Mar-26 Update: Part 2 : Transforming JSON to CSV with the help of Flatten task in Azure Data Factory - Part 2 (Wrangling data flows) I like the analogy of the Transpose function in Excel that helps to rotate your vertical set of data pairs ( name : value ) into a table with the column name s and value s for corresponding objects. And when this vertical JSON structural set contains several similar sets (array) then ADF Mapping Data Flows Flatten does a really good job by transforming it into a table with several rows (records)...

Setting Variables in Azure Data Factory Pipelines

(2018-Oct-15) Working with Azure Data Factory you always tend to compare its functionality with well established ETL packages in SSIS. Data flow task have been recreated as Data Copy activities; logical components have found they cloud-based siblings; as well as new kids on the block, such as Databricks and Machine Learning activities could boost adoption rate of Azure Data Factory (ADF) pipelines. Support for local variables hasn't always been available in ADF a nd was only recently introduced to already available pipeline parameters. This addition makes more flexible to create interim properties (variables) that you can adjust multiple times within a workflow of your pipeline.

Excel Power Map vs. Power BI Globe Map visualization

[ 2016-02-02 ] PowerMap in Excel is a great tool with good design and comprehensive user experience. I’ve started using it about a year ago when our team participated at first Canadian Microsoft Big Data hackathon ( http://blogs.technet.com/b/cansql/archive/2015/04/13/big-data-hackathon-a-story-from-a-winning-team.aspx ).

Send a Database Query Result as Email via Logic App in HTML format using Azure Data Factory

(2020-June- 07) It is a very simple concept, you have a database query result, either a direct database table output or a stored procedure returned result that needs to be sent as email notification. This email could serve as a status report of your daily data processing job or alert notification with some metrics that require immediate attention, and you want to be as user-friendly as possible with this message aka HTML format for your dataset is required.

Azure Data Factory: Extracting array first element

(2019-Apr- 28) Full credit goes to Microsoft for the latest efforts updating Azure products with new features and documenting corresponding changes for the end users. Azure Data Factory (ADF) is a great example of this. A user recently asked me a question on my previous blog post ( Setting Variables in Azure Data Factory Pipelines ) about possibility extracting the first element of a variable if this variable is set of elements (array). So as a spoiler alert, before writing a blog post and adding a bit more clarity to the existing Microsoft ADF documentation, here is a quick answer to this question. The first element of an ArrayVariable is extracted by this expression: @variables(' ArrayVariable ')[0] . I created a simple ADF pipeline to test this out:

SSRS report: Multi-value parameter going after Data-driven subscription

It is always good to enrich SSRS reports with multi-value parameters; users have more flexibility to combine different reporting scenario with no need to run a report for different parameter values separately. On the other hand, developers can craft their skills to embed multi-value parameters into report datasets; referencing different rows, labels and values from those parameters into some custom controls within reports is another competence which can also be quickly obtained.

Twitter Widget

Tweets by RayisImayev

Search This Blog

Posts

Posts

All Comments

All Comments

Powered by Blogger