Azure Data Factory: Extracting array first element

(2019-Apr-28) Full credit goes to Microsoft for the latest efforts updating Azure products with new features and documenting corresponding changes for the end users. Azure Data Factory (ADF) is a great example of this. 

A user recently asked me a question on my previous blog post (Setting Variables in Azure Data Factory Pipelines) about possibility extracting the first element of a variable if this variable is set of elements (array).

So as a spoiler alert, before writing a blog post and adding a bit more clarity to the existing Microsoft ADF documentation, here is a quick answer to this question. The first element of an ArrayVariable is extracted by this expression: @variables('ArrayVariable')[0].

I created a simple ADF pipeline to test this out:


And here the list of variables that I plan to use in this testing ADF pipeline:


1) I Set my ArrayVariable to two characters "A" & "B": @createArray('A', 'B') 
2) Then I extract the first element of this array into ArrayElement_1:=@variables('ArrayVariable')[0]
3) Second element of this array is extracted into ArrayElement_2:=@variables('ArrayVariable')[1]
4) Also, I check an out-of-range behavior of the ArrayVariable by setting ArrayElement_OutOfRange: =@variables('ArrayVariable')[2]

Which gives me the following error message and confirms that I can't make a reference to an element in my array variable that is outside of its boundaries.


Summary:
The use-case of referencing array elements by using square brackets is quite broad. And now it will be easier to use another activity output, like child items of GetMetaData activity task. You either can scan this list of values using ForEachLoop container or in case if you know the exact position of your array elements, then you can just extract those values with square brackets referencing.

Code of this blog post ADF pipeline can be accessed in my personal GitHub repository:
https://github.com/NrgFly/Azure-DataFactory/blob/master/Samples/pipeline/bdata_adf_variable_array_many_elements.json

Happy data adventures!

Comments

  1. This would be the same if you where using a Logic App.

    ReplyDelete
    Replies
    1. But this does help to use more Azure Data Factory resources internally.

      Delete
  2. how should we add elements with special characters in array

    ReplyDelete
    Replies
    1. Try to use backslash ( "\" ) character is a special escape character.

      Delete

  3. it doesn't work for me, I get that error. :( "The expression 'variables('ArrayVariable')[2]' cannot be evaluated because array index '2' is outside bounds (0, 0) of array.",

    ReplyDelete
    Replies
    1. Seems like that your array variable hasn't been populated yet. Also, try to test lower array elements ID.

      Delete
  4. This is my problem:
    https://stackoverflow.com/questions/62182125/azure-datafactory-set-variabe-array-and-for-each-exe-pipelines-activities

    can you give me another solution?

    ReplyDelete
    Replies
    1. Can you share you code repository for this ADF pipeline? Also can you explain in more details what you're trying to achieve? Screenshots don't provide sufficient information.

      Delete

Post a Comment