SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 

Azure ARM template function internals

I have been working with azure arm functions for quite a while and they are exceptionally useful but also quite a pain to work with, visualizing what it is that the functions will resolve to is often hard.

If you haven't used an ARM template, ARM templates are JSON documents which are effectively a whole load of properties made up of a name and a value. The value can either just be a value or it can include arm template functions. If we look at this example:


"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"name": "[variables('storageAccountName')]",
"apiVersion": "2016-01-01",
"location": "[resourceGroup().location]",
"sku": {
"name": "Standard_LRS"
},
"kind": "Storage",
"properties": {}
},

The value for type, apiVersion, sku name and kind are standard values, standard hardcoded values. The value for name and location are determined at runtime - you can tell the difference because the functions are wrapped in [ and ]. In this case "[variables('storageAccountName')] just grabs a value variable. Variables are defined by a json object called "Variables". Location's [resourceGroup().location] is a little more complicated. It calls a function "resourceGroup()" which returns an object and then the value of location is used.

I originally thought that Microsoft must be using something like node or something that parsed the functions using javascript as the language was very similar to javascript, or so I thought. When I wrote my vs code extension to parse these functions so I could test them locally rather go through a full publish cycle that takes forever in azure; That is exactly what I did, implemented all the functions like 'resourceGroup' and 'variables' as javascript functions and then called 'eval' to execute them.

When I was doing this, two things really stood out as not being quite right, the first was the 'greater' function - what this does is:

"Checks whether the first value is greater than the second value."

The function takes two parameters which could be an int or a string. Now the int version is fairly simple - 100 is bigger than a 1. The string version not quite as simple, you see normally when you compare strings the language follows the ASCII table so an upper case 'A' is less than a lower case 'a' as 'A' maps to number 65 and 'a' maps to 97. In my javascript version, I use "param1 > param2" which works exactly like that but the example on the docs page works the other way around:

https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-g...

'A' is greater than 'a'

So that was the first thing, if they just used javascript to eval the code then they would need to have implemented a custom version of > and I couldn't see why you would do that.

The second thing that was odd is that what I did was implement every function such as greater and equals and when I called eval, those functions were called great but 'if' is a reserved statement in javascript so you can't just call a function 'if' (even if it is a method on a class, I tried) because it is reserved. I had to use a regex to convert all calls to 'if' to 'if_'. This worked great for me but felt a little hacky and I wasn't sure if that is what Microsoft would have done.

So, I implemented my javascript version of the functions and forgot about it for a week or so but I couldn't help think that the greater function was odd (I have only implemented the wrong version so far so I know I need to change it at some point). Anyway, after thinking about this off and on for a little while I woke up one morning and realised that I can probably find out exactly how these were implemented if I used Azure Stack. Azure Stack is a version of Azure that runs on-premise.

So, I grabbed a copy of the Azure Stack SDK and started the download, I thought it would be an installer, install some stuff and then grab the files but it turned out to be a little bit harder.

The setup for Azure Stack is basically:

- Download VHDX of Windows 2016 with a Setup folder
- Run a script that:
- Validates you meet the minimum requirements
- Creates some virtual machines
- Creates a storage cluster
- Does some other stuff

I initially grabbed started the machine and started the installation and it failed pretty quickly because I didn't have a butt load of ram on my laptop and I wasn't running on bare metal and I didn't have enough disk space. I found that these checks were pester tests so I deleted them and the install continued merrily along (albeit slowly, you need hours to install Azure Stack). I also had to enable nested virtualization on the Azure Stack VM on my laptop using:

Set-VMProcessor -VMName stacky -ExposeVirtualizationExtensions $true

Eventually, I got the install running and after a while, I noticed that there was a vhdx being attached to one of the new virtual machines that was called "CommonNugets". Being a nosy sort of a fellow I stopped the install and the Virtual Machine, mounted the original VHDX that had been created by the AzureStack installer and had a poke around until I found this nuget file:

CloudDeploymnet\NuGetStore\AzureStack.Setup.5.14.723.nupkg

This nuget file contained, drum roll please....

Content\Setup\Packages\AzureStack-ResourceManager.msi

Then I grabbed uniextract to extract the contents and I was a little shocked to not see any javascript files but instead .net dll's. This one in particular:

Microsoft.Data.Edm.dll

I have tried to search for some reference to EDM but it seems it is Microsoft internal as I cannot find any reference to it at all.

So, inside this dll, it shows that they use Antlr.Net to parse the code values in the text. I was particularly pleased that we both worked out if a value was a standard value or a template function, in the same way, basically doing:

(expression.First() == '[')) && (expression.Last() == ']'));

Now, 'greater' how is it implemented by Microsoft?

First, they check if either the first or second parameters are strings if they are then if one is a string and one is not a string then they throw an exception.

Assuming that both are strings then they use the .net string.Compare to return an int, if string.Compare returns 0 or greater then the first value is greater than the second.

"string.Compare" is implemented by "System.Globalization.CompareInfo" because as well as the two strings they also pass in "StringComparison.InvariantCulture".

If we look at https://docs.microsoft.com/en-us/dotnet/standard/globalization-localizat... we can see that "FILE" is greater than "file" when using "StringComparison.InvariantCulture" so what we have isn't a straight map to the ASCII table but the sort order that windows uses.

Once I knew they had their own parser then the whole if / if_ regex thing became clear. I guess now I can fix greater at some point - I won't rush as calling greater on a pair of strings just seems a bit odd to me and I probably won't use it but at least now I can sleep without wondering how 'greater()' was implemented.

Tags: 

Ed Elliott's Sql Developer Blog

Ed is a Sql developer who has a mixed background in support, as a dba and as a developer working with a number of languages c, c#, vb, go, assembly with a variety of technologies and is currently trying to make the sql developer community a little bit more agile, one build step at a time!

Comments

Leave a comment on the original post [feedproxy.google.com, opens in a new window]

Loading comments...