Understanding Undocumented ARM Oddities

Over the past year I’ve been working pretty heavily with Azure Resource Manager (ARM) templates to create safe, reusable and consistent deployments of virtual infrastructure. When producing ARM templates, it’s important to understand what resource types are available, and what values to use in your template. I always use the Azure Template Reference to understand exactly how to define a certain resource type. However, sometimes you will run into situations where the Azure Template Reference is missing something that can be done in the Azure portal. So, how do we figure out how to template it if it’s not in the reference documentation?

Export Templates – Perhaps the quickest way to solve this problem is to use the native ‘Export Template’ blade in the Azure portal. For this, you will need to deploy your resource and configure it as you would like, using the Azure Portal. Once you have your resource ready, open the Export Template blade on your resource. This will create an automatically generated ARM template based on the current running state of your resource. From here, you can inspect the generated template and see if your undocumented settings or configuration has been captured in the generated template.

Download template

Azure Resource Explorer – Next stop is the Azure Resource Explorer which provides a visual interface for you to examine the Azure API’s. With the Azure Resource Explorer, you can explore the current running state of an Azure environment in JSON format. This can be very useful when attempting to reverse engineer an existing resource or environment. While Azure Resource Explorer isn’t returning data that can be directly used in an ARM template, it can be used as a mechanism to learn the syntax of resource properties that are missing from the Azure Template Reference.

This image has an empty alt attribute; its file name is 13nov1.png

When issues are encountered with undocumented resources, typically the fastest way to resolve the issue is by manually deploying the resources using the Azure Portal (clicky clicky) then reverse engineering the ARM template with a combination of using the Export Templates function and Azure resource explorer. Going down the route of doing everything in ARM templates, can lead to a lot of trial and error before getting a fully automated deployment, for now at-least.

Cheers,

Joel


Undocumented ARM Oddities – .Net Core App Services

Every once in a while, when working with ARM templates you come across something that is missing from the official Microsoft ARM template reference. In my case yesterday, I was looking to update the configuration of an Azure App Service to use the DotNetCore stack (rather than .NET 4.8).

While I initially thought this would be a quick job to simply look up the ARM reference and make the required changes, I found that there was nothing about DotNetCore in the ARM reference. Funny enough, there is a value for “netFrameworkVersion”, but don’t be deceived, if you are looking to setup DotNetCore – this value is not for you (this is for regular .Net only).

To better understand the problem, I Clickly Clikcy’d in an App Service and configured it for DotNetCore (Clickly Clicky is our lingo for deploying infrastructure using the portal rather than a CLI or template). With this, I attempted my usual trick of exporting a template and observing the JSON it spits out. However, much to my amazement I couldn’t see any reference to dotnetcore in there at all.

In the end it was the Azure Resource Explorer which came to my rescue. Used the tool to explore the example I created and found a value called “CURRENT_STACK” in the properties of the “Microsoft.Web/sites/config” resource type.

After playing this this for a while, I was able to translate this into my ARM template with the following JSON.

{
    "type": "Microsoft.Web/sites",
    "name": "[variables('WebSiteName')]",
    "apiVersion": "2020-06-01",
    "location": "[resourceGroup().location]",
    "kind": "app",
    "properties": {
        "siteConfig": {
            "metadata": [{
                "name": "CURRENT_STACK",
                "value": "dotnetcore"
            }]
        },

Hopefully this helps anyone who encounters this problem.

Cheers,

Joel


Azure Bastion – Unable to query Bastion data.

I’ve recently setup Azure Bastion to give external users/vendors access to resources via RDP or SSH following these instructions:

https://docs.microsoft.com/en-us/azure/bastion/tutorial-create-host-portal

The key permissions outlined in the prerequisites at point 3 are:

  • A virtual network.
  • A Windows virtual machine in the virtual network.
  • The following required roles:
    • Reader role on the virtual machine.
    • Reader role on the NIC with private IP of the virtual machine.
    • Reader role on the Azure Bastion resource.
  • Ports: To connect to the Windows VM, you must have the following ports open on your Windows VM:
    • Inbound ports: RDP (3389)

My scenario is to invite a guest AAD account, add them to a group and grant the group access as per below:

  • Grant Contributor role to the resource group that has the VM’s for the application.
  • Grant Reader role to the resource group that has the Bastion host.

This way the guest user logs into the Azure Portal complying with our conditional access policy and then they are presented with only the resources they have read or higher access too. In this scenario that is the two resource groups outlined above.

The guest user locates the virtual machine they wish to connect and then chooses Connect > Bastion > Use Bastion the following error message is presented.

Error Message:

“Unable to query Bastion data”

Initially working with Microsoft support we found that granting reader access at the subscription level gave the user permission to in-act the Bastion service, which simply give a username and password input.

These permissions were too lacks as a workaround and exposed a lot of production data to accounts that didn’t really have any business looking at it.

Workaround

[12/11/2020] The case is on-going inside Microsoft and I will leave a definitive response when I get the information. I’ve done some further investigation what would be the least amount of additional ‘Reader‘ permissions are required. I’ve found the following permissions are required in my scenario:

  • Reader permissions on the Virtual Network that has the ‘AzureBastionSubnet‘ subnet.
  • Reader permissions on the Virtual Network that has the connected virtual machine network interface.

In my scenario, the virtual machines are located in a development Virtual Network that is peered with the production Virtual Network which has the subnet ‘AzureBastionHost‘. So I had two sets of permissions to add. After applying the permissions you may need to get a coffee and return to the portal as it took 5-10 minutes to kick in for me.

Hope this helps someone that has done some googling but is still scratching their head with this error message.


Get all restore points for an Azure VM

Getting restore points out of Azure can be like getting blood from a stone. The portal likes to always set a custom filter showing only ~90 days and your Powershell cmdlet only allows for a 30 day interval for retrieval dates. When running ‘Get-AzRecoveryServicesBackupRecoveryPoint’ you get the following:

Get-AzRecoveryServicesBackupRecoveryPoint : Time difference should not be more than 30 days 

Sigh.. I just want all my restore points for a virtual machine please! All of them, because its my butt if for some reason I don’t have them. Using something like this can be useful to audit your backups against business needs for data retention.

Example: Get recovery points from the last two years for a single VM

# ------Variables--------------#
$retentionDays = 730
$vaultName = "PROD-RSV"
$vaultResourceGroup = "PROD-RSV-RG"
$friendlyName = "Server1"
#------------------------------#


$vault = Get-AzRecoveryServicesVault -ResourceGroupName $vaultResourceGroup -Name $vaultName 
$Container = Get-AzRecoveryServicesBackupContainer -ContainerType AzureVM -Status Registered -FriendlyName $friendlyName -VaultId $vault.ID
$BackupItem = Get-AzRecoveryServicesBackupItem -Container $Container -WorkloadType AzureVM -VaultId $vault.ID

$startingPoint = -25
$finishingPoint = 0
$jobsArray = @()

Do {

$StartDate = (Get-Date).AddDays($startingPoint)
$EndDate = (Get-Date).AddDays($finishingPoint)
$RP = Get-AzRecoveryServicesBackupRecoveryPoint -Item $BackupItem -StartDate $Startdate.ToUniversalTime() -EndDate $Enddate.ToUniversalTime() -VaultId $vault.ID 
$jobsArray += $RP
$startingPoint = $startingPoint - 25
$finishingPoint = $finishingPoint -25
}until($startingPoint -le -($retentionDays))

$jobsArray | FT -AutoSize -Property RecoveryPointid, RecoveryPointTime, RecoveryPointType 


The example above will go back 2 years (730 Days). This outputs to a table but you can quiet easily export to a CSV via:

$jobsArray | Export-Csv c:\temp\restores.csv -NoTypeInformation 

Enjoy.


Azure Application Insights – No Client Source IP Address

Working with one of your customers this week who is implementing Azure API Management alongside their web applications. We are funnelling all the request logs into an Application Insights services to manage visibility of the end-to-end transaction data. We noticed that all the client GET requests had ‘0.0.0.0’ in Client IP Address.

Request PropertiesValue
Client IP address0.0.0.0

I since learned that Microsoft obfuscate this data from Azure Monitor as it’s ingested into Applications Insights for what I call a ‘privacy policy‘. As this was a corporate application anonymity wasn’t needed and the development team wanted to understand when a request was made from their application either from inside corporate network or an unknown internet address.

A good habit to get into is first do a quick review of the latest API version for ‘Microsoft.Insights/components’ which does show a boolean value for DisableIpMasking.

{
  "name": "string",
  "type": "Microsoft.Insights/components",
  "apiVersion": "2020-02-02-preview",
  "location": "string",
  "tags": {},
  "kind": "string",
  "properties": {
    "Application_Type": "string",
    "Flow_Type": "Bluefield",
    "Request_Source": "rest",
    "HockeyAppId": "string",
    "SamplingPercentage": "number",
    "DisableIpMasking": "boolean",
    "ImmediatePurgeDataOn30Days": "boolean",
    "WorkspaceResourceId": "string",
    "publicNetworkAccessForIngestion": "string",
    "publicNetworkAccessForQuery": "string"
  }
}

Reviewing the property values for ApplicationInsightsComponentProperties object DisableIpMasking gave the following short but sweet answer.

NameTypeRequriedValue
DisableIpMaskingbooleanNoDisable IP masking.

Yeah I reckon that is worth a shot!

Update ApplicationInsightsComponentProperties value DisableIpMasking

As this value only seems to be exposed through the API we have to either push a new incremental ARM template through the sausage maker or perform a API request directly. An API request seems like the quicker request method, but doing this in a script with authentication and correct structure takes time. I have a nice trick when wanting to update or add a value to an object when either of those feel like overkill.

  1. Navigate to the Azure Resource Explorer
  2. Find the Application Insights Resource Group
  3. Select Providers > Microsoft.Insights
  4. Select Components > ‘Application Insights Name

You will be shown the JSON definition of your Application Insights Object. You can tell this by the line:

"type": "microsoft.insights/components"

To know your in the right place, under properties there will be many values, we should see Application_Type, InstrumentationKey, ConnectionString, Retention, but what will be missing is DisableIpMasking. So it’s as simple as adding it.

  1. Up the top of the page toggle the blue switch to ‘Read/Write’ from ‘Read Only’.
  2. Select ‘Edit‘.
  3. Remember to add a ‘,’ to the previous last line (in my case “HockeyAppToken“) before adding your new property.

The final step is to use the PUT button to update the object. Which intern has authenticated you to the API using your existing login token, constructed the JSON object and is sending a ‘POST’ method to the API endpoint for ‘management.azure.com/subscriptions/<subscriptionId>/resourceGroups/<rgName>/providers/microsoft.insights/components/<resourceName>?api-version=2015-05-01‘. Much simpler than doing a Powershell or Bash script, what a clever little tool it is.

The result will be that new request in Application Insights will have the source NAT IP address. Unfortunately all previous requests will remain scrubbed with ‘0.0.0.0’.

Closing thoughts

This is a great way to tweak services while attempting to understand whether it’s the correct knob to turn in the Azure service. But while it’s quick, it isn’t documented. If you have a repository of deployment ARM templates make sure you go back and amend the deployment JSON. The day will come when it gets re-deployed and it wont come out the sausage maker the same. The finger will get pointed back at that Azure administrator who doesn’t follow good DevOps practices.