Backup Palo Alto VM Series Config with Azure Automation

If you have implemented a VM-Series firewall in Azure, AWS or on-premises but don’t have a Panorama Server for your configuration backups. Here is a solutions for getting the firewall configuration into an Azure Blob Storage, this could be done similarly with Lambda and S3 using python and the boto3 library.

Why Do This?

If there are multiple administrators of the firewall and configuration changes are happening frequently you may want a daily/hourly backup of the configuration to restore in the event that a recent commit has caused unwanted disruption to your network.

Azure Automation is a great place to start, we will have to interact with the API interface of the firewall to ask for a copy of the XML. Generally speaking we don’t want to expose the API interface to the internet, nor is it easy to allow on the Azure Automation public IPs, so in this case a Hybrid Worker (VM inside your trusted network) can execute the code against the internal trusted interface that has the API listening.

Depending on your version of PowerShell and Invoke-WebRequest you may not be able to ignore a certificate error coming from the API interface. Which is why I’m updating system .Net class for X509 certificates policies.

The steps are pretty simple

  1. Create a directory on the file system (I’m using the Azure VM with temporary D drive local storage)
  2. Request the XML from the URL
  3. Login to Azure with service credentials
  4. Map to the cold storage account i’m putting the files in
  5. Copy the file

add-type @"     using System.Net;     using System.Security.Cryptography.X509Certificates;     public class TrustAllCertsPolicy : ICertificatePolicy {         public bool CheckValidationResult(             ServicePoint srvPoint, X509Certificate certificate,             WebRequest request, int certificateProblem) {             return true;         }     } "@ [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Ssl3, [Net.SecurityProtocolType]::Tls, [Net.SecurityProtocolType]::Tls11, [Net.SecurityProtocolType]::Tls12 
$todaydate = Get-Date -Format yy-MM-dd
$File = "PaloConfig-"+$todaydate+".xml"  
$FilePath = "D:\Palo\"+$File 
#Create Directory 
New-Item -ItemType directory -Path D:\Palo -Force 
#Download Config
Invoke-WebRequest -Uri "https://PaloIPAddress/api/?type=export&amp;category=configuration&amp;key=<ONETIMEKEY&gt;=" -OutFile $FilePath 
#Login with service principal account 
$TenantId = 'AzureTenantID' 
$ApplicationId = 'ServiceID' 
$Thumbprint = (Get-ChildItem cert:\LocalMachine\My\ | Where-Object {$_.Subject -match "CN=AzureHybridWorker" }).Thumbprint 
Connect-AzureRMAccount -ServicePrincipal -TenantId $TenantId -ApplicationId $ApplicationId -CertificateThumbprint $Thumbprint 
#Get key to storage account 
$acctKey = (Get-AzureRmStorageAccountKey -Name StorageAccountName -ResourceGroupName ResourceGroupName).Value[0] 
#Map to the backup BLOB context 
$storageContext = New-AzureStorageContext -StorageAccountName StorageAccountName -StorageAccountKey $acctKey 
#Copy the file to the storage account 
Get-ChildItem -LiteralPath $FilePath | Set-AzureStorageBlobContent -Container "paloconfigbackup" -BlobType "Block" -Context $storageContext -Verbose 

If you are not currently using a Hybrid Worker in your subscription, create one from the below link:

https://docs.microsoft.com/en-us/azure/automation/automation-hybrid-runbook-worker

Paste the code into an Azure PowerShell Runbook and create a re-occuring schedule.

You’ll have backups saved in cold storage for as long as you would like to retain the data. Create a storage policy can help you


Azure MFA with Palo Alto Client VPN

Client VPNs have come along way in recent years and are still a necessity for organisations protecting their backend services that cannot be published to the public internet securely. The nirvana is having data presented by web applications and use SAML authentication to any good identity provider that supports MFA. This world still doesn’t exists in its entirety and you wont be the first or last to say that you have some old application that runs as a fat client on desktop machines with non standard protocols. Client VPNs will remain relevant until all these applications adopt modern web authentication frameworks. 

Azure Active Directory is a great cloud based identity and authentication provider with lots of built in functionality to explore in the security space. If you’re using Office 365, then you already have one, and more bells and whistles can be turned on to include features like Multi Factor Authentication.  

Most good firewall products will have a Client VPN that supports radius as a second factor of authentication, but sometimes that’s where the documentation finishes.

How do I get my firewall VPN authentication to talk to Azure MFA if all I have available is radius?

The article today talks explicitly about Palo Alto Global Protect client and VM Series firewall, but there is no reason if other firewall VPN supports radius that you couldn’t perform the same architecture. 

Palo Alto Configuration

The following authentication settings needs to be configured on the Palo Alto firewall. I won’t bore you with every step, cause if you administering a firewall then I’ll assume a certain level of knowledge. The authentication flow will be:

  1. LDAP authentication with username and password (connected to Active Directory)
  2. Radius authentication using the NPS Azure MFA Extension

LDAP Authentication

Configure LDAP as per normal, nothing special to note here. This will be the first factor of authentication in the VPN login sequence. 

Radius Server Profile

  1. Device > Server Profiles > Radius and Add a profile.
  2. Enter a Profile Name to identify the server profile.
  3. Enter a Timeout interval in seconds after which an authentication. request times out (default is 3; I’ve changed this to 30 seconds to allow for cloud connectivity and then messages to client devices).
  4. Select the Authentication Protocol (PAP) that the firewall uses to authenticate to the RADIUS server.
  5. Add a new RADIUS server and enter the IP, Secret and Port (1812).

Radius Authentication Profile

  1. Select DeviceAuthentication Profile and Add a profile.
  2. Set the Type to RADIUS.
  3. Select the Server Profile you configured.
  4. Select Retrieve user group from RADIUS to collect user group information from VSAs defined on the RADIUS server. I’d recommend an ‘All Staff’ approach as this won’t be the restrictive policy to allow VPN.

Network Policy Server Configuration

Follow Micrsoft’s guide to deploying the NPS:

https://docs.microsoft.com/en-us/azure/active-directory/authentication/howto-mfa-nps-extension

The incoming request condition for policies can be the NAS Identifier which will be the name of your authentication profile in the Palo Alto, the example I named profile ‘Radius Authentication’ so therefor this will be presented on incoming connection from the Palo to NPS.

NPS Nas Identifier

Make sure for your authentication method you have PAP selected. The rest of the settings can be default. (CHAP/MS-CHAP while more secure was problematic in my deployment, therefor PAP was used). The traffic flows from the internal trusted interface of a firewall to a port on a trusted network segment encryption is not the major security control for this request. 

NPS Request Authentication Settings

After you install the Azure NPS Extension (make sure you reboot). This extension as great as it is, isn’t heavily customisable, which is why I strongly suggest this be a seperate radius server. Think of this NPS server as the MFA radius server as the extensions will intercept all requests regardless of policy. You don’t want this extension on an existing radius server that maybe used for WiFi authentication using certificates (EAP) for domain joined workstations etc. 

MFA using Azure Authenticator App
MFA using Azure One Time Password (OTP)

Test the solution

Before you test end to end, a simple test of only the Radius configuration for MFA can be done by the firewall CLI. Log in via SSH and test the profile. 

test authentication authentication-profile "Radius Authentication" username test@cloudstep.io password  

If it fails, do a quick sanity check on your test user:

  1. Synced to Azure Active Directory.
  2. Assigned a MFA license (P1 etc).
  3. Enrolled in MFA (https://aka.ms/MFASetup).

If you find that you’re not successfully completing the above CLI in the Palo SSH session, then here are some places to look for troubleshooting. 

Follow the Radius Authentication Flow

Live monitor

Do a quick visual traffic check in the GUI to make sure the radius authentication is leaving to the correct destination:

Monitor > Logs > Traffic > Enter this search criteria ( port.dst eq 1812 )

Do a packet capture

Monitor > Packet Capture add a filter, just choose the interface that the Palo should talk to the radius server on. Leave everything else blank and it just look like this afterwards.

VM Series Capture Filter

Under the capture configuration, we need to choose a ‘stage’ and give the file a name, I’ve left packet count and byte count blank.

VM Series Capture Stage

What we want to find is source IP of the Palo (.4) with destination port 1812. If its not here then its going out a different interface. We can modify the default interface for a service in the Palo if incorrect using service routes.

Wireshark Capture UDP Request

Here we can see the radius protocol inside the UDP packet. If we aren’t seeing the request make it to the radius server we may not be allowing 1812 on UDP on the ACL between the firewall appliance and radius server.

Wireshark Capture Radius Request

If we see the Access-Request on its way to the radius server we can do a few checks inside the Windows Server OS to validate the NPS configuration. 

Check the radius server

Event Viewer > Custom Views > Server Roles > Network Policy and Access Service review the log entries for each authentication request. If the request is being denied by the NPS server then make sure it’s being handled by the correct policy. A failed authentication request will show you which profile determined it was a failure, if it isn’t matching your NPS rules for connection request and network policy review the NAS Identifier the request is sending in the authentication packet. 

Review the MFA extension logs via Event Viewer > Applications and Services > Microsoft > AzureMFA.

If all is successful we would see a return UDP packet in the capture with response ‘Access-Accept’ from the radius (.10) to Palo (.4). 

Wireshark Capture Radius Response

In the above image you can see the value for ‘Time from request: 11.78794200 seconds‘, this is why I recommend a longer timeout duration on the Radius Server Profile in the Palo Alto configuration. I did it in 11 seconds, but your end users will probably be less efficient than I am.

MFA Options

You can complete the MFA via the Authenticator application on your mobile device via an ‘Approve/Deny’ choice in the notification area or if you’re using SMS code (OTP) the Global Protect client will prompt after successful username and password which is nicely named by default.

Global Protect OTP Request