Mastering Azure Update Manager – Part 2

In Part 1 we saw how to onboard machines to Azure Update Manager on a per machine level or on demand which is fine for a small number of machines to deal with.

In this part we will see how we can onboard machines at scale to Azure Update Manager by using Azure Policy.

Azure Policy

As mentioned, while onboarding machines on a per machine level to Azure Update Manager is fine for a small number of machines, this is getting quite inconvenient when the number of machines will raise.

Previously we configured periodic assessment and patch orchestration by using the Update Settings on a per machine level.

Now we want to do the same by using therefore Azure Policy which will scale these settings automatically.

Under the Get started section we see a notification that we can also assign built-in policies for regularly checking updates at scale.

To navigate to these built-in policies we can either click above on this link or navigate to Manage -> Policy within the Azure Update Manager.

Finally we will focusing here on the following three built-in policies.

Configure periodic checking for missing system updates on azure virtual machines
Schedule recurring updates using Azure Update Manager
Configure periodic checking for missing system updates on azure Arc-enabled servers

Below I am highlighting the Configure periodic checking for missing system updates on azure virtual machines policy which will replace our periodic assessment (per machine level) update configuration we previously configured in Part 1 for each machine.

So far this policy is not assigned to any resources.

This policy will force the machines to check for updates every 24 hours.

I will assign this policy to my subscription and specific resource group with the desired machines I want to manage updates.

We can also exclude resources here.

By default, this assignment will only take effect on newly created resources. Existing resources can be updated via a remediation task after the policy is assigned.

To also update my existing virtual machines, I will also create a remediation task below.

Here I can see all assigned scopes for this policy.

So far the policy isn’t remediated on my existing virtual machines.

Still in state evaluating.

Now in progress.

And finally completed.

We can scroll down to see all machines to which the policy is now associated.

Looks good!

The second setting we previously configured in Part 1 on a per machine level by using the Update Settings is Patch orchestration.

Therefore we can also use a built-in policy named Schedule recurring updates using Azure Update Manager like shown below.

As the name already implies, this will finally enable our machines to be updated by using the Azure Update Manager.

So far this policy is also not associated to any resources.

I will also assign this policy to my subscription and resource group as previously for the periodic assessment.

On the Parameters tab we need to provide the Maintenance Configuration ARM ID.

A mandatory parameter to be provided. It denotes the Azure Resource Manager (ARM) ID of the schedule that you want to assign to the machines.

The Maintenance Configuration ARM ID you will find on the properties blade of your Maintenance Configuration as shown below.

I will therefore first create a new Maintenance Configuration.

For testing purpose and to see the results the next day immediately, I will configure to repeat this schedule on a daily basis.

For testing purpose I will also select the resources directly and don’t use dynamic scopes for.

How to use dynamic scopes we already saw in Part 1.

I will also include all updates and not filter for specific, finally I will create the schedule and copying the Maintenance Configuration ARM ID, which is needed for the Schedule recurring updates using Azure Update Manager policy.

So after inserting the Maintenance Configuration ARM ID for the Parameters tab, we can go to the next tab.

Here I will also create a remediation task to update my existing virtual machines.

Finally we can assign the policy.

So far neither the Patch orchestration was changed to Custom Managed Schedules nor our schedule (Maintenance Configuration ARM ID) is associated with our virtual machines.

I will also here check if the remediation is still running for this policy. Therefore we click on the policy and under the Assignments tab on the assignment name.

Here we see within the Remediation tab that the remediation task is still evaluating. By clicking on it we will see more details.

Now in Progress and will be associated with my virtual machines.

Finally the remediation task was applied successful to all virtual machines besides one which is an Azure Arc-enabled server.

!! Note !!
Patch orchestration is not applicable to Arc-enabled servers. But we also can enable periodic assessment for them to check for updates every 24 hours and schedule when patches and updates are applied to Arc-enabled servers. For the update source they will use their already configured source on the machine itself (usually Windows Updates directly or WSUS).

Even though patch orchestration is not available, you can still use Azure Update Manager to manage updates for Arc-enabled servers. While patch orchestration is unavailable, you can still manually schedule when patches and updates are applied to Arc-enabled servers using Azure Update Manager. You can create custom maintenance windows and specify which updates should be applied.

Azure Update Manager can still provide compliance reports for Arc-enabled servers, showing which updates are missing and which have been applied. This helps with visibility into the update status of your Arc-enabled machines.

You can assess whether Arc-enabled servers are up to date by using Azure Update Manager to perform an update assessment, showing missing critical, security, or feature updates.

To avoid this error we can configure exclusions for this policy. For the policy assignment click on Edit assginment.

Because for the scope I selected my subscription and resource group in which also this Azure arc-enabled server was placed, now I need to add here an exclusion.

Finally I can save the assignment to apply this exclusion.

In the Azure Update Manager blade under Resources -> Machines we can see that the built-in policy named Schedule recurring updates using Azure Update Manager aka Patch orchestration was associated successfully.

Further we can see that our newly created schedule aka Maintenance Configuration now is also associated along with the Patch orchestration policy.

Create Alert Rule (Preview)

Logs created from patching operations such as update assessments and installations are stored by Azure Update Manager in Azure Resource Graph (ARG). You can view up to last seven days of assessment data, and up to last 30 days of update installation results.

To enable alerts (Preview) with Azure Update Manager through Azure portal, under Monitoring, select New alerts rule (Preview) to create a new alert rule.

We can either create a custom query or using pre-defined queries. Below I will use the pre-defined queries named Patch installation failures.

Click on Continue to create a new alert rule.

Aggregation granularity defines the time interval over which the collected values are aggregated. For example, if the aggregation granularity is set to 5 minutes, the alert rule evaluates the data aggregated over the last 5 minutes. If the aggregation granularity is set to 15 minutes, the alert rule evaluates the data aggregated over the last 15 minutes. It is important to choose the right aggregation granularity for your alert rule, as it can affect the accuracy of the alert.

Source: https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/tutorial-log-alert#configure-condition

Also on the conditions tab we need to enter a treshold value for the alert logic.

Operator -> The query results are transformed into a number. In this field, select the operator to use for comparing the number against the threshold.
Threshold value -> A number value for the threshold.
Frequency of evaluation -> How often the query is run. You can set it anywhere from one minute to one day (24 hours).

In the alert logic, configure the Operator and Threshold value to compare to the value returned from the measurement. An alert is created when this value is true. Select a value for Frequency of evaluation which defines how often the log query is run and evaluated. The cost for the alert rule increases with a lower frequency. When you select a frequency, the estimated monthly cost is displayed in addition to a preview of the query results over a time period.

For example, if the measurement is Table rows, the alert logic may be Greater than 0 indicating that at least one record was returned. If the measurement is a columns value, then the logic may need to be greater than or less than a particular threshold value. In the following example, the log query is looking for anonymous requests to a storage account. If an anonymous request is made, then we should trigger an alert. In this case, a single row returned would trigger the alert, so the alert logic should be Greater than 0.

Source: https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/tutorial-log-alert#configure-alert-logic

On the Details tab we need to use for the Identity either System assigned managed identity or User assigned managed identity.

I will use here a System assigned managed identity.

We also need to grant permissions to this identity after creating the rule as shown further below.

The created alert rules we will find under Monitor -> Alerts -> Alert rules as shown below.

As mentioned further above, we also need to grant permissions to the identity we are using for this rule. Therefore click above on the rule and navigate to Settings -> Identity.

Click on Azure role assignments.

Click on Add role assignment and select the desired subscription.

In order our new rule have access to the Azure Resource Graph logs at least we need to grant reader permissions.

https://learn.microsoft.com/en-us/azure/update-manager/query-logs

Create pre and post events

Pre and post events allows you to execute user-defined actions before and after the scheduled maintenance configuration. 

Azure Update Manager leverages Event Grid to create and manage pre and post events. For more information, go through the overview of Event Grid.

Hotpatch

Hotpatch is a feature that allows you to apply security updates and patches to your Windows Server virtual machines (VMs) without requiring a reboot. This significantly reduces downtime and disruption to workloads by allowing critical updates to be installed while the system continues to run, particularly in scenarios where uptime is crucial.

It works by patching the in-memory code of running processes without the need to restart the process.

Hotpatch is supported only on VMs and Azure Stack HCI created from images with the exact combination of publisher, offer and sku from the below OS images list. Windows Server container base images or Custom images or any other publisher, offer, sku combinations aren’t supported.

Source: https://learn.microsoft.com/en-us/windows-server/get-started/hotpatch#supported-platforms

Below for example you will see the settings for the image Windows Server 2022 Datacenter: Azure Edition on which hotpatch is not supported.

Below you will see the supported image Windows Server 2022 Datacenter: Azure Edition Hotpatch shown below.

More about you will find in the following article https://learn.microsoft.com/en-us/windows-server/get-started/hotpatch.

Azure Update Manager Pricing

Azure Update Manager is available at no extra charge for managing Azure VMs and Arc-enabled Azure Stack HCI VMs (for which Azure Benefits are enabled).

For Arc-enabled Servers, the price is $5 per server per month (assuming 31 days of usage).

Source: https://learn.microsoft.com/en-us/azure/update-manager/update-manager-faq#what-is-the-pricing-for-azure-update-manager

Azure Arc Pricing

Azure Arc is a bridge that extends the Azure platform to help you build applications and services with the flexibility to run across datacenters, at the edge, and in multicloud environments.

As already mentioned Azure-Arc basically is free when not using the separate additional capabilities for which we need to pay like Defender for Cloud for Azure-Arc enabled server. In this case you just pay for the Azure service you will consume with your Azure-Arc enabled server.

Source: https://azure.microsoft.com/en-us/pricing/details/azure-arc/data-services/

Add-on Azure management services, such as Azure Update Manager, Azure Policy guest configuration, Azure Monitor, Microsoft Defender for Cloud, and Microsoft Sentinel, are charged for Azure Arc-enabled server when enabled.

The total monthly price of Azure Arc-enabled servers depends on the number of Azure management and security services you run on each server and the plan or SKU purchased.

In Part 3 I will focus on how to updating SUSE Linux Enterprise by using the Azure Update Manager.

Links

Azure Updates
https://azure.microsoft.com/en-us/updates

About Azure Update Manager
https://learn.microsoft.com/en-us/azure/update-manager/overview

Assessment options in Update Manager
https://learn.microsoft.com/en-us/azure/update-manager/assessment-options

Azure Update Manager 2023 Edition (John Savill)
https://www.youtube.com/watch?v=LVmNh5YgBOQ

Azure Update Manager frequently asked questions
https://learn.microsoft.com/en-us/azure/update-manager/update-manager-faq#what-is-the-pricing-for-azure-update-manager

Deploy updates now and track results with Azure Update Manager
https://learn.microsoft.com/en-us/azure/update-manager/deploy-updates?tabs=install-single-overview

What is OVAL and how can I use it to learn about security issues?
https://access.redhat.com/solutions/4161

What is Azure Resource Graph?
https://learn.microsoft.com/en-us/azure/governance/resource-graph/overview

Azure Monitor overview
https://learn.microsoft.com/en-us/azure/azure-monitor/overview

Overview of Log Analytics in Azure Monitor
https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-overview

Log Analytics tutorial
https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-tutori

Quickstart: Run Resource Graph query using Azure portal
https://learn.microsoft.com/en-us/azure/governance/resource-graph/first-query-portal