Manage Azure Monitor agent on virtual machines

• Author
• Recent Posts

Baki Onur Okutucu

Onur is a DevOps Engineer and Subject Matter Expert for Azure, and PowerShell. He is the founder of Clouderz Ltd, a cloud consultancy based in London. For 13 years in a row, Microsoft has recognized him as an MVP (Most Valuable Professional). You can follow Onur on Twitter: @BakiOnur.

Latest posts by Baki Onur Okutucu (see all)

• Create and manage append blobs with PowerShell - Wed, Oct 12 2022
• Permanently delete a Key Vault in Azure using PowerShell - Fri, Feb 4 2022
• Restore Azure Files with PowerShell - Fri, Jan 28 2022

AMA supports multihoming for both Linux VMs and Windows VMs. What that means is that we can send the logs to multiple locations, such as Log Analytics workspaces. This way, for example, one team can access security logs, while the other team works with the application logs of certain VMs. In this post, we will install and configure the AMA on Azure VMs using PowerShell.

AMA concept

AMA has a data collection method that is managed by a resource called the "Data Collection Rule" (DCR). This rule stores all the required info, such as which VMs or scale sets to monitor, where to send the logs, and which types of logs to collect. So, instead of deploying the extension with customized configurations such as a Log Analytics workspace ID or workspace key onto the VMs, a DCR is used to store all this information. Once a DCR is created, VMs and VM scale sets can be added as data sources. With this method, VM extensions can be installed automatically when a VM or VMSS is added to a DCR. Alternatively, the AMA extension can be installed using PowerShell without any specific configuration.

While AMA is a more flexible method for collecting logs from virtual machines, there are also some limitations, such as the inability to use storage accounts and event hubs as destinations and the lack of support for private links.

Installing the AMA

Agents can be installed either manually or via DCRs. To install the AMA agent on a VM, we can use the following commands:

For Windows VMs

Set-AzVMExtension -Name AzureMonitorAgent -ExtensionType AzureMonitorWindowsAgent -Publisher Microsoft.Azure.Monitor -ResourceGroupName latest -VMName latest0001 -Location northeurope -TypeHandlerVersion 1.0

Azure Monitor Agent extension on a Windows VM

For Linux VMs

Set-AzVMExtension -Name AzureMonitorAgent -ExtensionType AzureMonitorLinuxAgent -Publisher Microsoft.Azure.Monitor -ResourceGroupName latest -VMName latest4 -Location northeurope -TypeHandlerVersion 1.5

Azure Monitor Agent extension on a Linux VM

Creating and configuring DCRs

DCRs are standalone objects in which we configure all the settings around the sources, destinations, and logs. So, to create a DCR using PowerShell, we need to prepare the rule file, which is in JSON format. This file contains all the information about which types of logs will be collected and which destinations will be used.

Creating the rule file

A sample rule file looks like the below. Within this file, three data sources are defined: Windows event logs, Linux syslogs, and performance counters. In addition, a Log Analytics workspace is specified as the destination. As you can see, we set the log destination on the DCR and not on the VM.

dcr.json
{
    "properties": {
        "dataSources": {
            "windowsEventLogs": [
                {
                    "name": "windowsSecurityEvents",
                    "streams": [
                        "Microsoft-Event"
                    ],
                    "xPathQueries": [
                        "Security!"
                    ]
                },
                {
                    "name": "windowsSystemAndAppEvents",
                    "streams": [
                        "Microsoft-Event"
                    ],
                    "xPathQueries": [
                        "System![System[(Level = 1 or Level = 2 or Level = 3)]]",
                        "Application!*[System[(Level = 1 or Level = 2 or Level = 3)]]"
                    ]
                }
            ],
            "performanceCounters": [
                {
                    "streams": [
                        "Microsoft-Perf"
                    ],
                    "scheduledTransferPeriod": "PT1M",
                    "samplingFrequencyInSeconds": 30,
                    "counterSpecifiers": [

                        "\\Processor(_Total)\\% Processor Time",
                        "\\PhysicalDisk(_Total)\\Avg. Disk Queue Length",
                        "\\Memory\\Committed Bytes",
                        "\\LogicalDisk(_Total)\\Free Megabytes"
                        
                    ],
                    "name": "performanceCounters"
                }
            ],
            "syslog": [
                {
                    "name": "linuxCronSyslogs",
                    "streams": [
                        "Microsoft-Syslog"
                    ],
                    "facilityNames": [
                        "cron"
                    ],
                    "logLevels": [
                        "Debug",
                        "Critical",
                        "Emergency"
                    ]
                },
                {
                    "name": "LinuxSyslogBase",
                    "streams": [
                        "Microsoft-Syslog"
                    ],
                    "facilityNames": [
                        "syslog"
                    ],
                    "logLevels": [
                        "Alert",
                        "Critical",
                        "Emergency"
                    ]
                }
            ]
        },
        "destinations": {
            "logAnalytics": [
                {
                    "workspaceResourceId": "/subscriptions/81391d2a-c61e-47f3-94db-eb5d319d509c/resourceGroups/LAtest/providers/Microsoft.OperationalInsights/workspaces/latest1",
                    "name": "centralLogAnalyticsWorkspace"
                }
            ]
        },
        "dataFlows": [
            {
                "streams": [
                    "Microsoft-Perf",
                    "Microsoft-Syslog",
                    "Microsoft-Event"
                ],
                "destinations": [
                    "centralLogAnalyticsWorkspace"
                ]
            }
        ]
    }
}

Creating the DCR

Once the rule file is ready, we can deploy the DCR using the following command:

New-AzDataCollectionRule -ResourceGroupName "latest" -Location northeurope -RuleName "DCRule01" -RuleFile ".\dcr.json" -Description "Data Collection Rule"

There are two main configs on a DCR resource: data sources and resources.

We configure log settings and destinations in data sources, and specify the actual VMs or VM scale sets to monitor in resources.

Data Collection Rule Data Sources

Each data source can be modified by editing logs or counters.

Performance counters can be configured in a data source

Logs and log levels for Windows event logs in a data source

Linux syslog can also be configured in a data source

Multiple destinations for the same logs can be configured on the DCR. This is also called multihoming. AMA supports multihoming for both Windows and Linux VMs.

Multiple destinations such as Log Analytics workspaces can be configured

Creating a data collection rule association

On a DCR, a rule association is needed to add the actual VMs to the scope. Only then will the VMs or VM scale sets be configured to send the required logs to the specified destinations.

The following commands can be used to create rule associations to add a Windows and a Linux VM to the DCR as resources.

$dataCollectionRule = Get-AzDataCollectionRule -ResourceGroupName latest -RuleName DCRule01
$WindowsVMId = '/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/LAtest/providers/Microsoft.Compute/virtualMachines/latest0001'
$LinuxVMId = '/subscriptions/ xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /resourceGroups/LAtest/providers/Microsoft.Compute/virtualMachines/latest4'
New-AzDataCollectionRuleAssociation -TargetResourceId $WindowsVMId -AssociationName "dcrAssoc" -RuleId $dataCollectionRule.Id
New-AzDataCollectionRuleAssociation -TargetResourceId $LinuxVMId -AssociationName "dcrAssoc" -RuleId $dataCollectionRule.Id

VMs and VM scale sets can be added as resources in DCRs

When a new VM is added, the AMA extension for that VM is also automatically installed.

Getting logs in the Log Analytics workspace

Logs sent to Log Analytics can be accessed using the following queries:

For Linux Syslog

Syslog
| project Computer, Type, SourceSystem

For performance counters

Perf
| project Computer, ObjectName, CounterName, CounterValue

For Windows events

Event

Logs can be checked in the Log Analytics workspace that is set as the destination

Conclusion

The AMA provides a more granular way of monitoring virtual machines using standalone DCR objects. Although it is currently limited to VMs, VM scale sets, and ARC-enabled servers, it is still useful when it comes to sending logs to different destinations, such as Log Analytics workspaces for both Windows and Linux VM resources.