Azure Automation Account Runbooks - Limitations and Performance Issues

[rogermoser]

Description of the issue

With one Azure Automation account, we have multiple deployments that export all configured components.
With the newer module, we are increasingly running into the knowing limits (400MB) [https://learn.microsoft.com/en-us/azure/automation/automation-subscription-limits-faq]
Based on this issue, we stripped the script to export components one by one - but still we get in this issue example 'AADAuthenticationMethodPolicy'

Microsoft 365 DSC Version

1.25.205.1

Which workloads are affected

other, Azure Active Directory (Entra ID)

The DSC configuration

Example for Export: 
Export-M365DSCConfiguration -Components @($ExportCompParam) -ApplicationId $ApplicationId -CertificateThumbprint $CertificateThumbprint -TenantID $TenantFQDN -Path $exportRootFolder -FileName $CFGFileName -ConfigurationName $CFGConfigurationNameInformation

Verbose logs showing the problem

RUN1

----------------------------------------------------

Starting Script to Export

2025-02-24 13:40

Module bevor connecting

Module after connecting


ModuleType Version    Name                                ExportedCommands                                              
---------- -------    ----                                ----------------                                              
Script     2.25.0     Microsoft.Graph.Authentication      {Add-MgEnvironment, Connect-MgGraph, Disconnect-MgGraph, Ge...

Script     1.1.36     MSCloudLoginAssistant               {Add-MSCloudLoginAssistantEvent, Assert-IsNonInteractiveShe...

2025-02-24,13:40:56,Success to cleanup Memory usage after Connect-M365Tenant : 25.18 MB

C:\app\


Path  
----  
C:\app

2025-02-24,13:40:58,Success to cleanup Memory usage after collecting all variable : 29.17 MB

trying to export the configuration

Exporting Microsoft 365 configuration for Components: AADAuthenticationMethodPolicy

Module after export


ModuleType Version    Name                                ExportedCommands                                              
---------- -------    ----                                ----------------                                              
Binary     1.0        Automation.Sandbox.AssetManageme... {Get-AutomationCertificate, Get-AutomationConnection, Get-A...

Binary     1.0.0.0    CimCmdlets                          {Export-BinaryMiLog, Get-CimAssociatedInstance, Get-CimClas...

Script     2.25.0     Microsoft.Graph.Applications        {Add-MgApplicationKey, Add-MgApplicationPassword, Add-MgSer...

Script     2.25.0     Microsoft.Graph.Authentication      {Add-MgEnvironment, Connect-MgGraph, Disconnect-MgGraph, Ge...

Script     2.25.0     Microsoft.Graph.Beta.Identity.Go... {Get-MgBetaEntitlementManagementAccessPackage, Get-MgBetaEn...

Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}     

Manifest   1.25.205.1 Microsoft365DSC                     {Add-M365DSCEvent, Add-M365DSCTelemetryEvent, Assert-M365DS...

Script     1.1.36     MSCloudLoginAssistant               {Add-MSCloudLoginAssistantEvent, Assert-IsNonInteractiveShe...

Manifest   1.1        PSDesiredStateConfiguration         {Invoke-DscResource, Publish-DscConfiguration, Set-DscLocal...

Manifest   2.0.0.22   ReverseDSC                          {Add-ConfigurationDataEntry, Add-ReverseDSCUserName, Conver...

2025-02-24,13:44:21,Success to cleanup Memory usage after Exporting Configuration : 543.35 MB

RUN2 

Fehler
Thread failed to start. (Thread failed to start. (Exception of type 'System.OutOfMemoryException' was thrown.))
----------------------------------------------------
Starting Script to Export
2025-02-24 15:33
Module bevor connecting
Module after connecting

ModuleType Version    Name                                ExportedCommands                                              
---------- -------    ----                                ----------------                                              
Script     2.25.0     Microsoft.Graph.Authentication      {Add-MgEnvironment, Connect-MgGraph, Disconnect-MgGraph, Ge...
Script     1.1.36     MSCloudLoginAssistant               {Add-MSCloudLoginAssistantEvent, Assert-IsNonInteractiveShe...
2025-02-24,15:33:43,Success to cleanup Memory usage after Connect-M365Tenant : 24.82 MB
C:\app\

Path  
----  
C:\app
2025-02-24,15:33:45,Success to cleanup Memory usage after collecting all variable : 29.57 MB
trying to export the configuration
Exporting Microsoft 365 configuration for Components: AADAuthenticationMethodPolicy
 
Authentication methods specified:
- Service Principal with Certificate Thumbprint
 
Connecting to {MicrosoftGraph}...
✅
[1/1] Extracting [
AADAuthenticationMethodPolicy
] using {
CertificateThumbprint
}...

Environment Information + PowerShell Version

Azure Automation Runbook PowerShell 5.1
Modules installed on Automation Account
Self-Signed certificate with KeyVault for authentication 
Storage Account to store exported configuration

[FabienTschanz]

@rogermoser We ran into the same issue, therefore we switched to an Azure Automation Hybrid Worker running on a Windows 11 VM. Not exactly a very cheap alternative, but for our company, this works very well. Performance is much better and you can export many components and workloads at the same time, given you assign at least 8GB of RAM to the machine.

Solely with the default Azure Automation runners, you most likely won't be able to do any proper things with Microsoft365DSC.

[FabienTschanz]

@rogermoser Does my answer help you in any way or do you require some further assistance?

[jackpoz]

I created #6571 as a VM is way too overkill in some cases, like when just exporting tenant-wide settings.