Staying in Control of your MFA rollout with Azure Monitor and Logic Apps

10 min readDec 9, 2020

In September I had written a blog about how to do an MFA Rollout and tried to cover as many aspects as possible. However, I would like to take a closer look at one aspect that is particularly important in larger projects — where does my rollout actually stand at the moment?

The Problem

Above a certain size, an MFA rollout runs in waves and we can usually only determine the start of a wave and then help a little with measures. But if there are follow-up activities like an Intune Rollout it is very important to know exactly where our rollout is and how high the coverage is.

To see how well the new registrations are doing I like to build a dashboard that shows the events around the setup and usage of MFA. Unfortunately, this data is not suitable for a precise location determination regarding the coverage of a rollout group — for this, a current status is needed and we need a possibility to filter by groups.

Due to the display options, especially with regard to the time history, I still want to use Azure Monitor and expand my dashboard.

The solution approach

With this challenge in mind I read Jan Bakker’s very good blogs about MFA registrations (here and here) on the one hand and the Azure Sentinel webinar about tackling identity (here) on the other hand and gradually a plan to solve my problem was born.

With the help of a Logic App I would like to read the data regularly from the AAD and write them into tables. Then I would like to link the tables and display a time history.

Logic Apps have a native connector to the AAD and can therefore read groups and their members. For the MFA Authentication Methods there is no option to use an existing connector but there is a method to get a report via MS Graph. That means I have to work on building a custom connector — luckily I can learn from Jan’s blog here.

In the first version, every 30 minutes the complete MFA report is to be placed in one table and all members of a group in another table. After that it is very easy to count a little bit by linking the two tables and to create an areachart that shows the time course. Furthermore I can imagine to use the two tables for some more use cases.

In my lab environment this approach worked great, because I had only ~100 users in the tenant and 10 users in the group. In a somewhat larger production environment I had to realize that there are limitations in many places and my Auth Method Report ended with just over 1000 users :-(

So my production approach unfortunately uses a loop, runs less often (every 2 hours) and therefore takes a little longer — but scales much better.

Implementation of the solution

The implementation is divided into the following components:

Preparations in AAD (Group and App Registration)
Creation of the Logic App (connector and process)
Creating charts with Azure Monitor (query and widget for the dashboard)

Preparations in AAD

The Group

Since it is generally useful to control rollout waves via groups, we also use AAD groups for filtering. The used group should be a Security Group. Whether it is a synchronized or cloud group and a static or dynamic group is irrelevant and should be based on your needs regarding the rollout strategy. Says: “If you want to go by location, it might be a good idea to use a dynamic group that evaluates the usage location.

For the first test I used a group of the Cloud-Assigned type and randomly added a few users. Important for the further process is the Object Id of the group (1).

The User

For the AAD Connector and the Custom Connector (I haven’t been able to get it to work with application permissions only) we still need a function user with the role Global Reader in whose context interfaces can be addressed.

For function users I currently follow the following approach:

Long passwords (min. 32 characters)
Clear naming, clear assignment (which Logic App?) and clear responsibility (which Admin? — use of the attribute Manager)
Inclusion in a Privileged Access Group to build an Access Review and to make any necessary exceptions to conditional access.
Monitoring for abnormal behavior — Sentinel ;-)

The App

For the custom connector of the Logic app we need an app registration to which we can delegate the necessary rights to access the MFA report. Beside the existing User.Read permissions I have added these Delegated Permissions:

Microsoft Graph — AuditLog.Read.All
Microsoft Graph — UserAuthenticationMethod.Read.All

After some time we had to notice that (apparently due to changes in this area) permission denied errors occurred more and more often. To avoid these, the Permission Directory Read All is necessary until further notice.

To use the app for the connector you also need a Client Secret and these Redirect URIs:

1 PowerAutomate (optional for future use)
https://global.consent.azure-apim.net/redirect
2 Logic Apps
https://logic-apis-westeurope.consent.azure-apim.net/redirect

Creating the Logic App — Overview

As described above our Logic App should read the members of a group every 120 minutes (1) generate an Auth Method Report for each user and write the combined data into a table in Log Analytics (3).

Creating the Custom Connector for the Logic App

To generate my report I decided to build a very simple connector, which only contains the method GetReport and does not do any further preparation of the data (there is still potential here).

To simplify the move from my test environment to production I have saved the finished connector as a template and describe here only the use of the template.

Logic Apps Custom Connector is an own resource type in Azure and via Add you can deploy a new empty connector to the resource group of your choice. After successful deployment, click Edit to get to the following dialog where you can upload the JSON file with the template (1):

Then we add an icon (1), a description (2) and a host (3). For the host I have chosen graph.microsoft.com

On the following security page, the link to the pre-configured app registration takes place. Here the Object ID of the app (1) and the generated Client Secret (2) are configured. The redirect URL for the Logic App (3) is also defined here. Important: If the URL differs from the one configured above (e.g. because your app is not located in Western Europe) you have to enter the app registration and correct the value again.

For the resource URL I have used https://graph.microsoft.com

Finally the method itself is defined and the request is defined. As I said, I kept it very simple and just configure a get against the URL (1) for the report: https://graph.microsoft.com/beta/reports/credentialUserRegistrationDetails

To filter the report the parameter filter is added (2).

Getting the Group Members

Our first action is to pick up the members of the group and then pass them to the following loop. To do this, only the Group ID is specified. Because the function only gives out a very clear number of group members (in my case it was 100) it makes sense to switch on pagination in the settings (2) and define a suitable size for the threshold.

The Loop

Now that we have all group members (1) we have to generate an MFA Auth Method Report for each user in a loop (2), parse it (3) and send it to Log Analytics (4).

The syntax for filtering the report is userPrincipalName eq ‘ UPN’ and is simply filled with the UPN of the current object. But before the output can be sent to Log Analytics it has to be parsed. To get the scheme for this we can use the Graph Explorer. Simply log in, query the endpoint (1) and if needed give a consent (2) gives a response (3) which can be used in the next step to recognize the schema.

The function Parse JSON is to take the body of the response from the custom connector (1) and split it into individual fields. The schema (2) of the data can be derived from the response of the Graph Explorer.

The last step takes the parsed values (1), defines the content as JSON (2) and passes it to a custom log (3).

The Query / Chart

After the first run of the Logic App, there is now a new table in the Custom Logs area that you can continue working with.

I am first interested in how many of the group members have already completed their MFA registration. All that is needed is a counting and you should make sure that the time interval matches the recurrence in the Logic App, otherwise double counts may occur.

let recurrence = 60m;
YourTableName
| summarize unregistered=countif(isMfaRegistered_s == “False”), xregistered=countif(isMfaRegistered_s == “True”) by length=bin(TimeGenerated, recurrence)
| render timechart with (title=”Users with successfull MFA registration”, legend=visible, yaxis=linear, ytitle=”users”, xtitle=”time”, kind=stacked )

Over time this will extend to a timechart and you can track the progress by pinning the query to your favourite dashboard.

An alternative approach

During my considerations and research I also found a way to achieve a similar result with Powershell instead of the Logic App. I didn’t pursue this any further, as I find the solution described here much more elegant. The main arguments for the Logic App are the graphical designer, better credential handling and integrated logging. But I still want to share the fragments here — maybe it will be of use to someone.

Microsoft provides a Powershell Script to retrieve the MFA report and generate a CSV. There is also a script in the PowerShell Gallery which imports a CSV into a Log Analytics. With this 3-liner the task can be fulfilled:

.\MfaAuthMethodAnalysis.ps1 -TenantId $TenantID -TargetGroup $TargetGroup -CsvOutput -Verbose$latest = Get-ChildItem -Path $dir | Where-Object {$_.Extension -eq '.csv'} | Sort-Object LastAccessTime -Descending | Select-Object -First 1Import-Csv $latest | Upload-AzMonitorLog.ps1 -WorkspaceId $WorkspaceId -WorkspaceKey $WorkspaceKey -LogTypeName $LogTypeName

Summary

I hope to get a high transparency for the rollout with this new data to learn how to improve the procedure for the following waves and future rollouts. I am also pretty sure that I will extend the “basic solution” created here with various things in the near future.

Especially important to me was the experience with the migration of the solution into a production environment, because it showed once again how fast one can run in limitations. The need to change the logic to a loop and the use of pagination are things you usually don’t learn in a lab. :-)

Independently of that use case, I like to take every opportunity to learn more about Kusto Query Language and Logic Apps, as this knowledge is also strongly required in security solutions like M365 Defender and Azure Sentinel.