Categories

TABLE OF CONTENTS

 

Introduction

 

CoreView can support monitoring your MS365 environment and put management of some behaviors on autopilot - wondering how to do that? With KPIs and Workflows!

In the 22.01 release, we introduced a new Workflow feature: an action called 'Operator approval with 'exception'. This feature is the keystone to fully automating policies and enabling complete employee self-service.

It is important to define what a Policy, Workflow, and KPI are before we get into how to set up a KPI.

 

Definitions:

  • Policy - This is a policy established by your organization. For example, your organization may have a policy that states that all users must have Multi-Factor Authentication (MFA) enabled.
  • Workflow - Workflows are repeatable processes that are automated using CoreView. They can be complex or straightforward, multi-step administrative activities. Workflow steps can be conditionalized, and rules developed to ensure that the execution of any workflow is consistent with your organization's practices. You can even introduce an approval step into a workflow, should that be needed.
  • KPI - Key Performance Indicators are essential metrics that you want to track for your organization’s Tenant(s). They are monitored via reports and can be saved to a dashboard so that you can easily track those metrics.
  • KPI Exception – A process to add an exception to a KPI when the KPI needs to be overridden for a set period of time, or if the rule should be ignored forever for a particular incident.

 

A user without CoreView access can approve, decline, or approve with an exception to remediate an issue. We will explain in this walkthrough how we set up a KPI exception.

In this article, we'll cover each step required to configure a process that:

  • Monitors your environment on defined criteria
  • Automatically starts a flow of actions to manage the items that result in a match for the defined criteria
  • Delegates the decision whether to proceed or not with such flow to a defined user of your company (without having them login into CoreView at any time)

 

Note: to follow all the steps on this walkthrough, your account needs to have the following roles: tenantAdmin, WorkflowEditor, WorkflowPublisher.

 

 

How to Configure a KPI

 

A KPI workflow serves to:

  • Monitor a policy you have put in place
  • Show items that are not compliant with the policy criteria
  • Allow a remediation action/workflow

 

KPI Exception:

This process will allow a workflow to have an exception to a workflow you have set. This will enable “nonstandard” (or exception) workflows to bypass the rule. If a workflow is bypassed, a reason will have to be stated, and a date will have to be set for how long the rule will be “bypassed.” That date can be either a set date or be set never to expire.

 

We're going to start configuring a KPI, but first - what's a KPI?

A KPI is an indicator that tracks items of your MS tenant (users, groups, …) that match the criteria (or query) they've been set to monitor. A KPI shows both the count and the list of such items. KPIs on CoreView can be configured starting from almost all reports and are monitored in the KPI dashboard and your custom dashboards. 

 

  1. Open the report you want to create the KPI from and apply the filters needed to filter that report to show only the items you would like to monitor in the KPI. In this example, we will be setting it up to look for inactive Teams groups.

 

Example: To monitor inactive Teams group, navigate to the 'Teams groups activity' report, make sure you add the 'Last activity date' column (if not already visible), and finally set the filter. Select 'Not in the last n days' from the dropdown and enter a value in the text field - This filter will show you only the Teams groups that have no activity in the last n days. We have chosen 90 days for this example

Table Description automatically generated

 

            2. Once the filtering is set, click 'Action' and 'Save 'KPI'.

Graphical user interface, table Description automatically generated

 

            3. In the 'Save KPI' panel, you can configure your KPI further. This panel has four tabs:

  • General tab
    • Type - lets you define whether you want to save this as a simple KPI or want to use it as a template
    • Report name
    • Description
    • Category - defines the category this KPI will be shown in the KPI Dashboard
    • Service - defines the service this KPI will be shown in the KPI Dashboard
    • Severity - predefines the values for the score of this KPI (in the Score tab)
    • Dashboards - defines the dashboard this KPI will be added to
    • How will this affect my users
    • Note

 

  • Score tab
    • This tab lets you configure the points assigned to the KPI. You can configure custom values or choose to use the predefined ones.
    • The KPI has three statuses: compliance, warning, and alert
    • The numbers at the top of the graph define the total score for the KPI - for the related status (I.e., in the example below, if the KPI is in the 'compliance' status, the score of the whole KPI will be 15)
    • The numbers at the bottom of the graph define the thresholds between statuses (I.e., in the example below, if the KPI has two items that match the criteria, it will have the 'warning' status)

 

Chart Description automatically generated with low confidence

 

  • Remediation tasks tab
    • Configure actions - This lets you configure a preferred action to be highlighted in the 'Manage' menu to all operators working with this KPI
    • Configure workflow - This lets you configure a preferred workflow to be highlighted in the 'Manage > Execute 'workflow' menu to all operators working with this KPI

 

  • Exceptions tab
    • In this tab, you can flag the attributes you want to monitor for exceptions: setting this puts in place a control on the items marked as an exception so that if the attribute flagged is changed for that item, the exception expires (and if the item matches the KPI criteria, it will appear back in the list of matching items of the KPI)

 

  • Review and complete

 

Example: Proceeding with the inactive Teams groups example, you want to fill the 'General' tab of the 'Save KPI' panel as shown in the picture:

Graphical user interface, text, application, email Description automatically generated

 

Leave the other tabs to default, proceed to the 'Review and complete' tab, and click save.

Graphical user interface, text, application, email Description automatically generated

  

How to Monitor and Manage a KPI

 

Now that your KPI has been configured, you can monitor it and manage it. To keep your dashboard tidy and compliant, you want to make sure to address all the items that match each of the KPI criteria so that your environment is safe and your KPI/policies are under control at all times. CoreView provides two ways of doing so:

  • Running remediating actions on items
  • Setting items as exceptions

In the example we're providing, the most recommendable remediation action to keep the KPI count low is to delete the Team’s groups that show in the KPI. Sometimes we don't want to delete those Teams, or take action.'Here is where exceptions come into play: setting an item as an exception hides that item from the KPI and excludes it from the count.

 

  1. Navigate to your KPI by going to 'Dashboard > KPI Dashboard'. Here you'll find the KPIs you have associated with this dashboard. You can access the detailed list of the items matching the KPI criteria by clicking on the KPI name.

 

Graphical user interface, application Description automatically generated

 

Example: by clicking on the KPI name 'Inactive Teams groups', you can open the list of the items matching the KPI criteria - in this case, the 16 Teams groups that have been inactive for more than 90 days.

 

Table Description automatically generated

 

  

2. The items in this list are worth your attention: these need to be either fixed with a remediation action or set as an exception.

 

  • To run a remediation action on the items, select the items you want to execute the management action on, then click on 'Manage' and select one of the management actions from the dropdown.
  • To set items as exceptions, select the items you want to set the exception on, then click on 'Action > Manage exception'. In the 'Manage exceptions' panel, select 'Set - Add exception to this KPI' from the first dropdown, and enter the notes and the optional expiry date.

 

Example: Let's fight our Teams sprawl! We can select the Teams groups from the KPI we want to take action on, select the 'Remove Teams group' management action and proceed.

 

Graphical user interface, table Description automatically generated

 

Even though we wish we could delete all these inactive groups, we sometimes just can't - we can select the groups we're aware of, and we're not going to take action on, and set them as an exception - so they're not counted anymore in the KPI. Select the groups you want to set as an exception and click 'Action > Manage Exception'.

 

Graphical user interface, table Description automatically generated

 

 

In the 'Manage exceptions' panel, select 'Set - Add exception to this KPI' from the first dropdown and enter the notes. You can even select an expiry date if you want the exception to last only a determined time - this is super useful when dealing with users on temporary leave.

 

Graphical user interface, text, application Description automatically generated

 

  


How to Automate KPI Remediation

 

At the beginning of this article, we claimed the process we were to build was 100% automatic, but so far, all the management we have shown on KPIs is manual - how can it be automated? The feature that we're looking for is 'Schedule', which is designed to forward a report to an email address (or a KPI in this case) on a recurring basis and optionally execute a workflow - to the items contained in the report (or KPI).

 

  1. From the KPI dashboard, open the list of items matching the KPI by clicking on the KPI name

 

Graphical user interface, application Description automatically generated

 

        2. From the detailed view of the KPI, click on 'Actions > Schedule'

 

Table Description automatically generated

 

 

    3. In the 'Schedule report' panel, you can set the schedule settings:

  • Schedule settings
    • Send when - Lets you define the criteria to send the KPI list to the email addresses (that can be configured in the 'Configure notification' section)
    • Recurrence - This lets you define the recurrence of the notification and execution of the workflow
    • Starting - Defines the start and end of the schedule

 

  • Actions
    • Configure workflow - Here, you can configure the workflow you want to be executed on a scheduled occurrence.  Selecting a workflow will let you map the inputs (or execution inputs) for the workflow - you can map each one of the shown columns to the workflow execution inputs.
    • Configure notification - Defines who is receiving the exported version of the KPI on each schedule occurrence, the format, and the email's body.

 

  • Review and complete

 

Example: To automate the clean-up of your inactive Teams groups, you need to create a new workflow that receives as input the name of the Teams group and has the action 'Remove Teams group'. So, let’s head to ‘Manage’ > Workflow and click “Create new Worflow”. Define the execution inputs and action as the following images:

  

Graphical user interface, application Description automatically generated

Graphical user interface, application Description automatically generated

Graphical user interface, text, application, email Description automatically generated

 

Once you're done with the workflow, save it, publish it and head back to your KPI dashboard. From the detailed list of the 'Inactive Teams groups' KPI, click 'Action > Schedule'. Define your schedule settings, the 'send when' policy (Is not empty makes sure the workflow will execute only when the KPI has Teams groups to be taken care of) and the recurrence.

 

Graphical user interface, text, application, email Description automatically generated

 

 

Proceed to the 'Configure workflow' tab, search from the workflow you just created, and click on it.

Graphical user interface, text, application, email Description automatically generated

  

To complete the workflow configuration, you need to map the fields to be used to run it. Click on the flash icon next to the 'Name' field and select 'GroupName', which is the unique identifier for the Teams groups, then apply changes. Proceed to 'Review and continue' and save your schedule.

  


How to Delegate KPI Management

 

Well done! You have taken control of your Teams group sprawl! Before your well-deserved victory lap, CoreView can help you push the automation one step forward: delegate the decision on whether to perform the remediation workflow or set the item as an exception to anyone from your organization. This is where our latest workflow feature, the new action 'Operator approval with exception', gets the spotlight. Let's see how to tweak your workflow to leverage it!

 

  1. Head to the workflow you want to implement this new feature to (or create a new one)
  2. Click ‘Add action > CoreFlow > Manual’ and select ‘Operator approval with exception’.
  3. In the 'Operator approval with exception' panel, you can configure the action which works similarly to the other 'Operator approval' actions. Here are the fields you can set:
  • Approved by email: defines the email address which will receive the approval request
  • Subject: it's the subject of the approval request email
  • Body: it's the body of the approval request email
  • KPI: this field is needed to tie the workflow to the KPI and be able to set the exception on that KPI
  • Userprincipalname: this field requires the unique identifier for the item the workflow is executed on
  • # of timeout second: defines the amount of time after which  the workflow can be interrupted if the approval is not granted or rejected.

  

Example: Let's modify the 'Clean-up Teams groups' workflow we used in the previous example to host the new action. This way, we can have the Teams group's owner receive the approval request for the deletion. We need to add a new execution input, 'TeamsGroupOwner', and add the 'Operator approval with exception' at the very beginning of the flow. The configuration is:

 

Graphical user interface, application Description automatically generated

Graphical user interface, application Description automatically generated

Graphical user interface, application Description automatically generated

 

Your new workflow is ready - save it and publish it!

  

You can go back to your KPI, schedule it attach this new workflow, and nail the mapping (just like it shown in previous point).

 

 

Approve, Reject or Set an Exception from the Approval Request

 

Now that your KPI is set, the workflow with the new approval flow is scheduled; you just need to relax and wait for CoreView to do all the work. Are you wondering what the user experience is for the users who are receiving the approval request? Here's what you are looking for:

 

  1. The email address set in the workflow  field 'Approved by Email' will receive the approval request email

  

Graphical user interface, application, Teams Description automatically generated

 

 

        2. Clicking on the blue check icon will open the approval request in the browser

Graphical user interface, text, application Description automatically generated

  

    3. The user once had reviewed the request, can proceed by:

  • Approving the request - the workflow continues, and the remediation action is executed on that item
  • Reject the request - the remediation action isn't executed

 

When clicking on 'Reject' to reject the request, the user will be prompted with one more question - whether that item should be marked as an exception or not.

  • If the user selects to mark that item as an exception - the KPI will save the item as an exception, and further execution of the scheduled workflow won’t ask again about that item. The user will also be able to set an expiry date for the exception.
  • If the user chooses not to mark that item as an exception - the item will stay in the KPI list, and the next occurrence of the scheduled workflow will trigger a new request.

 

Graphical user interface, text, application, email Description automatically generated

 

You have now set up an KPI Exception for monitoring.

 

 

Other Use Cases

 

KPIs, and KPI Exceptions are a great way for you to monitor your Tenant(s). Here are a few other Use Cases that you can also apply this process too:

Use Cases:

  • Users without Multi-Factor Authentication (MFA) enabled
  • Clean up MS365 groups without owners
  • Monitor high-risk users
  • Inactive guest users
  • Guest Users with assigned licenses
  • Administrators with licenses
  • Users with external email forwarding