How to Create a KPI Exception and How to use it
Modified on Thu, 07 Apr 2022 at 10:00 AM
- CoreView Release Notes May 2023
- CoreView Release Notes April 2023
- CoreView Release Notes March 2023
- CoreView Release Notes February 2023
- CoreView Release Notes January 2023
- CoreView December 2022 Release Notes
- CoreView November 2022 Release Notes
- CoreView October 2022 Release Notes
- September 2022 Release Notes
- August 2022 Release Notes
- Release 22.06 Key Features
- Release 22.05 Key Features
- Release 22.04 Key Features
- Release 22.03 Key Features
- Release 22.01 Key Features
- Release 21.12 Key Features
- Release 21.11 Key Features
- Release 21.10 Key Features
- Release 21.09 Key Features
- Release 21.08 Key Features
- Release 21.07 Key Features
- Release 21.05 Key Features
- Release 21.04 Key Features
- Release 21.03 Key Features
- Release 21.02 Key Features
- Release 21.01 Key Features
- Release 20.12 Key Features
- Release 20.11 Key Features
- Release 20.10 Key Features
- Release 20.09 Key Features
- Release Information
- Getting Started with Customer Care
Getting Started with CoreView
- Configuration Overview (New UX)
- Configuration Overview
- Creating CoreView Tenant Administrators (New UX)
- Creating CoreView Tenant Administrators
- CoreView Operator Uses Cases & Dependencies
- Creating a License Pool
- Understanding Virtual Tenants
- Frequently Asked Configuration Questions
- "Send As" DNS Requirements for CoreAdoption Campaigns (Optional)
- How to enforce MFA on CoreView service accounts
- Creating a License Pool
A Quick Tour of CoreView
- A Quick Tour of the CoreView Interface
- Introducing the CoreView New User Experience
- CoreView New UX FAQ
- CoreView Dashboards
- Using CoreView Reports
- How to use CoreView Management Actions (New UX)
- How to use CoreView Management Actions
- Understanding CoreView Releases
- Creating your first Workflow - A Practical Exercise
- Introducing the new site for Partner customer management
- How to check and analyze the Message Trace
- How To Configure Email Forwarding
- How to convert a Shared Mailbox to a User Mailbox
- How to convert a user mailbox to a shared mailbox in Exchange Online
- How to Create Microsoft 365 Groups for Improved Collaboration
- How To Create Shared Mailbox
- How To Create User Mailbox
- How To Grant Access To Mailbox
- How to List all the Mailboxes a User has access to in Microsoft 36
- How to remove delegates from Mailbox
- How to remove user access to Mailbox
- How to review and manage Exchange online mailbox permissions
- How to verify if a user has updated the Password
- Read Permission for Mailbox
- What are security groups and How to create it
- What is a Distribution Group and How to create it
- Exchange Online
- Custom Actions Library
- Getting Started with CoreHybrid
Understanding CoreView - Quick Start Guides.
- CoreView Quick Start Guide Overview and Index - Tenant Admins
- CoreView Quick Start Guide Overview and Index - Operators
- Understanding CoreView Tenant Configuration – Management
- Understanding the CoreView Operator Profile
- Understanding CoreView Operator Roles
- Understanding CoreView Operator Delegation
- Understanding CoreView - Report Column Filtering
- Understanding CoreView - The User Interface
- Understanding CoreView Tenant Configuration - V-Tenant User Filters
- Understanding CoreAdoption – Templates and Campaigns
- Understanding CoreLearning - Content Hierarchy
- Understanding CoreView Tenant Configuration - Portal Information
- Understanding CoreView Tenant Configuration - CoreLearning
- Understanding CoreView Tenant Configuration Options
Troubleshooting Common Issues
- Unable to see OneDrive, SharePoint and Exchange Data (New UX)
- Unable to see OneDrive, SharePoint and Exchange Data
- Remote Office 365 PowerShell session can Conflict CoreView Management Actions
- Why I cannot save the changes on existing License pool?
- Error when attempting to perform a Management Action (New UX)
- Error when attempting to perform a Management Action
- Unable to modify the Assigned Licenses in my License Pool Report
- Enabling Permission for Endpoint Manager Actions (New UX)
- Enabling Permission for Endpoint Manager Actions
- How to enable permission for BitLocker keys report (New UX)
- How to enable permission for BitLocker keys report
- How to recreate Admins Read-only (New UX)
- How to recreate Admins Read-only
- How to add an operator to the portal?
- How to enable and configure CoreView management session (New UX)
- How to enable and configure CoreView management session Current UX
- How to provide a consent to activate Azure AD Reports Feature and activate Partial Import?
- Tips & Tricks: Leverage Pivot Reports to Prototype License Pool Criteria Filter
- Tips & Tricks - How to manage email notifications for newly added Operators.
- Disable MFA from Read Only Service Accounts
- How To: Report on "Consumed Portal Licenses" (New UX)
- How To: Report on "Consumed Portal Licenses"
- How to Configure Allowed IP Addresses for CoreView Service Accounts
- Tips & Tricks: How to merge License Pools
- How to Use CoreView's Global Report Filters
- How to use the What If tool to check Azure AD conditional access policies
- How to Configure Allowed IP Addresses for CoreView Service Accounts
- How to Archive a Teams Group
- How to Restore a Teams Group
- On-demand Import for a Single Device in Endpoint Manager (Intune)
- Custom Actions using the Microsoft Graph API
- How to set up your tenant for the switch to Microsoft Graph API
- GraphAPI configuration: How to get Client ID and Client Secret
Reporting and Analytics
- How do I Check and Manage Calendar Permissions for a User? (newUX)
- How CoreView can help you with your Microsoft 365 Chargeback Goals.
- New UX: Understanding the new License Centers
- Understanding the Savings Opportunities Dashboard
- Understanding the License Optimization center
- Understanding License Pool Snapshots report
- Understanding Call quality dashboard
- Understanding Call quality report
- Understanding User call quality report
- Understanding Teams groups activity report
- Understanding Teams Adoption Growth Report
- Understanding Endpoint Manager reports
- Understanding Teams dashboard
- Understanding Risky Users report
- Understanding KPI dashboard
- Understanding Storage Dashboard
- Troubleshoot Active Users (License Usage) data
- Legacy Protocol Management
- Report Columns: Is active 30/60/90
- Quarantined Messages Report - Understanding The Reports
Managing and Administration
- Teams Voice: Direct Routing Support
- How to enable management function?
- CoreView Playbooks Overview
- CoreView Playbook Policy Overview
- Forward SMTP Address vs Forward Address management actions
- How to add the users in bulk while executing Users management actions?
- How to Create & Manage Custom Actions (New UX)
- How to Create & Manage Custom Actions
- How to schedule a report to be sent automatically, and how to modify its scheduling options?
- How to schedule an alert report for the License Count
- Tips & Tricks – How to read and modify license pool report?
- Overview of CoreView Workflow
- How to delegate Workflow management using roles
- How to configure CoreView and ServiceNow integration
- How to Enable Multi Factor Authentication for Operators and Admins who Access the CoreView Portal (New UX)
- How to Enable Multi Factor Authentication for Operators and Admins who Access the CoreView Portal
- How Can I Migrate from Group-Based Licenses to Direct Licenses Managed by CoreView?
- Naming convention rules
- Custom Actions: Forbidden and Warning Values
- How to add users to Distribution Group in bulk using via CSV
- Not able to manage licenses error (New UX)
- Not able to manage licenses error
- Using custom action json output as an input in the workflow
- Setting the Sensitivity Label on SharePoint as a Mandatory Field
- DistinguishedName vs OnPremisesDistinguishedName
Customer Engineering Workshop
- Migrating from Azure Group Based Licensing to CoreView
- Customer Engineering Workshop - Teams Voice
- Customer Engineering Workshop - Playbooks – policy, perfected
- New UX Workshop - General Overview Session 2
- New UX Workshop - General Overview Session 1
- What’s new in License reporting – the new user interface and the License Center
- CoreView and ServiceNow – Integrating Workflows with ServiceNow
- Advanced Workflows & Custom Actions
- Customer Engineering Workshop: Reports, Dashboards, and Alerts
- Limiting M365 Admin Access with Permission Roles, V-tenants, and License Pools
- Customer Engineering Workshop Global - Group Licensing with CoreView
- Customer Engineering Workshop Global - Filters and Custom Actions
- Understanding CoreView - Quick Start Guides.
- Internal Customer Care Resources
- Service Issues
CoreView Product Manual
- KPI Dashboard
- Operational Reports
- License Reports
- User Reports
- Mail Traffic Reports
- Exchange Reports
- Skype for Business Reports
- Teams Reports
- Group Reports
- Device Reports
- Endpoint Manager Reports
- Security reports
- SharePoint Reports
- Aggregation Reports
- OneDrive Reports
- Yammer Reports
- Report Actions
- Getting Started with CoreLearning
- Getting Started with CoreScan
- Getting Started with CoreTag
- Getting Started with CoreSaaS
- Learning Platform
TABLE OF CONTENTS
- How to Configure a KPI
- How to Monitor and Manage a KPI
- How to Automate KPI Remediation
- How to Delegate KPI Management
- Approve, Reject or Set an Exception from the Approval Request
- Other Use Cases
CoreView can support monitoring your MS365 environment and put management of some behaviors on autopilot - wondering how to do that? With KPIs and Workflows!
In the 22.01 release, we introduced a new Workflow feature: an action called 'Operator approval with 'exception'. This feature is the keystone to fully automating policies and enabling complete employee self-service.
It is important to define what a Policy, Workflow, and KPI are before we get into how to set up a KPI.
- Policy - This is a policy established by your organization. For example, your organization may have a policy that states that all users must have Multi-Factor Authentication (MFA) enabled.
- Workflow - Workflows are repeatable processes that are automated using CoreView. They can be complex or straightforward, multi-step administrative activities. Workflow steps can be conditionalized, and rules developed to ensure that the execution of any workflow is consistent with your organization's practices. You can even introduce an approval step into a workflow, should that be needed.
- KPI - Key Performance Indicators are essential metrics that you want to track for your organization’s Tenant(s). They are monitored via reports and can be saved to a dashboard so that you can easily track those metrics.
- KPI Exception – A process to add an exception to a KPI when the KPI needs to be overridden for a set period of time, or if the rule should be ignored forever for a particular incident.
A user without CoreView access can approve, decline, or approve with an exception to remediate an issue. We will explain in this walkthrough how we set up a KPI exception.
In this article, we'll cover each step required to configure a process that:
- Monitors your environment on defined criteria
- Automatically starts a flow of actions to manage the items that result in a match for the defined criteria
- Delegates the decision whether to proceed or not with such flow to a defined user of your company (without having them login into CoreView at any time)
Note: to follow all the steps on this walkthrough, your account needs to have the following roles: tenantAdmin, WorkflowEditor, WorkflowPublisher.
How to Configure a KPI
A KPI workflow serves to:
- Monitor a policy you have put in place
- Show items that are not compliant with the policy criteria
- Allow a remediation action/workflow
This process will allow a workflow to have an exception to a workflow you have set. This will enable “nonstandard” (or exception) workflows to bypass the rule. If a workflow is bypassed, a reason will have to be stated, and a date will have to be set for how long the rule will be “bypassed.” That date can be either a set date or be set never to expire.
We're going to start configuring a KPI, but first - what's a KPI?
A KPI is an indicator that tracks items of your MS tenant (users, groups, …) that match the criteria (or query) they've been set to monitor. A KPI shows both the count and the list of such items. KPIs on CoreView can be configured starting from almost all reports and are monitored in the KPI dashboard and your custom dashboards.
- Open the report you want to create the KPI from and apply the filters needed to filter that report to show only the items you would like to monitor in the KPI. In this example, we will be setting it up to look for inactive Teams groups.
Example: To monitor inactive Teams group, navigate to the 'Teams groups activity' report, make sure you add the 'Last activity date' column (if not already visible), and finally set the filter. Select 'Not in the last n days' from the dropdown and enter a value in the text field - This filter will show you only the Teams groups that have no activity in the last n days. We have chosen 90 days for this example
2. Once the filtering is set, click 'Action' and 'Save 'KPI'.
3. In the 'Save KPI' panel, you can configure your KPI further. This panel has four tabs:
- General tab
- Type - lets you define whether you want to save this as a simple KPI or want to use it as a template
- Report name
- Category - defines the category this KPI will be shown in the KPI Dashboard
- Service - defines the service this KPI will be shown in the KPI Dashboard
- Severity - predefines the values for the score of this KPI (in the Score tab)
- Dashboards - defines the dashboard this KPI will be added to
- How will this affect my users
- Score tab
- This tab lets you configure the points assigned to the KPI. You can configure custom values or choose to use the predefined ones.
- The KPI has three statuses: compliance, warning, and alert
- The numbers at the top of the graph define the total score for the KPI - for the related status (I.e., in the example below, if the KPI is in the 'compliance' status, the score of the whole KPI will be 15)
- The numbers at the bottom of the graph define the thresholds between statuses (I.e., in the example below, if the KPI has two items that match the criteria, it will have the 'warning' status)
- Remediation tasks tab
- Configure actions - This lets you configure a preferred action to be highlighted in the 'Manage' menu to all operators working with this KPI
- Configure workflow - This lets you configure a preferred workflow to be highlighted in the 'Manage > Execute 'workflow' menu to all operators working with this KPI
- Exceptions tab
- In this tab, you can flag the attributes you want to monitor for exceptions: setting this puts in place a control on the items marked as an exception so that if the attribute flagged is changed for that item, the exception expires (and if the item matches the KPI criteria, it will appear back in the list of matching items of the KPI)
- Review and complete
Example: Proceeding with the inactive Teams groups example, you want to fill the 'General' tab of the 'Save KPI' panel as shown in the picture:
Leave the other tabs to default, proceed to the 'Review and complete' tab, and click save.
How to Monitor and Manage a KPI
Now that your KPI has been configured, you can monitor it and manage it. To keep your dashboard tidy and compliant, you want to make sure to address all the items that match each of the KPI criteria so that your environment is safe and your KPI/policies are under control at all times. CoreView provides two ways of doing so:
- Running remediating actions on items
- Setting items as exceptions
In the example we're providing, the most recommendable remediation action to keep the KPI count low is to delete the Team’s groups that show in the KPI. Sometimes we don't want to delete those Teams, or take action.'Here is where exceptions come into play: setting an item as an exception hides that item from the KPI and excludes it from the count.
- Navigate to your KPI by going to 'Dashboard > KPI Dashboard'. Here you'll find the KPIs you have associated with this dashboard. You can access the detailed list of the items matching the KPI criteria by clicking on the KPI name.
Example: by clicking on the KPI name 'Inactive Teams groups', you can open the list of the items matching the KPI criteria - in this case, the 16 Teams groups that have been inactive for more than 90 days.
2. The items in this list are worth your attention: these need to be either fixed with a remediation action or set as an exception.
- To run a remediation action on the items, select the items you want to execute the management action on, then click on 'Manage' and select one of the management actions from the dropdown.
- To set items as exceptions, select the items you want to set the exception on, then click on 'Action > Manage exception'. In the 'Manage exceptions' panel, select 'Set - Add exception to this KPI' from the first dropdown, and enter the notes and the optional expiry date.
Example: Let's fight our Teams sprawl! We can select the Teams groups from the KPI we want to take action on, select the 'Remove Teams group' management action and proceed.
Even though we wish we could delete all these inactive groups, we sometimes just can't - we can select the groups we're aware of, and we're not going to take action on, and set them as an exception - so they're not counted anymore in the KPI. Select the groups you want to set as an exception and click 'Action > Manage Exception'.
In the 'Manage exceptions' panel, select 'Set - Add exception to this KPI' from the first dropdown and enter the notes. You can even select an expiry date if you want the exception to last only a determined time - this is super useful when dealing with users on temporary leave.
How to Automate KPI Remediation
At the beginning of this article, we claimed the process we were to build was 100% automatic, but so far, all the management we have shown on KPIs is manual - how can it be automated? The feature that we're looking for is 'Schedule', which is designed to forward a report to an email address (or a KPI in this case) on a recurring basis and optionally execute a workflow - to the items contained in the report (or KPI).
- From the KPI dashboard, open the list of items matching the KPI by clicking on the KPI name
2. From the detailed view of the KPI, click on 'Actions > Schedule'
3. In the 'Schedule report' panel, you can set the schedule settings:
- Schedule settings
- Send when - Lets you define the criteria to send the KPI list to the email addresses (that can be configured in the 'Configure notification' section)
- Recurrence - This lets you define the recurrence of the notification and execution of the workflow
- Starting - Defines the start and end of the schedule
- Configure workflow - Here, you can configure the workflow you want to be executed on a scheduled occurrence. Selecting a workflow will let you map the inputs (or execution inputs) for the workflow - you can map each one of the shown columns to the workflow execution inputs.
- Configure notification - Defines who is receiving the exported version of the KPI on each schedule occurrence, the format, and the email's body.
- Review and complete
Example: To automate the clean-up of your inactive Teams groups, you need to create a new workflow that receives as input the name of the Teams group and has the action 'Remove Teams group'. So, let’s head to ‘Manage’ > Workflow and click “Create new Worflow”. Define the execution inputs and action as the following images:
Once you're done with the workflow, save it, publish it and head back to your KPI dashboard. From the detailed list of the 'Inactive Teams groups' KPI, click 'Action > Schedule'. Define your schedule settings, the 'send when' policy (Is not empty makes sure the workflow will execute only when the KPI has Teams groups to be taken care of) and the recurrence.
Proceed to the 'Configure workflow' tab, search from the workflow you just created, and click on it.
To complete the workflow configuration, you need to map the fields to be used to run it. Click on the flash icon next to the 'Name' field and select 'GroupName', which is the unique identifier for the Teams groups, then apply changes. Proceed to 'Review and continue' and save your schedule.
How to Delegate KPI Management
Well done! You have taken control of your Teams group sprawl! Before your well-deserved victory lap, CoreView can help you push the automation one step forward: delegate the decision on whether to perform the remediation workflow or set the item as an exception to anyone from your organization. This is where our latest workflow feature, the new action 'Operator approval with exception', gets the spotlight. Let's see how to tweak your workflow to leverage it!
- Head to the workflow you want to implement this new feature to (or create a new one)
- Click ‘Add action > CoreFlow > Manual’ and select ‘Operator approval with exception’.
- In the 'Operator approval with exception' panel, you can configure the action which works similarly to the other 'Operator approval' actions. Here are the fields you can set:
- Approved by email: defines the email address which will receive the approval request
- Subject: it's the subject of the approval request email
- Body: it's the body of the approval request email
- KPI: this field is needed to tie the workflow to the KPI and be able to set the exception on that KPI
- Userprincipalname: this field requires the unique identifier for the item the workflow is executed on
- # of timeout second: defines the amount of time after which the workflow can be interrupted if the approval is not granted or rejected.
Example: Let's modify the 'Clean-up Teams groups' workflow we used in the previous example to host the new action. This way, we can have the Teams group's owner receive the approval request for the deletion. We need to add a new execution input, 'TeamsGroupOwner', and add the 'Operator approval with exception' at the very beginning of the flow. The configuration is:
Your new workflow is ready - save it and publish it!
You can go back to your KPI, schedule it attach this new workflow, and nail the mapping (just like it shown in previous point).
Approve, Reject or Set an Exception from the Approval Request
Now that your KPI is set, the workflow with the new approval flow is scheduled; you just need to relax and wait for CoreView to do all the work. Are you wondering what the user experience is for the users who are receiving the approval request? Here's what you are looking for:
- The email address set in the workflow field 'Approved by Email' will receive the approval request email
2. Clicking on the blue check icon will open the approval request in the browser
3. The user once had reviewed the request, can proceed by:
- Approving the request - the workflow continues, and the remediation action is executed on that item
- Reject the request - the remediation action isn't executed
When clicking on 'Reject' to reject the request, the user will be prompted with one more question - whether that item should be marked as an exception or not.
- If the user selects to mark that item as an exception - the KPI will save the item as an exception, and further execution of the scheduled workflow won’t ask again about that item. The user will also be able to set an expiry date for the exception.
- If the user chooses not to mark that item as an exception - the item will stay in the KPI list, and the next occurrence of the scheduled workflow will trigger a new request.
You have now set up an KPI Exception for monitoring.
Other Use Cases
KPIs, and KPI Exceptions are a great way for you to monitor your Tenant(s). Here are a few other Use Cases that you can also apply this process too:
- Users without Multi-Factor Authentication (MFA) enabled
- Clean up MS365 groups without owners
- Monitor high-risk users
- Inactive guest users
- Guest Users with assigned licenses
- Administrators with licenses
- Users with external email forwarding