Power Automate an alternative to Read and Understood App?

Introduction

Power Automate, since it’s recent inception, has been a tool to provide non-developers with the mechanism to define business processes.

Out of the box, Microsoft provide some useful templates to facilitate automation tasks in SharePoint such as Approval; the ability to approve / reject edited content, such as policies and procedures.

About this Article

For all business software solutions, it is very much the case of ‘the right tool for the job’. Power Automate and SharePoint are fantastic tools, but they have limitations. We have have been asked many times why we don’t just use Power Automate instead of developing an App like Read and Understood.

In this article describes an approach for how you might try to produce a Power Automate version of Read and Understood so you can understand the challenges and limitations.

SharePoint Structure

Before we embark on creating our Flows for Power Automate we’ll need to get some structure in SharePoint. To begin with, you may have more than one document library. It’s important to note that for every library you will need one Flow to monitor the changes, be it document Create or Modified.

User Acknowledgement List

You’ll need to keep a record somewhere of every user that is required to ‘acknowledge reading’ of a document. Remember, we will want to capture not only the fact that a user has acknowledged but to also know who was asked to but didn’t - because there will be some that don’t.

The structure of the list will require a couple of custom columns. We can use the out of the box Title field for storing Document Name we’re asking them to acknowledge. You will need a People column to store the employee you’re asking to do the acknowledgement. Also, a Choice field, set required with a minimum of one option ‘Acknowledged’. You might consider an ‘acknowleded with comments’ option so that users can feed back into the editorial process if they spot ommisions or errors. Why not a boolean (yes/no) column? The problem is making the field required, you can’t because the value will always be either Y/N and we need to use to select a value.

Definition List

This is optional, but you might want to think about how you might apply different rules against different content. For example, if you have Policies, Procedures and Guides, you might only want to capture acknowledgements for the Policies and Procedures only.

By using a custom column to store groups you can define the groups of users that need to participate in the acknowledgement process. A special mention for the SharePoint group ‘Everyone except external users’. Be warned this group does not phyically exist anywhere in SharePoint or Azure Active Directory so you can’t parse it for group membership.

If you are to create rules based on content and you’re using content types, then storing the Content Id’s would be a good place to start.

Document Library

One problem with SharePoint Online is that when users edit content there is an auto-save feature in office to keep regular saves of your work. The downside is that this creates multiple versions. If our Power Automate job is going to fire on the change event then this will be a problem.

Consider enabling Major and Minor versioning in your library. You can also include publishing approval. With Major / Minor versioning we can now control the PA Flow to only run if a Major version is created.

Automation

Before you start creating the Power Automate flows consider the following:

1. Child Flows are a great feature, create small reusable flows rather than trying to create one large flow.

2. If you have Dynamics 365 with multiple environments, then you have a great advantange if you have a Dev / UAT / Prod as you can manage the development / test / live use of your flows

3. If you don’t have Dynamics 365 then consider how you can have a Dev site in parellel to your production flows. The same goes for SharePoint site, duplicate your production site to test and build your automation.

User Acknowledgement Request

The first part of the automation is to store those users that are being asked to acknowledge reading the document in the ‘Acknowledgement list’. This PA Flow would be an ideal Child flow taking the following values:

• a comma separated string of all the users that are going to participate in the acknowledgement. This can be converted to an Array so we can loop over each user in turn.

• document sharing link,

• the document name, and

• document version

Now we have the data, it’s a matter of looping over each user and:

1. create a list item in the ‘Acknowledgement list’ setting the value of the columns using the Account, Document Name and Version

2. Next, we need to create a link to the record created in step 1

3. Send the user an email providing instructions and a link to the list item + document.

Document Definition Flow

Assuming you have different rules for different documents in your library, you may want to create a child flow to determine the documents to process. This would take a parameter of the Content Type Id of the document being processed. With this we perform a query looking for record in our ‘Definition list’ that has the same id.

The record that is returned has a People/Group picker that defines the groups we wish to use. By using groups we have a simplified approach to storing users. In our flow, to get the members of a group we can use Microsoft Graph.

For each group defined in our column, repeat the process of querying Graph for the users and store them in an array that we can pass back to the parent flow.

Document Major Version Flow

Bring it all together we can now have a single flow that will manage the process of calling the child flows. Firstly, we need the flow to run when a document is modified, however, as pointed out earlier we only want to process Major versions so if it is a minor version update - simply exit.

When we know we have a major version to process. we can:

1. Create a sharing link - be careful as you could be giving access to users that don’t have access, intentionally.

2. Call the child flow for the Definition list - this will return our users that we need process

3. Call our User Acknowledgement Request flow with:

   1. The list of users,

   2. Sharing link

   3. Document Name, and

   4. Document version

Challenges

Permissions

While we create list items for users to capure their acknowledgement of reading a document, it’s possible that everyone to see each others items. In addition, another user could acknowledge a document on someones behalf. One solution would be to secure the item to that user, which could be done having created the record. See securable item limit below.

List view threshold

SharePoint invokes a list view threshold and while the modern interface is great at dealing with this, if you want to produce reports across your User Acknowledgement List you may find that your list breaks and you can’t produces views of data e.g. Document ABC version 1.0 even though it might be for 10 or so records, if the list view is exceeded you can’t apply the filter.

Another Version Published

It happens. A major version gets published and the emails go out only for someone needing to make a change or add something else. This will trigger a new set of emails and list items. What happens to the previous items? You could delete them.

New Users added to Groups After Document Version Published

When you have new employees join or people move between departments they’re likely to move between AAD Groups. The issue here is that we want our automation process to pick these users up as being new members of the group and get them to read and acknowledge the documents.

Error Handling

Often, this gets forgotten but it has to be considered. Microsoft document transient errors and cloud computing. When something goes wrong, and it will, you need to be able to recover a situations. This may mean rerunning an automation job but have it pick-up from where it left off - this is difficult in Power Automate Flows without parsing what was done before..

Read and Understood Solution

All of what is described above, and more, is achieved by Read and Understood. We choose not to use SharePoint lists to store data, but instead use a dedicated SQL database to avoid Microsoft limitations.

Start by reading an introduction to Read and Understood…

Controlling which Documents Employees must Acknowledge having read

When developing Read and Understood we wanted to keep administration for our customers very simple, our thinking was that you set something up and leave it to take care of doing the job.

Within the App we created a concept of ‘Document Selectors’, the rules by which, when a document was updated or created, if the rule matched, an Acknowledgement Request was sent. The same document selectors would also define, how long an employee was required to read the document by and who the employees (recipients) required to acknowledge.

Email Recipients

We strongly recommend the use of either SharePoint or Azure Active Directory (AAD) Groups, which can be enabled within the Read and Understood App. Typically, companies will provide access to SharePoint sites through Group membership, either SharePoint or AAD; the App can use those same groups to determine who the Acknowledgement emails should be sent to. As groups are updated with new users to access the content those same users will automatically be added to the acknowledgement requests.

Library Document Selector

This ‘Document Selector’ will, once a library is enabled, be created for you to send a request to the recipients every time a new or existing file is published as a major version, you may not want. We can fine tune the rules applied by using a combination of SharePoint features and more ‘Document Selectors’ to define those rules.

SharePoint Content Types

By using SharePoint Content Types we can create additional rules against the Library content. Content Types have been part of SharePoint for a very long time and not used as often as they could. This Microsoft article details what they are and how they can help with good Document Management practices.

If you don’t use Content Types then consider the following scenario. A library will be made up of many document types, which could be classified as ‘Content Types’ such as Policy, Procedure and Guide. The types of documents may align to the types of notifications you want to send. If not we can take a slightly different approach for managing the type of notification Read and Understood will use.

Assuming you are not using Content Types for your library, then we could consider two scenarios:

1. Documents requiring employee acknowledgement, and

2. Documents that we need to inform employees the document has updated but doesn’t require them to acknowledge.

There might be a scenario where you don’t want to let users know at all that a document has been created or changed.

Before you create any Content Types

It’s a good idea to get a view of which documents in your library do or do not require acknowledgement, or you just want to send a notification only; again to simplify administration.

Assuming there are three types of documents in the library, those:

• Requiring Acknowledgement,

• Requiring Notification, and

• No action.

We could achieve this by creating in our Site two new content Types ‘Read and Understood Acknowledge’ and ‘Read and Understood Notification’.

Create a New Content Type

When the Content type has been created and added to your library, you can ‘bulk change’ multiple files in a SharePoint library to switch the Content Type.

Important: Ideally you should update all the documents before enabling Read and Understood as not to trigger lots of document updates to users because the Content Type has changed.

Document Selectors

The library will have a ‘default’ Document Content type which we can use for those documents we do not want to send either an Acknowledgement or a Notification email. For all the other documents set the content type based on whether you want to send an Acknowledgement or Notification.

In the Read and Understood App we create two new Document selectors, one for each of the two new content types created above. For the ‘Read and Understood Acknowledge’ we would configure the Acknowledgement section of the Document Selector. For the ‘Read and Understood Notification’ we would only configure the Notification section.

Other settings, such as inherited from - choose the Library Document Selector from the dropdown.

Testing

If you are still in Trial mode you can use the fast track feature, otherwise you will have to wait for the overnight process to run that sends the emails.

Microsoft Flow Template for SharePoint Approval, where Major and Minor versioning is used, does not Publish the File

What’s the Issue?

The out of the box approval flow template for SharePoint is missing a vital part of the process to get the File Metadata - ETag and then setting it when the item is approved. ETag is a special identifier for the Item version.

Solution

The article below is an excellent solution to the problem experienced by one of my clients, hopefully it can help other is you get stuck with the same problem…

https://blog.leitwolf.io/approve-a-sharepoint-document-with-flow-set-content-approval/

Microsoft New Employee Onboarding hub with Read and Understood Onboarding

Microsoft New Employee Onboarding hub with Read and Understood

In October 2020 Microsoft released the New Employee Onboarding hub for SharePoint online to:

‘…help organizations improve their onboarding process and experience the benefits of a well-planned new employee onboarding process‘.

The information below is taken from the Microsoft Article on New Employee Onboarding hub article on the importance of onboarding new employees:

Why invest in a new employee onboarding site?

New employee onboarding (NEO) should be a strategic process that integrates new employees into an organization and its culture while providing the knowledge and tools needed to become fully contributing team members. NEO processes often fall short for both the new hires and the organization. Only 12% of employees strongly agree their organization does a great job of onboarding new employees. An engaging and well organized NEO process can make all the difference in helping a new hire navigate through an exciting – but stressful - career journey, and it can have major organizational benefits.

Strategically planned NEO experience can:

Improve new hire performance and time to productivity - Organizations with a standard onboarding process report 50% greater new-hire productivity.

Improve new employee retention - Based on our research, 69% of employees are more likely to stay with a company for three years if they had a great onboarding experience. Employees who have a negative onboarding experience are twice as likely to look for new opportunities in the near future.

Onboarding with Read and Understood

Building on the NEO site template, use the new Read and Understood Onboarding feature to increase productivity, ensuring new employees read the documents that are important to your company and equally important to help them fulfil their new role.

A Quick Introduction to Read and Understood

Read and Understood targets some or all documents within SharePoint libraries using ‘Reading Categories’. The Reading Categories define who should be Notified when documents have been added / updated. Reading Categories also:

• Select the employees required to ‘Acknowledge’ reading documentss

• Set an ‘Owner’, who report on employee document Acknowledgements

• Set the duration (days) by which employees are required to acknowledge updated documents

• Send reminders at intervals to prompt those employees who have yet to complete the Acknowledgement

How Onboarding Works for Employees

Read and Understood tracks employee membership of SharePoint and/or Azure Active Directory Groups. When a new employee is added to a group, a Notification (email) is generated for each current document version targeted by a Reading Category. When a new employee completes an Acknowledgement request, their comments are recorded in Read and Understood against that document version.

Keeping existing employees Notified of new or updated documents is equally important as informing new employees. Existing group members are automatically updated as and when documents are changed ensuring all employees read the documents that are important in fullfilling their role.

Not Everything is a Document in SharePoint

It is common for companies to provide training or induction videos. If the videos are in SharePoint then they too can be part of the onboarding process with Read and Understood.

Embracing Microsoft Teams Sites

Many companies we work with are embracing Microsoft Teams. Files stored within Microsoft Teams Sites can be used with Read and Understood.

Reports to reduces corporate risk to stay audit ready

A full audit history for each document version of new, though onboarding, and existing employees who did or did not acknowledge reading.

Documents in Other Sites

Files you want onboarding employees to read do not have to reside within the new onboarding hub but can be located in any SharePoint site library. Read and Understood brings information together to make it easier for employees.

“Well informed employees make better business decisions“

Enable SharePoint Site Scripts with PnP PowerShell and a Windows Form

Updates: 25/01/2021

Script now includes support for PowerShell 7 and the latest release of PnP.PowerShell. The script detects if the modules are available and if not, installs them, which will require ‘Run as Administrator’.

Introduced ThreadJob to work-a-round an issue with PnP.PowerShell (7) when getting a connection to the tenant, it just hangs. Problem only occurred using credentials or -UseWebLogin from within a form. FYI. Start-Job deserialized the connection response so wasn’t usable.

A simple System.Windows.Forms.ListBox is used to display messages and errors.

Background

Occasionally with SharePoint, we need to enable Custom Scripts for a Site (Site Collection). In the old admin centre we were able to enable the option, as you can see in the image below:

Old admin centre - enable Custom Scripts

SharePoint admins cannot be reliant on customising SharePoint through the UI and instead have to result to using PowerShell. Two tools I particularly like to use are:

• PnP SharePoint Online PowerShell 5.1 … https://github.com/pnp/PnP-PowerShell,

• PnP.PowerShell PowerShell 7 https://github.com/pnp/powershell, and

• PowerShell Pro Shell Tools… https://ironmansoftware.com/powershell-pro-tools/

The latter I like to use if I’m providing a client with a more friendly UI to interact with PowerShell, however, while the tool provides a mechanism to output an exe file, my preference is to give them something they can, should they wish, tweak when they become more familiar with SharePoint PS Admin.

Enable/Disable Custom Script

There are many articles on the internet that demonstrate how, using the command-line, admins can enable or disable custom scripting for Sites in SharePoint. This article summarises how a more user friendly user interface was created using a windows form over the top. A copy of the script I produced is here:

Overview of how it works

Much like many of the scripts online, it simply connects to the Tenant, retrieves all sites and organises them into two lists, those enabled and those disabled.

Having downloaded my copy of the script, you can either manually install the required scripts or, run the PowerShellFormSiteScripting.ps1 as an Administrator and it will install the required components. Unzip my script file where you will see there are three files…

Date modified are likely to be newer as it gets updated.

Right click on the PowerShellFormSiteScripting.PS1 file and click Run with PowerShell.

Note: if using PowerShell 7 the option is slightly different and says ‘Run with PowerShell 7’

First thing you must do is authenticate. Provide a User Name, Password and your Tenant Domain Name:

You only need to provide your tenant domain name not the complete URL.

Once the details are entered, click the

Click the button. You can use this button to refresh the lists at anytime.

Assuming the details are entered the two lists will be populated with Site URLs where Custom Scripts is either enabled or disabled. If any of the information provided is incorrect then a warning will appear at the bottom of the form.

Enable or Disable Custom Script for Site

Simply select the Site you wish to Enable or Disable and click the arrow in the directional arrow. The video below demonstrates it working…

Under the hood

As i mentioned at the beginning there are many articles on this subject and the approach I take only really differs by the use of a Form.

First, it checks that the PnP modules are installed… if not the script will attempt to install them for you.

Next, when the user clicks Query SharePoint sites for Script Enablement, get all tenant sites, I chose to filter those out that had a different domain name e.g. My Sites.

Note: for PS 7 it’s using as ThreadJob to get the connection. This a work-a-round to an issue within using Connect-PnPOnline with Credentials within a Windows Form.

Unlike PS 7 this is example is just doing the connection within the Form itself

Previously, when the User click the arrows to either enable or disable scripting we processed the change by executing the code below:

Instead we can do this in a single line with PnP.PowerShell:

Set-PnpSite -Identity SiteUrl -NoScriptSite True/False -Connection ConnectionVariable

Managing Policies and Procedures and other key company documents in SharePoint

Having worked with SharePoint since 2003 we have seen organisations implement the document management features of SharePoint in many ways. Some organisations just opt for version control, others go much further by implementing records management and other SharePoint features. In many cases the use of document libraries is prolific with the focus being on the association of content based on the perspective of its origins (often departmental). This is where most organisations stop, with the implementation viewed as a job done because all the documents are now in SharePoint and covered by one or more of SharePoint’s document management features. 

But no matter how well controlled documents are in respect to their publication, review and editorial cycle; it's very often that the end user, the consumer of the content, that is left to find publications that are new or updated on their own devices. 

SharePoint does offer solutions to this with Alerts, custom Workflows or Microsoft Power Automation - Flow. These are either too granular to configure and administer on a large scale to cover all users or require skills that most end users just don’t have. Therefore, after working with SharePoint for over 17 years, we decided to develop Read and Understood for SharePoint. 

Read and Understood implements processes that keep users informed of new and updated policies, procedures and any other key documents without the complexity of custom workflow or arduous administrative overhead to manage Alerts on a company wide scale. 

Q. In your opinion what are the main challenges faced by organisations needing to publish Polices and other key documents? 

To save repetition the term key documents will be used in the remainder of this post to refer to Policies, Procedures and key company documents. 

1. Ensuring the right employees read the key documents 

As soon as a key document is approved and published its good practice to ensure that relevant employees have a chance to read it. Using SharePoint to control and publish is a great start but you still have no idea who’s read the document. Sending paper copies used to be a solution, getting users to sign the document, in years gone by but they typically just piled up on desks due to time constraints. Read and Understood was created so organisations maintain complete control over key documents by having controls to target specific groups (SharePoint or Azure Active Directory) of users with email-based notifications. Notification is the Read and Understood equivalent of a SharePoint Alert, with some significant enhancements. Management of Notifications is configured through a Settings List to simplify administration. Configuring a notification is as simple as writing an email; use relevant groups or individual employees to receive the email. Our quick start templates for the notification emails will provide the user with the information, provided in the template, including a link to the document. 

Read and Understood provides two levels of Notification: 

Notification - an email with details of the key document that’s been added or updated. 

Acknowledge reading – the user is required, using a link in the email, to confirm they have Read and Understood the document. Through the Settings List you can define rules that sets how long users will have to acknowledge the document, during that time if they have not acknowledged then reminders can be sent.  Read and Understood keeps a full audit trail of each document and its version cycle detailing who did or did not acknowledge within the allocated time. A report email is sent to key document owners summaries the information at the end of the cycle. 

2. Smart user management for new starters and internal movements 

All Read and Understood features can use a combinations of Azure Active Directory (AAD) user accounts or groups and/or SharePoint groups to notify users.  

When Read and Understood enters its notification cycle, triggered by a new or updated document, it will ensure any new members of the groups are included in the notifications.  

3. How much effort does it take to get Read and Understood add-in up and running? 

With over 30 years’ experience in implementing 3rd party solutions we're acutely aware that the setup for any solution can be arduous and time-consuming. Read and Understood administration has been designed to ensure that implementation effort is kept to a minimum but of course this does depend on how your content is structured in SharePoint. 

Read and Understood is administered through the creation of items in a Read and Understood Settings List working with one or more (no limit) document libraries in a site collection. 

Read and Understood is an add-in that that is added to the SharePoint App Catalog. Once added and deployed, sites can be added to the sites you wish to enable it. 

To enable Read and Understood notifications involves the following steps to be taken: 

• Identify all the document libraries that contain your key documents 

• Add the Read and Understood for SharePoint Add-in 

• Enable Read and Understood on the libraries identified in step 1 

• Define the type of notification you want to send, who should receive it by assigning the appropriate groups or users. If you want to, tailor the email message using the rich text editor to style it or get the message you want across. 

That’s the end of this post and I hope you’ve found it informative and thought provoking.