Smart scanning of receipts with Power Apps

In my previous posts, I have presented a few solutions built with Power Platform for use at Forward Forever. For example, we handle hour reporting and the generation of invoice attachments with Power Platform.

Now it’s time to modernize the expense handling.

The current process is straightforward. The employee saves the receipts in a SharePoint document library, from which they are manually processed forward monthly.

First, let’s provide the employees a more pleasant way to enter receipts with descriptions in the right place. As the number of receipts increases, we can also automate their processing and transfer to the payment system. However, let’s not get ahead of ourselves.

SharePoint document library

Receipts are still stored in a SharePoint document library. Let’s add some metadata to the library to help with the processing.

• Total amount of expense (Total value)
• Description (description)
• Place of purchase (Vendor)
• Has the expense been paid to the employee (Paid)

Now we can move on to the actual application.

Power Apps

Let’s create a new application (mobile layout) and add a gallery to it. The gallery shows user-created expenses that have not yet been paid.

Items = Filter (Expenses, Author.Email = varCurrentUser.Email And Paid = false)

VarCurrentUser is the variable whose value is set when the application is started (App OnStart).

Set(varCurrentUser, User())

An AI Builder control is added that can interpret the contents of a receipt (Receipt processor).

The control works cleverly. The user selects an image (Scan receipt), either from the device’s image library or by taking a new photo. The image is analyzed and then displayed to the user. You can see directly from the image which parts of it have been identified.

The receipt can be a photo of the receipt (even in the wrong direction) or, for example, a receipt in pdf format from an online store.

Correcting / adding information on receipt

After scanning the receipt, we need to check the information. At the same time, the user completes the fields that were left blank.

A new screen (Receipt Info) is created with a form connected to our SharePoint library. If we are entering a new expense (form status <> FormMode.Edit ), we use the values ​​parsed by the AI ​​as the default values ​​for the fields.

The form should of course be in New mode. Let’s force this during the navigation.

But…

We also want to save the actual receipt in the document library. This is still not possible directly from the form.

So we need to create a flow, which handles the saving of the receipt.

Flow – Saving the receipt

The flow is triggered from Power Apps. First, the values ​​obtained from PowerApps are stored in their own variables.

• File content (file content), Note: Today this is easier with Power Apps V2 trigger
• File type (File extension)
• Title (Title)
• Description (Description)
• Place of purchase (Vendor)
• Total amount of expense (Total value)

The user’s profile is retrieved, after which a new file is created (Create file) in the expense library. The metadata of the created file is updated immediately after this (Update file properties).

Now we have created the file in SharePoint.

For the file preview to work, it must have the correct file extension. Flow gets this as a parameter (FileExtension) and the extension is determined in Power Apps by examining the contents of the file.

If (
  Find ("application / pdf", ReceiptProcessor3.OriginalImage)> 0,
    "pdf",
  Find ("image / gif", ReceiptProcessor3.OriginalImage)> 0,
    "gif",
  Find ("image / jpeg", ReceiptProcessor3.OriginalImage)> 0,
    "jpg",
  Find ("image / png", ReceiptProcessor3.OriginalImage)> 0,
     "png",
     "unknown"
  )

Editing unpaid expenses

Let’s make a few more adjustments, so that the user can edit their added (but not yet paid) expenses.

Let’s add navigation from the main screen expense gallery to the Receipt Info screen. At the same time, let’s force the form into edit mode (EditForm (frmExpense)).

Saving the expense happens differently depending on whether it is a new expense or an edit.

• When creating a new expense, the flow is called
• When editing, the information is updated using the form (SubmitForm)

If (
  frmExpense.Mode = FormMode.Edit,
    SubmitForm (frmExpense),
    Set (varReturnvalue, 'PA - Add Expense'.Run (
       ReceiptProcessor3.OriginalImage,
       DataCardValue6.Text,
       inpTotalValue.Text,
       DataCardValue8.Text,
       inpVendor.Text,
       lblFileExtension.Text)
    )
  )

Displaying the receipt

It would be nice to show the user the added receipt when filling in the information. The challenge is that the new receipt can be either an image or a pdf file.

Let’s add separate controls (Image and PdfViewer) to preview the different types of files. The correct one is displayed according to the situation.

The content is the same in both controls.

If (frmExpense.Mode = FormMode.Edit,
     galExpences.Selected. Thumbnail.Large, 
     ReceiptProcessor3.OriginalImage
  )

Finishing touches

Finally, let’s tidy up the look of the app and change the colors and fonts to the correct ones. At the same time, scanning the receipt is moved to a separate screen. This allows for more space in the first screen for unpaid receipts.

Summary

The Receipt processor is still in preview. It recognizes information about receipts with varying degrees of success. Surprisingly well to say the least.

Publicly Available (GA) AI Builder components are additional features of the Power Platform. Their use requires AI Builder credits, which can be purchased in sets of 5 million for $500/month.

In our own tenant, there is one Flow per user with attended RPA license that includes 5,000 AI Builder credits. Nowadays you get also 250 credits for each Power Apps per App license and 500 credits for each Power Apps User license.

We will soon see if they are enough for us to analyze the receipts of expenses.