Power Apps and Mixed reality

(Canvas) Power Apps recently received several mixed reality features. They seemed super cool, but what kind of useful solutions can we actually build with them?

It’s time to find out.

First we’ll create a blank new Power App. Mixed reality features are in preview and therefore we have to enable them. It can be done on Settings -> Advanced settings -> Experimental features.

Mixed Reality section now appears into the Insert menu.

2D and 3D models

The whole idea idea of mixed reality is to show 3D (or 2D) models in the real environment. So first we need some models.

The models need to be in glb format. It was a bit difficult to find usable and free 3D models, so I decided to use the robot arm model Microsoft provides plus one photograph (Husky).

The models aren’t uploaded into Power Apps, but they have to be accessible from within the app. The easiest way is just to create a SharePoint list and add the required models as list item attachments.

Now we have all the data in place and it’s time to start reviewing the mixed reality / 3D controls for Canvas apps.

View in 3D

Want to show a 3D model inside the app? Just add the View in 3D control.

This control obviously needs a 3D model to show. Let’s add one gallery below the View in 3D control and connect this with the SharePoint list we just created.

Now we can reference the attachment from the selected item of the gallery. Just use it as a source of the View in 3D control.

First(gal3Dmodels.Selected.Attachments).Value

And this is what it looks like!

Nice, but not necessarily very useful yet.

View in mixed reality

Next we’ll replace the View in 3D control with the View in MR control.

Again we’ll use the model from the SharePoint list as the source data for this control. The end result is much more interesting this time. We have just created a mobile application for exploring how a 3D model looks like in the natural environment.

How would this robot arm look in our backyard?

I am impressed.

We can do same thing with 2D images. Would a Forward Forever poster look nice on the wall of our garden shed?

View Shape in MR

Let’s continue our journey in mixed reality and add the View Shape in MR control into our Power App.

With this control you can add a virtual cube into the real environment. But why on earth would somebody want to do that?

Let’s imagine that I want to buy a new table for our backyard. But will that new table I’ve got in mind actually fit there? We can create a cube with exactly the same size as the new table and validate how much space will it occupy in our garden.

This is a handy way to fit virtual things into the physical world when there is no 3D models available of the object in question.

Here’s a real life story from our office in Kamppi, Helsinki. We were discussing about the options for installing a big screen in our meeting room. But would a 65″ TV screen be way too big for the relatively compact space?

Instead of just guessing, we built a Power App with MR feature and put a virtual TV on the wall for everyone to have a look at.

65″ looks fine to me.

Measure in mixed reality

The last new control we have left is Measure in MR.

You can use this control for measuring distance, area or volume.

The control in action works pretty much the way you would expect. Very similar to the iPhone’s Measure app you might be familiar with.

Why should I use this instead of my phone’s existing measurement app? Because when using this control you can use the measures taken in your app and apply them for a specific use case or process.

In the screenshot below I have measured the height and width of the wall with Measure in MR control. After capturing these variables, the Power App calculates automatically the area of the wall.

Next we could add selection for the wallpaper and calculate the price for the needed amount of selected wallpaper, by referencing a data source like CDS that the app can natively talk with.

Summary

It’s just amazing how easily you can nowadays build apps utilizing basic mixed reality features.

But that’s what Power Platform is all about. Amazing, easy and for everyone.