Azure OpenAI Service and AI Builder – first impressions
Azure OpenAI Service is a new feature that lets you use GPT models on Azure. Its usage in Power Platform services is still in preview mode and available only for environments in the North America geography. Let’s have a look at how we can benefit from these latest generative AI features in our Power Platform low-code solutions.
Where to find it and how to use it
On the Power Automate page, under the AI Builder section there’s a banner with an access to the preview feature:
Instead of the traditional AI Builder screen, where users select a pre-defined AI Builder model, the preview for using generative AI will lead us to a ChatGPT type of an experience:
Let’s go through the different components on the above screenshot.
- Templates – there’s a variety of templates where generative AI performs different text analysis like summarize, classify, sentiment analysis, etc. In this post I selected Extract information from text.
- Instructions – this area expects 2 types of text: a set of instructions and a sample text where the instructions will test the model. In a real scenario, the sample text will be replaced by an input parameter. The set of instructions tells the generative AI model exactly what to do with the input text it will receive. In the image above, the model is instructed to extract container and registrations numbers from a bunch of unstructured text.
- Response – this will preview in real-time the result of the instructions for the given sample text. This is quite useful to optimize the instructions so that the output fits with your needs. You will spend a few iterations here until the response has all the information you need but trust me, this is worth spendings some time on. You will save plenty of steps in the cloud flow (spoiler alert for the example that will follow).
- Use prebuild model – When you are happy with the set of instructions for the sample text it is time to proceed to leveraging the model in a cloud flow or an app.
Alternatively
You can skip the steps above and go directly to the cloud flow and select the action Create text with GPT on Azure OpenAI Service (preview). In this case, the refinement of the instructions will happen inside the cloud flow.
Here’s the cloud flow that is automatically generated:
A couple of thing to note about the generated cloud flow:
- The use of the AI Builder action Create text with GPT on Azure OpenAI Service (preview) with an input parameter and instructions as part of the action;
- An approval process is added to the cloud flow and cannot be removed. It’s a useful requirement that all generative AI model requests will need to be approved by a human being. If not approved, the user that ran the cloud flow will receive a notification of the rejection.
Now that we have the basics in place, it is time to build a solution.
Rebuild previous AI Builder solution
Have you read my earlier post where I used AI Builder and Power Automate to extract unstructured information from an image and convert it into calendars events? This post was written back in September of 2022 and I thought it would be a good exercise to compare the differences of what was built nine months ago and how the solution will look like using the new AI generative model.
The steps used to configure the document processing model for the original solution were:
Then, on the cloud flow, I had to:
- use 5 variables to store, year, month, an array of codes, a date and a counter
- create a couple of Apply to each controls so that I could gather all the info from a person and all the codes
- use a few If/Switch conditions in order to filter persons and create a proper date based on codes
- write Power FX formulas to build the dates based on the year, month, day and time.
Using the new Azure OpenAI Service
First, we need a sample text so we can start to test our instructions. I copy and paste the text directly from a PDF work shift file (from original solution) an use it as the sample text. For the instructions I used natural language and described in detail what I expect as response from the AI model.
Extract Year and Month as fields
Extract "name of person" and "codes" as a table for the person "Tiago Neves".
Return the data as a JSON object with the name and the codes
Each code will need a start datetime and end datetime
For the AM code, use the time range 9h to 17h
For the PM code, use the time range 12h to 19h
For the EV code, use the time range 15h to 23h
The outcome is a response in JSON format with an array of all the codes for the person and the date fields already calculated.
{
"name": "Tiago Neves",
"codes": [
{
"code": "AM",
"start": "2023-01-01T09:00:00",
"end": "2023-01-01T17:00:00"
},
{
"code": "PM",
"start": "2023-01-01T12:00:00",
"end": "2023-01-01T19:00:00"
},
{
"code": "OFF",
"start": "2023-01-01T00:00:00",
"end": "2023-01-01T00:00:00"
},
{
"code": "EV",
"start": "2023-01-01T15:00:00",
"end": "2023-01-01T23:00:00"
}
]
}
All I need now is to adapt the pre-created cloud flow to properly address the list of codes provided. No formulas needed on the cloud flow, I just use the fields directly from the JSON to create the calendar events!
- Parse the JSON
- An Apply to each action in order to go over all codes
- For each code, use the start date and end date to create the events in Calendar
The result is the creation of the calendar events in a cleaner and simpler manner, compared to the original solution.
Conclusions
So, analysing this new feature from a citizen developer or Power Platform professional perspective, would I consider use generative AI models to build my future solutions? The answer is definitely yes! Once you get proficient in designing prompts to instruct the AI models, it is quite impressive how much this can simplify the low-code solution development steps needed.
Comparison
New solution
A simple cloud flow that uses the new AI generative model and a simple condition control in order to create all the events.
Original solution
The solution uses multiple variables, cycles and conditions. Additionally, string manipulations to build the dates is needed.
Final considerations
The feature is in preview and I found a few problems using the AI model, especially with large instructions and large sample texts. For that reason, I only used a subset of codes in my example.
OpenAI generative models (today) work with text only. As stated above, I had to copy and paste the text from a PDF file as the input for the AI model. I would expect in the future that this feature will allow other types of inputs like images or files.