Plugin actions in Copilot Studio: how to integrate with external services

The last 12 months have been full of excitement about ChatGPT and announcements of various Microsoft copilots. This time, in the context of Power Platform, we will get to know one of their most interesting applications. Microsoft Copilot Studio’s plugin actions.

What are plugin actions about? In short, they serve as integration tools that allow your chatbot/copilot to reach out to external services. Thanks to the powers of generative AI, it will “understand” how and when to run a specific cloud flow to be able to respond to the user’s request.

These actions are going to transform the way chatbots are built. But how do they work in practice? Also, how capable is this feature, which is currently in the preview phase?

Let’s find out.

Example: train timetable query

In 2018, I built a chatbot that can tell you when the next train leaves from station A to station B. The service utilized the schedule search offered by Digitraffic. As you can see from the earlier blog post, it was not a low-code solution.

How could a similar service be implemented in 2024 using the latest Power Platform technology? That is, Copilot Studio and its new generative AI powered features. Today we’re going to need:

• Power Automate cloud flow, which fetches the next train departures of the current day from station A to station B. The interface requires the stations as station codes (PKO, HKI, PSL, etc.)
• Another flow that returns the station code corresponding to the given place (Tampere).
• With Copilot Studio, a chatbot for which the previous flows can be used (as plugin actions)

Thanks to AI, everything just works magically!

Or does it really?

Creating a new copilot

First, let’s create a solution where every component needed will be stored in:

Then we’ll open Copilot Studio and create a new copilot. Don’t be surprised if things look familiar. In this context, copilot means the old Power Virtual Agent, i.e. a traditional chatbot.

First, we give a name for the copilot:

Let’s then go to advanced settings to change the solution to the one we just created:

Next, for this exercise, we get to the most important copilot setting. Let’s go to the Generative AI section to turn on dynamic chaining (Dynamic chaining with generative actions):

Now, when we add plugin actions to the copilot, for example those implemented with flow, the copilot tries to find one among them that can solve the user’s problem. All on its own.

Please note that dynamic chaining is currently only available in US environments.

Those flows are still needed for the copilot.

Flow: search station shortcodes

Let’s create a flow to be started from copilot, which receives the location name as a parameter. Our flow:

• searches for all train stops with their short codes using the readymade connector
• filters the ones you want
• returns the found shortcode. If the code is not found, the text “not found” is returned.

Copilot cannot utilize the end user connection. Let’s change our flow’s Run only users settings to use a preconfigured connection:

Adding functionality to the copilot

How can the copilot make use of the flow we just created? By adding a new plugin action to it:

Let’s search for our flow and select it. Note that it takes a while before the created flow appears for selection:

Next, we tell the copilot what the function does in practice. This is done in the Action details section.

It is worth investing time in typing detailed descriptions for the action since they define how well the copilot plugin is used.

The flow’s parameters should also be described. The location entered by the user is forwarded as is (Identify as User’s entire response):

The value returned by flow is also described:

This time the copilot is free to formulate the answer:

Let’s create a function (click Finish). Ensure that dynamic chaining is enabled:

Does it work?

Let’s turn on Tracing mode, so we can see how copilot works in the background.

Ask for Parkano’s short code and you’ll find it there!

We finally get to the nitty-gritty of using plugins. We can ask right after “what about Kupittaa?”. Copilot knows the context of the conversation and concludes that we are asking Kupittaa for the shortcode.

Flow – Schedule search

Let’s build another flow. This one takes the starting point (From) and the destination (To) as parameters and searches for the next departing trains. Let’s keep the solution simple, i.e. we don’t send flow the day whose schedules we are interested in. So far, we are only searching for the current day’s schedules.

Add flow copilot as a plugin action. Try again to describe its operation to the AI ​​as comprehensible as possible:

The parameters describe the flow’s desired departure and destination stations as shortcodes:

Ready!

Can copilot use two plugin actions together?

Next, we test how the solution works. When the user asks for the schedule between Parkano – Pasila, can copilot automatically first use the flow, which searches for the station code, and then do the actual schedule search with the received shortcodes?

No:

No matter how I tweaked the descriptions of plugin actions and their parameters, I couldn’t get this to work. Copilot does search for the shortcode of the station (usually only the destination), but it does not know how to use the shortcodes it receives in the actual schedule search. In addition, copilot needlessly tells the user which shortcodes it has found.

Combining two plugin actions on your own seems to be (still) too difficult for the copilot. Let’s make the task a bit easier.

Flow – Schedule and short code search in the same workflow

Let’s build a flow, which again receives the starting station and destination as parameters. The flow contains the following steps:

• retrieves station shortcodes
• performs a schedule search for the current day with the found shortcodes
• Returns the next departing trains

The flow ends up looking like this:

Let’s add the flow as a plugin action. We finally managed to find trains leaving from place A to place B:

The context is still preserved, and we can continue by asking about trains to Tampere:

Those with a keen eye noticed that the answer claims the trains go from Tampere to Pasila. Actually, the answer data is for trains from Pasila to Tampere. And the times are all in the wrong time zone.

Small flaws still.

On the other hand, you can naturally ask the copilot for only a few next trains and it knows how to filter the answer it receives from the flow.

Best of all, flow can independently ask the user for the information required for the search. If the user doesn’t enter the starting point, the flow will ask for it.

Impossible dates

In practice we would like to ask for train schedules freely for different days. But here suddenly a wall comes up against me.

Copilot Studio chatbots have long been able to ask the user for the day value, using the ready-made Date type (identify as Date). In this case, the user can enter the day in any format (tomorrow, next Monday, first day of June, 1.1.2024, etc.) and copilot always creates a date from it:

But this doesn’t (yet) work with plugin actions. Let’s add the departure date as a parameter to the flow that retrieves the schedule. Let’s try it first as a string:

Let’s define the parameter type as date (Identify as Date):

However, the result is not as desired. If the user says that he is traveling “next Monday”, the flow gets exactly that as a parameter. And not a corresponding date, as you might expect:

Let’s change the type of flow’s departure date parameter to date. Nothing works after this. When Copilot tries to pass the date to flow as a string:

Finally, the plugin is set to transmit the received date to the flow as it is (User’s entire reponse):

This works. As long as the user provides the date in the format yyyy-mm-dd. Otherwise, the flow will fail.

This is still not convenient.

Summary

Despite the setbacks, I’m excited about this way of building chatbots. That is, copilots. If the technology develops a little more, the dates are successfully recognized and language support is expanded – the possibilities are amazing.

You just need to keep a few things in mind.

First, the flow’s performance can be really slow. After the flow’s execution, the language model takes its own time to think about what to answer to the user based on the result. It is therefore worth optimizing the flows used for this purpose. If the performance is not sufficient for your use case, look for alternative technologies.

Secondly, the language patterns sometimes feel like they’ve got a mind of their own. For example, the copilot may start asking for information that was already included in the first input. Or it unexpectedly reverses the source and destination stations in the response. You need to pay attention to testing your scenarios extensively enough, not just relying on the solution when you get a first result that seems good enough.