Function calling

In addition to providing answers to customers’ questions, Markprompt is able to detect underlying intents and trigger code accordingly, such as to create a case. This is a concept know as function calling . The AI can detect a user’s intent (such as “I want to speak with an agent”), make sure the user provides all the required details for the function to run, and display an adequate UI, e.g. to confirm an action.

Anatomy of a function

Here is a basic example of a function definition:

1{
2  "name": "createCase",
3  "description": "Creates a case automatically when the user asks to create a ticket/case or when they ask to speak to someone."
4}

If consists of two mandatory fields:

  • A function name
  • A function description

The description is important. This is how the LLM is able to match a user’s intent with a function call. Make it as clear and explicit as possible, and keep it concise.

Functions with parameters

Some functions require additional parameters. For instance, if the user is asking to cancel their subscription, the function may need to know about the end date:

1{
2  "name": "processRefund",
3  "description": "Cancel auto-renewal and process a refund",
4  "parameters": {
5    "type": "object",
6    "properties": {
7      "cancellationDate": {
8        "type": "string",
9        "description": "The date at which the auto-renewal should end"
10      }
11    },
12    "required": ["cancellationDate"]
13  }
14}

In this example, you will notice that the cancellationDate field is marked as required. Thus, if a user asks the question:

I want to cancel my renewal and get a refund

the AI will ask a follow-up question, inquiring about the desired cancellation date:

Sorry to see you go! When would you like your subscription to end?

Triggering a function call

Now that the function’s intent has been specified, it is time to bind it to a function, which will be invoked when the intent is detected.

Using the SDK

The Markprompt React SDK provides helpers to easily setup callback functions that get triggered when a function call intent is detected. Here is an example using the ChatProvider component:

1import { useCallback, useMemo, type ReactNode } from 'react';
2import { Markprompt, ChatProvider, ChatViewMessage } from '@markprompt/react';
3
4const Home = ({ children }: { children: ReactNode }) => {
5  const submitCase = useCallback(() => {
6    // Create a case
7    // ...
8  }, []);
9
10  const chatOptions = useMemo(
11    () => ({
12      toolChoice: 'auto',
13      tools: [
14        {
15          tool: {
16            type: 'function',
17            function: {
18              name: 'createCase',
19              description:
20                'Creates a case automatically when the user asks to create a ticket/case or when they ask to speak to someone.',
21            },
22          },
23          call: async () => {
24            // This code gets called when the user confirms the call
25            await submitCase();
26            return 'Generating case details for you.';
27          },
28        },
29      ],
30    }),
31    [submitCase],
32  );
33
34  return (
35    <ChatProvider projectKey="<YOUR-PROJECT-KEY>" chatOptions={chatOptions}>
36      {children}
37    </ChatProvider>
38  );
39};

Alternatively, function calls can be set up in the Markprompt dashboard via assistants.

Showing a confirmation UI

In some situations, it may not be desirable to automatically call the function without an explicit user confirmation. The SDK allows you to define a confirmation UI to show before the function is actually called. This is done via the ToolCallsConfirmation prop. Here is an example:

1import { useCallback } from 'react';
2import { Markprompt, ChatProvider, ChatViewMessage } from '@markprompt/react';
3
4const Home = ({ children }) => {
5  // ...
6
7  return (
8    <ChatProvider
9      chatOptions={{
10        // ...
11        tools: ['...'],
12        ToolCallsConfirmation: ({
13          toolCalls,
14          toolCallsStatus,
15          confirmToolCalls,
16        }) => {
17          // Define a custom UI allowing the user to confirm their intent
18          const toolCall = toolCalls[0];
19          if (!toolCall) {
20            return <></>;
21          }
22
23          const status = toolCallsStatus[toolCall.id]?.status;
24          return (
25            <>
26              <p>Please confirm that you want to create a case:</p>
27              <button onClick={confirmToolCalls} disabled={status === 'done'}>
28                Confirm
29              </button>
30            </>
31          );
32        },
33      }}
34    >
35      {children}
36    </ChatProvider>
37  );
38};

Using the API

The Markprompt API also supports function calling. Instead of calling code, it will return a fully-specified JSON object with the required information to take adequate action:

1curl https://api.markprompt.com/chat \
2  -X POST \
3  -H "Authorization: Bearer <TOKEN>" \
4  -H "Content-Type: application/json" \
5  -H "X-Markprompt-API-Version: 2024-05-21" \
6  -d '{
7    "messages": [
8      { "content": "I need to speak with an agent", "role": "user" }
9    ],
10    "toolChoice": "auto",
11    "tools": [
12      {
13        "tool": {
14          "type": "function",
15          "function": {
16            "name": "createCase",
17            "description": "Creates a case automatically when the user asks to create a ticket/case or when they ask to speak to someone.",
18          }
19        }
20      }
21    ]
22  }'

If a function call intent is detected, the response will include a toolCalls entry:

1{
2  "toolCalls": [
3    {
4      "id": "call_id",
5      "type": "function",
6      "function": {
7        "name": "createCase"
8      }
9    }
10  ]
11}

Biasing behavior towards function calling

Markprompt can be instructed to favor function calling over responding in text. For instance, if the user has an urgent billing issue and you would like this to trigger a case creation without the user explicitly asking for it, you can provide additional system prompt instructions stating this behavior. For instance:

  • You must always prioritize creating a function call over returning a text response if the user enters a request that a function can handle.
  • If the user has an urgent billing issue, immediately trigger a call to the createCase function.

Examples

Below are some examples making use of function calling: