Using OpenAI Function Calling to have agents interact with your own API

As of June 2023, function calling was added to the OpenAI API when using the /v1/chat/completions endpoint in GPT4 & GPT3.5 turbo. Function calling is the ability for developers to describe functions to the model, make calls out to external APIs and infer from input if a function call needs to be made, and if all the requirements for the call have been met (prompting for additional information if required).

Examples could include asking an agent to send an email, query a database, connect to an API or simply respond a certain way to particular queries.

In this post I am aiming to have a GPT agent intepret text, infer if we need to make an API call and finally executing custom code before returning a message to the user.

To simplify the process I will be using the openai-function-calling from Jake Cyr, which facilitates the generation of JSON schema dictionaries, simplifying the implementation of custom functions.

For transparancy: This is more a mental note of what I was doing in case I come back to this later. My openAI knowledge is still developing and my Python knowledge is…. questionable. Don’t take what I say here as a guaranteed tutorial !

Pre-Requisites

I am assuming an envirionment already set up for Python development, access to the OpenAI API, (including setting your API keys up) and ideally access to GPT-4.

I am using the following packages, installed with pip.

• openai
• openai_function_calling
• typing
• json

So our imports look like this:

import openai
from typing import Callable
from openai_function_calling import Function, Parameter, FunctionDict
import requests
import json

Instructing the model on your function

We start by defining the function itself (as in the action you want to take) – Just a placeholder for the moment.

def create_new_system_user(username: str, password: str, firstName: str, surname: str) -> str:
    # Actions here, for now return
    return "User Created"

Now, using the openai-function-calling library, we define an OpenAI “Function,” where we specify the function we want to call, its description, constraints, and any follow-ups the agent should consider. We also provide an array of parameters required for the function:

Prompt engineering is fairly important here and being verbose is encouraged. I found that without instructing it to only act if the user was explitly implying they wanted to create a user it would often attempt to call the API incorrectly.

Note: I did have a lot of issues with the above when using the GPT-3.5-turbo model. Whilst function calling is supported it really struggled to ask for missing parameters or determine if the API should be called when ambiguous language was used. For example when I asked it to “create sausages using blah and blah” it still tried to call the user create API. GPT-4 did not have this issue.

create_new_system_user_function = Function(
    "create_new_system_user",
    "When explicitly asked to create a new user, and you are given all the required paramters. A user will be created within the system. Do not take action if the prompt does not explicitly ask to create a new user. If you do not have the required parameters, ask the user for them.",
    [
        Parameter("username", "string", "The username of the new user."),
        Parameter("password", "string", "The password of the new user."),   
        Parameter("firstName", "string", "The password of the new user."),         
        Parameter("surname", "string", "The password of the new user."),         
    ])

Finally we want to expose the FunctionDict which is essentially a JSON schema for the Function we defined.

create_new_system_user_function_dict: FunctionDict = create_new_system_user_function.to_json_schema()

At this point you could run the Python file with a print(create_new_system_user_function_dict) at the end and see the JSON definition it creates, for example:

{'name': 'create_new_system_user', 'description': 'When explicitly asked to create a new user, and you are given all the required paramters. A user will be created within the system. Do not take action if the prompt does not explicitly ask to create a new user. If you do not have the required parameters, ask the user for them.', 'parameters': {'type': 'object', 'properties': {'username': {'type': 'string', 'description': 'The username of the new user.'}, 'password': {'type': 'string', 'description': 'The password of the new user.'}, 'firstName': {'type': 'string', 'description': 'The password of the new user.'}, 'surname': {'type': 'string', 'description': 'The password of the new user.'}}}}

Wiring up a basic chat loop

Now we can just add some standard boilerplate code for calling the Completions API on a while loop (Warning: I didn’t bother putting an exit condition in for this!). This code will simply make an array of messages which we will use to store the input as we go so the agent remembers context (as the message array is sent each time), and then have a while loop which will simply execute infefinitely and keep taking input from the user.. If you do not have access to GPT-4 you can change your model name here -i.e. GPT-3.5-turbo-0630

messages: list[dict[str, str]] = [
    {
        "role": "system", 
        "content": "You are a helpful assistant."
    } 
]
def DoChat():
    while True:
        message = input("User: ")
        messages.append({"role": "user", "content": message})
        response = openai.ChatCompletion.create(
            model="gpt-4",
            messages=messages,
            functions= [create_new_system_user_function_dict],
            function_call="auto",
            temperature=0.0,
        )
        response_message = response["choices"][0]["message"]
        print(response_message)       


DoChat()

Running this now won’t do much however, and the agent will continuously ask for more information.

Now we need to see if the message has been intepreted as a function call, then have the agent execute the relevant function, before responding to the user. This can be achievved by adding the following code after we set the response_message variable.

if("function_call" in response_message):
            ## Call the function
            available_functions: dict[str, Callable] = {
                "create_new_system_user": create_new_system_user
            }
            function_name = response_message["function_call"]["name"]
            function_args = json.loads(response_message["function_call"]["arguments"])
            function_to_call: Callable = available_functions[function_name]
            function_response= function_to_call(**function_args)
            print(f"{function_response}")
            messages.append({"role": "assistant", "content": function_response})

Now if we run the code with a prompt like I want to make a new user it will prompt for more information, and when it thinks it has met all the variables, execute the function at which point we see the print statement in the create_new_system_user method.

Finally, we just need to edit the code within the create_new_system_user method to call out to our API and then return an appropriate string – Which I have omitted here as its system specific.

The final code

import openai
from typing import Callable
from openai_function_calling import Function, Parameter, FunctionDict
import json


def create_new_system_user(username: str, password: str, firstName: str, surname: str) -> str:
   # Make an API call (omitted for this example)
   # Handle response as success or false
   # Return an appropriate message to the user
   # For now just return a placeholder message
   return "User has been created"

create_new_system_user_function = Function(
    "create_new_system_user",
    "When explicitly asked to create a new user, and you are given all the required paramters. A user will be created within the system. Do not take action if the prompt does not explicitly ask to create a new user. If you do not have the required parameters, ask the user for them.",
    [
        Parameter("username", "string", "The username of the new user."),
        Parameter("password", "string", "The password of the new user."),   
        Parameter("firstName", "string", "The password of the new user."),         
        Parameter("surname", "string", "The password of the new user."),         
    ])

create_new_system_user_function_dict: FunctionDict = create_new_system_user_function.to_json_schema()


messages: list[dict[str, str]] = [
    {
        "role": "system", 
        "content": "You are a helpful assistant."
    } 
]
def DoChat():
    while True:
        message = input("User: ")
        messages.append({"role": "user", "content": message})
        response = openai.ChatCompletion.create(
            model="gpt-4",
            messages=messages,
            functions= [create_new_system_user_function_dict],
            function_call="auto",
            temperature=0.0,
        )
        response_message = response["choices"][0]["message"]
        print(response_message)
        if("function_call" in response_message):
            ## Call the function
            available_functions: dict[str, Callable] = {
                "create_new_system_user": create_new_system_user
            }
            function_name = response_message["function_call"]["name"]
            function_args = json.loads(response_message["function_call"]["arguments"])
            function_to_call: Callable = available_functions[function_name]
            function_response= function_to_call(**function_args)
            print(f"{function_response}")
            messages.append({"role": "assistant", "content": function_response}) 


DoChat()

Note: Sometimes odd things happen. Occasionally it would think it had met all the criteria and attempt to call my API with a missing param. Still early days, or perhaps more error checking… I did say this wasn’t a robust tutorial 🙂