Function Calling and Structured Outputs with the OpenAI API

This is the second post in our OpenAI API fundamentals series. Start with your first completion if you're joining partway through.

A plain text response is fine for a chatbot a human reads directly. It's a genuinely fragile foundation for anything where your own code needs to parse the model's output and act on it — a prompt that says "please respond in JSON" produces output that's usually valid JSON, which is a meaningfully different guarantee than always. Function calling and structured outputs are the API's actual mechanisms for closing that gap.

Function calling: letting the model request an action

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_course_availability",
            "description": "Check available seats for a specific training course",
            "parameters": {
                "type": "object",
                "properties": {
                    "course_name": {"type": "string", "description": "Name of the course"},
                    "date": {"type": "string", "description": "Preferred start date, YYYY-MM-DD"},
                },
                "required": ["course_name"],
            },
        },
    }
]
 
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Is there space in the Terraform course starting in March?"}],
    tools=tools,
)
 
message = response.choices[0].message
if message.tool_calls:
    for call in message.tool_calls:
        print(call.function.name, call.function.arguments)

get_course_availability {"course_name": "Terraform", "date": "2026-03-01"}

Critically, the model never actually calls the function itself. It decides a function call is warranted, and returns the function's name along with arguments matching the schema you defined — your own code is what actually runs get_course_availability(...), using those extracted arguments. This is the entire mechanism: the model is reliably turning unstructured natural language ("is there space in the Terraform course starting in March?") into a structured, schema-conforming function call your code can execute directly.

Fig. 1 — the function-calling round trip: the model requests, your code executes, the result comes back for a grounded reply

Completing the loop: feeding the result back

import json
 
# Your own actual function
def get_course_availability(course_name, date=None):
    return {"course_name": course_name, "seats_available": 4, "next_cohort": "2026-03-09"}
 
tool_call = message.tool_calls[0]
args = json.loads(tool_call.function.arguments)
result = get_course_availability(**args)
 
follow_up = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Is there space in the Terraform course starting in March?"},
        message,
        {
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(result),
        },
    ],
)
 
print(follow_up.choices[0].message.content)

Yes — there are 4 seats available in the Terraform course, with the
next cohort starting March 9th, 2026.

The second call feeds the function's actual result back in as a role: "tool" message, tied to the original tool_call_id — this is what lets the model produce a natural-language answer grounded in your system's real data, rather than a plausible-sounding guess. This two-step round trip (model requests a call → your code executes it → result goes back to the model) is the actual pattern behind every "AI assistant that can look things up or take actions" system, not a special separate capability layered on top of chat completions.

Parallel tool calls: more than one function in a single turn

A single user message can trigger the model to request several tool calls at once, not just one — genuinely common for a question that requires combining several pieces of real data:

message = response.choices[0].message
if message.tool_calls:
    results = []
    for call in message.tool_calls:
        args = json.loads(call.function.arguments)
        if call.function.name == "get_course_availability":
            result = get_course_availability(**args)
        elif call.function.name == "get_course_price":
            result = get_course_price(**args)
        results.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })
 
    follow_up = client.chat.completions.create(
        model="gpt-4o",
        messages=[*original_messages, message, *results],
    )

A question like "is there space in the Terraform course starting in March, and how much does it cost" can trigger both get_course_availability and get_course_price as separate tool calls in a single model response — your code needs to handle iterating over message.tool_calls as a list from the start, rather than assuming exactly one call per turn, since a real production system routinely needs to feed back multiple tool results before the model can produce its final answer.

Structured outputs: guaranteeing schema conformance directly

Function calling is the right tool when the model needs to request an action. For simpler cases — extracting structured data from text, with no actual function to call — structured outputs enforce a JSON schema directly on the model's response:

from pydantic import BaseModel
 
class CourseInquiry(BaseModel):
    course_name: str
    preferred_date: str | None
    urgency: str  # "low", "medium", "high"
 
response = client.chat.completions.parse(
    model="gpt-4o",
    messages=[{"role": "user", "content": "I need Kubernetes training ASAP, ideally next month"}],
    response_format=CourseInquiry,
)
 
inquiry = response.choices[0].message.parsed
print(inquiry.course_name, inquiry.urgency)

Kubernetes high

Unlike a prompt that merely asks for JSON, response_format with a defined schema (a Pydantic model, here) makes the API guarantee the response conforms to that exact structure — not "usually valid JSON matching roughly what I asked for," but a genuine contract your code can rely on without a try/except around a json.loads() call that might fail unpredictably.

Nested and more expressive schemas

Structured outputs aren't limited to flat fields — a Pydantic model can express nested structure and enums, and the guarantee still holds:

from enum import Enum
from pydantic import BaseModel
 
class Urgency(str, Enum):
    low = "low"
    medium = "medium"
    high = "high"
 
class CourseInquiry(BaseModel):
    course_name: str
    preferred_dates: list[str]
    urgency: Urgency
    contact: dict[str, str]
 
response = client.chat.completions.parse(
    model="gpt-4o",
    messages=[{"role": "user", "content": "..."}],
    response_format=CourseInquiry,
)

An Enum field constrains the model to one of a fixed set of literal values rather than free text — genuinely useful for anything downstream that switches behavior based on the exact value (routing a "high" urgency inquiry to a different queue than a "low" one), where a model that occasionally returned "Urgent" or "HIGH" instead of "high" would silently break that routing logic.

Choosing between the two

Function calling: the model needs to trigger a real action or look up real data from your system — checking availability, sending an email, querying a database.
Structured outputs: you need reliably-shaped data extracted from a response — classifying a support ticket's urgency, extracting fields from a user's free-text message — with no actual function being called at all.

Both replace "ask for JSON in the prompt and hope" with an actual, enforced contract — they solve two related but genuinely distinct problems.

Handling a model choosing not to call any tool

Not every user message actually warrants a tool call — a genuinely conversational message ("thanks, that's helpful") shouldn't trigger the model reaching for get_course_availability just because tools happen to be available. Code that assumes message.tool_calls will always be present needs to handle the case where it isn't:

message = response.choices[0].message
if message.tool_calls:
    # handle tool calls as shown earlier in this post
    ...
else:
    print(message.content)  # the model responded directly, with no tool call at all

This is a genuinely common oversight in a first implementation — code that only handles the tool-call branch and assumes it always fires will crash or silently drop the response entirely on the (frequent, in a real conversation) turns where the model correctly decides no tool is actually needed.

Forcing a specific tool call, when you already know which one is needed

The examples so far let the model decide whether and which tool to call. tool_choice overrides that, useful when your application logic already knows a specific tool must be used for a given code path:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Kubernetes course, please."}],
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "get_course_availability"}},
)

Forcing a specific tool is worth reaching for when your application has already narrowed down the intent through other means (a menu selection, a specific UI action) and the model's only remaining job is extracting well-structured arguments for a call you already know needs to happen — genuinely different from the general case of letting the model decide whether a tool call is warranted at all.

What to actually remember from this post

The model never calls a function itself — it returns a name and matching arguments; your own code executes the actual function.
Feed a tool's result back in as a role: "tool" message, tied to the original tool_call_id, to complete the loop and get a grounded natural-language answer.
A single turn can trigger multiple parallel tool calls — handle message.tool_calls as a list, not a single expected call.
Structured outputs guarantee schema conformance directly — a genuinely different guarantee than a prompt merely asking for JSON, and nested schemas with enums extend that guarantee to more expressive data shapes.
Function calling is for triggering actions; structured outputs are for extracting reliably-shaped data — related, but solving different problems.

Next in the series: Managing Context, Tokens, and Cost in Production OpenAI Applications, where we cover what happens once real conversations get long enough for tokens and cost to become genuine engineering constraints.