Getting Started with the OpenAI API: Your First Completion

This is the first post in our OpenAI API fundamentals series. Later posts cover function calling, context/token/cost management, RAG, and fine-tuning vs. prompting.

Every OpenAI API tutorial starts with essentially the same five-line example, and it's genuinely a reasonable place to start — the API's core shape really is that simple. What most tutorials skip past is everything around that example that actually matters once you're building something real.

The basic request

from openai import OpenAI
 
client = OpenAI()  # reads OPENAI_API_KEY from the environment
 
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant for a technology training company."},
        {"role": "user", "content": "Explain what a load balancer does, in two sentences."},
    ],
)
 
print(response.choices[0].message.content)

A load balancer distributes incoming network traffic across multiple
servers so no single server becomes overwhelmed, improving both
reliability and response times. If one server fails, the load balancer
routes traffic to the remaining healthy servers automatically.

The messages array is the entire conversation the model sees, every time — there's no server-side memory of previous requests at all. Three roles matter here: system sets behavior and context for the whole conversation, user is the actual prompt, and assistant (not used yet in this first example) is how the model's own previous replies get fed back in for multi-turn conversations, covered directly in the context management post later in this series.

Never hardcode an API key

import os
from openai import OpenAI
 
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

The official SDK actually reads OPENAI_API_KEY from the environment automatically, so OpenAI() with no arguments already does the right thing — the explicit version above is worth showing once just to make clear where that value has to come from. An API key hardcoded into source code is the exact same category of mistake as a database password committed to git, covered repeatedly throughout this blog for Terraform, Ansible, and Kubernetes: it ends up in version control history, in logs, in a screen-share, permanently.

Parameters that actually change behavior

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    temperature=0.2,
    max_tokens=300,
)

• temperature controls randomness in the model's output. Lower values (0.0–0.3) produce more consistent, deterministic-leaning output — appropriate for factual tasks, structured extraction, code generation. Higher values (0.7+) produce more varied, creative output — appropriate for brainstorming or creative writing, not for a task where you need the same input to reliably produce a similar answer.
• max_tokens caps how long a response can be. Leaving it unset risks an unexpectedly long (and correspondingly expensive) response for an open-ended prompt; setting it too low truncates a genuinely necessary answer mid-sentence. Getting this right for a specific use case usually means testing against real example prompts, not guessing.

Two more parameters are worth knowing beyond these first two:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    temperature=0.2,
    max_tokens=300,
    top_p=1.0,
    presence_penalty=0.0,
)

• top_p is an alternative way to control randomness (nucleus sampling), restricting the model to the smallest set of most-likely next tokens whose combined probability reaches top_p. OpenAI's own guidance is to adjust temperature or top_p, not both simultaneously — they interact in ways that are hard to reason about together, so pick one lever and leave the other at its default.
• presence_penalty (and its sibling frequency_penalty) discourages the model from repeating the same tokens or topics — useful for longer generative outputs that otherwise tend to loop back on the same phrasing, rarely necessary for short, factual responses.

Streaming: responding before the full answer is ready

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a short paragraph about Kubernetes."}],
    stream=True,
)
 
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Without streaming, a user waits for the entire response to generate before seeing anything at all — for a long response, that can be several real seconds of a blank screen. stream=True yields the response incrementally, token by token, as it's generated — the mechanism behind the "typing" effect in every chat interface built on this API, and a genuinely significant perceived-latency improvement even though the total generation time is identical either way.

Handling errors and rate limits properly

import time
from openai import RateLimitError, APIError
 
def get_completion(messages, retries=3):
    for attempt in range(retries):
        try:
            return client.chat.completions.create(model="gpt-4o", messages=messages)
        except RateLimitError:
            wait = 2 ** attempt
            time.sleep(wait)
        except APIError as e:
            if attempt == retries - 1:
                raise
            time.sleep(1)
    raise RuntimeError("Exhausted retries")

A production application calling this API needs to handle rate limits (a RateLimitError, resolved by backing off and retrying, ideally with exponential backoff as shown) and transient API errors as a matter of course, not as an edge case discovered after a real outage. A demo script that calls the API once and prints the result never needs this; anything running unattended, at real volume, needs it from the first version.

Rate limits themselves come in two distinct dimensions worth knowing: RPM (requests per minute) and TPM (tokens per minute). An application making many small requests can hit an RPM limit well before its actual token throughput is anywhere near the TPM limit — worth checking which limit a given account or use case is actually constrained by, rather than assuming both scale together proportionally.

Setting a request timeout explicitly

The SDK's default timeout is generous enough that a genuinely stuck request can leave your own application hanging far longer than acceptable for a real user-facing path:

client = OpenAI(timeout=20.0)
 
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    timeout=10.0,  # overrides the client-level default for this specific call
)

Setting an explicit, deliberately-chosen timeout — shorter for a user-facing request where a slow response is worse than a fast failure, longer for a background batch job where completing correctly matters more than completing quickly — is worth doing from the first version of any production code path, rather than relying on the SDK's default and discovering it's too generous only after a user complains about a request that hung for a genuinely long time.

Choosing a model deliberately, not just defaulting to the newest

gpt-4o isn't the only option, and defaulting to whichever model a tutorial happens to use is the same mistake as defaulting to an oversized cloud instance because a tutorial used it. OpenAI's lineup spans a genuine capability-and-cost range:

response = client.chat.completions.create(
    model="gpt-4o-mini",  # smaller, cheaper, faster — fine for simple tasks
    messages=[...],
)

We cover this in real depth in a later post in this series, but it's worth flagging even at this introductory stage: a classification task, a simple extraction task, or an internal tool rarely needs the same model as a nuanced, customer-facing conversation. Treating model choice as a deliberate decision per use case, from the very first prototype, avoids a costly habit that's harder to unlearn once an application has many call sites all defaulting to the same, most expensive model out of inertia.

Reading usage data from every response

Every response includes a usage object worth logging from the very first version of any real application, not added retroactively once cost becomes a visible problem:

print(response.usage.prompt_tokens, response.usage.completion_tokens, response.usage.total_tokens)

Logging this alongside every call from day one is what makes the cost-management practices covered later in this series — trimming history, routing to cheaper models — measurable rather than guessed at, the same "instrument before you optimize" discipline as any other performance or cost work.

What to actually remember from this post

• messages is the entire conversation state, sent fresh on every request — the API itself has no memory between calls.
• temperature and max_tokens are the two parameters worth deliberately setting for almost any real use case, not leaving at their defaults; adjust temperature or top_p, never both at once.
• Streaming improves perceived latency, not actual total generation time — genuinely worth it for anything with a human watching in real time.
• Handle rate limits and transient errors from the start — with exponential backoff, not a single unguarded try — and know whether your use case is RPM-bound or TPM-bound.

Next in the series: Function Calling and Structured Outputs with the OpenAI API, where the model stops just returning text and starts reliably returning data your code can act on directly.