Build Your First Real App on the OpenAI API
Calling the OpenAI API from a script is trivial. Building a real application around it, one that handles structured output, errors, retries, cost, and streaming, is where the actual craft lives. In this tutorial you will go past the hello-world chat call and build a small but production-minded application: a support-ticket classifier that reads an incoming message, extracts structured fields, and streams a suggested reply. You will learn the API patterns that separate toy scripts from software you would be comfortable running in front of users, and you will come away with code you can immediately extend.
The single most important mindset shift is this: treat the model output as data, not as final text. Real applications need the model to return fields you can parse and act on, which is exactly why structured output matters.
Set Up the Client the Right Way
Install the official SDK and configure it once. Keep your key in an environment variable, never in source code, and create a single shared client your whole app reuses.
pip install openai
# .env
OPENAI_API_KEY=sk-...
python
from openai import OpenAI
client = OpenAI() # reads OPENAI_API_KEY automatically
A shared client is not just tidy; it reuses connections and lets you centralize timeouts. If you instantiate a new client inside every request handler, you throw away connection pooling and your latency and cost both creep up.
Call the Chat Completions Endpoint
The core call accepts a list of messages and returns a completion. You will use the messages list to hold system instructions and user content, which is where all of the prompt design happens.
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You classify support tickets. Be concise."},
{"role": "user", "content": msg},
],
temperature=0.2,
)
answer = resp.choices[0].message.content
Notice temperature=0.2. For classification you want deterministic output, so you keep creativity low. For a creative-writing feature, you would raise it. Choosing temperature per task, instead of using one setting everywhere, is a small habit with a big effect.
Get Predictable JSON with Structured Output
Parsing free text is brittle. Instead of asking for JSON in a prompt and hoping, use the API's built-in structured output support, where you declare a JSON schema and the model returns exactly that shape. This is the difference between building a fragile string parser and shipping real software.
resp = client.responses.create(
model="gpt-4o-mini",
text={"format": {"type": "json_schema", "schema": { "type": "object",
"properties": {
"category": {"type": "string"},
"urgency": {"type": "string"},
"summary": {"type": "string"},
},
"required": ["category", "urgency", "summary"]
}}},
input="User message: " + msg,
)
I am using the responses API here because it makes structured output and built-in tools cleaner to express. Whatever endpoint you choose, the principle is identical: declare the schema up front, then treat the output as typed data you can route on. Your code can now branch on category and urgency without a single regex.
Stream the Reply for a Better Feel
Users experience streaming as speed. When you add stream=True, the API sends tokens as they are generated, and your front end can render them live instead of making the user stare at a spinner for ten seconds.
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
In a web framework, you turn that generator into a server-sent events stream or an async generator. The pattern lets one model call serve many users without blocking your request handlers.
Handle Errors, Retries, and Costs
Real apps must fail gracefully. The SDK surfaces specific errors for rate limits, invalid API keys, and timeouts, and you should retry only the transient ones with exponential backoff, never the permanent ones like an invalid key.
- Retry on rate limits and timeouts with backoff and jitter; do not hammer the server.
- Never retry on auth or schema errors; fix the input instead.
- Log every call's model and token usage so you can spot runaway costs before the bill is a surprise.
Rate-limit errors are a signal to back off and slow down, not to hammer harder. A well-built retry loop with exponential backoff and random jitter prevents most outages before your application ever sees an error.
For cost, always set the smallest adequate model, monitor token usage per request, and cache answers for repeated questions. In many support tools, deduplicating common questions cuts API spend by a large fraction.
Put It Together and Think About Prompts
You now have the four pillars: a shared client, structured output to route on, streaming for responsiveness, and disciplined error and cost handling. Assemble them into one endpoint and spend the rest of your energy on the system prompt, because that is what your end users actually interact with. Write clear instructions, give examples of good and bad classifications, and iterate based on real tickets.
Choosing a Model and Budgeting Tokens Wisely
The endpoint you now call is one of several model sizes, and choosing among them is a real cost and latency decision rather than a detail. For simple classification or short replies, a small, fast model usually delivers excellent results at a fraction of the token price and response time. Reserve the larger and more capable models for genuinely complex reasoning, long-form writing, or when the cost of a wrong answer is high. Testing the same prompt across two model tiers on your own data is the fastest way to learn how much capability you actually need.
On the cost side, streaming and structured output change the economics in useful ways. Streaming makes the reply feel instant and lets you render partial text, and with a cap on the number of tokens you can keep runaway responses from inflating a bill. Setting a reasonable max_tokens guardrail on every call is cheap insurance, and logging the token counts returned by the API lets you spot prompt bloat before it becomes a monthly expense line. A call that used to sit quietly under your threshold is the first place cost creep shows up.
The production discipline of an API integration is not about the first call; it is about knowing what each call costs and whether it earned that cost.
Set a simple monthly budget target for the integration and review token usage against it weekly at first. When usage moves predictably, revisit prompt lengths and model tier once more before scaling, because those two levers dominate most of the cost you can control.
Once the classifier works, the natural extensions are adding a built-in retrieval tool so the classifier can look up a customer's order history, and promoting to the async client for concurrency. Both build directly on the foundation you just laid, and both will feel straightforward because you already understand how the API behaves under real application conditions.
