A Full-Stack Python Developer’s Guide to Shipping AI Projects

A model call is just another dependency. It is slower than your database, less reliable than your database, priced per token, and it can return something syntactically perfect and factually wrong. Every design decision in this post follows from those four properties.

I have shipped full stack Python AI projects for teams in the US, UK and UAE: CV screening, multi-tenant WhatsApp SaaS, natural-language lead generation, clinical decision support, computer vision APIs. The components barely change between them. What changes is the plumbing around the request path once a language model is inside it, and that is where most builds go wrong.

What stack do full stack Python AI projects actually run on?

A deliberately unremarkable one. FastAPI for the API, Pydantic for every boundary, Celery with Redis for long work, Postgres with pgvector for rows and embeddings, React with TypeScript on the front end, Docker for build parity, and CI that runs more than unit tests. The interesting engineering is not in choosing these; it is in deciding where the model call sits relative to them.

"The model is not your architecture. It is one slow, expensive, non-deterministic dependency that your architecture has to survive."

What changes when AI enters the request path?

Four assumptions break at once: latency moves from milliseconds to seconds, failure becomes partial rather than binary, output is non-deterministic so tests cannot assert equality, and every call carries a price. A CRUD app quietly depends on all four being false.

If the model is also choosing its own steps rather than answering once, the operational surface grows again, and I cover that separately in building production-ready AI agents with Python. This post assumes the simpler and far more common case: an application that makes model calls as part of otherwise normal features.

How do you handle timeouts on slow model calls?

Pick one of three shapes per endpoint: answer inline under a hard timeout, stream tokens so the connection stays alive, or hand the job to Celery and return a job ID. My rule is that anything that can plausibly exceed thirty seconds goes to the queue, no exceptions.

Celery's task documentation says task functions should ideally be idempotent, and presents acks_late as an option for tasks that are, because a terminated worker process can cause the same task to run again. With model calls that matter twice over: a retried task is a second bill and a second side effect. I key every AI task on a deterministic idempotency key derived from the input, and store the result against that key before doing anything visible with it.

Why does Pydantic matter more once a model generates your data?

Because the model has become an input source you do not control, producing data you intend to store. Pydantic is where you catch it. The same discipline you already apply to request bodies applies to completions, and the failure mode without it is a plausible-looking record with a hallucinated enum value.

import json
from typing import Literal
from pydantic import BaseModel, Field, ValidationError


class LeadScore(BaseModel):
    tier: Literal['hot', 'warm', 'cold']
    score: int = Field(ge=0, le=100)
    reason: str = Field(max_length=280)
    evidence_ids: list[str] = Field(default_factory=list, max_length=5)


def score_lead(client, payload: dict, attempts: int = 2) -> LeadScore:
    schema = LeadScore.model_json_schema()
    messages = [
        {'role': 'system', 'content': 'Score the lead. Return JSON matching the schema.'},
        {'role': 'user', 'content': json.dumps(payload)},
    ]
    for attempt in range(attempts):
        raw = call_model(client, messages, schema=schema, timeout=20)
        try:
            return LeadScore.model_validate_json(raw)
        except ValidationError as exc:
            messages.append({'role': 'assistant', 'content': raw})
            messages.append({
                'role': 'user',
                'content': 'That failed validation: ' + exc.json() + '. Return corrected JSON only.',
            })
    raise ValueError('model could not produce a valid LeadScore')

Three things are doing the work there. Literal instead of str means an invented tier fails loudly rather than landing in the database. The validation error is fed back to the model as text, which repairs most single-field mistakes on the second attempt. And the retry count is finite, so a confused model becomes an exception instead of a loop.

Provider-side constraints help too. OpenAI documents structured outputs that constrain generation to a JSON Schema you supply, which removes most shape errors before they reach you. Treat that as a first line of defence and keep the Pydantic validation anyway, because schema conformance is not the same as semantic correctness. Practical patterns for wiring that into a business system are in connecting the OpenAI API to your CRM and on the OpenAI integration page.

Do you need a vector database, or is Postgres with pgvector enough?

For nearly every build I have shipped, Postgres with pgvector is enough. One store, one backup, one migration path, and metadata filters live in the same query as the similarity search, which is exactly what multi-tenant applications need.

Move to a dedicated vector service when you genuinely outgrow it: very large corpora where index build time dominates, or sustained query volumes where you want that workload isolated from your transactional database. Below that, a separate vector store is one more thing to keep in sync and one more place for stale documents to hide. The harder problem is almost always the ingestion pipeline, which is why chunking, deduplication and re-embedding on change belong in a proper data engineering design rather than in a helper function. The Airflow pipeline behind my analytics build exists for exactly that reason.

How do you stream tokens to a React and TypeScript front end?

Server-sent events over a FastAPI StreamingResponse, read on the client with fetch and a ReadableStream reader. WebSockets are more machinery than a one-directional token stream needs, and SSE survives ordinary HTTP infrastructure better.

import json

from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse

app = FastAPI()


async def token_stream(request: Request, prompt: str):
    usage = {}
    async for chunk in call_model_stream(prompt):
        if await request.is_disconnected():
            break
        if chunk.delta:
            yield 'data: ' + json.dumps({'delta': chunk.delta}) + '\n\n'
        if chunk.usage:
            usage = chunk.usage
    yield 'event: done\ndata: ' + json.dumps({'usage': usage}) + '\n\n'


@app.post('/api/answer')
async def answer(request: Request, prompt: str):
    return StreamingResponse(
        token_stream(request, prompt),
        media_type='text/event-stream',
        headers={'Cache-Control': 'no-cache', 'X-Accel-Buffering': 'no'},
    )

How do you keep AI costs from surprising you?

Cost equals tokens per call multiplied by calls per user action multiplied by volume. Log token usage per request from the first commit, attach it to a tenant and a feature, and enforce caps in code rather than relying on a billing alert that arrives after the damage.

Providers price per million input and output tokens, usually with cached input billed lower than fresh input, and those rates change often enough that any figure in a blog post is stale. Check the official pricing page for the model you have chosen, then multiply by token counts you measured in your own logs. That measured number, not the headline rate, is what you build a margin on.

What do evals and CI look like for full stack Python AI projects?

Three tiers. Fast unit tests with the provider mocked, a small contract test that hits the real API, and an eval suite of real cases scored against known-good outcomes that runs nightly or on prompt changes. Without the third tier, a prompt edit is an unreviewed production change.

Everything ships in Docker with the provider SDK version pinned. Model deprecations and SDK changes are a real source of breakage, and a pinned image plus a nightly contract test is how you find out on your schedule rather than during a customer demo.

What would I cut from a first build?

Most of it. The first version of nearly every AI feature I have shipped was one FastAPI endpoint, one Pydantic model, one Postgres table and a Celery task, with no framework in between.

The pattern held on AgentFlow, a visual AI agent builder, where the visual layer was the last thing built rather than the first, and on a YOLO-based detection API, where the queue and the result store mattered more than the model wrapper. Good full stack Python AI projects look like good Python projects with one awkward dependency handled carefully. If you want that shape built properly the first time, that is what my full-stack Python development work is for.