AI Product Engineering

Independent engagements building full-stack web applications with generative AI capabilities, from first specification through to production deployment.

Client names and project specifics are confidential. This case study describes the recurring engineering problems and the approach, not any individual engagement.

Most requests to 'add AI' to a product are really requests to remove a manual step. The engineering work is rarely in the model call itself; it is in everything around it — getting reliable structured output, handling the cases where the model is wrong, and integrating the result into a workflow that already exists.

• LLM output is probabilistic while application logic expects determinism
• Cost and latency both scale with usage in ways that surprise stakeholders
• Failure modes are unfamiliar to teams used to conventional software
• The valuable step is often narrow and specific, not a general assistant

• Identify the specific step where automation delivers value, rather than adding a general chat interface
• Produce structured, validated output that downstream code can rely on
• Keep cost and latency bounded and predictable
• Deliver a complete application, not a prototype
• Leave a codebase the client can maintain

• Small budgets and short timelines typical of early-stage products
• Model behaviour changes between provider versions
• Client teams may have limited capacity to operate complex infrastructure
• Confidentiality over client data and business logic

1. 01

   Client Application

   React and TypeScript interface

2. 02

   Authenticated API Layer

   FastAPI or Flask, request validation

3. 03

   Business and Workflow Services

   Application logic and AI-assisted steps

4. 04

   Database and External Integrations

   Persistence and third-party services

5. 05

   Monitoring, Notifications and Background Jobs

   Async processing and observability

• AI calls isolated behind an internal interface so the provider can be changed
• Structured output enforced with schema validation rather than parsed from free text
• Retry with a repair step when validation fails, bounded by a retry limit
• Deterministic fallback path when the AI step cannot produce valid output
• Long-running work moved to background jobs so requests stay fast

• Cheaper models used for narrow classification steps, stronger models reserved for the steps that need them
• Caching of results for identical inputs
• Prompt and context size controlled deliberately, since context length drives cost
• Batching where the workload allows it

• Managed hosting appropriate to the client's operational capacity
• Environment separation and automated deployment from version control
• Configuration and API keys supplied through the environment

• Schema validation on every AI response before it reaches application logic
• Explicit fallback behaviour rather than propagating a failed AI call to the user
• Rate limiting and timeout handling on all external calls
• Monitoring of failure rates on AI-assisted steps specifically, since they fail differently from conventional code

AI-assisted steps are evaluated on a fixed set of representative inputs, measuring the rate of valid structured output and the rate of correct output separately — a response can be well-formed and wrong, and only the second rate reflects usefulness.

Client outcomes and business metrics are confidential and are not published here.

Benchmarking in progress. Results will be published with the hardware, model, runtime and measurement conditions that produced them.

• The hard part of shipping AI features is the surrounding engineering, not the model call.
• Treat model output as untrusted input. Validate it exactly as you would validate a request from a client.
• A narrow automated step that works beats a general assistant that mostly works.
• Defining the fallback path first makes the whole feature simpler to reason about.

• Continued work on evaluation harnesses for AI-assisted steps
• Reusable patterns for structured output and repair across engagements