AI Integration

I've shipped production AI systems that a business actually depends on: a Claude-powered pipeline that ingests messy real-estate listings into structured data, and a competitor price-monitoring system that posts market intelligence to Slack. Neither is a chatbot. Both taught the same lesson: integrating an LLM is a data-engineering problem with a probabilistic component in the middle.

The trust boundary

The architecture that survives contact with production treats the model as an untrusted client. It proposes; your code disposes.

Nothing the model says touches the database until it passes typed validation and domain rules (an apartment isn't 4 m², rent isn't ¥8). Failures retry with the validation error in context. Models are remarkably good at fixing their own output when told exactly what's wrong:

const Listing = z.object({
  layout: z.enum(["1R", "1K", "1DK", "1LDK", "2LDK", "3LDK"]),
  areaSqm: z.number().min(8).max(500),
  rentYen: z.number().int().min(20_000).max(5_000_000),
  confidence: z.number().min(0).max(1),
});
 
async function extract(source: string, feedback?: string) {
  const raw = await claude(EXTRACTION_PROMPT, source, feedback);
  const result = Listing.safeParse(raw);
  if (!result.success) {
    return extract(source, formatZodError(result.error)); // one retry, then queue
  }
  return result.data.confidence >= 0.85
    ? commit(result.data)
    : humanReviewQueue.push(result.data); // the uncertain 10%
}

Don't automate the last 10%

The confidence gate is the most important line in that snippet. Full automation was achievable and would have been wrong. Routing low-confidence extractions to an audit-logged human queue keeps the error rate near zero while still eliminating about 90% of the manual work. As a bonus, the review queue is a free labeled dataset: every human correction tells you exactly where your prompts underperform.

Cost is architecture

Frontier models over every document, every day, is real money, and I managed this budget with the same discipline that cut our infra bill 80%. The levers, in order of impact:

1. Don't call the model. Dedupe sources, diff before re-extracting, cache by content hash.
2. Right-size the model. Extraction and classification rarely need the largest tier.
3. Batch and cap. Schedule pipelines, set hard monthly budgets, alert on anomalies (the same bill-as-telemetry habit that once caught a leaked API key costing ¥100k/month).