Building reliable systems around models.

Classic ML engineering starts with training a model. AI engineering often starts with a powerful existing model, then builds the product scaffolding around it.

The daily work changes. You still care about data, tests, latency, cost, reliability, and monitoring. But now the uncertain component is a general-purpose model that can reason, hallucinate, follow tools, ignore instructions, or behave differently after a model upgrade.

A foundation model is a general model trained on broad data, then adapted by prompting, tools, retrieval, or finetuning.

If you cannot tell whether the AI system got better or worse, you do not have an engineering loop yet.

Evals are test suites for model behavior. They include inputs, expected facts, unacceptable behaviors, graders, human review, and production feedback. The goal is not perfect truth. The goal is to catch regressions before users do.

{
  "input": "How do I reset my warehouse password?",
  "expected_facts": [
    "route through internal identity provider",
    "never expose raw database credentials",
    "link to approved runbook"
  ],
  "checks": [
    "factuality",
    "policy_compliance",
    "citation_quality",
    "tone"
  ]
}

• Use deterministic checks for schema, citations, and policy rules.
• Use model graders for fuzzy qualities like helpfulness or completeness.
• Keep a small golden set that humans trust.
• Add failed production examples back into the eval set.

A prompt is not magic wording. It is the interface contract between your software and the model.

RAG gives the model relevant material at answer time instead of trying to bake all knowledge into model weights.

user question
  -> query rewrite
  -> retrieve candidate chunks
  -> rerank
  -> build grounded prompt
  -> generate answer
  -> cite sources
  -> evaluate answer quality

The hard parts are chunking, metadata, retrieval quality, reranking, citation behavior, freshness, and knowing when no answer should be given. A weak retriever makes even a strong model look careless.

An agent is a loop where a model can decide what to do next, often by calling tools and observing results.

Agentic systems are useful when the path is not fully known in advance: research, debugging, triage, multi-step operations, or workflow automation. They are risky when actions are expensive, irreversible, security-sensitive, or hard to verify.

Finetuning changes model behavior. Retrieval changes model context. Most teams need better context and evals before they need finetuning.

• Use prompting when behavior changes are small and easy to state.
• Use RAG when the model needs private or changing knowledge.
• Use finetuning when you need repeated style, format, or task behavior.
• Use distillation when a smaller model should imitate a larger one.

Dataset engineering is the unglamorous core: collect examples, label them consistently, remove leakage, balance edge cases, and keep train/eval splits honest.

Production AI is constrained by waiting time and money. Every token costs something, and users feel every delay.

A reliable AI product is a normal software system with a probabilistic component inside it.

1. Receive the request and classify intent.
2. Load user, policy, and product context.
3. Retrieve documents or call tools if needed.
4. Generate structured output.
5. Validate output before showing or acting.
6. Log enough detail to debug failures.
7. Feed errors back into evals and product design.