Mission 10 of 12 · Updated November 2026

Make the Agent Reliable: Tracing, Evals & Safety

Agents fail quietly. Trace your Mission 9 agent end to end, find at least two distinct failure scenarios, build a 10 to 15 case evaluation dataset in Langfuse, run experiments, fix what breaks and prove the improvement. Measure latency and cost, and defend against prompt injection and unauthorised tool use.

By the end of this mission: “I can find why an AI agent misbehaves, fix it, and prove the fix with repeatable evaluations.”

The tech stack for this mission

Langfuse logoLangfuseOpenTelemetry logoOpenTelemetryVitest logoVitestOpenAI logoOpenAINode.js logoNode.js

What you learn, checkpoint by checkpoint

  1. Checkpoint A

    Trace full agent runs and find the root cause of failures

    • Observability vs evaluation - and why you need both
    • Add custom spans for tools, retrieval and agent steps with Langfuse and OpenTelemetry
    • Read a trace to find the root cause of wrong agent behaviour

  2. Checkpoint B

    Build an evaluation dataset and run repeatable experiments

    • Build an evaluation dataset of 10 to 15 test cases
    • Run experiments in Langfuse and deterministic checks in Vitest
    • Score pass/fail outcomes and run regression tests after every change

  3. Checkpoint C

    Fix, re-evaluate, measure cost and defend against prompt injection

    • Fix the failures you found and re-run the evals to prove it
    • Measure and document latency, token usage and cost
    • Defend against prompt injection, unauthorised tool use and sensitive-data leaks

AI companion mode · AI as Product

AI runs inside what you build

AI lives inside the product you build. Treat its output as untrusted - validate it, trace it, evaluate it, and design for uncertainty.

What proves you’re done

Before-and-after eval results + failure analysis + latency and cost notes + prompt-injection defence demo

Every mission ships something real you can show. If you can’t explain it, you can’t submit it.

Deliberately left out of this mission

Advanced LLM-as-judge pipelines · Enterprise observability setups · Distributed tracing architecture · Self-hosting Langfuse

Depth is set on purpose: you master what this mission needs, not everything at once. The advanced material lives in optional self-study after Mission 12.

Free · no card needed

Reading is not building.

This mission is part of the paid 12-week programme - but the way we teach is free to try. The free trial puts you inside Sigmo, the learning portal our students use daily, with the 6 Projects in 6 Days precourse: real projects, real code, published to your own GitHub.

Start the free trial

Mission 9

Build an AI Recruitment Agent with Tools & MCP

Mission 11

Build Your AI-Native Capstone

Ready for the real thing?

The AI-Native Software Development Programme - all 12 missions with a live cohort, daily Buildroom, and mentors watching your back.

Apply nowSee the whole curriculum