CIS 4394 Agentic AI · Fall 2026 · Dr. Xinyu Fu
CIS 4394 Agentic AI · Fall 2026 · Dr. Xinyu Fu
A model produces one output you can grade. An agent produces a path: a dozen decisions, each one able to poison everything downstream, none of them guaranteed to repeat. Before you can pick a metric, you have to understand what makes the measurement hard.
Ordinary software is a function: the same input gives the same output, so one passing test is proof. An agent samples from a distribution — same input, different path, sometimes a different answer. A single passing run proves the system can succeed, never that it will.
One wrong step doesn't cost you one step. It poisons every step after it: the agent reasons confidently on top of a bad observation, and the trace still looks coherent. Reliability multiplies down the chain — see the calculator below.
“Summarize this ticket,” “draft a reply,” “research three vendors” have no unique correct string to diff against. Exact-match tests, the cheapest and most trustworthy kind, simply don't apply to most of what agents produce — so you need graders that are themselves fallible.
The agent changes the world as it goes: it writes files, sends messages, books things. You can't rerun step 7 in isolation, and a “correct” final answer reached by an unacceptable route (six needless paid API calls, a policy violation, a lucky guess) is not actually a pass.
Below is a fixed, scripted episode of the Week 2 Job Search Agent on one identical input — the same posting, the same prompt, the same tools. Press Run the task five times. Everyone in the class sees the same five outcomes, because the lesson shouldn't depend on your luck.
Suppose every single step of your agent is independently reliable at some rate. Drag the number of steps and watch what happens to the probability that the whole task comes out clean. Two agents: one with 95% steps, one with 99% steps.
Whole-task success = 0.9510
Whole-task success = 0.9910
Steps aren't all equally reliable. Say: read the ticket 0.99 → look up the order 0.97 → check the refund policy 0.95 → compute the amount 0.98 → issue the refund 0.96.
Every step looks fine in isolation. Nobody on the team would flag a 0.95. The product still fails roughly one refund in seven. (Illustrative per-step numbers — an original teaching example, not measured data.)
Reliability is engineered per step and experienced per task. Two design consequences follow, and both are decisions you own rather than model choices:
Shorten the chain. Fewer steps means fewer multiplications. A deterministic tool or a plain workflow branch that replaces three agent steps buys more reliability than a bigger model usually does.
Verify inside the chain. A check after the fragile step (did the lookup return anything? does the amount match the policy?) stops the poison from propagating. That's the reflection pattern from Week 4 in its most boring, most valuable form.
You cannot evaluate what you did not record. This is the evidence standard from Week 2, and it is the minimum unit of an agent trace: six fields, every step, every run. Click each one for what it buys you.
Why agent debugging needs traces and spans rather than classic metrics and log lines — and how an eval set gets wired to them. Watch this before page 02.
1. What a span contains that an ordinary log line does not, and why nesting matters when one bad observation sits eight steps upstream of the wrong answer.
2. Where the dataset of test cases lives relative to the running agent — the harness anatomy you'll build on page 03: dataset → agent under test → evaluator → report.
3. How an eval turns into a regression test: the same dataset, rerun on every prompt or code change, so “I improved it” becomes a claim with evidence instead of a memory of one good run.
| Outcome (final-response) eval | Trajectory (path) eval | |
|---|---|---|
| Question asked | Is the final answer right? | Did it get there acceptably? |
| Typical checks | Exact match, numeric match, required fields present, rubric score on the reply | Which tools were called, in what order, with what arguments; steps used; whether a forbidden action was taken; whether the policy was followed |
| Catches | Plainly wrong output | Right answer for the wrong reason; wasted or duplicated calls; policy and safety violations; the lucky guess |
| Misses | Everything about how — including a run that got lucky, and one that quietly cost 10× more | A beautiful, policy-perfect path to a wrong answer |
| Cost of running | Cheap — you already have the answer | Needs the full trace stored and a way to assert over it |
A correct answer that arrives after the customer left is a failure with good manners. Latency belongs in the eval report, not in a separate performance ticket.
Tokens and tool calls per run. An agent that is 2 points more accurate and 5× more expensive is a business decision, not an obvious upgrade — and you can only have that conversation with numbers.
Four dimensions — final-response quality, trajectory and tool use, latency, cost (Agents Companion, Google/Kaggle). Right, efficient, and affordable. Reporting only the first is how teams ship something nobody can pay for.