Georgia State University — J. Mack Robinson College of Business PATH — Pathways for AI Training & Hiring CIS 4394 Agentic AI  ·  Fall 2026  ·  Dr. Xinyu Fu
CIS 4394 · Week 8 ⌂ Course 01Why Demos Lie 02pass@k, pass^k & Benchmarks 03Practice & Case
Week 8 · Module 3 · Evaluation

It worked when I demoed it.

One successful run tells you almost nothing about a system that will run a thousand times. This week you turn “it seemed to work” into a number a manager can act on — and learn why the number that sells a demo is not the number that ships a product. Don't trust the demo; test the behavior.

📝 This week's logistics: Quiz 3 is this week — in class, closed-book, 5 multiple-choice. Scope: the Weeks 6–8 readings. Sample questions are on the Week 7 page. Group Assignment 3 releases this week, and capstone Milestone I releases (due Week 9). Agent Radar continues — the presenter slot is open.
The big question
Discussion questionYour agent completed the task perfectly when you demoed it to your manager. What is the chance it completes the next eight customer runs perfectly?
From one run you cannot know — that's the whole point. One success is a sample of size 1 from a probabilistic system, and you chose the sample. But you can reason about it: if each run independently succeeds 90% of the time — a number most teams would be thrilled with — then all eight succeeding is 0.98 = 0.4305, about 43%. The customer who runs your agent eight times is more likely than not to see it fail at least once. That gap between “90% reliable” and “43% of customers get a clean week” is what this whole week is about, and page 02 gives it a name: pass^k.
The hook

The demo was run #1

What your manager saw
✓

One run. Your laptop, your test input, your network, your eyes on the screen. It worked. You said “it works.” Everyone believed you — including you.

What the next five runs looked like
✓
✓
✗
✓
✗

Same agent, same prompt, same task — five more times. Nothing broke, nothing was “deployed wrong.” The system is simply non-deterministic, and you had been quoting a sample of size one. Run it yourself on page 01.

The course line, one more time: in Week 2 you evaluated an agent instead of admiring it; in Week 3 you proved a prompt change helped with a reliability check instead of a vibe. This week that instinct becomes an engineering discipline with metrics, harnesses, and benchmarks — the thing that stands between a prototype and a deployment.
Interactive · click each gap

Four reasons the demo lied to you

A demo is not a small production run. It differs from production in four specific, nameable ways — and each one hides a class of failure that an average-accuracy number can never show you.

The umbrella term: distribution shift — the inputs your agent meets in production are drawn from a different distribution than the ones you tested on. Evaluation is how you find out how different, before your customers do.
This week's pages

Work through in order

01 Why demos lieWhy evaluating an agent is harder than evaluating a model · run the same task five times · compounding error across steps · what to log per step · outcome vs trajectory. → 02 pass@k, pass^k & benchmarksThe two metrics people confuse · a calculator that shows one curve rising and the other collapsing · SWE-bench, τ-bench, WebArena · LLM-as-judge and its biases. → 03 Practice & caseWrite an eval set for your Week 2 agent · measure your own pass^5 · the 92%-demo support case · interview check · readings. →
Why managers care

Reliability is the gap between a tool people trust and one they turn off

Cost

Every failed run costs twice: the wasted tokens, and the human who now has to redo the work and check what the agent did. A 90%-reliable agent doesn't save 90% of the labor — it saves whatever is left after someone verifies all of it.

Trust

Adoption is not decided by your average. It's decided by each person's own experience of the tool. Users who hit two failures in their first week stop using it — and they tell their team, which is why quiet failure rates turn into abandoned deployments.

Auditability

“How often does it fail, and what does failure cost?” is a question a manager is entitled to ask before signing. Without an eval harness and stored traces, the honest answer is “we don't know” — which is a no.

The deployment gate: evaluation isn't the homework you do after building. It's the control that decides whether the thing ships, at what level of autonomy, and with which human gate in the loop. If you can't measure it, you can't deploy it.
← CourseAll weeks