How to run a 14-day coding-agent harness pilot that survives a Q2 procurement review

Problem this solves

TechTimes ran a piece on May 13 framing harness engineering as the fourth paradigm of AI engineering, after prompt, context, and agent engineering. The framing is not the news. The news is that engineering leaders finally have a name for the thing they have been doing in the dark for six months, and procurement does not have a name for it yet.

That gap is where Q2 pilots are dying. A CTO walks into procurement with a Cursor seat invoice, a Claude Code license line, or a Copilot CLI rollout plan, and procurement asks model questions. Token rates. Vendor stability. Data residency. All fine. None of them are the harness. The model is the horse. The harness is the saddle, the reins, the stirrups, and the fence around the pasture. Procurement has been pricing horses.

The approach

A 14-day pilot has one job: produce four lines a Q2 procurement reviewer can carry back to legal and finance without rewriting. Not a slide deck. Four lines. Here is the shape that consistently lands them on time.

1. Name the harness in the procurement memo, not the model.

   "We are piloting Cursor v3.3 with cloud agents and Bugbot at default effort" beats "we are piloting AI coding tools." The harness is the unit procurement will sign for. Pick the version number. Write it down.

2. Anchor the 14 days to a real failure mode, not a workflow.

   "Faster PRs" is not a failure mode. "Junior engineers spending three days on a flaky test a senior engineer can fix in twenty minutes" is a failure mode. Pick one. The pilot exists to remove it once, repeatably.

3. Wire the governance primitives before day one.

   GitHub Copilot CLI v1.0.46 shipped on May 12 with read-only gh auto-approval and CLI deprecation warnings as first-class features. Cursor's May 13 cloud environments shipped with version history, audit logging, and environment-scoped secrets. Turn them on before any engineer logs in. Pilots that retrofit governance on day twelve fail procurement on day fifteen.

4. Pick one metric procurement cannot dismiss.

   Acceptance rate is dismissable. Time-to-merge is dismissable. A metric that maps to a line item on the engineering P&L is not. Cost per merged PR with token spend attached. Verified-output rate at seven days. Hours of senior-engineer review per agent-authored PR. One number. Defended in plain English.

5. Rehearse the kill switch in week two.

   On day nine, simulate a credential leak or a sensitive-repo commit and walk the runbook end to end. Who pages whom. How fast credentials rotate. Whether the audit trail survives the rollback. Procurement will ask. The answer should already be on paper.

6. Write the four-line procurement answer on day eleven.

   One line on what the harness is and which version. One line on the failure mode it removed and the evidence. One line on the governance posture and the kill-switch rehearsal date. One line on the metric and its baseline. Leave three days to revise.

Why most teams get this wrong

The most common pilot I see in the wild is a 90-day evaluation that bakes in three things procurement quietly hates.

First, it picks the harness by model. Someone reads a SWE-bench leaderboard, picks the top score, and tells procurement “we are piloting GPT-5.5.” Procurement responds with a 40-question security questionnaire about OpenAI, which is the wrong vendor. The pilot stalls for six weeks while a CISO and a vendor risk analyst argue about a model that the harness might swap out next month.

Second, it measures the easy thing. Acceptance rates. Suggestions per developer. Lines generated. The Harness study reported by TechTimes is the cleanest data on why this fails: 89 percent of leaders say their current metrics accurately reflect AI’s impact, and in the same survey 94 percent admit those metrics miss tech debt, validation time, and burnout. Both numbers from the same people. The metric procurement will defend is the one the same leaders just told themselves does not exist.

Third, it treats the harness as static. Cursor flipped Bugbot from $40 per seat to usage billing on May 11. Copilot CLI shipped a deprecation-warning system on May 12 because base models are about to flip on May 17. The harness moves every Tuesday. A pilot that finishes its rubric on day thirty is auditing a tool that no longer exists.

The fix is small. Pick a 14-day clock. Pick one failure mode. Name the harness version. The discipline writes the procurement answer in advance, instead of catching up to it.

The numbers

That sentence is the shape of the procurement question Q2 actually asks. Not what the seat costs. What the unit of work costs. Bugbot at default effort finds 0.7 bugs per run; at high effort it finds 0.95 bugs per run, with resolution rate holding at 79 percent. Those are auditable numbers. Procurement signs auditable numbers.

Cursor’s May 13 multi-repo cloud-agent environments cut Dockerfile-based builds by 70 percent through layer caching and build secrets. That kind of number lands in a pilot rubric because it touches the engineer’s wall-clock day. The pilot’s job is to surface three numbers exactly like those by day eleven.

Ship it

Mitchell Hashimoto’s harness principle, the one OpenAI and Anthropic both cite when they talk about this discipline, is one sentence: any time an agent makes a mistake, engineer a solution so it never makes that mistake again. A 14-day pilot is the smallest container for that principle that procurement will read.

Red Hat’s Matt Hicks said at Summit this week, on May 13: “One year ago, you might have argued the risk was okay. AI has made that not okay to skip now.” He was talking about patching, but the sentence applies to harness pilots too. The teams that finish Q2 with a procurement-ready harness are the ones who stopped treating the pilot as a science project and started treating it as four lines on a memo.

Pick the harness. Pick the failure mode. Wire the governance primitives. Write the four lines. Two weeks.