TransformationActive

AI Delivery Harness: Measured Outcomes at Enterprise Scale

50+ measured business outcomes across eight teams in 60 days, at ~$51 of AI spend each

The Business Challenge

An enterprise technology department had more demand than developers, a portfolio of applications whose original builder had moved on, and a leadership team that wanted AI to help, with evidence rather than enthusiasm. The question was not whether one person could move faster with AI. It was whether AI-assisted delivery could run as a governed, measurable operating practice: every outcome dated, attributed to a team served, and costed.

Discovery & Collaboration

I started where I always do: with how the work actually happens. The daily reality was requests, incidents, analysis for leadership decisions, integrations, and governance work arriving from many teams at once, each needing discovery before anything was built. So the harness was designed around that intake: capture the request, pull the context, decide what is worth doing, do it, and log the outcome with a source note.

Under the hood it's an engineered harness, not a chat window. Everything it touches (work logs, tickets, notes, prior decisions) is indexed for semantic search, so the right context surfaces in seconds by meaning rather than keyword, and a golden-set retrieval evaluation keeps that search honest: one number decides whether quality held after any change. Review verdicts are scoped: they say what was actually checked and what was delegated to a human, so a reviewer's confidence can't quietly inflate. Headless builders run inside fail-closed containment that confines them to their own workspace and denies everything else. Token spend is reserved before a run and reconciled against actuals afterward, so a failed turn still counts. Build-versus-buy and model-routing decisions are made on cost and evidence, and written down.

The Strategic Decision

Measure everything, claim only what the record supports. Each outcome was catalogued from daily work logs with a date, a team served, and a source note; internal tooling work was excluded from the count; no hours-saved estimates were used. AI spend came from the organization's own metered cost report, not from my estimate. That discipline is the point: an AI program a finance leader can audit is one a finance leader will fund.

The AI delivery harness flowyesno12345?RequestContext pulledWorth doing?BuildScoped reviewHuman checkarrivessemantic searchin a containedworkspaceLogged and declinedOutcome loggeddateteam servedsource notemetered cost

The Build

Across one 59-day window, the harness delivered 53 outcomes serving eight teams: identity and security, leadership and portfolio reporting, media operations, business validation, privacy and compliance, broadcast technology, digital and QA, and forms and data. By type: 16 stakeholder requests fulfilled, 14 analyses for leadership decisions, 9 pieces of work shipped, 6 governance or process builds, and the rest automation, incidents, defects, and migration steps.

A few of them, in shape:

  • A migration operating model covering more than 400 application records, with checkpoint-based workflows and auditable changes made available to nine contributors across three functions. My part was the architecture and the AI integration, not personal execution of every migration.
  • A stalled governance integration brought to a working end-to-end test, then a 13-page configuration-and-handoff guide so the receiving team could extend it.
  • A business-application prototype, iterated with stakeholders and carried through production delivery in partnership with engineering, with 13 fixes deployed to production in one release, with business validation following.
  • A rationalization assessment of nearly 60 web properties (keep, archive, or retire), based on business ownership and actual publishing activity rather than assumption, which surfaced live business content that needed a migration path before anything was retired.
  • A same-day fix to a live-broadcast defect during a first-year event.
  • A seven-page AI-governance guidance prototype for three employee audiences, translating policy input into tool guidance, data-handling caveats, and support routes.

The Outcome

53 outcomes. $2,684 of metered AI spend. About $51 per outcome, roughly 12% of one developer role's cost over the same period. When leadership asked about spend, I didn't argue; I sent an executive summary with the numbers, then cut weekly spend 45% and cost per outcome 22% in three weeks by routing routine steps to cheaper models and fixed scripts, with the main model kept for judgment and drafting.

What the record supports is architecture, governed AI workflows, technical delivery, cross-functional coordination, and handoffs other teams could pick up. What it doesn't claim is enterprise-wide adoption or savings nobody measured. That line between what was designed, built, delivered, and adopted is the one I hold, because it's the one that makes the rest believable.

Tech Stack

ClaudePythonGitHubJiraSQLite