Harness Engineering for AI-Native Software: The Controls Matter More Than the Demo

An engineering leader's view of how to make agent-built systems testable, governable and affordable.

An engineering leader’s view of how to make agent-built systems testable, governable and affordable

There is a moment in almost every AI demonstration when someone asks, “Can we put this into production?” The answer is rarely about whether the model can produce an impressive response. It is about what happens on a bad day: the market feed is stale, an internal service times out, a policy changes, or two agents confidently agree on the wrong answer.

That is the problem I want harness engineering to solve.

I use harness here in a broader sense than a test runner. In traditional engineering, a harness provides fixtures, mocks, data, environments and execution controls that allow teams to test software consistently. With AI agents participating in design, code generation, testing and operations, that concept needs to extend into the agent runtime itself. It must define the boundaries of autonomy, provide independently checked evidence, and make failures reproducible.

A well-designed harness does not make an agent infallible. It makes unsafe behavior harder, failures more visible, and recovery more predictable.

A financial advisory example exposes the problem

Consider a hypothetical portfolio advisory application assembled largely through AI-assisted and agent-driven development. A customer asks: “I’m overexposed to technology stocks. What could a more balanced allocation look like?”

The happy-path demo is easy to imagine. One agent retrieves holdings, another interprets market information, and another writes a polished explanation. But a polished answer can still be unsuitable. The wrong account could have been retrieved. A market-data timestamp might have been ignored. The allocation could breach a firm’s concentration policy. Or the explanation could state a level of certainty that the evidence does not support.

None of these problems is reliably solved by adding “be careful” to a system prompt. Nor should a second LLM’s approval be the sole control: two models can share assumptions and repeat one another’s mistakes.

For this example, I would separate reasoning from authority. An agent may propose an allocation, but it should not decide whether a customer is eligible for a product, whether a trade can execute, or whether a policy exception is acceptable. Those decisions belong to independent services and, where required, accountable people.

The architecture I would actually build

I would start with two layers, not a grand autonomous platform.

1. Bound the agent before it runs

Each agent receives a specific responsibility, a defined set of tools, and an execution budget. A portfolio-analysis agent gets read-only access to authorized holdings and approved market feeds. It does not receive trade execution privileges “just in case.” Tool schemas restrict arguments; authorization is checked at the service boundary, not inferred from the prompt.

I would also limit the number of handoffs, retries, elapsed execution time, and model spend per request. These controls are easy to overlook during prototyping, especially when one agent can ask another agent to redo its work. Without limits, a modest customer interaction can turn into a costly loop.

Prompts are useful for expressing intent. They are not a security boundary. A production harness has to enforce permissions and budgets in code or managed infrastructure.

2. Validate outside the model

Before the agent sees input, check account entitlement, schema, source provenance, and freshness. After it proposes a result, run deterministic checks on calculations, mandatory disclosures, and applicable business constraints.

Take a simple illustration: a proposed allocation totals 103%, or a sector allocation exceeds a policy threshold configured for that customer and product. A validator should reject it regardless of how compelling the narrative sounds. The agent may get a bounded chance to correct a formatting or arithmetic issue, but it should not be allowed to rewrite the policy that rejected its answer.

Not everything can be expressed as an if statement. Investment suitability can require context, professional judgment and regulatory interpretation. The harness should route those cases for qualified human review instead of inventing a false impression of automated certainty.

Six controls I would prioritize before scaling

First: data trust. Check whether every retrieved fact belongs to the right account, comes from an approved source, and is sufficiently current for its purpose. A market price and a customer’s long-term risk preference have different freshness requirements. Treating them identically is poor design.

Second: clear output contracts. Have agents return structured fields—proposed weights, assumptions, source timestamps, risk labels, and citations to policy identifiers—rather than only prose. JSON schema compliance does not prove correctness, but it lets ordinary software inspect what the model claims.

Third: independent calculations. Use deterministic services for percentages, exposure aggregation, eligibility and threshold checks. Asking a language model to recompute basic arithmetic on every turn adds cost without adding assurance.

Fourth: constrained execution. Apply per-task token and cost budgets, timeouts, restricted tools and bounded retries. Track the full cost of a successfully validated request, not merely the price of one model call.

Fifth: adversarial evaluation. Test prompt injection, conflicting instructions in retrieved documents, stale data, missing holdings, partial tool failures and inconsistent responses from collaborating agents. These tests need expected outcomes and clear fail criteria.

Sixth: decision evidence. Preserve the request ID, model and prompt versions, tool calls, source timestamps, validation results and human approvals in an appropriately protected audit trail. Do not log raw sensitive customer information simply because logging is convenient.

Where the harness belongs in the SDLC

I do not see this as a late-stage QA activity. The same principles should follow an AI-generated feature from idea to production.

During requirements, establish the business invariants: what must never happen, what requires approval, and what evidence will prove compliance. During code generation, apply secure coding checks, dependency analysis and tests that are independent of the code-writing agent. During pull requests, use change impact to select relevant test suites, then let CI enforce mandatory gates. During integration testing, simulate adversarial inputs and failure of downstream services. In production, monitor both system health and agent behavior.

A change to portfolio allocation logic, for example, should trigger tests for boundary percentages, risk categories, rounding, authorization and downstream rendering. An agent can recommend extra coverage after examining the code diff, but the final test gates should not depend on that agent remembering to recommend them.

This distinction matters. Agent-generated tests can accelerate engineering; they are not automatically independent evidence. A shared misunderstanding can contaminate the implementation and the tests that appear to validate it.

Using Bedrock Guardrails without overestimating them

For an AWS implementation, Amazon Bedrock Guardrails can support baseline model-interaction protections, depending on the chosen features and configuration. That is a sensible starting point for content safety and some sensitive-information or grounding-related checks.

It is not a substitute for the business rules of a financial institution. Guardrails do not inherently know a customer’s risk agreement, a firm’s changing concentration policy, or who is authorized to approve an exception. I would implement those in dedicated policy and entitlement services, tested and versioned like other critical software.

The intended path is straightforward: validate the request; authenticate and authorize access; retrieve trusted data; run bounded agent reasoning; validate the resulting proposal; apply business-policy checks; and send the outcome either to the user or an appropriate reviewer. Telemetry should cover every handoff, not just the final answer.

The feedback loop should be governed, not magical

The phrase “self-improving harness” is attractive, but it can conceal a dangerous assumption: that an agent should change its own controls when something goes wrong.

Suppose a production review discovers a recommendation based on a delayed price feed. I would want the team to reproduce the failure, add a regression scenario, tighten the freshness rule, validate the change against known cases, and release it through normal approval processes. AI can help identify likely causes and propose new tests. It should not silently relax a policy because the original answer failed validation.

This is a practical learning loop: observe, reproduce, improve, verify and deploy. It is slower than an autonomous self-editing story, but substantially easier to audit and trust.

What I would measure

A dashboard showing only “number of agent runs” or “tokens saved” will tell very little about engineering quality. I would use a smaller set of measures that explain both value and risk:

Dimension Practical measure
Reliability Validated completion rate, timeouts and failed handoffs
Safety and quality Critical rule violations, unsupported claims and adversarial-test outcomes
Economics Cost and latency per validated business outcome
Delivery Change-validation time, escaped defects and coverage of enforced gates
Governance Trace completeness, policy version visibility and review/escalation rates

These numbers need interpretation. More rejections may mean the agent has become worse—or that the validator has become better. Lower cost may hide a drop in quality. Reviewing the measures together is part of the leadership responsibility.

The leadership question

The most important design decision is not which model or orchestration framework to choose. It is deciding where autonomy ends.

Some work is well suited to agents: interpreting ambiguous requirements, proposing test cases, finding likely change impacts and summarizing evidence. Other work demands stronger boundaries: enforcing customer entitlements, calculating exposure, applying financial limits, authorizing trades and approving exceptions.

I would invest in a shared engineering capability—tool permissions, evaluation datasets, output contracts, telemetry and reusable policy checks—rather than letting each team assemble its own fragile set of prompts. That is how harness engineering becomes a platform discipline rather than another QA utility.

The goal is not to make every AI decision autonomous. The goal is to make each permitted decision explainable, testable and safe enough for its business context.

AI agents may accelerate the act of building software. Harness engineering is what helps us take responsibility for the software they build—and the decisions it makes.

Note: The portfolio advisory system described here is a hypothetical architecture used to illustrate engineering trade-offs, not a claim of a deployed financial advisory product. This article reflects an engineering perspective rather than investment advice.

  • Agentic AI
  • AI Quality
  • Engineering Leadership

Harness Engineering - Beyond the AI Model

AI can identify a failure and suggest a fix. Should we trust it to fix the problem on its own? Why AI-driven remediation needs production-grade discipline.

· 1 min read