Building an Advanced Agentic Harness

Building an Advanced Agentic Harness

• ID: 212cb586
• Original URL: https://data4sci.com/blog/building-an-advanced-agentic-harness
• Author(s): Data4Sci
• Date: 2026-07-15
• Category: agents
• Source type: article
• Tags: agent-harness, typed-tools, planning, memory, evaluation
• Quality score: 4/5
• Fetched at: 2026-08-06T15:43:42+00:00
• Obsidian evidence: OpenClaw定时任务/ClawFeed24小时高价值一览/2026-08-06-ClawFeed24小时高价值一览.md

中文导读

Data4Sci 把生产级 agent harness 拆成可组合原语：Pydantic typed tools、Planner 生成依赖 DAG、Worker 按 ready set 并发执行、分层 memory、确定性校验优先再上 LLM judge，以及多维 budget / pressure 降级。它还把错误分成 transient、tool misuse、missing info、fatal，强调缺信息才 replan，不应盲目重试。文章适合从框架黑盒退回到可审计的 agent 运行时设计。

为什么值得关注

直接服务 AAIF 的 agent 工程实践线：计划、执行、验证、预算和追踪都可落地。

English Summary

The article develops an advanced agentic harness with typed tools, plan DAGs, parallel workers, layered memory, deterministic validation plus LLM judging, budget pressure controls, tracing, and explicit error classification for replanning.

原文摘要 / Source Excerpt

Building an Advanced Agentic Harness

发布时间: 2026-07-15

原文链接: https://data4sci.com/blog/building-an-advanced-agentic-harness

That Basic Harness loop is correct, but _naive_. A lone pilot in a well-built jet might win a dogfight, but nobody runs an air campaign that way. Real operations add mission planners who decide what sorties to fly before anyone takes off, squadrons that fly independent sorties in parallel, fuel budgets and bingo calls that force a return to base before the tanks run dry, flight recorders that make every mission reconstructible after the fact, and after-action reviews that decide whether the mission actually succeeded. None of these replace the pilot. They wrap the pilot in structure so that the whole system stays fast, safe, debuggable, and measurable.

Claude Code, Devin, Cursor, Hermes, and other production agents do exactly the same thing to the basic loop. In this post we upgrade every piece of our basic harness toward that production shape, without hiding any of the mechanics behind a framework. The guiding question for the whole exercise is a simple one:

How do you turn a single LLM call into a reliable system that can plan, act, recover, and prove it did the right thing?

Our answer is composition. We build small, testable primitives: typed tools, a plan DAG, tiered memory, a verification hierarchy, budgets, and a tracer, and wire them together with a deliberately thin orchestrator. Each primitive exists because naive agents fail in a specific, predictable way. LLMs invent invalid tool arguments, so we add typed tools with Pydantic validation. Everything runs sequentially, so we add a dependency graph and parallel execution. The context window fills with junk, so we add multi-tier memory under a retrieval budget. Bad outputs propagate silently, so we add a verification hierarchy. One prompt tries to do everything, so we split it into _Planner_ , _Worker_ , and _Critic_ roles. Costs run away, so we add multi-dimensional budgeting with graceful degradation.

Proving the harness _usually_ works, with an eval suite, retrieval benchmarks, and specialized worker pools will get a full fledge treatment in a future post.

The running example

Throughout the post we build a city comparison agent: given a list of cities, it produces a report comparing them on population, timezone, and a short narrative summary of each. The task looks almost insultingly simple, but it was chosen carefully. Each city-attribute lookup is independent of every other one, which means a three-city request naturally decomposes into nine tool calls that could all run at the same time. The final report, on the other hand, depends on all of the lookups finishing first, so we’re. well beyond a flat list of steps. We can programmatically check that every requested city actually appears in the report to verify the results. And the tools have wildly different costs: population and timezone lookups are in-memory dictionary reads, while the per-city summaries and the final aggregation each call the LLM, which gives us realistic budget pressure to manage.

For the sake of reproducibility, lookup tools read from a small mocked dictionary, _CITY\_FACTS_ , so the notebook is fully reproducible without network access. The LLM-backed pieces can run against either a real Anthropic model or a deterministic mock, which brings us to the first primitive.

A pluggable brain

Every component we are about to build eventually calls an LLM: the planner, the summarizer, the aggregator, the critic. If that call is hard-wired to one SDK, the entire harness becomes untestable and vendor-locked.

So before anything else, we define a base class that provides an abstraction over the details of the various LLM calling APIs

class LLMProvider:
    """Shared interface. Subclass to plug in a different backend."""

    def complete(self, system: str, user: str, role: str = “default”) -> str:
        raise NotImplementedError

    async def acomplete(self, system: str, user: str, role: str = “default”) -> str:
        # Wrap sync call in a thread; works for any SDK.
        return await asyncio.to_thread(self.complete, system, user, role)

We also implement a _MockProvider_ for testing and debugging purposes that returns deterministic, role-aware responses: a canonical plan w

...[excerpt truncated, fetched body length=26863 chars]...