Google open-sources Harness²: AI agents that improve without retraining

What this is about

Google Research has released Harness² (also written Harness2) as open source under the Apache 2.0 license. A paper titled "Harness²: Recursive Agent Harnessing for an Open World" and a project page accompany it. The framework is designed to improve AI agents without retraining the underlying language model. Instead of the model weights, it rewrites the environment around the model.

What a "harness" is

A harness is everything that turns a model into an agent: prompts (instructions), tools, memory, skills (reusable instructions) and orchestration. Today, teams mostly tune this by hand: a developer watches the agent fail, edits the prompt, adds a tool and tries again. That works for anticipated tasks, but according to the paper it hits limits in an "open world," where agents run into unexpected tasks.

Harness² automates this fine-tuning work while the model weights stay fixed. There is no fine-tuning and no new training data. Eight components can be modified:

According to the README, the first four make up the model's context, tool hints and scripts form the tool interface, subagents handle delegation and plugins are the middleware.

Two loops

The superscript 2 stands for two levels of recursion.

Loop 1 (task level): Propose–Probe–Compose. An editor proposes general ("invariant") and domain-specific changes. The agent re-runs the task with each variant. A composer compares the runs and merges what helped. The key point: learning comes from behavioral differences between runs, with no reference solution. That's practical, because in the real world nobody hands you an answer key.

Loop 2 (harnessing level). After each task, an "Experience Reflector" stores the adopted harness and evidence-backed lessons in a persistent improvement harness. This harness guides the next round. So it's not only the agent that gets better, but also the process of improving agents.

Budget and cost

Loop 1 runs with a budget k and a mode. In parallel mode, k invariant and k domain-specific candidates are proposed, followed by k independent compositions – a broader search. In sequential mode, k Propose-Probe-Compose steps run one after another, each on the refined harness – deeper refinement. Each task takes 4k agent rollouts and 3k improvement sessions (via run_sweep.py); --k 3 means 12 rollouts and 9 sessions. The LAB benchmark alone has 1,143 tasks, so costs add up fast.

Results

Google tested GLM-5 and Gemini 3.5 Flash in four model-harness combinations on three benchmarks. Even a single step (k = 1) improved both models over their original harness.

These are maximums, not averages, and the LAB figure refers to the partial score, meaning the share of evaluation criteria passed. According to the README, a higher k brings further gains; per-configuration details are in the paper. The numbers come from Google itself, and independent reproductions are still pending.

Why it matters in practice

Improving a model usually requires training data, GPUs and access to the weights. If you only use a model through an API, you typically don't have those. Harness² stores every improvement in readable files: rewritten prompts, fixed scripts, new guardrails. You can inspect them, compare them with the original, and the framework documents its reasoning. The agent ("executor") and the improver are set by model ID in a configuration file. The release pairs GLM-5 or Gemini 3.5 Flash as agents with Vertex Gemini or Claude as the improver.

The free demo

The demo runs with deterministic editors and real Python executions, with no model access. You need Linux (WSL on Windows, a VM or container on macOS), Python 3.12 (3.13 is supported according to pyproject.toml), Astral's package manager uv and git. The steps: install uv, clone the repository from github.com/google-research/harness2, create an environment with uv venv --python 3.12 .venv, install with uv pip install -e '.[render]' (the extra pulls in openpyxl and python-pptx) and start it with harness2-demo --output runs/demo --k 2. Without --mode, both modes run. In the KDnuggets author's test, this took about two seconds.

The demo includes two bookkeeping tasks (ledger-1, ledger-2). The starting harness has a bug: the script summarize.py only adds positive amounts. For a ledger with 100.25, −20.10 and 5.00, it returns 105.25. The script adopted by Harness² removes the filter and correctly outputs 85.15 – without ever knowing the right value. The framework also writes an experience file (EXPERIENCE.md) containing the adopted change (c1), rationale, observation, transferable lesson, evidence and high confidence.

Takeaway

Harness² fits a trend: instead of retraining models, the surroundings are optimized automatically. That matters most to teams that use models through APIs. Caveats remain: the gains are "up to" figures from Google's own tests, the demo with scripted editors proves only the mechanism, not the quality with real models, and higher budgets mean far more model calls and therefore higher costs. The release includes no model weights to download; it is a framework.