Authors: Siwei Wu$^{2,3}$, Jincheng Ren$^{ 4 ,6}$, Yizhi Li$^{\dagger*3}$, Haau-Sing Li$^{3}$, Chengran Yang$^{3}$, Weicheng Gu$^{3}$, Yuxuan Zhang$^{3}$, Jian Yang$^{\diamond1}$, Riza Batista-Navarro $^{2}$, Chuanyi Zhang$^{6}$, Ming Zhou$^{5}$, Bryan Dai $^{3}$, Chenghua Lin$^{\diamond2}$
<aside> 🏗️
TL;DR
Modern agents are more than their foundation models: their capabilities also depend on the harnesses that control interaction, observations, context, tools, and task completion. While recent work [1, 7, 10, 11, 22] shows that agents can improve such harnesses from execution experience, existing approaches often exhibit limited generalization: they typically evolve directly on benchmark instances and optimize from individual trajectory outcomes, making it difficult to distinguish reusable harness improvements from benchmark-specific solution patterns.
We study Harness RSI as a general self-improvement problem: can an agent improve its harness from independent experience and transfer these improvements to unseen tasks, domains, and foundation models? To this end, we construct a 2,000-instance evolution pool fully disjoint from downstream benchmarks and introduce ModularRSI, which decomposes the harness into functional modules, uses contrastive trajectories to diagnose recurring mechanism-level weaknesses, independently evolves each module through scoped code modifications, and integrates only improvements that pass validation gates.
Under this benchmark-independent evaluation protocol, ModularRSI improves Terminal-Bench 2.0 accuracy from 47.57 to 52.43, while consistently transferring across unseen task domains and different foundation models. We release the evolution dataset, code, and evaluation toolkit as an open playground for studying generalizable Harness RSI.
</aside>
GitHub: <https://github.com/IQuestLab/ModularRSI>
Dataset: <https://huggingface.co/datasets/IQuestLab/ModularRSI_2000_Instances>
Paper: TBD
A central challenge in Harness RSI is what should be learned from execution experience. Existing approaches often update the harness directly from individual trajectories, which can retain successful prompting patterns, tool-use strategies, or task-specific solution procedures. Such updates may improve similar tasks while capturing benchmark-specific recipes rather than reusable harness deficiencies.
ModularRSI instead uses contrastive trajectories to identify behavioral differences that consistently separate successful and failed executions. These recurring patterns are more likely to reveal general weaknesses in the mechanisms governing agent behavior, making the resulting improvements more transferable beyond the experience that produced them.
The remaining challenge is localization: even when a recurring deficiency is identified, it is difficult to determine which part of a monolithic harness should be modified. ModularRSI therefore decomposes the harness into five functional modules—Agent Loop, Observation Management, Tool Use, Context Management, and Task Completion Detection—providing clear diagnostic boundaries for mapping trajectory-level evidence to targeted module-level evolution.
Figure 4. Overview of ModularRSI. Functional modules evolve independently through trajectory collection, diagnosis, constrained modification, and validation before being integrated into the final harness.
Overview of ModularRSI. Functional modules evolve independently through trajectory collection, contrastive diagnosis, scoped modification, and validation before being integrated into the final harness.
Beyond facilitating evolution, modularity also provides a practical engineering principle for harness design. Explicit functional modules allow the system to adaptively select and compose task-relevant capabilities during inference, avoiding unnecessary functionality while providing the same clear boundaries for diagnosis during RSI. In this sense, modularization improves both where the harness should evolve and what functionality should participate in execution.
We next examine whether the improvements learned under this design generalize beyond the tasks, domains, and foundation model used during evolution.
The central question of Harness RSI is not whether performance improves on the experience used for evolution, but whether those improvements survive beyond the experience that produced them. Following the strict evolution–evaluation separation introduced earlier, we evaluate ModularRSI along three increasingly stronger dimensions of transfer: unseen tasks, unseen domains, and unseen foundation models.