# Harness Engineering: Context Overview

## What it is

Harness engineering is the discipline of designing, optimizing, and managing the structured software infrastructure surrounding a foundation model. The harness includes everything except the model weights. The term was coined in February 2026 and formalized by Lee et al. at Stanford in March 2026.

## Why it matters

The central empirical result: changing only the harness around a fixed LLM can produce a 6x performance gap on the same benchmark (Lee et al., Meta-Harness, arXiv:2603.28052). A harness optimized for one model transferred to five held-out models, including models released after the harness was designed. Harness investment is durable across model generations.

## Historical progression

Three nested phases:

- **Prompt engineering (2023-2024).** Control surface is the individual instruction. Single-turn optimization.
- **Context engineering (2024-2025).** Control surface is the full information environment presented at each step.
- **Harness engineering (2026).** Control surface is the complete system lifecycle: state persistence across sessions, tool orchestration, verification loops, observability, modularity. The integration layer becomes a first-class optimization target.

## Relationship to prior LLM-for-science work

The individual components (RAG, tool calling, memory, planning) are not new. ChemCrow (2023), Coscientist (2023), and the AI Scientist (2024) implemented them. Three things are substantively novel:

1. **The integration layer as an optimization target.** Prior systems treat the scaffold as fixed. Meta-Harness demonstrated it can be searched over and optimized, producing the 6x gap.
2. **Lifecycle management across sessions and campaigns.** RAG+Tools+LLM operates within a single session. The harness addresses cross-session state persistence, context compaction, session handoff, and cumulative learning.
3. **Separation of concerns between reasoning and optimization.** In prior systems, the LLM is both cognitive and execution layer. The scientific harness separates these: the LLM reasons about what to test, a mathematically principled engine (e.g., BAX) handles how to test it. The harness is the interface contract between them.

The honest framing: harness engineering is not a new set of components. It is a new discipline for treating those components as a system and optimizing them across the lifecycle of a scientific campaign.

## The scientific harness architecture

Developed through iterative stress-testing, the architecture has five components. Two original candidates (context engineering and modularity) failed the stress test — they are emergent properties, not components you build.

1. **Tool Integration** — MCP (Model Context Protocol) connections: callable interfaces with documentation and permissions. Holds the interface, not the tool. Swap the instrument without changing the harness. MCP turns tool descriptions into callable connections.
2. **Memory & State** — three tiers: short-term (context window, promote or lose it), medium-term (Obsidian, Claude memory, project docs; survives sessions), long-term (graph DB, experimental repositories; outlives people and projects). *Field-refined:* separate the **machinery** (skills, tools, guardrails, master instructions) from the **knowledge** (all work products and notes), and make the medium-term store the single container for the knowledge so every project artifact lives in it by construction. This removes the copy-and-drift failure mode; there is one memory of record, not a primary copy and a stale mirror. The real capture flow is often analog-first (physical notebook -> LLM transcription -> markdown note), which the harness should encode as it is, not as an idealized digital-native flow.
3. **Planning & Decomposition** — hypothesis object specifications, static blueprint, acceptance criteria, trigger specifications. The scientific PRD. BAX is a tool, not a planning function. Campaign orchestration is a skill, not a planning function.
4. **Verification & Guardrails** — persistent adversarial policy independent of executing components. Safety interlocks, audit logging, escalation to human oversight. The component that says "no" when everything else says "go." *Field-refined:* guardrails are **scoped** — a writing-voice standard is a hard floor for written products but must not be imposed on campaign orchestration or planning-doc structure, which answer to scientific and process standards instead. Institutional policy is **ingested, not authored from memory**: a source -> staged vault note -> distilled hard-stop -> promoted guardrail-with-citation pipeline gives every institutional rule provenance, which is itself a guardrail (no conclusions from unverifiable sources). For experimental science, the sharpest guardrail is a **computed-standard test** for a result: a discrimination claim must be data-carried, not prior-carried, and not an artifact of a boundary condition.
5. **Skills / Encoded Methodology** — reusable domain methodology. Campaign orchestration, analysis protocols, pre-delivery scans. What a new postdoc takes six months to learn, encoded as a file.

**Modularity & Extensibility** is an emergent property: well-designed interfaces mean you swap instruments, skills, and hypothesis classes without touching the harness architecture. *Field-refined:* the property shows up concretely as (1) **tool interface contracts** — a literature source returns a fixed record shape, so OSTI or Scopus slot in behind the same interface a skill already calls, without editing the skill; and (2) **skill-to-skill contracts** — one skill emits a structured object (e.g., a literature skill's dissenting perspective) that another consumes (an adversarial-review persona), so skills compose without either knowing the other's internals. Distinguish a single containable *skill* from a multi-component *business system*; the latter may contain skills but is not one.

## The autonomy progression

The five components are the same at all three levels. What changes is who initiates and who reviews:

- **Human-in-the-loop:** researcher initiates, reviews, decides. The harness provides context and methodology.
- **Human-on-the-loop:** a trigger initiates (Claude Code headless mode via `claude -p` + cron job). Tools called via MCP connections; `--allowedTools` sets the permission boundary. Researcher reviews high-stakes decisions. Key step: add a trigger. *Field-refined:* triggers live in the **automation layer** — hooks in settings.json that the harness executes on lifecycle events (PreCompact, SessionEnd, PostToolUse, and the like). This layer is distinct from the five components: policy (guardrails) is obeyed by the model, methodology (skills) is followed by the model, but only hooks fire on their own. Automating cumulative memory (capture state before compaction) is the first and most natural use, and it is what moves a harness from human-in-the-loop toward human-on-the-loop.
- **Human-over-the-loop:** the harness runs autonomously. Researcher sets specs, reviews outcomes. Key step: trust the verification layer. Earned by observed performance, not declared by policy.

The progression cannot be skipped. The harness is the prerequisite for safe autonomy.

## The bootstrap

The minimum viable harness requires Claude Code, one skill file (grill-me), and one afternoon. The grill-me skill is the bootstrap artifact: one skill in, five components out. A self-executing setup guide conducts a structured interview and collaboratively builds all five components from the researcher's answers. *Field-refined:* in practice the researcher brings a small starter set beyond grill-me (a literature skill, and reference files capturing who they are, their writing voice, and their institutional assets), and the setup connects at least one real MCP tool so Tool Integration is demonstrated live, not merely specced. The strongest demo does not pre-author the first project plan; it *generates* it by running the front of the researcher's own idea-to-result pipeline (grill-me -> literature research) so the plan is a product of the harness, exercising Skills, Tool Integration, and Memory together.

## The scientific harness gap

As of July 2026, the entire published harness engineering literature focuses on software development. No published work applies formal harness engineering to physical experimental science. The Genesis Mission proposal architecture (LLM reasoning + BAX active learning + modular experimental platform) is the first scientific harness. This gap is both the opportunity and the positioning.

## Connection to SLAC institutional positioning

The harness is where institutional value accrues. SLAC does not build LLMs and should not try. SLAC builds the scientific harness: the structured interface between foundation models and experimental science. This requires co-located domain science, active-learning methodology, and experimental infrastructure. A national laboratory is the natural home for this discipline.

## Key sources

- Lee et al. (Stanford, 2026). Meta-Harness. [The 6x result, cross-model transfer.]
- Zaharia et al. (Berkeley, 2024). Compound AI Systems. [Conceptual foundation.]
- Young (Anthropic, 2025). Effective Harnesses for Long-Running Agents. [Practitioner patterns.]
- Picrew et al. (TMLR, 2026). Agent Harness Engineering: A Survey. [110+ papers.]
- Weng (Thinking Machines Lab, July 2026). Harness Engineering for Self-Improvement. [RSI framing.]
- Yang et al. (2026). ARIS. [Only paper applying harness engineering to scientific research, limited to computational ML.]