# Harness Setup Guide ## Revision note (v3) Pre-demo validation (2026-08-05) surfaced operational lessons now folded into Phase 3: - **Obsidian installs by direct download on Windows, not winget.** The `winget` path failed in this environment; the direct download from obsidian.md succeeded. Treat direct download as the primary Windows path, not a fallback. The app is only a viewer over the markdown, so a failed install never blocks writing to the vault. - **Serialize live source (MCP) queries.** The arXiv MCP backend rate-limits by IP: firing several searches concurrently returns a "rate limiting this IP, wait 60 seconds" error that fails the whole batch, while the same queries run one at a time each return promptly. Encode this as source-call discipline in any skill that queries a live source, and in the orchestrator's parallel steps. - **Full-text deep reads use the arXiv LaTeX-source tools, not `download_paper`.** The server's PDF-conversion extra is not installed, so `download_paper`/`read_paper` fail on PDF-only papers; `list_paper_latex_sections` + `get_paper_latex_section` work. - **Pre-warm and re-warm the MCP server.** Cold start builds the package (30-60s). Trust and the cached package persist across sessions, but the running server process dies with the session, so fire one throwaway query at the start of any live/timed session. ## Revision note (v2) This guide was revised after a live setup session surfaced several improvements. Changes from v1: - **Build in place.** If the session already runs inside a folder named `harness`, build the components directly in it. Do not nest a second `harness/` inside (the literal v1 `mkdir` did). - **Vault as the single container for knowledge.** Project workspaces live *inside* the vault (`vault/projects//`), not in a separate root `projects/` tree. Machinery (skills, tools, guardrails, CLAUDE.md) stays at root. This makes the vault the single memory-of-record — every project `.md` is in it by construction, so no copy rule and no drift. - **Context/reference files are prerequisites.** `about-me.md`, `voice.md`, and `my-company.md` (who the researcher is, their writing voice, their institutional assets) are read at the start and seed the voice of every file. If a researcher has them, ingest them before Phase 1. - **Guardrails are scoped.** Writing-voice standards are guardrails for *written work products* only, not for campaign orchestration or planning-doc structure. - **Institutional policy is ingested, not authored.** A source -> vault -> distill -> promote pipeline populates policy hard-stops with provenance, rather than summarizing from memory. - **Tool Integration is shown, not just specced.** Where a skill needs an external source, connect a real MCP server (the session used the arXiv MCP server at project scope) so the demo exercises Component #1 concretely. - **Adversarial review is its own skill**, distinct from grill-me. See Phase 5. - **Automation lives in hooks, not guardrails.** Event-triggered behavior (e.g., capture session state before compaction) is configured as a hook in `.claude/settings.json`, because guardrails and memory cannot self-trigger. See the new "Automation layer" note after Phase 6. Honest limit: a hook cannot make the model synthesize at SessionEnd (no further model turn), so intelligent capture is nudged at PreCompact and the transcript is archived deterministically at SessionEnd. ## Instructions for Claude You are conducting a guided setup session to build a scientific harness for a researcher. This document is your playbook. Follow each phase in order. Do not skip phases. Do not proceed to the next phase until the current phase produces its deliverable. At each phase, interview the researcher, then collaboratively build the output file. Show them the draft, incorporate their corrections, and confirm before moving on. Write all output files in the researcher's voice and domain — not in generic language. If you don't yet know their voice, match the register and vocabulary they use in their answers. --- ## Phase 0: Orient (5 minutes) Explain to the researcher what we're building and why: "We're going to set up a scientific harness — a structured project that makes your AI use cumulative instead of disposable. By the end of this session, you'll have a folder structure, a set of initial configuration files, and the beginning of your first skill. Everything we build will persist across sessions and improve over time. The session takes about 2 hours. We'll work through it together — I'll ask questions, you'll answer, and we'll build each file collaboratively. You can stop after any phase and still have something useful." Then create the folder structure. If you are already inside a folder named `harness`, create these in place (do not nest another `harness/`). Note that `projects` lives inside the vault, not at root: ``` mkdir -p skills tools guardrails mkdir -p vault/{projects,literature,meetings,people,policies} ``` If the researcher has context/reference files (`about-me.md`, `voice.md`, `my-company.md`), place them at the harness root and read them before Phase 1 — they seed the voice of every file built. Create the stub CLAUDE.md: ```markdown # CLAUDE.md — Harness Master Instructions ## Status: Setup in progress ## Folder Structure Machinery (root): /skills, /tools, /guardrails, CLAUDE.md. Knowledge (the vault, single memory-of-record): - /vault/projects — One subfolder per active project. Each has plan.md, outputs/, delivered/. - /vault/literature, /vault/meetings, /vault/people, /vault/policies. ## Memory - Short-term: This conversation. Promote important findings before session ends. - Medium-term: Obsidian vault at ./vault. [Capture flow the researcher actually uses.] - Long-term: Graph DB [TBD — added later; may be deferred for initial setup]. ## Voice and Preferences [To be populated from setup interview] ## How to Work - Start every session with a planned deliverable, not open-ended chat. - Run the relevant skill for the task type. - Check guardrails before shipping anything. - Promote results to medium-term memory before closing. ``` Confirm with the researcher that the folder is created and the stub is in place. Then proceed. --- ## Phase 1: Domain Interview (30-40 minutes) This is the core of the setup. You are interviewing the researcher to understand their work deeply enough to build their harness components. Ask each question, listen carefully, and take notes. The answers seed every file we'll build. ### Block A: What you work on Ask these questions one at a time. Wait for answers. Follow up where useful. 1. "What is your primary research area? Describe it the way you would to a colleague in a related but different field." 2. "What are you actively working on right now — the 2-3 projects or questions that take most of your time?" 3. "For your most important current project: what is the scientific question, and what would a good answer look like?" 4. "What tools, databases, instruments, or software do you use regularly in your research? Include anything you interact with at least weekly." 5. "What external data sources or literature databases do you search when starting a new project or reviewing a field?" ### Block B: What goes wrong 6. "When you've used Claude or other AI tools, what has gone wrong? What did you have to fix or redo?" 7. "What mistakes would be most costly in your domain? What should an AI tool never do when working in your area?" 8. "Are there safety constraints, ethical boundaries, or institutional policies that apply to your work? Things that should be hard stops, not suggestions." 9. "When you review your own work before it goes out — a paper, a proposal, a presentation — what do you check for? What are your personal quality standards?" ### Block C: How you work 10. "Describe how you typically move from a new idea to a funded project or published result. What are the stages?" 11. "What tasks do you do repeatedly that feel like they could be systematized? Literature reviews, data analysis, proposal sections, experimental planning?" 12. "How do you currently keep track of what you've learned across projects? Notes, files, memory, conversations with colleagues?" 13. "If a new graduate student or postdoc joined your group tomorrow, what would you need to teach them about how you work? What takes the longest to transfer?" ### Block D: What matters most 14. "Looking at everything we've discussed — what would be the single highest-value thing for an AI system to help you with? The thing that would save you the most time or produce the best improvement in quality?" 15. "What's the second-highest?" After completing the interview, summarize what you've learned back to the researcher and confirm accuracy. Then proceed to building components. --- ## Phase 2: Build guardrails.md (15 minutes) Using the answers from Block B (questions 6-9), draft `guardrails/guardrails.md`. **Scope the quality standards.** Writing-voice standards (no inflation, external-driver openings, no em dashes, prose-carries-argument) are guardrails for *written work products* only. State this explicitly. They do not govern campaign orchestration or the internal structure of planning docs; those answer to scientific and process standards instead. **Separate the scientific-integrity standard.** If the researcher does experimental science, capture their computed-standard test for a result (what must hold before a claim stands) as its own block. Note any dependency on a tool that does not yet exist, so the gap is visible rather than silent. **Institutional policy is ingested, not authored.** Do not summarize institutional policy from memory. Instead, record a promotion pipeline: ingest source -> stage in vault `/policies` (tagged `#policy-draft`) -> distill enforceable hard-stops -> promote into guardrails with a citation back to the source note (flip to `#policy-active`) -> set a review cadence. List the policy *domains* pending ingestion. Run the actual ingestion as a follow-up session. **Add memory discipline.** A guardrail that work products are created directly in the vault and never authored-then-copied (single memory-of-record). This replaces any "always copy files to the vault" rule, which would create drift. Structure: ```markdown # Guardrails — Verification & Adversarial Policy ## Hard Constraints [From Q7-8: things the model must never do in this domain] ## Quality Standards [From Q9: the researcher's personal pre-delivery checklist] ## Known Failure Modes [From Q6: what has gone wrong before, encoded as rules] ## Escalation Rules - If uncertain about [domain-specific risk], stop and ask. - Never [domain-specific prohibited action] without explicit confirmation. - Always [domain-specific required check] before finalizing any deliverable. ## Pre-Delivery Checklist [Synthesized from Q9 and Q6: what to verify before anything ships] ``` Show the draft to the researcher. Incorporate corrections. Save the file. --- ## Phase 3: Build first tool specs + stand up the vault (15 minutes) Using the answers from Block A (questions 4-5), create tool interface specifications for the 2-3 most-used external resources. **Prioritize what the demo/first use actually exercises.** Do not spec every tool the researcher named. Park tools the near-term work will not touch (say so explicitly and revisit later), and spec the ones a live session will use. **Connect one real MCP server (Tool Integration, shown not just specced).** Pick a source the researcher's first task needs, prefer one with no auth for a first demo (the reference session used the arXiv MCP server), and add it at **project scope** so it travels with the harness as a demo artifact: `claude mcp add --scope project -- `. Verify the exact package before wiring it in. Project-scoped servers require a one-time trust approval on next interactive launch; that is expected, not an error. Write the tool spec around a stable *interface contract* so other sources (e.g., OSTI, Scopus) slot in behind it later without editing the skill. Two operational notes for any live source, learned from the arXiv MCP server (2026-08-05): the backend rate-limits by IP, so the skills that call it must issue queries **sequentially, never fanned out** (concurrent calls fail the batch with a 60-second rate-limit error); and cold start builds the package (30-60s), while the running process dies with the session, so **pre-warm with one query** at the start of any live or timed session even though trust and the cached package persist. For each tool, create a file in `tools/`: ```markdown # [Tool Name] ## What it is [One sentence: what this tool does in the researcher's workflow] ## Endpoint / Access [How to reach it: URL, API, local path, or manual access method] [Note: when ready for automated use, this becomes an MCP server endpoint] ## Valid Inputs [What you can ask it for, in what format] ## Outputs & Meaning [What it returns, and what the results mean in the researcher's scientific context] ## Permissions [Who/what should call this tool, and any restrictions] ## MCP Status [Current: documentation only (human mediates) | Future: MCP server (callable connection)] ``` Also stand up the vault and write its spec. Create the actual `./vault` folders now (Phase 0 did this) so a live session can land real output in `vault/literature/`. Install the Obsidian app and open `./vault` as a vault. On Windows, install by **direct download from obsidian.md**: `winget` failed in the reference environment (2026-08-05), and the direct download succeeded. The app is only a viewer over the markdown, so a failed or deferred install never blocks writing to the vault. Write the spec to reflect the *actual* capture flow the researcher uses (the reference session: physical notebook -> LLM transcription -> markdown note in the vault), and mark the long-term graph DB deferred if it is not yet live. Create `tools/obsidian-vault.md`: ```markdown # Obsidian Vault — Medium-Term Memory ## What it is Linked markdown notes serving as persistent memory across sessions and projects. ## Location ./vault (inside the harness folder). Open this folder as a vault in the Obsidian desktop app. ## Structure /projects — one subfolder per active project (plan.md, outputs/, delivered/) /meetings — one note per significant meeting /literature — one note per literature scan or synthesis /people — one note per key collaborator or contact ## How to Use - At end of each session, promote key findings to the relevant note. - Link notes with [[double brackets]] when entities connect. - Start each session by reading relevant project and meeting notes. ``` Show drafts to the researcher. Incorporate corrections. Save files. --- ## Phase 4: Build first project plan (15 minutes) Using the answers from Block A (question 3) and Block C (question 10), create the first project workspace and plan. ``` mkdir -p vault/projects/[project-name]/{outputs,delivered} ``` **The plan can be the demo's output, not a pre-authored input.** If the session runs the front of the researcher's idea->result pipeline live (e.g., grill-me then literature-research), the project `plan.md` is the natural *product* of that run. Prefer generating it that way — it turns the demo from "here is a harness I built" into "watch the harness produce a real work product," exercising Skills + Tool Integration + Memory at once. Confirm the seed idea with the researcher first. Draft `vault/projects/[project-name]/plan.md`: ```markdown # [Project Name] — Plan ## Scientific Question [From Q3: what is the question, stated precisely] ## What Success Looks Like [From Q3: what would a good answer look like] ## Approach [From the interview: methods, instruments, constraints] ## Design Space [If applicable: what variables are being explored] ## Constraints [Timeline, budget, equipment access, personnel] ## Acceptance Criteria [How do we know when this is done or when a hypothesis is validated/refuted] ## Current Status [Where the project stands right now] ``` Show the draft. Incorporate corrections. Save. --- ## Phase 5: Build first skill (20 minutes) Using the answers from Block C (question 11) and Block D (questions 14-15), identify the researcher's highest-value repeatable task and encode it as a skill. **Distinguish skills from larger systems.** A skill is one containable methodology file. Multi- component business systems (e.g., sponsor-profile assembly, opportunity matching, EH&S workflows) are not skills and should not be forced into one file. Record that boundary. **Skills compose; do not reimplement.** If the highest-value task is an orchestrator (e.g., idea -> execution plan), it should *call* existing skills (grill-me for concept refinement, a literature skill for grounding, an adversarial-review skill for work-product critique) at the right stages, not reimplement them. Note skill-to-skill contracts explicitly (e.g., a literature skill emits a structured dissenting perspective that an adversarial-review skill consumes as a persona) — these contracts are what make instruments swappable (the modularity property). **grill-me and adversarial-review are different skills.** grill-me interviews the *researcher* to remove ambiguity from a plan. adversarial-review is a reviewer *persona* that critiques a finished work product, with context injectable from prompt, from specific vault notes, or from a literature- derived dissenting perspective. Build/spec them separately. **Foreground judgment, not just sequence.** For the slowest-to-transfer skills (the "six-month postdoc" knowledge), encode the *quality bars* at each stage, not only the step order. A new person can follow the steps and still produce weak work; what transfers slowly is what makes each stage good. Ask: "You said [highest-value task from Q14] is where AI help would matter most. Let's build a skill for that. Walk me through how you do it now — step by step, including the things you check for and the mistakes you've learned to avoid." Listen to their description. Then draft `skills/[task-name].md`: ```markdown # [Task Name] Skill ## When to Use [One sentence: what triggers this skill] ## Methodology [Step-by-step process, encoded from the researcher's description] [Include the checks, the failure modes, the quality standards] [Reference guardrails where applicable] [Reference relevant tools where applicable] ## Inputs Required [What the researcher needs to provide to start] ## Expected Output [What the skill produces, in what format] ## Quality Checks [Pre-delivery verification steps specific to this task type] ``` Show the draft. This is the most important file — spend time getting it right. The researcher should recognize their own methodology in it. Incorporate corrections until they say "yes, that's how I do it." Save the file. --- ## Phase 6: Expand CLAUDE.md (10 minutes) Using everything learned in the session, expand the stub CLAUDE.md into a full master instruction file. Update the Voice and Preferences section from how the researcher spoke during the interview — their register, their level of formality, their domain vocabulary, whether they prefer concise or detailed responses. Update the How to Work section with methodology preferences that surfaced during the interview. Add a section listing what exists in the harness: ```markdown ## Current Harness Contents - skills/grill-me.md — Adversarial interview skill [provided] - skills/[first-skill].md — [description] [built today] - tools/[tool-1].md — [description] [built today] - tools/[tool-2].md — [description] [built today] - tools/obsidian-vault.md — Medium-term memory spec [built today] - guardrails/guardrails.md — Domain verification policy [built today] - .mcp.json — Connected MCP server(s) at project scope [built today] - vault/projects/[name]/plan.md — First project specification [built today] ``` Also update the Memory section with the researcher's real capture flow, and the Folder Structure to reflect vault-as-container. Add a "Known Gaps" list (tools/skills specified but not yet built, and anything deferred) so the gaps are tracked rather than forgotten. Show the draft. Incorporate corrections. Save. --- ## Automation layer (optional, after Phase 6) The five components are policy, methodology, memory, tools, and planning. Automation is a distinct layer that ties them together: **hooks** in `.claude/settings.json` that the harness executes on lifecycle events. Use this when the researcher wants something to happen *automatically* on an event, not on request. The canonical example is session capture: - Build a `session-capture` skill (methodology: what a good state note contains). - Wire a `PreCompact` hook that nudges the model to run it before context is lost, and a `SessionEnd` hook that archives the raw transcript. - Route auto-captures to a `vault/sessions/` holding area, tagged distinctly, so auto-generated notes do not pollute the curated memory-of-record. Promotion to project/meeting/literature notes is a human step. Do not encode automation intent as a guardrail line ("at X% context, do Y") — guardrails are read and obeyed by the model, they do not fire on their own. Only hooks fire on events. And be honest about the limit: no model synthesis is possible at SessionEnd, so synthesis is nudged at PreCompact and SessionEnd does deterministic archiving. ## Phase 7: Wrap Up (5 minutes) Summarize what was built: "Here's what we created today: - A folder structure mirroring the five harness components - guardrails.md with your domain constraints and quality standards - [N] tool interface specifications - Your first project plan - Your first skill, encoding your methodology for [task] - An expanded CLAUDE.md with your preferences and harness inventory Next time you open Claude Code in this project, it will load CLAUDE.md automatically and know your structure, your tools, your constraints, and your methodology. Three things to do this week: 1. Use the [first skill] on a real task. Notice what it gets wrong and update the skill file. 2. Create your Obsidian vault at the path we specified. Add one note for your current project. 3. Optional but high-ROI: run the voice profiling interview to capture your writing style in detail." Ask if they have questions. End the session. --- ## Notes for the ALD This guide is designed to be handed to any directorate researcher along with one file: `skills/grill-me.md`. The researcher loads this document into a Claude Code session and follows the prompts. The session produces a functioning harness in approximately 2 hours. The guide should be updated as the harness architecture evolves. Key dependencies: - The grill-me skill must be provided separately (not embedded in this document) - The folder structure matches the five-component architecture from the Scientific Harness Concept document - The interview questions in Phase 1 are designed to seed every subsequent phase — changing the questions changes what gets built The voice profiling interview is referenced but not included. It should be offered as a separate follow-up session.