A Full Training Run

Lesson 4 · SkillOpt & Agent Harnesses · ~25 minutes · hands-on

Warm-up: the backend boundary

From Lesson 1, you learned that when wiring pi or Kiro into SkillOpt, the piece you build is the backend. Let's make sure that boundary is sharp before you use a real one:

What does a SkillOpt model backend take in and return?

A task description in; a score out
Chat messages in; a ModelResponse (content + usage) out
A skill document in; an edited skill document out

Exactly. The backend is the model interface — it takes chat messages and returns generated text + token usage. The env adapter is what handles tasks and scoring. Two distinct roles. Today you'll use the shipped claude_code_exec backend, which wraps Claude Code as the target model.

The backend is simpler than that — it's the model interface. It takes chat messages (list of role/content dicts) and returns a ModelResponse with content and usage. Scoring and task handling live in the env adapter, a separate component. Re-read the new-backend guide — it's only 130 lines.

What you'll do

Run SkillOpt's full training loop — the paper's actual optimization algorithm — against a shipped benchmark using the claude_code_exec backend. This produces a real best_skill.md artifact: a learned skill document trained on actual tasks with the validation gate enforcing quality.

This is different from SkillOpt-Sleep (Lesson 3): Sleep harvests your past usage and consolidates. The full training loop actively runs new rollouts on a defined task set, reflects on failures, proposes edits, and gates them — the full SkillOpt paper algorithm with train/val/test splits.

Step 1 — Pick a benchmark

SkillOpt ships six benchmarks. For this lesson, use SearchQA — it's well-documented, fast to run with small batch sizes, and the scoring is deterministic (exact-match on extracted answers):

# Check the shipped envs
ls $(python -c "import skillopt; print(skillopt.__path__[0])")/envs/

You should see: searchqa, officeqa, swebench, spreadsheetbench, and others.

Step 2 — Configure and run

The shipped configs live in the SkillOpt repo. Clone it for configs (you already have the package installed via pip):

# Get the configs and data scripts
git clone https://github.com/microsoft/SkillOpt.git ~/skillopt-repo
cd ~/skillopt-repo

# Download the SearchQA data split
python scripts/download_data.py --env searchqa

Now run with a small batch to keep costs manageable:

# Set your API key (the optimizer model)
export OPENAI_API_KEY="your-key"

# Run training — small batch, 2 epochs
python scripts/train.py \
  --config configs/searchqa/default.yaml \
  --train.batch_size 8 \
  --train.num_epochs 2 \
  --gradient.minibatch_size 4

Cost control

With batch_size=8 and 2 epochs, expect ~$1–3 total spend (optimizer + target). The optimizer proposes edits; the target (Claude Code in exec mode) runs tasks. You can further reduce cost by setting --model.reasoning_effort low.

Step 3 — Read the training log

As training runs, watch for:

Epoch/batch progress: Each batch triggers: rollout → reflect → propose edit → gate check
Gate decisions: ACCEPTED or REJECTED for each proposed edit. Count the ratio — a healthy run accepts 30–60% of proposals.
Val scores: The held-out validation score should trend upward epoch-over-epoch. If it's flat, the gate is correctly rejecting unhelpful proposals.
The learning rate: optimizer.learning_rate: 4 means the edit can change at most 4 "lines" of the skill per proposal. This bounds drift.

Step 4 — Inspect the artifact

After training completes, find your best skill:

# The output directory (check the config or the last log line)
ls outputs/searchqa/skills/

# Read the trained skill
cat outputs/searchqa/skills/best_skill.md

This is the core output — a text document the gate validated improves held-out accuracy. Apply the rubric: does each learned rule encode a failure mechanism with a remedy? Is it step-level specific?

Step 5 — Evaluate on test

Run the final evaluation on the held-out test split to confirm the gain is real:

python scripts/evaluate.py \
  --config configs/searchqa/default.yaml \
  --skill outputs/searchqa/skills/best_skill.md \
  --split test

Compare this score to the baseline (empty skill). The difference is your trained lift — the empirical proof that the optimization worked.

What if the gain is small?

With only 2 epochs and batch_size=8, expect modest gains (2–5 percentage points). The paper runs 4 epochs with batch_size=16 for full results. Small gains are normal for a first run — the point is proving the pipeline works end-to-end, not maximizing accuracy today.

The training run accepted 2 of 6 proposed edits. The val score went from 0.72 to 0.76. What does this tell you?

The optimizer is too weak — it should accept more edits
The gate is too strict — you should lower the threshold
The system is working as designed — most proposals don't help, but the ones that pass the gate deliver real lift

Yes. A 33% acceptance rate with +4pp lift is textbook SkillOpt. The gate's job is to reject the ~67% of proposals that look plausible but don't actually improve held-out performance. Quality over quantity — every accepted edit is empirically validated.

The gate isn't "too strict" — it's doing its job. Accepting 2/6 with a +4pp gain means the 4 rejected proposals would have been neutral or harmful. The validation gate prevents the 25% negative transfer documented in SkillLens. You don't want to accept more — you want the right ones accepted.