Glossary — SkillOpt Harness

B

Backend (Harness)

The pluggable execution target that runs tasks with the current skill document and returns a trajectory or ModelResponse to SkillOpt's engine.

Bounded Edit

A constrained text modification (add, delete, or replace) applied to the skill document, limited in size by the textual learning rate.

C

Contrastive Dreaming

Running each task multiple times during replay to produce both successful and failed trajectories, giving the optimizer clearer signal about what specifically caused success or failure.

D

Dream Rollout

An offline re-execution of a task using the current skill, performed during the sleep cycle's replay phase to generate training signal without live user interaction.

E

Env (Environment)

The combination of a task dataset, a scoring function, and a seed skill that defines what SkillOpt trains against — analogous to a training benchmark.

Experience Archive

A persistent store of past tasks and trajectories that accumulates across nightly sleep cycles, enabling associative recall of relevant history into future consolidation runs.

Experience Replay

The technique of pulling similar past tasks from the archive into the current night's dream batch to provide richer context for the optimizer's reflection.

M

Meta-Skill

A quality rubric embedded into the optimizer's own prompt that instructs it what constitutes a good skill rule, yielding higher-quality proposals.

Model Backend

The interface class requiring a single generate(messages, temperature, max_tokens) → ModelResponse method that wraps any target agent (CLI or API) for SkillOpt.

N

Negative Transfer

When an extracted or proposed skill rule makes the agent perform worse than having no skill at all, measured at 25% of ungated cases in SkillLens research.

Noise Floor

The variance in evaluation scores across repeated identical runs, below which measured lift cannot be confidently distinguished from random variation.

O

Optimizer Model

The separate LLM that reads scored trajectories, reflects on failures, and proposes bounded edits to the skill document — distinct from the target model that executes tasks.

S

Saturation

The state where consecutive nightly proposals are all rejected by the gate, indicating the skill has reached ceiling performance for the current task distribution.

Seed Skill

The initial markdown document (can be empty or minimal) that serves as the starting point before any optimization edits are applied.

Skill Document

The plain-text markdown file that serves as the trainable state in SkillOpt — analogous to neural network weights — injected as a system prompt at zero inference-time cost.

Skill Transfer

The property that a skill document optimized in one harness or model can improve performance in a different harness or model, though gains vary and the validation gate should be re-run on the new target.

Sleep Cycle

The nightly automated pipeline of harvest → mine → replay → consolidate → stage → adopt that extracts improvements from real agent sessions without active user involvement.

Staged Proposal

A candidate set of skill edits that passed the validation gate and awaits explicit user adoption (or auto-adoption) before being merged into the deployed skill.

T

Task Mining

The process of analyzing harvested agent session transcripts to identify recurring task patterns that can be replayed and optimized against.

Textual Learning Rate

The edit budget that limits how much of the skill document can change per optimization step, preventing catastrophic drift — analogous to learning rate in gradient descent.

V

Validation Gate

The strict empirical check that accepts a proposed skill edit only if it improves the score on held-out tasks never used during training, preventing drift and negative transfer.