Multi-Task Skills & Saturation

Warm-up: retrieve

What you'll learn

Your first skill was trained on one task domain (Lesson 5). But your agent does many things — code review, test writing, refactoring, documentation, debugging. This lesson covers: training across multiple task types, deciding whether to compose separate skills or merge into one, and what to do when a skill plateaus.

One skill or many?

Two architectures for multi-domain skills:

Step 1 — Add task types to your env

In your dataloader (Lesson 5), each item has a task_type field. Add items from a second domain:

# data/my_env_split/train/tasks.json
[
  {"id": "review_001", "task_type": "code_review", "input": "...", "ground_truth": "..."},
  {"id": "review_002", "task_type": "code_review", "input": "...", "ground_truth": "..."},
  {"id": "test_001", "task_type": "test_generation", "input": "...", "ground_truth": "..."},
  {"id": "test_002", "task_type": "test_generation", "input": "...", "ground_truth": "..."}
]

Your adapter's get_task_types() returns all types present — the optimizer sees them all and can learn type-specific rules.

Step 2 — Train on the combined set

python scripts/train.py --config configs/my_env/default.yaml

Watch the per-type scores in the training log. The optimizer may learn rules that help one type and are neutral for others (good), or rules that help one and hurt another (the gate will reject these).

Step 3 — Diagnose conflicts

If you see many rejections after adding a second task type, check whether:

1. The rules conflict: "Always include type annotations" helps test generation but hurts commit message formatting. Solution: split into separate skills.
2. The scoring functions conflict: Different domains need different evaluation. Solution: ensure your _score() function handles each type appropriately.
3. The data is imbalanced: 90% of one type drowns out the other. Solution: balance your train split or use stratified sampling.

Step 4 — Breaking through plateaus

When a skill saturates (Lesson 9), you have four levers:

Step 5 — The skill composition pattern

If you do split into multiple skills, deploy them as layers:

# For pi:
~/.pi/skills/
├── 00-base-coding.md      # Universal coding patterns
├── 01-code-review.md      # Review-specific rules
└── 02-test-generation.md  # Test-writing rules

# For Kiro:
.kiro/steering/
├── skillopt-base.md
├── skillopt-review.md
└── skillopt-tests.md

Number-prefix ordering ensures the base skill loads first. Type-specific skills add rules that only apply in their domain. The agent sees all of them — the rules are additive.

The end state

After this course, you have a system that:

1. Harvests your real agent sessions nightly
2. Mines recurring tasks and replays them with contrastive dreaming
3. Proposes skill edits bounded by a textual learning rate
4. Gates every edit against held-out tasks (no negative transfer)
5. Stages proposals for your review (or auto-adopts if you trust the gate)
6. Deploys as zero-overhead system prompt text
7. Covers multiple task types with a unified or composed skill architecture

Your agent gets better the more you use it. No weight training. No inference overhead. Just validated text edits that encode what works.