Wire In Your Own Harness (pi or Kiro)

Warm-up: capstone check

Before you build a backend from scratch, let's confirm the boundary is clear:

What you'll do

Implement a SkillOpt ModelBackend for your preferred agent — pi, Kiro, or another CLI agent. You'll read the shipped claude_code_exec backend as your template, implement the contract, register it, and run a training loop through your own agent.

Step 1 — Read the template

The best reference is the codex_exec backend — it wraps the Codex CLI as a subprocess:

# Find it in the installed package
python -c "import skillopt; print(skillopt.__path__[0])"

# Read the exec-style backend
cat $(python -c "import skillopt; print(skillopt.__path__[0])")/model/codex_exec.py

Note the pattern:

1. __init__ — store config, validate the agent CLI is available
2. generate() — format messages into a prompt, invoke the agent CLI as a subprocess, capture output, return ModelResponse
3. Token counting — approximate (word count × 1.3 is fine for exec backends)

Step 2 — Implement your backend

Create ~/skillopt-repo/skillopt/model/pi_exec.py (or kiro_exec.py):

import subprocess
import json
from skillopt.model.base import ModelBackend, ModelResponse


class PiExecBackend(ModelBackend):
    """Execute tasks via the pi coding agent CLI."""

    def __init__(self, cfg: dict):
        super().__init__(cfg)
        self.model_name = cfg.get("model_name", "pi")
        self.timeout = int(cfg.get("timeout", 120))
        # Verify pi is available
        subprocess.run(["pi", "--version"], capture_output=True, check=True)

    async def generate(
        self,
        messages: list[dict],
        temperature: float = 0.7,
        max_tokens: int = 4096,
        **kwargs,
    ) -> ModelResponse:
        # Format messages into a single prompt for the CLI
        prompt = self._format_messages(messages)

        # Run pi with the skill as system prompt + task as user message
        result = subprocess.run(
            ["pi", "--non-interactive", "--prompt", prompt],
            capture_output=True,
            text=True,
            timeout=self.timeout,
        )

        content = result.stdout.strip()
        # Approximate token count
        token_est = len(content.split()) * 1.3

        return ModelResponse(
            content=content,
            usage={
                "prompt_tokens": int(len(prompt.split()) * 1.3),
                "completion_tokens": int(token_est),
            },
            model=self.model_name,
        )

    def _format_messages(self, messages: list[dict]) -> str:
        """Collapse chat messages into a single prompt string."""
        parts = []
        for msg in messages:
            if msg["role"] == "system":
                parts.append(f"[System]\n{msg['content']}\n")
            elif msg["role"] == "user":
                parts.append(f"[Task]\n{msg['content']}\n")
        return "\n".join(parts)

Step 3 — Register

Add to skillopt/model/__init__.py:

from .pi_exec import PiExecBackend

BACKEND_REGISTRY = {
    # ... existing backends ...
    "pi_exec": PiExecBackend,
}

Step 4 — Test standalone

Before running full training, verify the backend works in isolation:

python -c "
import asyncio
from skillopt.model.pi_exec import PiExecBackend

backend = PiExecBackend({'model_name': 'pi', 'timeout': 60})

async def test():
    resp = await backend.generate(
        messages=[
            {'role': 'system', 'content': 'You are a helpful assistant.'},
            {'role': 'user', 'content': 'What is 2+2?'}
        ]
    )
    print(f'Content: {resp.content}')
    print(f'Usage: {resp.usage}')

asyncio.run(test())
"

If this prints a response, your backend contract is satisfied.

Step 5 — Run training through your agent

Update your config to use the new backend:

# In configs/my_env/default.yaml
model:
  backend: pi_exec
  model_name: pi
  timeout: 120

Then run:

python scripts/train.py --config configs/my_env/default.yaml

You're now training a skill for your agent, using your agent as the execution target, on your tasks. This is the full mission: SkillOpt optimizing a skill document that makes your daily agent measurably better.

The Kiro alternative: MCP shell

If you prefer Kiro over pi, the approach is different. Kiro doesn't have a simple subprocess CLI — it's an MCP-native agent. The pattern from plugins/copilot/mcp_server.py is:

1. Register a SkillOpt MCP server that exposes skillopt-sleep tools
2. Kiro calls these tools during sessions (harvest is automatic)
3. The consolidation happens server-side, same as Copilot's integration

The missing piece for Kiro is a transcript harvester — Kiro doesn't yet write ATIF-v1.7 session logs the way Devin does. You'd need to write a harvester that reads Kiro's session state and converts it to SkillOpt's expected format. The Devin plugin's harvester (plugins/devin/harvester.py) is the reference.

Deploying your skill

Once you have a validated best_skill.md, deploy it:

Then schedule nightly sleep cycles to keep improving it:

skillopt-sleep schedule  # installs a cron entry for this project