🧠 Concepts on LLM Wiki — Agentic AI Landscapehttps://blog.imfsoftware.com/llm-wiki/docs/concepts/Recent content in 🧠 Concepts on LLM Wiki — Agentic AI LandscapeHugoen-us<link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent-skills-standard/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent-skills-standard/</guid><description><h1 id="agent-skills-standard">Agent Skills Standard<a class="anchor" href="#agent-skills-standard">#</a></h1> <p>An open standard (agentskills.io) for packaging reusable capabilities that AI agents can load dynamically. Initiated by <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/anthropic/">anthropic</a>, designed for cross-tool compatibility.</p> <h2 id="specification">Specification<a class="anchor" href="#specification">#</a></h2> <p>A skill is a directory with a <code>SKILL.md</code> file:</p> <pre tabindex="0"><code>skill-name/ ├── SKILL.md # Required: YAML frontmatter + markdown instructions ├── scripts/ # Optional: executable code ├── references/ # Optional: documentation ├── assets/ # Optional: templates, resources</code></pre><h3 id="required-frontmatter">Required Frontmatter<a class="anchor" href="#required-frontmatter">#</a></h3> <ul> <li><code>name</code>: Max 64 chars, lowercase + hyphens, must match directory name</li> <li><code>description</code>: Max 1024 chars, what it does and when to use it</li> </ul> <h3 id="optional-frontmatter">Optional Frontmatter<a class="anchor" href="#optional-frontmatter">#</a></h3> <ul> <li><code>license</code>, <code>compatibility</code>, <code>metadata</code>, <code>allowed-tools</code> (experimental)</li> </ul> <h2 id="progressive-disclosure">Progressive Disclosure<a class="anchor" href="#progressive-disclosure">#</a></h2> <p>The key design principle — skills scale without blowing up context:</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent-state-model/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent-state-model/</guid><description><h1 id="agent-state-model">Agent State Model<a class="anchor" href="#agent-state-model">#</a></h1> <p><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a> tracks agent state across three dimensions:</p> <h2 id="phase-lifecycle">Phase (Lifecycle)<a class="anchor" href="#phase-lifecycle">#</a></h2> <p>The infrastructure lifecycle of the agent container:</p> <p><code>created → provisioning → cloning → starting → running → stopping → stopped (or error)</code></p> <h2 id="activity-cognitive-state">Activity (Cognitive State)<a class="anchor" href="#activity-cognitive-state">#</a></h2> <p>What the agent is doing within the <code>running</code> phase:</p> <p><code>idle | thinking | executing | waiting_for_input | blocked | completed | limits_exceeded | offline</code></p> <ul> <li><strong>Sticky activities</strong>: <code>completed</code>, <code>blocked</code>, and <code>limits_exceeded</code> persist until explicit restart or stop</li> <li><strong>blocked</strong>: Set by agents themselves when waiting for an expected event (e.g., child agent completing) — prevents false stall detection</li> <li><strong>offline</strong>: Occurs when heartbeat is lost, often due to auth token refresh failure. Fix: stop and restart the agent.</li> </ul> <h2 id="detail-freeform-context">Detail (Freeform Context)<a class="anchor" href="#detail-freeform-context">#</a></h2> <p>Freeform text about the current activity — tool name, message, task summary.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/</guid><description><h1 id="agent">Agent<a class="anchor" href="#agent">#</a></h1> <p>In <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a>, an Agent is an isolated process running an LLM + <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/harness/">harness</a> loop against a task. It is the fundamental unit of execution.</p> <h2 id="properties">Properties<a class="anchor" href="#properties">#</a></h2> <p>Each agent has:</p> <ul> <li><strong>Identity</strong>: Unique name, container ID, and (in hosted mode) a UUID and URL-safe slug</li> <li><strong>Home directory</strong>: Mounted at <code>/home/&lt;user&gt;</code> — contains harness config, credentials, <code>agent-info.json</code></li> <li><strong>Workspace</strong>: Mounted at <code>/workspace</code> — a dedicated git worktree or cloned repo</li> <li><strong>Template</strong>: The <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/template/">template</a> blueprint that seeded its configuration</li> <li><strong>Harness</strong>: The LLM-specific adapter (Claude, Gemini, OpenCode, Codex)</li> </ul> <h2 id="state-model">State Model<a class="anchor" href="#state-model">#</a></h2> <p>See <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent-state-model/">agent-state-model</a> for the full three-dimensional state tracking system.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/context-management/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/context-management/</guid><description><h1 id="context-management">Context Management<a class="anchor" href="#context-management">#</a></h1> <p>Strategies for managing the limited context window available to LLM agents, especially in multi-skill pipelines where different agent personas must share information efficiently.</p> <h2 id="why-it-matters">Why It Matters<a class="anchor" href="#why-it-matters">#</a></h2> <p>Context windows are finite. Every skill, instruction, tool definition, and conversation turn consumes tokens. Without management, agents hit limits, lose early instructions, or waste budget on irrelevant context.</p> <h2 id="progressive-disclosure">Progressive Disclosure<a class="anchor" href="#progressive-disclosure">#</a></h2> <p>The <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent-skills-standard/">agent-skills-standard</a> and <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/claude-code/">claude-code</a> both use this pattern:</p> <ul> <li><strong>Metadata</strong> (~100 tokens): Loaded at startup for all skills — just name + description</li> <li><strong>Instructions</strong> (&lt;5000 tokens): Loaded only when skill activated</li> <li><strong>Resources</strong>: Loaded only when specifically needed</li> </ul> <p><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/claude-code/">claude-code</a> extends this: MCP tool definitions are deferred by default (only names consume context until a tool is used).</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/frontier-agent/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/frontier-agent/</guid><description><h1 id="frontier-agent">Frontier Agent<a class="anchor" href="#frontier-agent">#</a></h1> <p>A term coined by <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/aws/">aws</a> for a new class of AI agents with three defining characteristics:</p> <ol> <li><strong>Autonomous</strong>: Direct them towards a goal, and they figure out how to achieve it</li> <li><strong>Massively scalable</strong>: Able to perform multiple concurrent tasks and distribute work across agents</li> <li><strong>Work independently</strong>: Operating for hours or days without intervention</li> </ol> <h2 id="distinction-from-regular-agents">Distinction from Regular Agents<a class="anchor" href="#distinction-from-regular-agents">#</a></h2> <p>Regular AI agents can plan and execute multi-step tasks with some autonomy. Frontier agents go further — they&rsquo;re designed for long-running, independent operation at scale, not just responding to individual prompts or short interactive sessions.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/grove/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/grove/</guid><description><h1 id="grove">Grove<a class="anchor" href="#grove">#</a></h1> <p>In <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a>, a Grove is a project workspace where <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/">agent</a>s live. It corresponds to a <code>.scion</code> directory on the filesystem.</p> <h2 id="scope">Scope<a class="anchor" href="#scope">#</a></h2> <ul> <li><strong>Project-level</strong>: Located at the root of a git repository</li> <li><strong>Global</strong>: In the user&rsquo;s home folder (<code>~/.scion</code>)</li> </ul> <h2 id="identification">Identification<a class="anchor" href="#identification">#</a></h2> <p>Every grove has a unique Grove ID:</p> <ul> <li><strong>Git-backed groves</strong>: Deterministic UUID v5 derived from namespace + normalized git URL — same repo always maps to same ID regardless of protocol</li> <li><strong>Hub-native groves</strong>: Random UUID v4</li> </ul> <h2 id="contents">Contents<a class="anchor" href="#contents">#</a></h2> <p>A grove contains:</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/harness/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/harness/</guid><description><h1 id="harness">Harness<a class="anchor" href="#harness">#</a></h1> <p>In <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a>, a Harness is an adapter that allows a specific LLM tool to run within the Scion orchestration layer. It handles provisioning, configuration, and execution for that tool inside an OCI container.</p> <h2 id="purpose">Purpose<a class="anchor" href="#purpose">#</a></h2> <p>The harness ensures that generic Scion commands (<code>start</code>, <code>stop</code>, <code>attach</code>, <code>resume</code>) work consistently regardless of the underlying agent software.</p> <h2 id="interface">Interface<a class="anchor" href="#interface">#</a></h2> <p>Each harness implements (Go):</p> <ul> <li><code>DiscoverAuth()</code> — Locate credentials on the host</li> <li><code>GetEnv()</code> — Map credentials to container environment variables</li> <li><code>GetCommand()</code> — Build the correct CLI invocation</li> <li><code>Provision()</code> — Harness-specific setup during agent creation</li> <li><code>PropagateFiles()</code> — Copy config files into agent home</li> <li><code>GetVolumes()</code> — Define volume mounts</li> </ul> <h2 id="supported-harnesses">Supported Harnesses<a class="anchor" href="#supported-harnesses">#</a></h2> <table> <thead> <tr> <th>Harness</th> <th>Target Tool</th> <th>Notes</th> </tr> </thead> <tbody> <tr> <td><code>gemini</code></td> <td><strong>gemini-cli</strong></td> <td>Default harness. API key / OAuth / Vertex AI auth</td> </tr> <tr> <td><code>claude</code></td> <td><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/claude-code/">claude-code</a></td> <td>Anthropic API key / Vertex AI auth. See <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/claude-code/">claude-code</a> for full capabilities.</td> </tr> <tr> <td><code>opencode</code></td> <td><strong>opencode</strong></td> <td>Experimental. No hook support</td> </tr> <tr> <td><code>codex</code></td> <td><strong>codex</strong></td> <td>Runs <code>--full-auto</code> by default</td> </tr> <tr> <td><code>generic</code></td> <td>Any CLI tool</td> <td>Fallback adapter</td> </tr> </tbody> </table> <h2 id="capability-matrix">Capability Matrix<a class="anchor" href="#capability-matrix">#</a></h2> <table> <thead> <tr> <th>Capability</th> <th>Gemini</th> <th>Claude</th> <th>OpenCode</th> <th>Codex</th> </tr> </thead> <tbody> <tr> <td>Resume</td> <td>✅</td> <td>✅</td> <td>✅</td> <td>✅</td> </tr> <tr> <td>Resume with Prompt</td> <td>✅</td> <td>✅</td> <td>✅</td> <td>❌</td> </tr> <tr> <td>Hooks</td> <td>✅</td> <td>✅</td> <td>❌</td> <td>❌</td> </tr> <tr> <td>OpenTelemetry</td> <td>✅</td> <td>✅</td> <td>❌</td> <td>✅</td> </tr> <tr> <td>System Prompt Override</td> <td>✅</td> <td>✅</td> <td>❌</td> <td>❌</td> </tr> </tbody> </table> <h2 id="extensibility">Extensibility<a class="anchor" href="#extensibility">#</a></h2> <p>New harnesses can be added via the <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/plugin-system/">plugin-system</a> (hashicorp/go-plugin over gRPC) without modifying the core codebase.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/hub/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/hub/</guid><description><h1 id="hub">Hub<a class="anchor" href="#hub">#</a></h1> <p>In <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a>, the Hub is the central control plane of the hosted (distributed) architecture. It coordinates state across multiple users, <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/grove/">grove</a>s, and <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/runtime-broker/">runtime-broker</a>s.</p> <h2 id="responsibilities">Responsibilities<a class="anchor" href="#responsibilities">#</a></h2> <ul> <li><strong>Identity &amp; Auth</strong>: Manages user identities via OAuth, issues tokens for brokers and agents</li> <li><strong>State Persistence</strong>: Stores definitive state of agents, groves, and templates in a central database (SQLite)</li> <li><strong>Orchestration</strong>: Dispatches agent lifecycle commands to appropriate <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/runtime-broker/">runtime-broker</a>s</li> <li><strong>Collaboration</strong>: Provides shared view via Web Dashboard and Hub API</li> </ul> <h2 id="communication-with-brokers">Communication with Brokers<a class="anchor" href="#communication-with-brokers">#</a></h2> <ul> <li><strong>Direct HTTP</strong>: When broker has a reachable endpoint</li> <li><strong>Control Channel (WebSocket tunnel)</strong>: When broker is behind NAT/firewall</li> </ul> <h2 id="agent-creation-flow-hosted">Agent Creation Flow (Hosted)<a class="anchor" href="#agent-creation-flow-hosted">#</a></h2> <ol> <li>CLI syncs grove with Hub</li> <li><code>POST /api/v1/groves/{groveId}/agents</code></li> <li>Hub selects a <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/runtime-broker/">runtime-broker</a></li> <li>Merges environment variables and secrets from all scopes (user → grove → broker)</li> <li>Resolves <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/template/">template</a> with content hash for broker-side caching</li> <li>Dispatches to broker</li> <li>Broker provisions and starts <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/">agent</a></li> <li>Status reported back via heartbeats</li> </ol> <h2 id="see-also">See Also<a class="anchor" href="#see-also">#</a></h2> <ul> <li><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a></li> <li><a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/runtime-broker/">runtime-broker</a></li> <li><a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/grove/">grove</a></li> <li><a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/">agent</a></li> </ul></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/kiro-powers/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/kiro-powers/</guid><description><h1 id="kiro-powers">Kiro Powers<a class="anchor" href="#kiro-powers">#</a></h1> <p>Specialized packages that enhance existing <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/kiro/">kiro</a> agents with prebuilt expertise for specific development tasks.</p> <h2 id="contents">Contents<a class="anchor" href="#contents">#</a></h2> <ul> <li>Curated MCP servers</li> <li>Steering files</li> <li>Hooks</li> <li>Can be dynamically loaded on demand</li> </ul> <h2 id="purpose">Purpose<a class="anchor" href="#purpose">#</a></h2> <p>Focus on providing domain-specific knowledge and best practices. Distinct from the <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/kiro/">kiro</a> autonomous agent — Powers enhance agents with expertise, while the autonomous agent is the execution engine that works independently.</p> <h2 id="see-also">See Also<a class="anchor" href="#see-also">#</a></h2> <ul> <li><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/kiro/">kiro</a></li> <li><a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/frontier-agent/">frontier-agent</a></li> </ul></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/llm-wiki-pattern/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/llm-wiki-pattern/</guid><description><h1 id="llm-wiki-pattern">LLM Wiki Pattern<a class="anchor" href="#llm-wiki-pattern">#</a></h1> <p>A pattern for building personal knowledge bases where the LLM incrementally builds and maintains a persistent, interlinked wiki from raw sources. Proposed by <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/andrej-karpathy/">andrej-karpathy</a>. This wiki is a running instance of this pattern.</p> <h2 id="core-insight">Core Insight<a class="anchor" href="#core-insight">#</a></h2> <p><strong>Wiki &gt; RAG.</strong> RAG rediscovers knowledge from scratch on every query — no accumulation. An LLM-maintained wiki compiles knowledge once and keeps it current. Cross-references, contradictions, and synthesis compound with every source added.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/mcp-protocol/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/mcp-protocol/</guid><description><h1 id="mcp-model-context-protocol">MCP (Model Context Protocol)<a class="anchor" href="#mcp-model-context-protocol">#</a></h1> <p>An open protocol that standardizes how applications provide context and tools to LLMs. Enables communication between AI agents and external services.</p> <h2 id="how-claude-code-uses-mcp">How Claude Code Uses MCP<a class="anchor" href="#how-claude-code-uses-mcp">#</a></h2> <p><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/claude-code/">claude-code</a> uses MCP as its primary tool integration mechanism:</p> <ul> <li>Connect to external data sources (Google Drive, Jira, Slack, custom tooling)</li> <li>Discover and install prebuilt plugins</li> <li>Create custom plugins</li> <li>Tool definitions deferred by default — only tool names consume context until Claude uses a specific tool (tool search)</li> </ul> <h2 id="usage-across-the-ecosystem">Usage Across the Ecosystem<a class="anchor" href="#usage-across-the-ecosystem">#</a></h2> <ul> <li><strong><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/claude-code/">claude-code</a></strong>: Primary tool integration. Supports prebuilt and custom plugins.</li> <li><strong><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/kiro/">kiro</a></strong>: <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/kiro-powers/">kiro-powers</a> contain curated MCP servers as part of expertise packages.</li> <li><strong><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a></strong>: Not directly mentioned, but the <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/plugin-system/">plugin-system</a> (hashicorp/go-plugin over gRPC) serves a similar role.</li> </ul> <h2 id="significance">Significance<a class="anchor" href="#significance">#</a></h2> <p>MCP is emerging as a shared standard across the coding agent ecosystem. Both Anthropic (Claude Code) and AWS (Kiro) use it, making it a potential interoperability layer between different agent tools. The <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent-skills-standard/">agent-skills-standard</a> (agentskills.io) is a complementary open standard: skills teach agents <em>how</em> to do things; MCP connects agents to external <em>tools and data</em>.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/multi-agent-orchestration/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/multi-agent-orchestration/</guid><description><h1 id="multi-agent-orchestration">Multi-Agent Orchestration<a class="anchor" href="#multi-agent-orchestration">#</a></h1> <p>The practice of coordinating multiple LLM-based <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/">agent</a>s to work on tasks concurrently, with isolation, specialization, and collaboration.</p> <h2 id="three-approaches-emerging">Three Approaches Emerging<a class="anchor" href="#three-approaches-emerging">#</a></h2> <h3 id="infrastructure-first-scion">Infrastructure-first: Scion<a class="anchor" href="#infrastructure-first-scion">#</a></h3> <p><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a> positions itself as a &ldquo;hypervisor for agents&rdquo; — providing the infrastructure layer (containers, isolation, lifecycle management) while treating higher-level concerns as orthogonal. Harness-agnostic. Emphasizes human interaction as imperative.</p> <h3 id="product-first-kiro-autonomous-agent">Product-first: Kiro Autonomous Agent<a class="anchor" href="#product-first-kiro-autonomous-agent">#</a></h3> <p><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/kiro/">kiro</a>&rsquo;s autonomous agent is an opinionated product — a <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/frontier-agent/">frontier-agent</a> that handles the full stack from task intake to PR creation. Coordinates specialized sub-agents internally. Emphasizes autonomy and independence.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/plugin-system/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/plugin-system/</guid><description><h1 id="plugin-system">Plugin System<a class="anchor" href="#plugin-system">#</a></h1> <p><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a> supports a plugin architecture built on <code>hashicorp/go-plugin</code> for extending system capabilities. Plugins communicate over gRPC.</p> <h2 id="plugin-types">Plugin Types<a class="anchor" href="#plugin-types">#</a></h2> <ul> <li><strong>Message Broker Plugins</strong>: Custom message delivery backends for agent notifications and structured messaging</li> <li><strong>Agent Harness Plugins</strong>: Custom <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/harness/">harness</a> implementations that integrate new LLM tools without modifying the core codebase</li> </ul> <h2 id="status">Status<a class="anchor" href="#status">#</a></h2> <p>Currently in foundational stage, with reference implementations available for both plugin types.</p> <h2 id="see-also">See Also<a class="anchor" href="#see-also">#</a></h2> <ul> <li><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a></li> <li><a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/harness/">harness</a></li> </ul></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/prompt-engineering-patterns/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/prompt-engineering-patterns/</guid><description><h1 id="prompt-engineering-patterns">Prompt Engineering Patterns<a class="anchor" href="#prompt-engineering-patterns">#</a></h1> <p>Structured techniques for crafting prompts within agentic skills, drawn from research and the <a href="https://blog.imfsoftware.com/llm-wiki/docs/sources/ten-pillars-agentic-skill-design/">ten-pillars-agentic-skill-design</a> framework.</p> <h2 id="chain-of-thought-cot">Chain of Thought (CoT)<a class="anchor" href="#chain-of-thought-cot">#</a></h2> <p>Structure prompts to encourage step-by-step reasoning. Break complex tasks into numbered steps. Wei et al. (2022).</p> <h2 id="react-pattern">ReAct Pattern<a class="anchor" href="#react-pattern">#</a></h2> <p>Integrate reasoning and acting in a loop (Yao et al., 2023):</p> <pre tabindex="0"><code>Thought: [reasoning about what to do] Action: [specific action to take] Observation: [result of the action] ... (repeat until task complete)</code></pre><h2 id="self-reflection-reflexion">Self-Reflection (Reflexion)<a class="anchor" href="#self-reflection-reflexion">#</a></h2> <p>Enable agents to learn from mistakes (Shinn &amp; Labash, 2023):</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/runtime-broker/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/runtime-broker/</guid><description><h1 id="runtime-broker">Runtime Broker<a class="anchor" href="#runtime-broker">#</a></h1> <p>In <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a>, a Runtime Broker is a compute node that registers with a <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/hub/">hub</a> to provide execution capacity for <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/">agent</a>s.</p> <h2 id="responsibilities">Responsibilities<a class="anchor" href="#responsibilities">#</a></h2> <ul> <li>Manages local lifecycle of agents dispatched from the Hub</li> <li>Handles workspace synchronization</li> <li>Template hydration (with content hash caching)</li> <li>Log streaming</li> <li>Reports status back via heartbeats and agent status updates</li> </ul> <h2 id="communication">Communication<a class="anchor" href="#communication">#</a></h2> <p>Connects to the Hub via:</p> <ul> <li><strong>Direct HTTP</strong> — when broker has a reachable endpoint</li> <li><strong>WebSocket Control Channel</strong> — when behind NAT/firewall</li> </ul> <h2 id="examples-of-broker-nodes">Examples of Broker Nodes<a class="anchor" href="#examples-of-broker-nodes">#</a></h2> <p>A server, laptop, or Kubernetes cluster — any machine that can run containers.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/runtime/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/runtime/</guid><description><h1 id="runtime">Runtime<a class="anchor" href="#runtime">#</a></h1> <p>In <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a>, the Runtime is the infrastructure layer responsible for executing <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/">agent</a> containers. Scion abstracts container execution behind a common interface.</p> <h2 id="supported-runtimes">Supported Runtimes<a class="anchor" href="#supported-runtimes">#</a></h2> <table> <thead> <tr> <th>Runtime</th> <th>Platform</th> <th>Notes</th> </tr> </thead> <tbody> <tr> <td>Docker</td> <td>Linux / macOS / Windows</td> <td>Default fallback. Supports remote Docker hosts</td> </tr> <tr> <td>Podman</td> <td>Linux / macOS</td> <td>Daemonless, rootless alternative</td> </tr> <tr> <td>Apple Container</td> <td>macOS</td> <td>Native Virtualization Framework, improved performance</td> </tr> <tr> <td>Kubernetes</td> <td>Any (via kubeconfig)</td> <td>Agents as Pods. Namespace isolation, resource specs, workspace sync via tar snapshots</td> </tr> </tbody> </table> <h2 id="runtime-selection">Runtime Selection<a class="anchor" href="#runtime-selection">#</a></h2> <p>Resolved by <code>GetRuntime</code> factory function:</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/skill-evaluation/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/skill-evaluation/</guid><description><h1 id="skill-evaluation">Skill Evaluation<a class="anchor" href="#skill-evaluation">#</a></h1> <p>Moving from &ldquo;it feels better&rdquo; to &ldquo;I have proof&rdquo; when measuring AI agent skill quality.</p> <h2 id="the-problem">The Problem<a class="anchor" href="#the-problem">#</a></h2> <p>LLM agents are non-deterministic. Manual testing captures one sample from a distribution. &ldquo;Vibes-based&rdquo; evaluation misses regressions, false positives, and edge cases. As Karpathy noted: &ldquo;The eval is often harder than the task itself.&rdquo;</p> <h2 id="three-tier-framework">Three-Tier Framework<a class="anchor" href="#three-tier-framework">#</a></h2> <table> <thead> <tr> <th>Tier</th> <th>Method</th> <th>Cost</th> <th>Frequency</th> <th>Catches</th> </tr> </thead> <tbody> <tr> <td>1</td> <td>Deterministic graders</td> <td>~$0</td> <td>Every commit</td> <td>Command execution, file existence, sequence, format</td> </tr> <tr> <td>2</td> <td>LLM-as-judge</td> <td>$0.01–0.20/eval</td> <td>PRs, nightly</td> <td>Code quality, conventions, readability (rubric-based)</td> </tr> <tr> <td>3</td> <td>Human review</td> <td>$0.50–5.00/eval</td> <td>Sparingly</td> <td>Calibration, edge cases, high-stakes decisions</td> </tr> </tbody> </table> <p>&ldquo;The best eval is one that actually gets run.&rdquo; — Anthropic</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/concepts/template/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/concepts/template/</guid><description><h1 id="template">Template<a class="anchor" href="#template">#</a></h1> <p>In <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a>, a Template is a versioned blueprint for creating an <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent/">agent</a>. It defines the base configuration, system prompt, tools, and initial state.</p> <h2 id="contents">Contents<a class="anchor" href="#contents">#</a></h2> <ul> <li><code>home/</code> directory tree — copied into the agent&rsquo;s home</li> <li><code>scion-agent.json</code> (or <code>.yaml</code>) — specifies <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/harness/">harness</a> type, env vars, volumes, command args, model overrides, container image, resource requirements</li> </ul> <h2 id="inheritance">Inheritance<a class="anchor" href="#inheritance">#</a></h2> <p>Templates support inheritance via a <code>base</code> field. Scion walks the chain and merges configurations bottom-up (base first, then overrides).</p> <h2 id="scopes">Scopes<a class="anchor" href="#scopes">#</a></h2> <ul> <li><strong>Project-level</strong>: <code>.scion/templates/</code></li> <li><strong>Global</strong>: <code>~/.scion/templates/</code></li> <li><strong>Hosted</strong>: Can be scoped as global, grove, or user, with visibility controls (private, grove, public)</li> </ul> <h2 id="management">Management<a class="anchor" href="#management">#</a></h2> <pre tabindex="0"><code>scion templates create scion templates clone scion templates list scion templates show scion templates update-default</code></pre><h2 id="defaults">Defaults<a class="anchor" href="#defaults">#</a></h2> <p>Scion ships default templates for each supported <a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/harness/">harness</a>: <code>gemini</code>, <code>claude</code>, <code>opencode</code>, <code>codex</code>. Users can create custom templates for specialized roles (e.g., &ldquo;Security Auditor&rdquo;, &ldquo;React Specialist&rdquo;).</p></description></item></channel></rss>