💡 Analyses on LLM Wiki — Agentic AI Landscapehttps://blog.imfsoftware.com/llm-wiki/docs/analyses/Recent content in 💡 Analyses on LLM Wiki — Agentic AI LandscapeHugoen-us<link>https://blog.imfsoftware.com/llm-wiki/docs/analyses/cross-source-themes/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/analyses/cross-source-themes/</guid><description><h1 id="cross-source-theme-analysis">Cross-Source Theme Analysis<a class="anchor" href="#cross-source-theme-analysis">#</a></h1> <p>16 sources, 8 tools, 2 standards, 3 methodologies, 1 practitioner account, 2 skill/eval resources. Here are the themes that appear across 3+ sources independently — not because they reference each other, but because they converged on the same ideas.</p> <blockquote class='book-hint '> <p><strong>Note</strong>: This analysis was originally written against 11 sources. The 5 newest sources (Paperclip, Spec Kit, BMad Method, Anthropic Eval Guide, Promptfoo) strengthen existing themes — particularly Theme 3 (human-in-the-loop spectrum) and Theme 7 (evaluation). A full refresh is recommended when the wiki reaches 20+ sources.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/analyses/how-to-eval-a-skill/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/analyses/how-to-eval-a-skill/</guid><description><h1 id="how-to-eval-a-skill-practical-guide">How to Eval a Skill (Practical Guide)<a class="anchor" href="#how-to-eval-a-skill-practical-guide">#</a></h1> <p>Anthropic&rsquo;s prompt evals measure whether a prompt produces good output. Skill evals are harder because a skill has more surface area: it needs to trigger correctly, execute the right steps, use the right tools, produce the right output, and NOT trigger on the wrong inputs.</p> <p>This guide maps Anthropic&rsquo;s eval methodology onto skills, drawing from the wiki&rsquo;s sources.</p> <hr> <h2 id="the-key-difference-prompts-vs-skills">The Key Difference: Prompts vs. Skills<a class="anchor" href="#the-key-difference-prompts-vs-skills">#</a></h2> <table> <thead> <tr> <th></th> <th>Prompt Eval</th> <th>Skill Eval</th> </tr> </thead> <tbody> <tr> <td><strong>What you test</strong></td> <td>Does this prompt produce good output?</td> <td>Does this skill trigger, execute, and produce correctly?</td> </tr> <tr> <td><strong>Input</strong></td> <td>A prompt + expected output</td> <td>A prompt + context + expected behavior chain</td> </tr> <tr> <td><strong>Failure modes</strong></td> <td>Bad output</td> <td>Wrong trigger, wrong steps, wrong tools, bad output, false positive activation</td> </tr> <tr> <td><strong>Non-determinism</strong></td> <td>Output varies</td> <td>Trigger, routing, tool selection, AND output all vary</td> </tr> </tbody> </table> <p>A skill eval must test the full chain: <strong>routing → activation → execution → output → side effects</strong>.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/analyses/key-insights-agentic-landscape/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/analyses/key-insights-agentic-landscape/</guid><description><h1 id="key-insights-the-agentic-ai-landscape-april-2026">Key Insights: The Agentic AI Landscape (April 2026)<a class="anchor" href="#key-insights-the-agentic-ai-landscape-april-2026">#</a></h1> <p>Synthesized from 16 sources across this wiki. This analysis captures the patterns, tensions, and emerging consensus visible when you look across the entire landscape rather than at any single tool.</p> <hr> <h2 id="1-five-layers-are-emerging">1. Five Layers Are Emerging<a class="anchor" href="#1-five-layers-are-emerging">#</a></h2> <p>The landscape has organized into five distinct layers:</p> <table> <thead> <tr> <th>Layer</th> <th>Representative</th> <th>Core Bet</th> </tr> </thead> <tbody> <tr> <td><strong>Company</strong></td> <td><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/paperclip/">paperclip</a></td> <td>Orchestrate agents into companies with org charts, budgets, governance.</td> </tr> <tr> <td><strong>Methodology</strong></td> <td><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/spec-kit/">spec-kit</a>, <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/bmad-method/">bmad-method</a></td> <td>Structure the development process. Specs before code, or adaptive agile workflows.</td> </tr> <tr> <td><strong>Infrastructure</strong></td> <td><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/scion/">scion</a> (GCP)</td> <td>The agent runtime is the hard problem. Be a hypervisor. Stay agnostic.</td> </tr> <tr> <td><strong>Product</strong></td> <td><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/kiro/">kiro</a> (AWS)</td> <td>Ship an opinionated end-to-end agent. Autonomy and scale matter most.</td> </tr> <tr> <td><strong>Tool</strong></td> <td><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/claude-code/">claude-code</a> (Anthropic), <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/fabric/">fabric</a></td> <td>Make the individual agent excellent. Let users compose upward.</td> </tr> </tbody> </table> <p>The methodology layer is new — <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/spec-kit/">spec-kit</a> (&ldquo;specs before code&rdquo;) and <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/bmad-method/">bmad-method</a> (&ldquo;expert collaboration over autopilot&rdquo;) represent two competing philosophies for structuring AI-assisted development. <a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/paperclip/">paperclip</a> adds the company layer above everything, orchestrating agents into organizations with budgets and governance.</p></description></item><item><title/><link>https://blog.imfsoftware.com/llm-wiki/docs/analyses/ten-pillars-evidence-map/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://blog.imfsoftware.com/llm-wiki/docs/analyses/ten-pillars-evidence-map/</guid><description><h1 id="evidence-map-supporting-the-ten-pillars-framework">Evidence Map: Supporting the Ten Pillars Framework<a class="anchor" href="#evidence-map-supporting-the-ten-pillars-framework">#</a></h1> <p>This analysis maps each pillar from <a href="https://blog.imfsoftware.com/llm-wiki/docs/sources/ten-pillars-agentic-skill-design/">ten-pillars-agentic-skill-design</a> against real-world evidence collected across 11 sources in this wiki. Your paper acknowledged &ldquo;no original controlled study&rdquo; as a limitation — the wiki now provides post-hoc validation from production implementations.</p> <hr> <h2 id="pillar-1-architecture-and-structure">Pillar 1: Architecture and Structure<a class="anchor" href="#pillar-1-architecture-and-structure">#</a></h2> <p><strong>Your claim</strong>: Organize content into clearly defined sections — metadata, interfaces, core logic, workflows, configuration.</p> <p><strong>Supporting evidence</strong>:</p> <ul> <li><strong><a href="https://blog.imfsoftware.com/llm-wiki/docs/concepts/agent-skills-standard/">agent-skills-standard</a></strong> codified this into a formal spec: SKILL.md with YAML frontmatter (name, description, license, compatibility, metadata, allowed-tools) + markdown body + optional scripts/references/assets directories. This is now an open standard at agentskills.io.</li> <li><strong><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/claude-code/">claude-code</a></strong> implements it: <code>.claude</code> directory with CLAUDE.md, <code>.claude/rules/</code>, <code>.claude/skills/</code>, <code>.claude/agents/</code>. Hierarchical, scoped (org → project → user → local).</li> <li><strong><a href="https://blog.imfsoftware.com/llm-wiki/docs/entities/pai/">pai</a></strong> takes it further: USER/ vs SYSTEM/ separation. Six layers of customization (identity, preferences, workflows, skills, hooks, memory). Upgrade-safe architecture.</li> <li><strong><a href="https://blog.imfsoftware.com/llm-wiki/docs/sources/skills-pipeline-sleestk/">skills-pipeline-sleestk</a></strong> follows the spec exactly: each skill is a directory with SKILL.md + references/ subdirectory.</li> </ul> <p><strong>Strength</strong>: Strong. Multiple independent implementations converged on the same structure. The Agent Skills spec formalizes what your paper recommended.</p></description></item></channel></rss>