Why verified skills beat generated skills, docs and bigger models
Any agent can write fluent Nim, and fluent advice about Nim. Neither is checked until the compiler runs it. In Verified Nim Skills, the compiler has already run every testable claim, and it runs them again on new builds.
The problem: Nim that looks right
Code models are weaker on low-resource languages, and Nim is one.9 The dangerous mistakes compile: ownership hooks that crash on legal reuse, copy guards that miss aliasing, fuzz harnesses that die on an empty input. They pass the happy-path test and fail in production.
An agent cannot catch these by reading. Someone has to run the code. See three real examples →
Self-written skills don’t check themselves
Claude Code and Codex both ship skill creators.1 The output reads well. It is not verified.
SkillsBench measured this directly. Curated skills raised task success by about 16 percentage points. Skills that models wrote for themselves gave no average gain.2 In the authors’ words, models “cannot reliably author the procedural knowledge they benefit from consuming.”
Generated skills improve only with a test loop.3 For Nim, that loop means test programs run against the real compiler, in debug and release, re-run on every new build. That loop is our product.
Our own carefully written drafts needed a correction or a caveat on about one claim in four. A self-generated skill carries the same kind of errors, unchecked. What the audit found.
Why our skills are small, targeted and tested
Bloated, unverified skill packs make agents worse:
- In SkillsBench, focused sets of two or three skills beat comprehensive bundles, which came out slightly negative. 16 of 84 tasks got worse with skills.2
- An ETH study found generic context files raised inference cost by over 20% without improving success.4 Model quality also drops as input grows.5
Each of our skills covers one topic, so the agent loads one or two, not twenty. Every edit is evidence-gated: we tighten or delete before we add. Trimming one verified skill that made agents over-measure cut mean agent time per task by 44%, at the same pass rate.
Skills also have to load. Agents skip unforced skills often,67 so onboarding includes a two-line AGENTS.md pointer that tells the agent to load them.
Docs say what exists, not what is safe
Context7 indexes Nim and Nimony, and nimlangserver has an MCP mode.8 Keep them. They answer “what API exists”.
They don’t answer “will this crash”. No doc page lists compiler bugs nobody has found yet. Our verification found one, a release build that silently drops errors, and got it fixed upstream. Raw docs also add the context volume the studies above warn about.
Keep a docs MCP for lookup, add Verified Nim Skills for what is safe, and put a two-line pointer in AGENTS.md so the agent loads them. Copy the pointer.
Cheaper models become viable
A bigger model costs more on every call. Claude Opus 5.5 lists at $4 input and $20 output per million tokens;11 DeepSeek-V4-Flash-class models list at about $0.14 and $0.28.10 And no model, however large, has seen the compiler build you installed last week.
With our skills, a low-cost model passed 18 of 20 hard Nim tasks. Without them, it passed 3 of 20. Each skill-guided attempt used about twice the tokens, yet each solved task cost about $0.05 instead of $0.21. A larger model without skills passed 0 of 6. That is why we build and test the skills on low-cost, fast models. See all 40 blind trials. Published work points the same way: smaller models with good skills compete with larger models without them.26
Why a subscription, not a download
Compilers move, and correct guidance goes stale without anyone touching it. A moved library path broke all 12 tests of one skill during our own verification. Newer rules required an explicit std/syncio import. An API a skill named had been renamed.
The test suite turns that drift into a list of failures we fix before your agent hits them. You pay for that stream. Cancel, and you keep what you received.
Sources
- Anthropic: Improving skill-creator; OpenAI: Build skills. ↩
- SkillsBench, arXiv:2602.12670 (v1 figures: 24.3% → 40.6% curated; −1.3pp self-generated; v1 HTML). Not a Nim-specific study. ↩
- SkillAxe, arXiv:2606.10546: evaluation-guided refinement closed 47–67% of the gap to human-written skills. ↩
- Evaluating AGENTS.md (ICLR 2026), arXiv:2602.11988. ↩
- Chroma: Context rot. ↩
- Tessl: A proposed framework for evaluating skills. ↩
- Vercel: AGENTS.md outperforms skills in our agent evals. ↩
- Context7 library search: nimony; nimlangserver MCP. ↩
- MultiPL-T, arXiv:2308.09895; survey of low-resource code generation, arXiv:2410.03981. ↩
- Aggregator figure, October 2026: CloudZero: DeepSeek pricing. ↩
- Anthropic pricing, October 2026. ↩