Back to posts
AINews

Turn the PDFs your agent guesses about into skills it loads on demand

An open-source converter with about 17,000 GitHub stars compiles a technical PDF, internal doc, or book into an agent skill — a 4K-token core plus 1K-token chapters loaded only when needed, instead of a 100K–250K-token context dump on every conversation. The install, the run, where chapter detection breaks, and when RAG is still the right call.

There is a category of document that your coding agent should treat as the authority and does not: the internal API standard, the style guide your team argues from, the compliance document, the technical book whose framework you want applied to your own work. Ask about it, and the model answers from training data — close enough to sound right, wrong on the specifics that matter.

The two usual fixes both cost more than they look. Pasting the PDF into context works once, then bills you again on every conversation: a 244-page book like Think Python is roughly 119K tokens, and a longer technical book runs 250K+ — paid per turn, while pushing the rest of your context out. Building RAG over the documents works too, but now you own an embedding pipeline, a vector store, and a retrieval quality problem, for what might be four PDFs that change twice a year.

There is a third route that fits that middle ground: compile the document into a skill. book-to-skill is an MIT-licensed converter that does exactly this, and it has moved fast enough to be hard to ignore — about 17,000 GitHub stars, roughly 4,600 of them added this week. It takes a PDF, EPUB, DOCX, Markdown file, or a whole folder, and produces a structured agent skill: a SKILL.md core of about 4,000 tokens holding the document's mental models and a chapter index, plus per-chapter files of about 1,000 tokens each that the agent opens only when a question needs them. Answering one targeted question then costs about 5,000 tokens of context instead of the full dump. The project measures that gap with tiktoken at 24–51× depending on book size, and the numbers are reproducible with a script in the repo — but the shape of the win matters more than the exact multiple: the context-dump cost recurs on every turn, the compiled skill's does not.

Install and run

The converter is itself a Claude Code skill (it also works with GitHub Copilot CLI and Amp). Installation is a clone into your personal skills directory:

git clone https://github.com/virgiliojr94/book-to-skill.git ~/.claude/skills/book-to-skill

Requirements are light: Python 3, plus format-specific extractors the skill offers to install when missing. Extraction runs locally with no API key — Calibre is the only hard dependency, and only for MOBI/AZW ebooks. Then, inside Claude Code:

# One document
/book-to-skill ~/docs/api-standard.pdf api-standard

# A whole folder, compiled into one unified skill
/book-to-skill ~/workspace/project-docs/ project-knowledge

# Fold a new document into an existing skill later
/book-to-skill ~/docs/new-revision.pdf ~/.claude/skills/api-standard

Two stages run under that command. First a deterministic Python extractor pulls clean text and chapter structure out of the file, with no model involved. We ran it on a 15-page arXiv paper to check: it reported 6,151 words, roughly 8K tokens, and warned that no table of contents was detected, so chapter mapping would rely on heading scans. Then the agent distills that text into the skill files, following the extraction spec in the converter's own instructions, which pushes for named frameworks, principles, techniques, anti-patterns, a glossary, and a cheatsheet rather than a chapter-by-chapter summary.

The result lands as a normal skill directory, and current Claude Code picks up new skills within the running session, no restart needed. From there it works like any skill you wrote by hand:

/api-standard                      # load the core mental models
/api-standard error envelopes      # find and explain one topic
/api-standard ch05                 # open a specific chapter

Where it breaks, and what to check before trusting it

Chapter detection needs real chapter markers. The extractor segments on Chapter N-style headings or a detectable table of contents. The project is upfront that Pro Git, which heads chapters with bare section titles, does not auto-segment — the same will be true of most internal docs written in a wiki export. The skill still generates, but as a flatter structure; run the analyze-only mode first (/book-to-skill with "analyze" in the request) to see what it found before committing.

Prose mode flattens tables and code. The default pdftotext extraction is instant but drops structure: on a 103-page technical PDF the project measured zero tables and zero code blocks surviving. Passing --mode technical routes extraction through Docling instead, which preserved 48 tables and 36 code blocks from the same file at a cost of roughly 1.5 seconds per page. For an API reference or anything with code samples, technical mode is the difference between a usable skill and a lossy one.

Spot-check the distillation. The compile step runs through the model, so the skill inherits the model's failure modes. Pick three specifics you know cold from the source document and ask the freshly generated skill about them. The repo ships tools/validate_skill.py, which audits the generated SKILL.md against the Agent Skills spec for your host — worth running, but it checks format, not fidelity. Fidelity is your three questions.

The description decides whether it ever fires on its own. A generated skill triggers automatically only when its description frontmatter matches what you are asking; the combined description text is truncated at 1,536 characters in the skill listing. If the skill works when invoked by name but never volunteers, the description is the problem — we covered the diagnosis and fix in a previous post.

When this is the wrong tool

A compiled skill is a snapshot of a stable document. That framing tells you when to reach for something else. A corpus that is large, changes weekly, or needs cross-document search — support tickets, a living wiki, hundreds of contracts — wants retrieval, not compilation; the decision order between a prompt, a skill, RAG, and a knowledge base is laid out in our decision guide, and the RAG-versus-index distinction in its companion. And two constraints from the project itself are worth honoring: extraction is local, but the distillation pass sends the text through whatever model your agent runs on, so check your provider terms before feeding it confidential material; and a skill generated from a copyrighted book is for your own use, not for redistribution.

The sweet spot is the document set almost every team has and almost no team has wired in: a handful of authoritative, slow-moving references that the agent should consult instead of improvise around. Compiling those costs one command each, and the next time the agent answers a question about your API's error envelope, it opens chapter five instead of guessing.

Get the next post when it ships

One email on Sunday with the new post and a short list of what shipped that week — new guides, tool updates, and a couple of links worth reading.