pdf-explore

# PDF Explore — navigate a PDF too big to embed

A 50-page PDF read in full is ~200K tokens of context. When the answer
draws on several sections at once (summarize the methods; compare section
3 and section 5), or when the answer is "every page" (list all the
datasets / citations / figures / benchmarks mentioned anywhere in this
document), reading the whole thing page-by-page is the expensive way to
get it. This skill parses the PDF **once** into persistent text with a
deterministic Python kernel, then lets you **narrow** — by outline, by
lexical scan, by regex — and read only the pages you actually need,
reasoning over them yourself. Nothing you read vanishes: it is ordinary
text and ordinary files.

## Setup (any agent, no API key)

This is a **pure skill** — `kernel.py` is deterministic Python and *you*
(the base model) do all the reasoning. There is no `host` runtime and no
LLM API. Load the helpers once per session in a Python cell:

```python
exec(open("skills/claude-science/pdf-explore/kernel.py").read())
# adjust the path to wherever this skill is installed
```

Nothing auto-loads it outside Claude Science. Then call the helpers
directly (no import). If a helper is "not defined", you haven't `exec`'d
`kernel.py` yet — go back and run the line above.

Dependencies: `pip install pypdfium2 pillow` (pillow does the PNG encoding
for `mode="image"`; it is not pulled in by the pypdfium2 wheel).

## Which helper

| | when | returns |
|---|---|---|
| **`pdf_pages(path, pages=[...], mode="text")`** | you need several pages/sections *at the same time* — summaries, comparisons, anything where the answer draws on more than one range | `[{page, text, n_chars}, ...]` — persistent text; write to a file then read it |
| **`pdf_outline(path)`** | structured doc (paper, report, book) with an **embedded** TOC | `[{page, heading, level}, ...]` — the embedded outline, or `[]` if the PDF has none |
| **`pdf_scan(path, query, top_k)`** | narrow a long doc to a handful of candidate pages for a query | `{hits: [{page, score, matched, text}], n_scanned}` — a **lexical pre-filter** (no LLM); *you* read the shortlist and judge relevance |
| **`pdf_grep(path, pattern)`** | **exhaustive** regex sweep (DOIs, accession ids, every "Table N", emails) | `[{page, matches, lines?}, ...]` — every match with its page |
| **`pdf_pages(mode="image", dpi=200)`** | read a small value, axis label, or legend off a **figure** | `[{page, image_path}, ...]` — open the PNG with your agent's image tool |

These come from `kernel.py` — load it via `exec` once per session (see
**Setup**), then call directly. `pdf_resolve(path)` normalizes a path
(a workspace path or a `~/`-expanded path); the helpers call it
internally, so `path` can be either form.

Note: the default backend is pypdfium2 (Google PDFium; permissive
Apache-2.0/BSD-3-Clause). PyMuPDF is honored as a fallback if already
installed, but it is AGPL-3.0-licensed (commercial licenses available from
Artifex): if you embed it in a network-accessible service, AGPL's
source-sharing terms apply to that service.

## Recipe — pull the sections you need as persistent text (synthesis)

For "summarize the methods" / "compare section 3 and section 5" / anything
where the answer draws on several page ranges at once, pull **all** the
pages you need in **one** python call, write them to a file, then read
that file:

```python
wanted = [5, 21,22,23,24,25, 62,63,64, 124,125,126]  # from pdf_outline
with open("sections.txt", "w") as f:
    for p in pdf_pages("paper.pdf", pages=wanted, mode="text"):
        f.write(f"\n── page {p['page']} ──\n{p['text']}")
import os; print(f"wrote {os.path.getsize('sections.txt'):,} bytes")
```

Then read `sections.txt` with your agent's file-read tool (in chunks if
it's large) and write the answer from that. It's ordinary text — one
parse, and you reason over it directly. **Don't `print()` a full chapter**
into the cell output: most agents spill large cell output to disk and make
you re-read it anyway, so writing + reading a file costs the same two steps
without the wasted preview. (For a quick look at ≤5 pages, printing is
fine.)

Text is ~800 tokens/page vs ~4,000 tokens/page as vision, and you pay it
once. Find the page numbers from `pdf_outline` (below) or the paper's own
table of contents first.

## Recipe — navigate by outline (try this first)

```python
for e in pdf_outline("report.pdf"):
    print(f"p{e['page']:>3} {'  ' * (e['level'] - 1)}{e['heading']}")
# → then pull the section you want with pdf_pages(pages=[...])
```

Free and instant when the PDF has an embedded outline (most LaTeX-compiled
papers do). `pdf_outline` reads the **embedded** TOC only — if the PDF has
none it returns `[]`. In that case **build the outline yourself**: pull the
first handful of pages (or a stride sample of a long doc) as text with
`pdf_pages` and pick out the headings by reading them. For a semantic
question the outline doesn't obviously answer ("where do they discuss
limitations"), fall through to `pdf_scan`.

## Recipe — find the pages relevant to a query

`pdf_scan` ranks pages by **lexical** overlap with your query — it is a
cheap **pre-filter**, not a relevance judgment. It narrows a long document
to a handful of candidate pages; *you* then read those pages and decide
which actually answer the question.

```python
r = pdf_scan("paper.pdf", query="batch-effect correction methods", top_k=8)
for h in r["hits"]:
    print(f"p{h['page']}  score={h['score']:.2f}  matched={h['matched']}")
print(f"[{r['n_scanned']} pages scanned]")
```

Then read the shortlist's text and make the final call yourself:

```python
for h in r["hits"]:
    print(f"\n── page {h['page']} ──\n{h['text'][:2000]}")
```

Keep the pages that genuinely address the query; discard lexical false
positives (a page that merely says "batch" in another sense). Because the
ranking is lexical, a synonym the query didn't use won't score — so lean on
your own reading, broaden `top_k` if the shortlist looks