SKILL 24 / DATA · DEEP-DIVE

Plan-Validate-Execute

Make the model hand in a plan a script can reject · The real validators inside the official pdf / docx / xlsx skills · Four validators, four different exit-code contracts

2026-09-25 · BigCat

Last issue was about which layer a reference belongs in. This one is about what actually goes in that scripts/ directory — the answer is validators. Every validation script in the official pdf, docx, xlsx and pptx skills, opened one by one, contradictory exit codes included.

Prerequisite → Skill 23, domain-split reference structure (the scripts/ slot of that three-layer split)

// WHY THIS MATTERS

Updating 50 form fields, redlining a contract, laying a grid of formulas into a financial model — what these share is that the model thinks and edits at the same time, and when it goes wrong you cannot tell which step went wrong. The docs have a name for this class of work: plan-validate-execute. Have the model write down what it intends to do as a structured file, let a script tear that file apart, and only touch the original once the file passes. Skill authoring best practices devotes a whole section to it, titled Create verifiable intermediate outputs. This issue does not stop at the doc — it opens the eight or nine validation scripts that really exist in anthropics/skills and looks at how each writes its errors, when it exits 1, and when it reports failures and exits 0 anyway.

// 01

What it kills for you: editing while still thinking, in bulk

Claim: what this pattern buys is not correctness, it is a rollback-able train of thought. Rewriting a JSON plan costs nothing; a corrupted original may not.

The scope the docs pin down

That section names four situations verbatim: Batch operations, destructive changes, complex validation rules, high-stakes operations. The worked example is concrete — updating 50 PDF form fields from a spreadsheet, where without validation the model "might reference non-existent fields, create conflicting values, miss required fields, or apply updates incorrectly." The flow is written as five steps: analyze → create plan file → validate plan → execute → verify.

Install (two plugins; all four skills here are in the first)

# pdf / docx / xlsx / pptx — every subject of this issue
/plugin marketplace add anthropics/skills
/plugin install document-skills@anthropic-agent-skills

# skill-creator / mcp-builder and others (used in section 03)
/plugin install example-skills@anthropic-agent-skills

You do not need the plugin: these four directories drop straight into ~/.claude/skills/ or .claude/skills/, and the Python files under scripts/ are standalone runnable anyway. When not to install: if you never touch Office or PDF files, all four are pure metadata tax. The code under scripts/ is still worth reading, which is what this issue is for.

The first usage you can run today: fill a fillable PDF

skills/pdf/forms.md opens with CRITICAL: You MUST complete these steps in order. Do not skip ahead to writing code. Four commands, run from skills/pdf/:

# 1. Probe: does the form have fillable fields (the doc drops the .py here, see 04)
python scripts/check_fillable_fields.py form.pdf

# 2. Analyze: extract the real fields as JSON — field_id / page / rect / type
python scripts/extract_form_field_info.py form.pdf field_info.json

# 3. Plan: you write field_values.json; field_id must match step 2
#    [{"field_id":"last_name","description":"...","page":1,"value":"Simpson"}]

# 4. Execute: the script checks ids and value domains first, exit 1 and no output
python scripts/fill_fillable_fields.py form.pdf field_values.json out.pdf

Step 4 is the whole pattern in miniature: fill_fillable_fields.py checks your plan against field_info.json before writing a single byte, and its errors read like ERROR: Invalid value "X" for radio group field "citizenship". Valid values are: [...]. It enumerates the legal values, which is exactly the docs' implementation tip made real: make errors verbose enough for the model to fix itself.

Sources · Anthropic · Skill authoring best practices, the Create verifiable intermediate outputs and Workflows and feedback loops sections (retrieved 2026-09) · script behavior read from the main-branch source of skills/pdf/scripts, 2026-09
// 02

Mechanism: turn "what I intend to do" into something set-comparable

Claim: this pattern holds on exactly one precondition — every key in the plan can be set-compared against a list that really exists. field_id can be. "Make the heading sound more professional" cannot.

The official directories hold two variants; don't mix them

A. Validate the plan (plan-first) — pdf extract_form_field_info.py -> you write field_values.json -> fill_fillable_fields.py checks ids/domains, exit 1, original untouched extract_form_structure.py -> you write fields.json (with bounding boxes) -> check_bounding_boxes.py validates it standalone -> fill_pdf_form_with_annotations.py -> convert_pdf_to_images.py to look at the result B. Validate the artifact (execute-then-validate) — docx / pptx / xlsx edit word/document.xml -> zip -> office/validate.py (XSD + rels + types) openpyxl writes formulas -> recalc.py (LibreOffice recomputes all) pptxgenjs writeFile -> office/validate.py (charts + slide XML)

The difference is the moment an error surfaces. Variant A costs you an enumerable list up front (the PDF's field table, a page's row boundaries); without that list the plan's keys have nothing to compare against and the pattern decays into ceremony. Variant B needs no list, and the cost is that the original has already been modified — docx covers for that with --original: python scripts/office/validate.py out.docx --original doc.docx baselines the XSD checks against the source, so errors the original already had do not read as yours, and --auto-repair fixes common issues.

Why the intermediate output must be JSON and not prose

Easy to get backwards: the intersection check in check_bounding_boxes.py uses both entry_bounding_box and label_bounding_box. Write the entry box and omit the label box and it can no longer catch text colliding with its own label — which happens to be the commonest fault of all. One missing field in the plan's schema blinds the validator in one eye, and it will not mention that it is blind.
// 03

Real usage and limits: five chains, each with its commands

Claim: where a validator sits is not arbitrary. Normalization comes before the plan, cleanup comes after the structure is final, and looking at the rendered output always comes last — get the order wrong and every step validates something that no longer holds.

1. Non-fillable PDFs: even the coordinates get validated first

python scripts/extract_form_structure.py in.pdf form_structure.json  # labels/lines/checkboxes
# you write fields.json; pdf_width/pdf_height means PDF coords, image_* means pixels
python scripts/check_bounding_boxes.py fields.json                    # validate, then fill
python scripts/fill_pdf_form_with_annotations.py in.pdf fields.json out.pdf
python scripts/convert_pdf_to_images.py out.pdf verify/              # look at it last

The same directory holds create_validation_image.py, which draws entry boxes in red and label boxes in blue over the page image. forms.md never mentions it, yet it is the most direct tool there is for letting the model validate a plan with its eyes. When not to use this chain: if the PDF has real fillable fields (run check_fillable_fields.py to find out), computing coordinates by hand is self-inflicted.

2. docx redlining: normalize before you write the plan

unzip -q doc.docx -d unpacked/ && find unpacked -type l -delete
python scripts/merge_runs.py unpacked/     # coalesce runs, or text isn't contiguous in the XML
# edit unpacked/word/document.xml — do NOT reformat or pretty-print
(cd unpacked && zip -Xr ../out.docx .)
python scripts/office/validate.py out.docx --original doc.docx --author "Reviewer"

What merge_runs.py kills is Word slicing one sentence across a pile of <w:r> elements, so a phrase you can see in the document does not exist as a contiguous string in the XML. It is a precondition of the plan: skip it and the anchor strings in your plan match nothing. --author is a neat plan check in its own right — it reports text you changed without wrapping it in <w:ins>/<w:del>, a mistake the docs describe as easy to do by accident and invisible in the accepted view. For a clean copy use accept_changes.py, though it and pandoc --track-changes=accept share one known defect (a deleted paragraph mark leaves an emptied paragraph behind), which the SKILL.md documents itself.

3. pptx: clean up only after the structure is final

In skills/pptx/scripts, the note on clean.py is one hard constraint: Run after <p:sldIdLst> is final — it deletes slides and media nothing references any more, so running it while the structure is still moving may delete what you need next. thumbnail.py is the look-first step for choosing layouts, but its second argument is mandatory: the default prefix thumbnails lets a second deck in the same directory silently overwrite the first deck's grids.

4. xlsx: the validator is mandatory, and it only proves things evaluate

recalc.py is the only validator in this issue the docs call mandatory — openpyxl writes formulas with no cached values, so until you recalculate every formula cell reads back as None to pandas. But the SKILL.md immediately hits the brake: A green recalc proves your formulas evaluate, not that they are right. An off-by-one range yields a clean, error-free file full of wrong numbers. The remedy it offers is behavioral: write two or three formulas and check the values they pull before building out the grid.

5. Turning the pattern on skills themselves

The SKILL.md of skill-creator gives two commands: python -m scripts.run_loop --eval-set <trigger-eval.json> --skill-path <skill> and python -m scripts.package_skill <path/to/skill-folder>; scripts/ also holds quick_validate.py, run_eval.py and aggregate_benchmark.py. The order it insists on is GENERATE THE EVAL VIEWER BEFORE evaluating inputs yourself — freeze the criteria into something runnable before you start judging. When not to bother: a single SKILL.md you alone use, where writing evals costs more than it returns.

6. The model will skip the validator — that is what hooks are for

Everything above depends on the model choosing to run things in order. To pin it down, move it outside the model's judgment: Claude Code hooks fire PreToolUse before a tool call and exit code 2 blocks it outright (verbatim: On events that can block, exit 2 blocks whether or not you print JSON), and the reason can be returned precisely with permissionDecision: "deny" plus permissionDecisionReason. PostToolUse cannot block — the tool already ran — and exit 2 there only hands stderr to the model as a warning. When not to use hooks: one-off tasks, or checks that need judgment. A hook is a deterministic gate, not a reviewer (see Skill 27).

Three criteria for choosing among the chains above

  1. Is there an oracle that does not depend on the model's judgment? An XSD, a field table, a formula error code all are. With no oracle, do not build this: you are only having the model grade its own paper, one extra round of tokens later.
  2. Can the plan's keys be set-compared against a real list? Yes (field_id, cell addresses, slideN.xml) → go variant A and validate the plan. Only "looks about right" (copy, visual balance) → go variant B or just look at the render; don't force a JSON into existence.
  3. Is failure reversible? Irreversible or bulk → variant A; the docs' four situations are exactly this criterion. Single file, re-runnable, five minutes to redo → variant B suffices, which is why docx and xlsx are variant B end to end.
// 04

Costs and alternatives: four official validators, four exit-code contracts

Claim: this is the one thing to take away — you cannot assume a validator exiting 0 means it passed. Four scripts in one repository answer that question differently, and none of them says so anywhere prominent.

The exit-code table, read off the source

script on finding problems exit ────────────────────────────────────────────────────────────────────── fill_fillable_fields.py prints ERROR, writes nothing 1 OK fail-closed office/validate.py reports the failures 1 OK fail-closed recalc.py JSON status: "errors_found" 0 !! silent check_bounding_boxes.py prints "FAILURE: ..." 0 !! silent recalc.py's last line is: sys.exit(1 if "error" in result else 0) -> only "nothing was recalculated at all" (an error key) exits 1; a workbook full of #REF! exits 0. check_bounding_boxes.py never calls sys.exit(1): fully overlapping boxes exit 0.

The consequence is concrete: write python scripts/recalc.py out.xlsx && echo OK in CI or a hook and a workbook riddled with #NAME? sails through green. The xlsx SKILL.md does warn about this — errors_found exits 0, so never treat a clean exit as a clean workbook — but the identical behavior in check_bounding_boxes.py carries no documented warning at all; you only see it by reading the source. Wire either into automation and you must parse stdout, not just check $?.

Two places where copying the doc fails

1. The first command in forms.md is missing its extension. It reads python scripts/check_fillable_fields <file.pdf>; the real filename is check_fillable_fields.py (all seven other scripts in that directory carry .py). Copied verbatim it is a No such file or directory.
2. Not one of the example script names in best-practices exists. That section's PDF workflow uses analyze_form.py, validate_fields.py, fill_form.py and verify_output.py; the real skills/pdf/scripts/ holds extract_form_field_info.py, check_bounding_boxes.py, fill_fillable_fields.py and convert_pdf_to_images.py. The doc teaches the pattern, not the commands — copy commands out of ls, never out of prose.

Three bills to pay

What you can do instead

  1. Variant B: just do it, then validate. The official docx, xlsx and pptx skills all work this way — they have no plan file at all. For re-runnable single-file work that is enough.
  2. Move the gate into a hook. PreToolUse exit 2, no opportunity for the model to skip (Skill 27).
  3. Use the tool's own dry run. If something upstream can do the --check for you, don't invent an intermediate format — that's one less schema to maintain.
  4. Have a second instance review it. A second pair of eyes with isolated context, at the price of coordination overhead (Skill 26).
  5. Just look at it. convert_pdf_to_images.py, thumbnail.py, soffice.py --convert-to pdf plus pdftoppm — all four official skills keep this route, because some things only an eye can check.
One last honest cost: this pattern improves when you find an error, not how many there are. A fields.json that passes every check can still put the customer's name in the date-of-birth box, because no script knows what belongs in that box. Validators guard the format, not the intent; the intent still needs your own eyes.

// ONE-PAGE PICK LIST

Read these three first: (1) skills/pdf/forms.md — the only file in the repo that writes plan-validate-execute out end to end, both chains (fillable fields and coordinate annotation). (2) fill_fillable_fields.py — 98 lines that make "check before writing" and "errors list the legal values" into the shortest possible example. (3) the Create verifiable intermediate outputs section — where the pattern is named and its four situations bounded.

Then as needed: check_bounding_boxes.py (how a standalone plan validator is written — note it exits 0) · xlsx/scripts/recalc.py (artifact validation, a JSON contract, and the exit-code trap) · docx/scripts/office/validate.py (--original baselining, --auto-repair, the --author redline check) · merge_runs.py (the normalization that precedes a plan) · pptx/scripts (the ordering constraint on clean.py, the prefix trap in thumbnail.py) · skill-creator (run_loop.py and package_skill.py, the pattern turned on skills themselves) · Claude Code hooks (the PreToolUse exit-2 hard gate).

Explicitly don't: force a plan file onto a task with no oracle · trust a validator's exit code without reading its stdout · copy script names out of prose (two are confirmed not to match the repo) · run clean.py before <p:sldIdLst> is final · write docx anchor plans before merge_runs.py · read "recalc came back green" as "the numbers are right".

// GLOSSARY

plan-validate-execute
The docs' name for the pattern: the model writes intended changes as a structured file, a script validates that file, execution follows only on a pass, and the artifact is verified afterwards. Meant for batch, destructive, high-stakes and complex-rule operations.
verifiable intermediate output
The JSON plan itself. The hard test for "verifiable" is that every key can be set-compared against a list that really exists.
oracle
Anything that can call something right or wrong without the model's judgment — an XSD, a form-field table, a formula error code. With no oracle the pattern is just ceremony.
fail-closed / fail-open
Refusing to continue on a problem (non-zero exit, no output written) is fail-closed; exiting 0 and leaving the caller to parse stdout is fail-open. The four official validators split evenly between the two.
execute-then-validate
The variant with no plan file, where the artifact is validated after the edit. docx, xlsx and pptx are all this, backstopped by --original or a full recalculation.
normalization
Putting the original into an addressable shape before the plan is written. merge_runs.py coalescing fragmented runs is the type case: skip it and the plan's anchor strings do not exist in the XML.
PreToolUse hook
The Claude Code hook that fires before a tool call, where exit 2 blocks the call outright. The only way to turn "run the validator first" from the model's good intention into a hard constraint.

// FURTHER READING