中文 EN
// Engineering Notes

One Human + a Fleet of Agents:
How This Hub Is Built

BigCat · 2026-07 · Tech Blog
BigCat's Learning Hub is a family of static sites I built for my own learning: mental models, AI/ML, system design, CS papers, philosophy, Buddhism, parenting, investing — thirty-odd subjects, each living in its own GitHub Pages repository. They update themselves daily, ship in both Chinese and English, and come with site-wide search, login-free comments, email subscribe/polls, and text-to-speech — and they install as a phone app that reads and plays offline and lets you highlight and take notes — while day-to-day operation requires almost none of my time. This post walks through how the whole pipeline is put together.

0. What it looks like

The core idea in one sentence: I only maintain each site's roadmap (TOPICS.md); production, publishing, post-processing, aggregation, and inspection are all automated — with the "write the content" step delegated to Claude agents running on a schedule in the cloud.

1. Overall architecture

Human (me) curate TOPICS.md · set caps · review suggestions Generation · cloud Claude routines ×20 staggered cron → mount repo → write per CLAUDE.md pick topic (fs-idempotent) → zh+en → publish.sh git push Repos · 20+ content repos (GitHub Pages) TOPICS.md · CLAUDE.md · .maxchars publish.sh gate (8 checks) on push Post-processing · per-repo Actions inject shared JS · Azure TTS bake (audio → R2) daily cron Aggregation · hub GitHub Actions refresh-hub landing · build-search Pagefind index GitHub Pages deploy Reader's browser · static site search / read / TTS / language toggle Governance · the system polices itself • cap reached → API pause (enabled=false) • *.en.html Chinese-leak fingerprint scan • monthly frontier refresh → ROADMAP-SUGGESTIONS.md (advisory) • ai-ml ⇄ super-individual cross-ref sync (no dup coverage) each layer trusts the layer below's artifacts, not its process — hence gates. Interaction backend · the one backend Cloudflare Worker: /subscribe · /vote · /comment D1 (SQLite): subscriptions · votes · comments double opt-in · anti-spam · CORS-locked Action bridges votes/comments → repo JSON /12h fetch

The layering principle: each layer trusts only the artifacts of the layer below it, never that layer's process. Generation can go wrong, so the repo layer has a validation gate; the gate can miss things, so the governance layer patrols.

2. Anatomy of a content repo: four files define a site

Besides the HTML pages, each content repo contains exactly four things, each with one job.

TOPICS.md — the human-curated roadmap (read-only to the bot)

The only part of the system that needs my ongoing attention. It lists, in order, every topic the site will cover — the length of that list is itself the cap: write it out and the site graduates (see §9). The key design decision is one-way authority: the routine may only read it — publish.sh flatly rejects any commit that modifies TOPICS.md. When topics run out, the routine is not allowed to extend the roadmap itself; it can only send me a push notification asking for a refill.

Drawing the line here is what keeps the whole system on course: the AI decides how to write; the human decides what gets written.

CLAUDE.md — the writing spec

Sites that need precise control over layout and depth (system design, paper deep-reads, daily book deep-reads) carry a detailed execution spec: target reader (senior engineers), length band, required section structure (the paper site's nine-part skeleton: one-liner → glossary → context → problem & motivation → core idea → key results → impact → limitations & critiques → takeaways), color palette (each site has its own visual signature — system design is dark cyan, papers are amber-copper), and honesty requirements — uncertain quotes must be marked as paraphrase, and limitations and counterarguments must be written.

The most useful trick in these specs: anchor on a published page. The prompt points at a specific article ("match read1 for depth, format, and voice") and says: do it like that. A concrete exemplar is far more stable than any prose description of style.

It later turned out to have a more important second identity: CLAUDE.md is the control plane for changing a routine's behavior without touching its prompt. Every cloud run auto-loads the repo-root CLAUDE.md, and it layers on top of the prompt — so to change one rule I don't have to edit that long, self-contained prompt (and editing trigger config carries the wholesale job_config replacement risk, see §9); a line in CLAUDE.md is enough. I verified this channel with a probe: I dropped "at the start of the run, write a probe file and push it" into one repo's CLAUDE.md, pushed only that repo, and fired the trigger manually — the probe file duly appeared on the remote, and the routine still wrote that day's page as usual.

.maxchars — the length ratchet

A one-line file containing a single number (3500/4000/5000). publish.sh counts the CJK characters of every new page and enforces this ceiling. This one was earned the hard way: LLM-generated series exhibit a "length ratchet" — each article comes out slightly longer than the last, because the model anchors on the most recent pages, and a few dozen days later the pages have bloated out of control. The fix clamps from both sides: a target band in the prompt, a hard ceiling at the gate.

publish.sh — the publishing gate

All repos share the same ~150-line bash script; the routine must publish through it. What it checks:

That last convention is no small thing — the commit message itself becomes a machine-readable publishing record; downstream completion detection and hub badges all work by parsing it.

3. The generation engine: cloud Claude routines

Content is produced by scheduled agents ("routines") running on claude.ai — currently 24 content triggers. Each trigger's job consists of:

Take the daily book deep-read routine. Its prompt is five steps:

  1. Pick a topic: choose the lowest-numbered entry in TOPICS.md not yet done — where "done" is determined by ls *-read*.html. The filesystem is the database: no state store anywhere, and repeated firings are naturally idempotent, because the next run sees a different file listing.
  2. Write: follow CLAUDE.md's section structure; err on the side of depth; gloss technical terms in English on first mention; never fabricate.
  3. Land both languages: {slug}-read{N}.html + {slug}-read{N}.en.html, each required to read natively rather than as a stiff translation, cross-linked via a language bar; both language index pages get updated too.
  4. Publish: run ./publish.sh; through the gate means live.
  5. Notify: a PushNotification lands on my phone — "updated + one-line essence + link."

The prompt's last sentence is "complete autonomously, wait for no confirmation" — nobody is present when a cloud routine runs, so any step that waits for approval is a deadlock.

Beyond the 20 content routines there are two meta-routines:

4. Post-processing: what happens after the push

The routine clocks out after pushing, but the page isn't in final form yet. Each repo's GitHub Actions take over for two kinds of post-processing.

Shared-script injection (all repos)

Comments, search, bilingual TTS, navigation buttons, and the lightbox are all provided by shared scripts hosted in the hub repo (comments.js, search.js, i18n-tts.js, index-button.js, lightbox.js). Content pages are forbidden from hard-coding these script tags (publish.sh blocks it); instead, an injection Action scans the HTML after each push, adds whatever is missing, and auto-commits as Auto-inject shared scripts.

Why injection instead of baking the tags into the generation template? Because infrastructure must be able to evolve independently of content. With 20+ repos and hundreds of pages, script tags hard-wired into templates would mean retraining 20 prompts and re-touching hundreds of pages just to upgrade the search script. Under injection, you change the shared script once and the whole fleet picks it up on the next pass — and old pages get retrofitted for free.

Azure TTS bake (pre-baked audio)

Every section of the Chinese pages is click-to-listen, with pre-baked audio:

Hash addressing makes the whole chain idempotent: unchanged content costs zero API quota, and editing one section re-bakes only that section. The hash is taken over the raw text, not over the normalised string that gets sent to Azure — the reading rules (symbol substitution, formula handling, emoji stripping) keep evolving, and if the hash tracked those rules, any small tweak would put the entire site's audio "out of match" and silently re-bake thousands of segments. A small blemish in old audio is far cheaper than re-baking everything.

The audio does not live in git. It sits in Cloudflare R2 (keyed as <repo>/<lang>/<hash>.mp3) and is served by a Worker. It used to be committed into the repos and served straight from Pages; the price was several GB of binaries in git history, with the largest sites closing in on the 1 GB Pages limit. Since the move, audio/ is gitignored everywhere.

Only Chinese is baked. English pages fall back to the browser's built-in Web Speech API — noticeably more robotic than Azure, but the quality gap is far smaller for English than for Chinese, and not worth the quota and storage.

5. Aggregation: how the hub tracks 20 repos beneath it

The hub repo is fully automated too, via two daily Actions.

refresh-hub.yml — re-rendering the landing page

Runs generate_hub.py, a textbook case of "one source of truth, two language renders": card metadata (titles, bilingual blurbs, palettes, sections) lives in a single CARDS array, and both the Chinese and English pages render from it — there are never two HTML files to keep in sync.

The dynamic parts come from the GitHub REST API:

Timing-wise, this Action runs about 45 minutes after all content routines, so the day's new pages always make it to the landing page.

build-search.yml — site-wide search

Static sites have no backend, so search is Pagefind: every day, clone all content repos into _src/ as one tree, run Pagefind to build a sharded index (Chinese and English indexed separately). The front-end search.js provides a floating search button and opens the search overlay in the page's language. Footers, navigation, and comment containers are excluded from indexing as noise.

The index does not live in git — same reckoning as the audio. It is 3,100 files and 46 MB, rebuilt and committed in full every night, which had taken .git to 713 MB against a 5 MB working tree: the repo was 99% history. It now goes to R2 and is served by a Worker under /pagefind/*, in exactly the URL shape it had when it was same-origin, so the front end only swapped an origin.

Going cross-origin has one trap worth recording: Pagefind resolves result paths against whatever origin the bundle is served from. After the move, every search result pointed at the Worker's domain — which serves nothing but /pagefind/* — so for a while every search result on the site was a dead link. The fix is stripping the bundle origin back off client-side. (The same batch cost me one more: the Worker forwarded the browser's conditional request headers wholesale to R2's onlyIf. A browser revalidating a cached module sends If-Modified-Since, R2 answers "not modified" with a bodyless object, and my code read that as a precondition failure and returned 412 — so import() failed outright while curl worked perfectly, because curl sends no validators. A bug you can reproduce with curl is a lucky bug.)

One special case: the Thinker Roundtable site (thinker-arena) is client-rendered — its content lives in JSON, invisible to a crawler. The fix is render_search_snapshots.py, which renders the JSON debates into plain HTML snapshots purely for Pagefind to consume before indexing — SSR for the crawler's benefit, just as a daily batch job.

Chinese search: the same mistake, three times

Searching 现象学 (phenomenology) returned 1,172 results — very nearly the whole site. Twenty-seven pages actually contain the word.

The cause was not ranking. It was that the index side and the query side disagreed about what counts as a word. To make 拓扑 (topology) match inside 拓扑直觉, the build had been inserting zero-width spaces between adjacent Chinese characters, forcing the index into single characters — while the query side still ran Pagefind's own word segmenter. So a multi-character query went looking for word tokens in an index that held none, and Pagefind drops terms it cannot find rather than returning zero. The query quietly collapsed to the single character 学 and swept up every page containing it.

Fixed on both sides. The query side now tries an exact phrase first and falls back when empty: quoting a Chinese term forces phrase matching, and 现象学 went straight from 1,172 to 27 — exactly its true page count. But quoting cannot be unconditional, proper names especially: 海德格尔 (Heidegger) gives 0 quoted and 15 unquoted, because query-side and index-side segmentation disagree on that string. So each term is probed once and the verdict cached.

The index side needed sub-word recall: 拓扑 never appears alone, only inside 拓扑学 / 拓扑指纹 / 拓扑不变, so 61 pages contain it and search returned 1. That calls for a hidden bigram shadow text — the classic dictionary-free CJK indexing scheme.

Then I fell into the same hole a third time. I wrote those bigrams as Chinese, assuming a space between them made two tokens. Pagefind's segmenter re-segments them too, dissolving the pairs back into single characters: 贝叶斯 (Bayes) then returned 75 results against the 24 pages that contain it. What fooled me was that 拓扑 happens to use rare characters and looked fine; only a word built from common ones exposed it. Encoding each bigram as opaque ASCII (bg8d1d53f6, the two codepoints in hex) finally made it hold: Bayes 24/24, topology 18/18, Kant 12/12 — matching grep exactly.

The lesson: those were one mistake wearing three costumes — the index side and the query side must agree on what a token is. When either side is "improved" alone, it does not fail loudly; it returns a pile of confident-looking nonsense. The build now stamps a scheme version into the shadow block and the client enables the matching query tier only when it finds that marker, so a cached front end and a re-encoded index can never half-agree.

Ask: letting the essays answer questions

Keyword search has a structural ceiling: it finds pages containing a word. It cannot answer "which pieces discuss decision fatigue" — the essays that discuss it need not use the phrase.

So the search modal gained a second tab. Retrieval is Cloudflare AI Search (formerly AutoRAG), which manages the whole pipeline — chunking, embeddings, vector store, generation, citations — with qwen3-embedding-0.6b for embeddings and qwen3-30b for answers.

What it needs is clean text, so build-search.yml grew a step that strips the aggregated HTML down to prose Markdown in R2. I did not point it at the HTML directly (it accepts .html): these pages are a hundred-odd divs of layout around a dozen paragraphs of prose, so a good share of every chunk would be furniture and the chunk boundaries would fall in markup rather than between ideas. File names mirror site paths, so whichever file it cites converts straight back into a link.

The results are worth quoting. Asked which pieces cover decision fatigue, it returns five — from mental models, book recommendations, neuroscience, the mental-model index, and investing: five different content repos, several of which never use the term. Asked whether willpower is finite, it answers that this is contested, and cites the 2016 pre-registered replication across 23 labs that took the ego-depletion effect to nearly zero — which is the argument of the essay on file, not the model's own background knowledge.

I measured the cost, since this is the one metered thing in the system: a full re-index is ~13,000 neurons (~$0.14), a question ~28 (~$0.0003). The free allowance is 10,000 a day, so roughly 350 questions daily. Both layers are incremental — the corpus sync uploads only files whose MD5 changed, and AI Search only embeds changed files (one sync processed 22, another 64, against a corpus of 2,690) — so steady state is the day's new articles, not a full rebuild every six hours.

Questions and their answers are kept in the browser, so revisiting one is a click. Replay makes no network call at all — it spends no allowance, and it sidesteps a privacy decision nobody asked for, namely a public site centrally logging what its visitors ask.

This layer's tuition was mostly "the docs and the API disagree": in a search result filename is a top-level field rather than the documented metadata.filename, and content is an array of parts rather than a string — written to the docs, every source link would have been built from undefined. I also gave the Worker the same name as the AI Search instance, and requests 404'd (Cloudflare 1042) before reaching any of my code. And the model will happily print internal paths at the reader — (corpus/mental-models/energy-attention-day27.md) — which the prompt forbids and the Worker strips anyway, because a prompt is a request rather than a guarantee.

6. Interaction layer: subscribe, comment, vote — the one backend

The five layers above contain not a single line of server code. But three things — email subscription, comments, voting — inherently write state, which pure static can't do. This is the one place in the whole system with a backend, kept as small as I could make it: one Cloudflare Worker (worker.js) plus one D1 (SQLite) database, entirely within the free tier (100k Worker requests/day, 5M D1 row-reads/day).

One Worker, three jobs

The three features share a single Worker and database, routed by pathname:

Data lands in three D1 tables: subscriptions (email+list as primary key, with a confirmed flag and language), votes (poll+voter as primary key, deduped by construction), and comments (page path, name, body, moderation flag, salted iphash). The front end is just two scripts — engage.js (subscribe box + poll) and comments.js — injected site-wide like the other shared JS; the Worker's CORS admits only whitelisted origins (now the custom domain hub.cissychen.com, plus the pre-migration cissy0802.github.io).

The cost of login-free: three anti-spam defenses

No login means a low barrier — but it also means losing the account gate that was the natural anti-spam wall: anyone can post, bots especially. So the self-hosted version builds in three defenses: a honeypot field (a hidden website input a human won't fill and a bot will — filled means silently dropped), per-IP rate limiting (max 3 per 60s, keyed on a salted hash of the IP, never the raw address), and optional Cloudflare Turnstile (once a secret is bound, the Worker auto-enforces it — turn it on if spam appears). Comment bodies are always rendered with textContent, never as HTML, closing off XSS at the root.

Double opt-in subscribe = the foundation for passwordless login later

Subscription uses double opt-in: /subscribe first stores an unconfirmed row and emails a confirmation link (via Resend's free tier, 3,000 emails/month); clicking it hits /confirm, which flips the row to confirmed=1. Only confirmed rows count as real subscribers. This isn't just about catching typo'd addresses — "email → verify" is exactly the foundation of passwordless login: to add login to comments/voting later, just issue a session token and check it at /comment and /vote — no schema redo. (That line was later cashed in — commenting and voting both require an account now, built on exactly this foundation; see §7.)

The weekly digest: turning static content into a push

Once you collect emails you have to be able to send. This chain deliberately never touches my machine:

Vote dedup at first relied only on a random front-end localStorage token — clear it and you could vote again. For a casual hub poll, honest-but-not-airtight was enough at the time; and when it did need to be strict, it was tightened exactly as that sentence promised (votes are now bound to an account, one per person — see §7). The trade-off runs through the whole layer: static wherever possible; for the one piece that must be stateful, cover it with the smallest possible backend on a free tier — and design in, at every point, how it could later grow stricter.

7. From website to app: offline, notes, accounts

The first six layers are about producing the content. This layer grew later, in a different direction: making it feel like an app — installable to a phone's home screen, readable and listenable offline, with highlight notes, and separate accounts per person. The key constraint didn't change: no new server, and not one line changed in any content repo. It all hangs off the existing injection layer (§5) and that one Worker + D1 (§6).

Install as an app + offline: one service worker over the whole site

Step one is going PWA: add a manifest.webmanifest and a root-scope service worker (sw.js), and "Add to Home Screen" in Safari/Chrome gives you a full-screen, icon-bearing app. Because the SW registers at the domain root, its scope naturally covers all 30-plus content repos beneath it — one registration, whole-site offline, no per-repo install.

The offline strategy is layered: the hub shell (landing page, shared JS) is cached automatically; HTML/JS are network-first, falling back to cache offline, so online you always get the latest; audio (pre-baked mp3, potentially hundreds of MB) is never cached automatically — only when you tap "⤓ Offline" on an article does that one article's HTML + its mp3 go into the Cache API. Per-article, a few to tens of MB, controllable.

I only wanted that download button for myself, which ran into an iOS gotcha: a home-screen PWA gets a storage jar completely isolated from Safari's, and "Add to Home Screen" strips the query string off the URL — so an initial ?me=1 switch to unlock it on-device could never reach the installed app. After a detour through an in-app gesture, the genuinely clean fix had to wait for the account system (below): make "who's the owner" not a flag living on the device but something tied to my account — the button appears only when the logged-in email is mine. So in the installed app you just log in once, with no query string or gesture, and a new device is just another login. Gotchas like "a desktop app's storage/URL behavior differs from the browser's" are all paid for in testing.

Highlight notes: re-locating, offline, grouping, comments

Notes are a notes.js, spread across the whole site by the injection layer just like the offline script (even the client-rendered roundtable-debate pages, which deliberately opt out of the hub nav, are covered — one bug lived here: injection used to sit after the "don't show nav buttons" early return, which disabled notes along with it; moving injection before that early return fixed it). Select any passage, a "+ Note" bubble pops up, one tap saves it.

The hard part is re-locating a sentence back in the article: on save it stores the highlighted passage plus ~40 characters of context on each side; on reopen it searches the page for "prefix + passage + suffix," and on a hit scrolls there and flashes a yellow highlight. The context matters because the same sentence can appear more than once on a page — only the surrounding text pins down the exact occurrence.

That machinery later got a second job: redrawing past highlights as underlines in the article itself. Which is where a quiet trap sat. The first implementation wrapped the passage with Range.surroundContents(), which throws on any range that crosses an element boundary — and I swallowed that exception. On sites with dense inline markup (neuroscience runs <strong> mid-sentence in 19 of 22 paragraphs) nearly every highlight silently drew nothing, with no error anywhere. Wrapping each intersecting text node separately fixes it: one note can own several <mark>s, visually continuous, with editing and deletion acting on all of them.

The underline then became an affordance itself: tap it and an editor opens right there for your thought on that passage, prefilled and editable if you've written one before. So a solid underline means "I wrote something here", a dotted one means "I only marked it" — you can see at a glance where you actually thought. Alongside those per-passage thoughts there's a second, coarser layer: a thought on the piece as a whole, writable both at the end of the article and beside it in the notes list, the same field in two places.

Notes are offline-first: written straight into a localStorage queue, live locally at once, and flushed to the backend when you're back online — highlight away on a plane or a subway. (A small bug hid here too: on a successful push I removed the note from the queue but forgot to put it in the local cache, so for a moment after saving the note existed nowhere locally and its underline only appeared after a reload.) The list page (/notes.html) nests two collapsible levels — by site, then by article — collapsed by default, so it opens as a map of where you've left marks; searching auto-expands whatever matched. Articles are deliberately ordered by the site's own index sequence (Day 01, 02, 03…) rather than by when you highlighted, so re-reading follows the same order as the site.

Read markers and an offline library

With this many sites the problem isn't running out of things to read — it's losing track of where you got to. So every article carries a read marker: a "mark as read" button at the end, plus two ways it marks itself — scrolling to the bottom, or listening to the last TTS segment (finishing it by ear counts too). A site's index ticks the ones you've finished and shows "12 / 68 read"; each card on the hub landing carries a count as well.

"Listening counts" came with a constraint: i18n-tts.js is a per-repo copy, and the versions have drifted — the injection layer spreads a script, not one shared instance — so changing it would mean touching thirty-odd repos. Instead of patching it, the read marker observes the progress readout it already maintains ("current segment / total") and calls it finished on the last one. Same craft as the iOS gotcha at the top of this section: when you can't change a thing, read its state from outside.

Last came the offline entry point. The service worker used to fall back to the hub landing for any uncached page — but the landing lists 30 sites as cards, and those sites' index pages had never been cached, so every link bounced back to the same landing and none of the downloaded articles could be reached at all. It now falls back to /offline.html, an "offline library" that reads the Cache API directly and lists what this device actually holds, with titles parsed out of the cached HTML on the spot (offline, you can't go ask for them again) and a flag for which pieces carry audio. Downloading a whole site now also caches that site's index, so a downloaded site keeps its own navigation.

Downloading itself got sanded down over three rounds of real use, each a different face of the same lesson: state has to live somewhere that outlives the page. Round one — a finished site showed ✓, then came back as ⤓ after a reload, because the button started from nothing every time and never asked what the cache already held; completing a site now writes a marker, and a later load restores from one scan. Round two — a download interrupted by navigating away didn't continue, because the loop runs in the page's JS and dies with the page; a run now also writes a "downloading" marker, and returning to the landing picks it up, skipping whatever is already stored. Round three wasn't a bug but a design mistake: one button meant both download and delete, so the same spot did opposite things depending on state. It's now two buttons, ⤓ and 🗑, each doing exactly one thing, with the bin appearing only when there's something to remove.

Deleting is easier to get half-right than it looks: drop only the HTML and the bulk of the space — the audio — stays. So a delete reads the cached HTML back to find which segments that piece referenced and clears them too, matched by filename so it covers both the old same-origin layout and the post-R2 URLs. (I paid tuition here as well: while splitting the buttons I deleted the two removal functions along with the old code, so the bin just sat at "…" forever — another silent failure.)

Multi-user accounts: cashing in the foundation §6 buried

Once notes need to cross devices — and let different people keep their own — you need real accounts. And this is exactly the prophecy buried at the end of §6: "'email → verify' is the foundation of passwordless login; issue a session token later and there's no schema redo." Now it's cashed in: still the same Worker, same D1, just two more tables (users / sessions) and a set of POST /auth-* endpoints.

No shortcuts on security, because registration is public:

With the whole layer added, the backend is still that one Worker + one D1, still within the free tier. Which loops right back to §6's trade-off: making subscription "email → verify" back then wasn't busywork — it laid the foundation for today's account system in advance. Today there was no rewrite, just another floor built on the foundation already there.

And with that, §6's two deliberately-loose spots got tightened

Once accounts existed, the two places §6 deliberately left loose closed themselves naturally — commenting and voting now both require a login:

The cost, honestly recorded: the participation bar did go up — a passing reader now has to sign up to leave a line. For a site built for my own learning and published as a side effect, I'd rather have a few traceable real comments than many anonymous ones of unknown provenance. And the reason this step was so cheap — one column, two endpoints, not a single existing table touched — is entirely that early double opt-in decision. The value of a seam you left open is only settled the day you use it.

8. Two sites that break the standard template: Thinking Hub and Deep Research

Everything above describes the standard content site — "a human sets the topic, the routine writes a page." Two sites don't follow that template; they're more complex, and they best show how far this architecture can stretch.

Thinking Hub (the Thinker Roundtable) — the audience votes on what gets written

On a standard site the human maintains TOPICS.md and the AI writes to it; this site hands topic selection to the audience. It's also client-rendered: each debate is a debates/*.json rendered into a chat stream by browser scripts (which is why site-wide search needs the snapshot SSR from §5 to crawl it at all).

The hard part is a wall: the cloud routine runs in a container behind an egress proxy that doesn't allow the self-hosted backend (hitting bigcat-engage…workers.dev returns 403 outright), so the routine can't read votes or comments itself. The fix is a bridge: a GitHub Action (its runner isn't behind that wall) writes the backend's net votes and comment snapshots back into the repo's ideas.json / audience_inbox.json every 12 hours. The routine then reads repo files only and has the backend data — without ever connecting to the backend.

That closes into an audience feedback loop: audience votes/proposals → Worker + D1 → the Action bridges every 12h → repo JSON → the routine opens a debate on the highest-net-vote topic (or a human-pinned queue), and while wrapping up mines audience comments for new candidates. The audience becomes part of the roadmap — the very authority the standard sites deliberately withhold from the AI is here handed safely back to a crowd.

Generation is heavier too: pick 6–10 thinkers from a 150+ persona registry (run diversity_report.py first and cast by the aggregate distribution — explicitly fighting the overfit of "copy the shape of recent debates"), then the routine writes three bilingual rounds itself and closes with a three-AI panel. Even the character-encyclopedia entries must be written "blind to this debate," so a profile never gets polluted by the topic it happened to be cast for.

Deep Research — multi-agent investigation + adversarial fact-checking

A standard site has one routine write one page; this site's content is the output of a multi-agent research pipeline. Every load-bearing claim is challenged by 3 independent verifier agents against primary sources (adversarial fact-checking) — a claim that doesn't survive either gets more evidence or gets cut.

It's also the only site with a build step: other sites have the routine emit HTML directly, but here the source is markdown (in sources/, each piece in a "plain" and a "deep" edition, both languages), which build.py renders into HTML via its registries — one piece = plain × deep × zh/en = 4 pages. To change content you edit the markdown and re-run build.py.

Why it earns its own entry: it upgrades "generation" from "write a page" to "research → adversarial verify → synthesize with citations" — the most research-intensive branch of the system, and the one that best fits this post's title, one human directing a fleet of agents.

9. Governance: making the system police itself

Getting it running is the easy part; running unattended for the long haul is the hard part. The lines of defense:

Caps + auto-pause

Every site's roadmap is finite — when the material is learned, the site should graduate, not pad itself for the sake of a daily streak. But the cap is not a hardcoded number — it floats, and equals however many entries remain unwritten in TOPICS.md. A daily inspection task computes two numbers per site — the highest published issue N_pub and TOPICS' highest number N_top — and syncs the cloud trigger's switch in both directions: written to the cap (remaining ≤ 0) and it goes enabled=false; append new topics to TOPICS (remaining > 0) and it switches back on automatically. So "add a few roadmap lines" is the only revival operation — I never touch a trigger.

The pause operation has two safety interlocks, both paid for with real mistakes:

  1. Before pausing, get the trigger and confirm that the repo it mounts really is the one being retired — protection against a trigger ID mapped to the wrong site;
  2. The update body sends only {"enabled": false}, because this API's update replaces job_config wholesale rather than merging — sending a partial job_config would wipe out the prompt, the mounted repo, and the model configuration.

The deploy watchdog

There's one more failure that's easy to overlook: the page is correct, it was pushed, and then the GitHub Pages deployment itself fails (intermittent 503s / build flakiness). The content sits in the repo while the live site still serves the old version — the sneakiest failure of all, because every check upstream reported "pass."

So a launchd watchdog runs locally, sweeping each repo's recent Pages deployments via gh every 20 minutes and re-running any that failed. It deliberately runs on my machine rather than in the cloud — the cloud sandbox's egress proxy blocks api.github.com, so my own laptop is the one that can reach it.

Audio backup: the one asset git cannot reach

Almost everything in this system can be restored from git: text, templates, scripts, workflows. Audio is the exception. It lives in R2, and audio/ is gitignored in every repo — so git pull can never bring audio down. It is also the only asset that cost real quota and cannot be rebuilt instantly from content.

So a second launchd job pulls R2 down incrementally every day as a second copy. A companion server can then run the local repos as a browsable site: it rewrites the pages' absolute hub URLs to local paths in the response, without changing a single byte on disk — these local clones are read-only mirrors, and the cap guard flattens any drift with git reset --hard, so editing the HTML would just be silently reverted. If R2 and the hub both vanished, the local clones plus audio would still serve the whole site.

There's a counter-intuitive trap here: the guard script must never use git clean -x. The -x flag deletes ignored files too — i.e. it silently wipes the entire audio backup, parts of which R2 itself no longer has. An action meant to "keep the working tree clean" would destroy the one asset that cannot be rebuilt.

Content-layer autonomy: pushing the checks down into each repo

Originally these content-layer checks — stop when the roadmap is done, backfill out-of-band pages into TOPICS, self-check English pages for Chinese leaks — were all done centrally by that external inspection task. They have since been delegated into each repo's CLAUDE.md: since every run auto-loads it, letting the site handle all three at generation time is more timely than patching them up afterwards from outside. The external inspection thereby slimmed down to nothing but the trigger switch. (The backfill exists to hold one invariant: TOPICS must be a superset of what's published. Otherwise a manually published day57 that never got recorded would collide with a Day 57 added to TOPICS later — which could then never be written.)

Of those, the Chinese leak is the most common degradation in bilingual generation: Chinese seeping into the English page's template slots (subtitles, tags, name fields). Each repo's CLAUDE.md hardcodes a set of fingerprint greps (like class="en">[一-鿿]) to self-check after generation and before publishing. The fingerprints are chosen carefully — legitimate Chinese, like Buddhist scripture alongside its English translation or glossed terms, doesn't use those classes and never false-positives.

But the switch itself cannot be delegated, and that's a hard boundary: most routines mount only Google Drive in the cloud, with no meta-tool that can write triggers, so they cannot change their own enabled; worse — a trigger that has been disabled cannot revive itself, so re-enabling can only ever come from outside. Hence the final shape: autonomy in the content layer, the switch stays out.

The quality ratchets

10. Design principles that emerged

Static first, with one small backend. On the content side: no database, no server, no build framework — HTML is the artifact, git is the CMS, GitHub Pages is the CDN, and search (Pagefind) and TTS (pre-baked mp3s) solve traditionally "needs a backend" problems with static answers. The only genuinely stateful piece — subscribe / comment / vote — is confined to one Cloudflare Worker + D1, entirely on the free tier. Static-first isn't dogma; it's keeping the backend's footprint as small as possible.
The filesystem is the state. Topic progress = the output of ls; the publishing record = commit messages; completion status = parsed from commits. There is no independently maintained state store anywhere — so nothing can drift out of sync with reality.
Every step is idempotent. A re-fired routine won't rewrite existing pages (the file already exists); a re-run TTS bake won't double-bill (hash hit); a re-run injection won't stack tags (only adds what's missing). For unattended systems, retries are the norm — idempotency is a precondition, not an optimization.
Separate generation from validation. Never assume LLM output is correct; publish.sh backstops it with the dumbest possible bash checks. The smarter the generator, the dumber and more deterministic the validator should be.
The human guards exactly one entrance. All my recurring effort converges on TOPICS.md and cap decisions; everything else derives from them. The AI expanding its own mandate (extending the roadmap) is mechanically forbidden — it can only file a request.
Convention over protocol. The Add #N commit format, the {slug}-day{N}.html naming, the one-line .maxchars file — components talk through these humble conventions. There isn't a JSON schema anywhere, yet every convention has at least two consumers.
Capabilities grow in the injection layer, not the content repos. Offline, highlight notes, accounts — all these "app-ification" capabilities hang off the single script spread over the whole site; not one line changed in any of the 30-plus content repos. To add a new capability to every site you change one shared JS file, not 30 repos.
Lay foundations for the future without over-building for it. Subscription was deliberately made "email → verify" — not because login was needed then, but to lay the foundation; when the account system was actually needed, the same Worker + D1 extended straight onto it with no rewrite. Reserve the seams, but don't build the floors early.
Graduate when it's time. Every site has a cap; when it's reached, the routine auto-pauses and the badge flips to Completed. This system serves my own learning — a finished field deserves closure, not hostage-taking by "consistent output."

11. Cost

One person's attention is the most expensive resource in this system. The design goal of the pipeline was never "full automation" for its own sake — it was to free my attention from production and operations and spend all of it on the only thing worth spending it on: deciding what to learn next.