Teaching a language model to teach

That structure could carry a model further than prompting could.

The first instinct in mid-2024 was to throw bigger prompts at GPT-4 and hope for the best. I was skeptical of that approach for the same reason I'm skeptical of most "just prompt harder" arguments: the interesting work in any content system happens upstream of the model, in how the inputs are structured. The publisher's skill graph already encoded the pedagogy. The job was to design a pipeline that let that structure do work at every level, as a hierarchy of constraints passed from parent to child to block.

The publisher's catalog had spent years getting the upstream right. The bet was that if you respected what they'd built, the model only had to be good enough at the last mile.

That was the thesis going in. The interesting parts of the project turned out to be the places where the thesis was right and the places where it wasn't quite enough.

Three moves, in order.

Most of the build collapses into three decisions about where the design work lived. The pipeline was the visible artifact. The real architecture was upstream of it and around it.

An Airtable proof-of-concept that started the project gets folded into step one; useful as a forcing function for thinking about structure, not durable as infrastructure.

From skill graph to draft lesson, one surface at a time.

The walkthrough below is the production flow as it ran during the pilot. Each surface answers a single question: what's the designer doing here, and what does the model owe them next?

The skill model was fleshed out in Airtable. Every parent skill, child competency, and learning objective is a node with attached prompting rules. The rules drive complexity downward: an objective inherits its parent competency's voice and scope, and adds its own about Bloom's level, activity type, and example density.

What worked, and where the model ran out of road.

The block-level generation was solid by the second month. With the full hierarchy of constraints in place, GPT-4 could produce paragraph-scale teaching content that was on-voice, on-standard, and structurally correct often enough that the editorial surface stopped feeling like a safety net and started feeling like a workspace.

The harder problem was holding a lesson together. Cross-block coherence — the thing that makes a twelve-block lesson read like one mind wrote it rather than twelve neighbors — was where 2024 hit its ceiling. Context windows and reasoning at that scale weren't quite there yet. The pipeline could generate twelve good blocks; it couldn't always make sure block seven didn't quietly contradict block three. The editorial surface absorbed that gap, which is part of why the surface mattered so much. But it meant the system was doing more reviewer-assisted than designer-replacing work, and the honest framing for the pilot was that.

The blocks were solid by month three. Stitching twelve of them into a lesson that read like one mind wrote it, which is where 2024 ran out of road.

The other thing the pilot clarified, and this matters more than the coherence story: the editorial UI was the product. The pipeline was infrastructure. The model was a vendor. What determined whether the system would get used by an instructional designer on a Tuesday afternoon was the texture of the review surface: how fast it was to regenerate one block, how hard it was to lose your place, how trustworthy the feedback signals were. That was design work, and it's what the pilot proved was buildable.