A Knowledge Substrate With No Opinions

A few weeks ago I ended a piece about LLM Wiki with an admission: whoever needs a harness built for long-arch intellectual and humanist work, one that treats disagreement as structure rather than defect, will have to build it themselves, from their own epistemology up.

I’ve started. The bigger effort is going to take more than one post, but this piece is about something narrower and quintessential: the floor everything else stands on. Before there is a harness, before there are agents with opinions and models with tempers, there has to be a place where the knowledge itself lives. Let’s call it the substrate, and the first design decision I made about it was to give it no opinions at all.

That sounds like an evasion. It isn’t. It’s a discipline, and it’s the hardest one in the whole design, because the natural drift of any knowledge system is toward doing more: a little inference here, a little reconciliation there, until one day the storage layer has quietly become a judge. I kept coming back to a plainer analogy. A repository is not the compiler, and nobody confuses the two. The repository holds what’s true about the code’s history (this line, this author, this moment) and doesn’t say anything about whether the code is good. The substrate is that, for thought. It ingests nothing, extracts nothing, decides nothing. It certifies form, not truth. Everything semantic, every judgment about what a passage means or whether two claims contradict, belongs to and is owned by the agents working over it, not the medium.

Key Features

The Surface

The core is almost insultingly plain and unoriginal: Markdown pages, a little structured front matter, typed links between them, basically Karpathy’s LLM Wiki pattern, taken at its word ¹. I’ve taken to calling the writing surface itself just that, the surface, and each individual document is just a page, which a person reads and a machine parses with nothing sitting between them. Each page carries a stable identifier in its front matter though, independent of its file path, the way a person keeps their name across a house move. That matters more than it sounds: derived claims anchor to this identifier, and a rename shouldn’t orphan a year of accumulated reading.

Every ornament I could add to the surface is a cost, so most of them lose the argument. The property I’m protecting is dual legibility. The same artifact is what I read and what the machine parses. Every clever addition is one more place where those two views can drift apart.

All writing, human or agent, passes through one door, the single surface every read and write goes through, where two hands reaching for the same page get sorted out. You write against the version you last read. If it’s moved since, your write is rejected and handed back with the current hash, so you can retry, merge, or walk away, in other words, concurrency control with optimistic locking. A page write and any provenance it carries commit together, so one never gets left without the other.

Persistence

The page is the document of record. Everything formal (structure, relations, indices) is derived from the prose. None of it is a second source of truth. If the derived layers vanished tomorrow, the knowledge would still be there, complete and readable, because the prose never depended on them to exist. That’s the property “persistence” is standing in for: the output of one round of work is available to the next without a translation step.

Provenance and Auditability: The Anchor and The Signature

This is the design decision I lost the most sleep over, because it splits into two things people conflate constantly. One is grounding, i.e. the anchor: every derived claim carries a pointer back to the page and the exact revision of it that licensed the claim, checkable by anyone, deterministically, forever. The other is attribution, i.e. the signature: which agent made the claim, under which model, which prompt, reading which revision, at what hour. One points to provenance, the other to auditability.

The anchor can be regenerated. Re-run another extraction and you get it again. The signature can’t. Re-run the same model against the same page tomorrow and you don’t recover yesterday’s reading. You produce a new one, a different act. An interpretation is an act, not a fact sitting there waiting to be photographed. So the signature lives in its own durable, append-only ledger, the marginalia, kept alongside the pages, the way an actual marginal note sits beside a text without altering it. A second extraction is a new entry, not a correction of the old one. History accumulates.

Change Control

Version control over the pages themselves tracks the prose through time: who changed what, when, with a diff and a rollback available. This is a separate axis from provenance and auditing, which traces a claim to its source rather than the page through its own history. The two are linked by content reference. A provenance entry names the revision it read, without the prose needing to know derivations exist at all.

When a page outgrows itself, an agent can split it: new, sharper pages get born, and the original becomes a redirect under its own old identifier. Anchors already pointing at it keep resolving to the exact revision they were reading, undisturbed, for as long as anything still depends on it. Nothing forces the split to be reconciled immediately. That’s on purpose.

Visibility: The Radar

This is the layer that answers everything I complained about last time, so I’ll spend the most time on it. It is called radar, because it’s built to catch what you weren’t looking for. Basically, it is an index! A dynamic, entirely disposable structure built by parsing the prose (entity graphs, embeddings, whatever the retrieval technology of the week wants to be). It’s re-buildable exactly one way, by re-reading the pages and letting extraction run again, which appends new entries rather than overwriting old ones.

I put one obligation on it: whatever it returns has to point at a resolvable page. That’s the entire coupling surface, and it’s what makes the radar replaceable without touching anything else. I put one more requirement on its behavior too: it has to surface low-centrality material as readily as the well-connected consensus reading. A structured query answers a question whose shape I already knew to ask. What I actually need is a counter-assertion by an author I wasn’t thinking about, a sibling claim two domains over, the tension I’d walk right past, surfaced without me knowing to look for it. A fact-store resolves that toward one truth. The radar’s job is to hand it to me instead, tension intact.

Integrity: The Proofreader

The proofreader runs deterministic, syntactic checks: malformed pages, dangling links, anchors that no longer resolve at the revision they claim. It only flags things. Repair happens like every other write, by whoever picks up the flag. The set of checks that ran is named and versioned, because a control nobody can name certifies nothing.

Technologies and Tools

Markdown and YAML front matter as the writing surface is plain, vendor-agnostic, and it connects to the actual practice of writing rather than a bespoke database formalism. Git or equivalent for change control, giving me diffs and history for free. These were the easy and obvious ones!

A replaceable retrieval index for the radar, chosen against a short list of requirements rather than a specific technology: resolvable anchors at page granularity, surfacing tuned toward low-centrality material rather than connectedness, targeted invalidation when prose changes, and recorded merges instead of silent shenanigans. Simple syntactic linters or LSP for the proofreader, structural and referential checks over an open-world corpus, flagging syntax without ever touching a semantic claim.

Why I Set the Semantic Web Aside

I gave the whole stack (RDF, SKOS, OWL) a real shot, in good faith, before setting it aside. It pays for itself when many parties agree a term means one thing everywhere it appears. That agreement is what lets a reasoner entail what nobody explicitly said. A corpus built from a lifetime’s contested reading doesn’t have that agreement to spend. Force one meaning and the actual disagreement flattens into whichever reading won the modeling argument. Keep the syntax and refuse the vocabulary, and nothing composes and nothing entails, which is the worse outcome, because the full modeling cost is still paid.

The same failure shows up with concepts that drift rather than concepts that disagree. “neoliberalism” in 1938 is not “neoliberalism” in 1985, and it’s one thing under transformation, not two. An identifier names one stable thing forever. A page can just say a term meant one thing in one decade and something else later, and mean it, at no engineering cost at all. Most importantly, and this was the last straw, the modeling decision the stack demands up front is usually the analytical conclusion, not an input to it, which means the stack asks you to finish the reading before you’ve started it.

Choosing and Distrusting LightRAG

Before landing on LightRAG, I gave the field a proper look. HippoRAG, Microsoft GraphRAG, and a couple of others each got read closely against my own requirements and set aside for reasons specific to what they optimize for. That search, and how LightRAG actually held up once I started running it against real material, deserves its own longer post, and it’s coming.

For the radar’s first real implementation I picked LightRAG ²: dual-level retrieval that matches specific queries against entities and thematic ones against relations, no community-detection step where a minority claim gets summarized away, cheap revision through delete-and-reinsert. I picked it on economics and update behavior, and I don’t fully trust it yet. More on LightRAG later.

Suffice to say that past a source-ID cap it silently stops attaching new material to a concept, and the concentration lands exactly on the hub concepts where surfacing matters most. Its entity deduplication merges look-alike entities across chunks with no record that a merge happened at all (the exact silent collapse that I wanted to prevent).

This is still just the floor. There’s no agent standing on it yet, no harness. The floor was the part that had to be argued for first, since everything a harness will later decide to believe about a page depends on whether the page underneath it was ever allowed to hold a contradiction in the first place. Mine can.