A local-first refinery for the corpus you've already read
Twenty years of reading produces a problem most people don't name out loud: you remember that you read something useful, you remember roughly what it was about, and you cannot find it. Filesystem search collapses on synonyms. Spotlight is fine for filenames and bad for content I read three years ago about a tangentially-related thing. Cloud search engines like ChatGPT's are answering a different question — "what does the internet think about this" — and they have no access to your PDFs anyway.
KnowledgeRefinery is a local-first macOS app that takes the problem seriously. The point isn't search. The point is a refinery — a workshop where you ingest a corpus, watch it get processed into something queryable, and then operate on it as raw material.
The architecture is local because the corpus matters and because trust matters. The PDFs you've collected, the notebooks you've taken, the slack archives, the email exports — these aggregate into a profile that's intimate enough that I'm not interested in uploading it to anyone. Local embeddings on Apple Silicon are now fast enough that you don't have to.
The three operations the app exposes:
- Ingest. Drag a folder onto the app. The refinery parses, chunks, embeds, and indexes. Supported formats are the ones that matter: PDF (including scanned-with-OCR), markdown, Jupyter, plain text, EPUB, HTML.
- Semantic search. Query in natural language. The query gets embedded, the index gets searched, and the result is a ranked list of chunks with the actual file they came from. Click through, jump to the page. The matches aren't keyword matches — they're concept matches, which is what you wanted when you were trying to remember "that thing about lattice-based signatures and zero-knowledge proofs in the context of regulator-facing audit trails."
- 3D concept visualization. The embeddings live in a high-dimensional space. The visualization projects the corpus down via UMAP and shows it as a navigable point cloud — clusters are concepts, dense regions are topics you've read about heavily, outliers are the weird papers that don't fit anywhere. You can grab a region, see what's in it, and discover the structure of your own attention.
The third one is the one I keep coming back to. Looking at the projection of my own corpus is a humbling experience. It surfaces patterns I didn't realize were patterns. The repository's name is intentional — the corpus is the ore, the app is the refinery, the queries are the assays.
What becomes possible: you have a working relationship with your own reading. The accumulated artefact gets to participate in the work, instead of being a pile of files that's growing faster than you'll ever read them.