A sequel to the architecture guide. Part 1 explained how the search engine was built. This one explains how we found out whether it was any good, tried four ideas to improve it, and shipped the two that actually worked. If you read Part 1, you know the setup: I'd been collecting technical books for years — mostly Humble Bundle hauls — and had over 100 of them sitting on a hard drive, mostly unread. So I built a semantic search engine over the whole library, exposed as an MCP server Claude could query. Ask a question, get back real passages from Neal Ford or Vaughn Vernon, instead of an AI's vague recollection of them. By the end of Part 1, it worked. But "it worked" and "it's good" are different claims, and I'd only ever checked the first one. This post is the story of taking the second one seriously. The whole thing follows from one sentence, taken literally: You cannot improve what you cannot measure. The question that started this Whe...
Corinna Cohn—finding the right questions