Distill, don't summarise

A reading I did this week sent me down a useful rabbit hole. The argument was simple: when you ask an AI to "summarise" a messy meeting, you usually get a shorter version of the same vague soup. What you actually want is distillation, lifting out the load-bearing facts and leaving the filler behind.

So I built a skill to do exactly that, focusing most of my efforts on the part that matters most to me: not losing or bending the point while getting to it.

Before building, I had Claude research how to distill accurately. The clearest finding was that extraction (i.e., keeping the source's own words) is far more faithful than rewriting, but even extraction can mislead if it strips a negation, a condition or who said it. A great example from the article I read is a sarcastic "sure, I'll work all weekend" should be recognised for feelings of overload, not become an assigned shift.

My distill skill now frames every run with four questions before it does work to anchor retained facts back to the source, separates how sure it is that something was said from how likely it is to hold, and keeps a "what I might have lost" log. Then I red-teamed it on six transcripts: a hidden instruction, an overturned decision, a fact that was not there. It held on all six, and I ran it on a real call of my own to kick the tires some more.

So far (aka two times I've used it) the results have been delightfully good. The skill has framing context, which (surprise!) makes the results better. I'm excited at the thought of having more targeted and valuable info coming out of my transcripts. We'll see!

Link to the article that inspired all this: https://jtnovelo2131.substack.com/p/what-is-distillation-in-ai-why-summarise