The knowledge acquisition bottleneck, forty years on
I have been telling people that the cost of writing a business down collapsed.
I want to correct that, because it is not quite true, and the version that is true is more interesting.
They tried this before, and they named why it failed
In the 1980s a large amount of money and talent went into expert systems. The idea was that if you could get an expert’s knowledge into a machine as rules, the machine could do the expert’s reasoning. Some of them worked. MYCIN, built at Stanford, diagnosed blood infections and recommended treatment. In a blinded evaluation published in JAMA in 1979, eight infectious disease experts rated its therapy recommendations acceptable 65 percent of the time, against a range of 42.5 to 62.5 percent for the Stanford faculty specialists it was compared with.
The reasoning was never the problem. The problem was upstream.
To build one of those systems, a knowledge engineer sat with an expert and asked them to state their rules. This turned out to be extraordinarily slow, and worse than slow, unreliable. Experts gave you the tidy version. They forgot the exceptions. They described what they believed they did rather than what they did. Every project spent most of its budget on the interview and still shipped with an incomplete model.
The field named this. They called it the knowledge acquisition bottleneck. By 1988, Cullen and Bryman were asking in the journal Expert Systems whether it was time for a reassessment. It was not a minor annoyance. It was the structural reason the category never reached the scale it promised.
Then the money left, the field went quiet, and the term mostly stopped being said out loud.
Which should give anyone pause, including me
So when I say that writing a business down got cheap, the correct response is that an entire industry bet on exactly that and lost.
If you are going to make a claim that has been falsified once already, you had better be able to say what is different, and “the models are better now” is not an answer. Reasoning was not the bottleneck in 1985 either. MYCIN’s inference worked. The knowledge did not arrive.
I did not know any of this when I started. I found it after the fact, which is a slightly embarrassing thing to admit and also the reason I trust the conclusion more than I did before.
What actually changed
The bottleneck did not get solved. It got routed around.
Knowledge acquisition failed because the method was articulation. You ask a person to state a rule, and people cannot state their rules. Not because they are hiding them. Because the knowledge is procedural, it has never been in words, and asking for it in words is asking for a translation nobody has ever performed.
Forty years of research confirmed that and did not fix it. I do not think a better interviewer fixes it either, human or otherwise.
What I did in my own operation was not articulation. I pulled the data out into a model I owned, and then I sat and looked at it while something reasoned over it with me. I was not remembering rules. I was being shown things and reacting.
I knew we had vehicles. I could not have told you how to define one. Put the record in front of me in a shape I could interrogate and the definitions fell out, along with a considerable number of rules I did not know we had.
That is a different cognitive act. Recall against recognition. Recall is what the knowledge engineer asked for and it is the thing people are bad at. Recognition is easy, it is fast, and it works reliably in people who cannot introspect at all. Show someone a fact about their own business that looks wrong and they will tell you why in about four seconds, and the reason is the rule.
Two things had to exist for that route to open, and neither did in 1985. The operational record has to be queryable, which means it has to be somewhere other than paper and a person’s memory. And something has to be able to reason over it conversationally, at the speed of a conversation, without a research team.
Both arrived recently. Neither was aimed at this problem.
This is not a lone insight, and it is worth saying so
The derivation half of this is already a real industry. It is called process mining, and it is a mature, well funded category that reconstructs how a business works from its own event logs rather than from what people say it does. It works, and it validates the part of my argument I was least sure of.
It is also not what I am doing, for two reasons that matter.
Process mining discovers processes. I am after constraints. Those are different objects. A process is a sequence: this happens, then that happens, and here is where it stalls. A constraint is a boundary: this cannot happen at the same time as that, this person is not allowed to do this, this needs an hour before it can go out again. Sequence tells you how work flows. Constraint tells you what will break. Almost all of the operating knowledge in a small service business is the second kind, and I have not found anyone modelling it there.
And it needs event logs. Those tools read structured event data out of enterprise systems. A two million dollar service business does not produce that. One of the vendor comparisons in this space concedes that around seventy percent of back office work happens between systems, in email and spreadsheets and messages, where the logs cannot see it. That gap is not an edge case for me. It is the entire customer.
Elsewhere the same architecture keeps appearing: machine proposes, human confirms. Data catalog companies are shipping agents that infer what a data field means and route the judgment to a person to certify. Others extract business rules out of legacy code the same way. The pattern is converging and I do not think it belongs to anyone. What differs is the object and the setting: operational constraints, in businesses with no event log and no source code, where the rules were never written anywhere at all.
Where it still does not work
I would rather say this myself than have it found.
Working back through my own operation, the constraints sort into about a dozen shapes. Most of them can eventually be proposed from the record. Three cannot, and it is worth knowing which.
Which two things should never be put together, when both are individually normal and the reason is judgment rather than policy.
What makes an operator uneasy without stopping them. The advisory threshold. It exists entirely in a person’s discomfort and leaves no trace.
What to assume when the data is missing. Every operation runs on a set of default assumptions that somebody made once and nobody has revisited. They are invisible, they are frequently wrong after a few seasons of drift, and no amount of looking at the record surfaces them, because the record is what the assumption produced.
Three shapes out of twelve. That is the part that is still an interview, and I suspect it stays one.
There is also a failure mode in the method itself that I take seriously. A rule derived from data can be false. Something that never happened might be forbidden, or might just never have come up. That distinction is invisible in a record and obvious to an operator, which is exactly why the human stays in the loop rather than being a courtesy.
What I am testing
I built a model of one operation over about two years while knowing the business completely. It holds well over a hundred distinct rules.
So I am pointing the analyses at that operation’s own data, with the rule tables deliberately withheld, and measuring what share of those rules come back with me out of the room. Candidates get written down before scoring, because I cannot blind myself to a business I run and pre-registering is the only honest substitute.
If most of them come back, the route around the bottleneck is real and it will work on a business I have never seen. If very few do, then what happened here was that I already knew every answer, and I should say so.
That is a strange thing to publish before knowing the result. It also seems like the only honest way to write about a claim that has been wrong once before. The current state of that test is on my now page.
If you work on getting rules or requirements out of people who cannot state them, in research or in a product, I would like to compare notes. Judging by how long this problem has had a name, there should be more of us talking. Get in touch.