Now
What I tested
I built a model of one operation over about two years, knowing the business cold. The frozen answer key holds 63 hard operating constraints, the ones that make a trip invalid, unsafe or non-compliant if broken. That is not the size of the rule set. It is the size of the part that stops a trip. Gear checklists, site practice and administrative policy take the working total to several hundred.
The question that decides everything else: can rules like those be found by analysis instead of by memory?
So I built a battery of analyses and pointed it at that operation's own data, with the rule tables deliberately withheld, then measured what share of the rules it recovered with me out of the room. Candidates were pre-registered before scoring, and I tracked three numbers rather than one: how many known rules it finds, how many of its proposals are real, and how many real rules it finds that two years of manual work missed. That third number turned out to be the interesting one.
This is requirements elicitation, and there is a research literature on it that has moved quickly in the last year. What that work mostly lacks is an operation with ground truth to test against. I have one, which is the only reason my version of this was worth running. The long version is here, including why this was tried once before and named as the reason it failed.
Recover most of them and the method transfers to businesses I do not know. Recover few and the value was my judgment in a room, which is a different business with a different price on it.
The number is ten percent. Six of the 63. By my own stated criterion, that is judgment rather than a method. The same run also surfaced 18 real constraints that two years of building had never written down, and 14 of those came from one place: the log of every time a person overruled the system.
A second experiment read nineteen days of one season's internal channels, cold, against a 39-rule baseline. It came back with 48 more. Nineteen days, one source, one season, and it listed more rules than the whole hard-constraint key. Whatever the real number is, no operating model I have built has been close to complete.
The whole thing is written up in Ten percent came back. The runs, the pre-registration, the scoring and the mistakes are at github.com/salmonriverops/constraint-battery.
What is live
Operations installs for service businesses, described on the work with me page.
Talks are being lined up with industry associations. Conversations in progress with two booking platform founders about what it would take for an outside system to act on their data rather than only read it.
What I got wrong recently
I spent a long time believing the hard part of this work was building the software. It is not, and it has not been for years. The hard part is writing down how a business actually works, and I could not see that because in my own operation I already knew every answer, so it cost me nothing.
I also assumed the expensive step was the interview. It is not quite that either. Nobody who runs an operation can recall their rules on request, myself included. What works is showing people their own data and letting them react to it, which is a far easier thing to ask.
What I am looking for
A service business where one person answers "can we do it, by when, for how much" all day from memory, and where things exist that can only be in one place at a time. Ninety minutes, and I will tell you what I find.
If you work on eliciting requirements or operational rules from people who cannot articulate them, whether in research or in a product, I would like to compare notes. There are not many of us and I do not think we are talking to each other.