If anyone has read James Surowiecki's book, the Wisdom of Crowds, they will be familiar with the story of the US submarine, Scorpion, that disappeared on its way back to land. Despite the Navy holding various bits of data about the sub, such as its location the last time it had made radio contact they had to find it in a possible circular area which had a 20 mile diameter and thousands of feet deep.

To solve this problem, a naval officer called John Craven created a series of scenarios to explain what had happened to the Scorpion. He then asked an eclectic mix of experts, such as mathmaticians, salvage men and submarine specialists to guess which of the scenarios was most likely. The experts guessed the speed the sub was going, why it ran into trouble and the steepness of the decent and other relevant information. Craven didn't select any one of the guesses, but took the guesses together and applied Bayes's theorum to them to get essentially the mean location of the sub.

The final location was a genuinely a collective judgement of where the submarine was but did not match any of the expets guesses exactly. Five monthas after the submarine disappeared a navy ship found it, just 220 yards fom where Craven's group had said it would be.

This story shows just how smart a crowd can be when it wants. Despite having little or no hard data, the experience and combined knowledge of the experts was accurate to a fraction of a percent.

A British psychologist, Gordon Rugg, has taken the theory behind Craven's success to another level. He has come up with the Varifier Approach. Rugg recognised that in all fields there is an 'expertise gap' that is often unrecognised and usually ignored. The varifier approach asks experts to draw a mental map of their field. From there Rugg puts different maps from different experts together to form an atlas of knowledge on any given subject. In fact 11% of papers published in Nature and the British Medical Journal had serious statistical errors, showing that experts often suffer fuzzy logic. Rugg explains his theory:


"You look for an area of overlap that doesn't contain much detail... If it turns out there's an adjoining area which everyone thinks is someone else's territory, then that's a potential gap."

Using this theory Rugg studied the Voynich Manuscript, a hand-lettered book written in an unknown code that has frustrated cryptographers since its discovery in an Italian villa in 1912. He discovered that if he focussed on all the thoughts that experts had ignored i.e. their knowledge gaps, he could see what they had missed and solve complex problems.


'In three months, he cooked up the most persuasive explanation yet for the 234-page text: Sorry, folks, there is no code - it's a hoax! Lifelong Voynichologists were impressed with his reasoning and proofs, even if they were a little chagrined. "The Voynich is such a challenge," says Rugg, "such a social activity. But then along comes someone who says 'Oh, it's just a lot of meaningless gibberish.' It's as if we're all surfers, and the sea has dried up."'

"The verifier method boils down to seven steps: 1) amass knowledge of a discipline through interviews and reading; 2) determine whether critical expertise has yet to be applied in the field; 3) look for bias and mistakenly held assumptions in the research; 4) analyze jargon to uncover differing definitions of key terms; 5) check for classic mistakes using human-error tools; 6) follow the errors as they ripple through underlying assumptions; 7) suggest new avenues for research that emerge from steps one through six."

The Varifier Approach demonstrates the power of the Medici Effect, combining more than two unrelated areas of expertise to create a more powerful and new idea, something that the best advertising and marketing does.

Rugg now hopes to apply the Varifier Approach to a range of subjects such as the origins of the universe and Alzheimer's disease.

Wired Article

Originally pulished on IF!