Thursday, March 13, 2014

David J. Hand's "The Improbability Principle"

David J. Hand is an emeritus professor of mathematics and a senior research investigator at Imperial College London. He is the former president of the Royal Statistical Society and the chief scientific adviser to Winton Capital Management, one of Europe’s most successful algorithmic-trading hedge funds. He is the author of seven books, including The Information Generation: How Data Rules Our World and Statistics: A Very Short Introduction, and has published more than three hundred scientific papers. Hand lives in London, England.

He applied the “Page 99 Test” to his new book, The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day, and reported the following:
Page 99 of my new book, The Improbability Principle, falls squarely in the middle of a chapter entitled The Law of Truly Large Numbers. This is one of the more accessible of the five fundamental laws of the improbability principle, so it’s a great place to start to describe what the book’s about.

This law says that, with enough opportunities, any outrageous thing is likely to happen. To get 20 heads in a row, just keep tossing that coin for long enough. Have enough dreams, and sooner or later you’re bound to dream about something which happens the next day. Deal enough poker hands, and a royal flush is pretty well certain to come up.

But page 99 takes the law a bit further. It applies the law to the question of whether the suspension of F-14 Tomcat fighter aircraft flights was justified after one of them crashed in February 1996. The answer depends on how many aircraft were built and on how many crashed. (In case you are wondering, 712 and 161.)

The law of truly large numbers is pretty straightforward. But even it can have subtle aspects. One which is fairly well-known is the birthday problem. This asks how many people must be in a room for it to be more likely than not that some pair of them will have the same birthday. The answer, surprising if you have not thought carefully about what exactly the question asked, is just 23. The reason is that with 23 people there are 253 pairs - if not a “truly large” number, large enough relative to the 365 days in a year to make it more likely than not that at least one pair will have a common birthday.

The other laws of the improbability principle are the law of inevitability, the law of selection, the law of the probability lever, and the law of near enough. Together these laws make up the improbability principle, which says extremely improbable events are commonplace. Without going into the mathematics, but illustrating with many real-life rare events and extraordinary coincidences, the book shows how the principle works.
Visit the official The Improbability Principle website.

--Marshal Zeringue