The probability puzzle that makes your head melt

Scott Lazarus our consultant managing the role
Posting date: 1/21/2015 1:38 PM

A reference in a recent Magazine article to the Monty Hall problem - where a contestant has to pick one of three boxes - left readers scratching their heads. Why does this probability scenario hurt everyone's brain so much, asks maths lecturer Dr John Moriarty.

Imagine Deal or No Deal with only three sealed red boxes.

The three cash prizes, one randomly inserted into each box, are 50p, £1 and £10,000. You pick a box, let's say box two, and the dreaded telephone rings.

The Banker tempts you with an offer but this one is unusual. Box three is opened in front of you revealing the £1 prize, and he offers you the chance to change your mind and choose box one. Does switching improve your chances of winning the £10,000?

Each year at my university we hold open days for hordes of keen A-level students. We want to sell them a place on our mathematics degree, and I unashamedly have an ulterior motive - to excite the best students about probability using this problem, usually referred to as the Monty Hall Problem.


This mind-melter was alluded to in an AL Kennedy piece on change this week and dates back to Steve Selvin in 1975 when it was published in the academic journal American Statistician.

It imagines a TV game show not unlike Deal or No Deal in which you choose one of three closed doors and win whatever is behind it.

One door conceals a Cadillac - behind the other two doors are goats. The game show host, Monty Hall (of Let's Make a Deal fame), knows where the Cadillac is and opens one of the doors that you did not choose. You are duly greeted by a goat, and then offered the chance to switch your choice to the other remaining door.

Most people will think that with two choices remaining and one Cadillac, the chances are 50-50.

The most eloquent reasoning I could find is from Emerson Kamarose of San Jose, California (from the Chicago Reader's Straight Dope column in 1991): "As any fool can plainly see, when the game-show host opens a door you did not pick and then gives you a chance to change your pick, he is starting a new game. It makes no difference whether you stay or switch, the odds are 50-50."

But the inconvenient truth here is that it's not 50-50 - in fact, switching doubles your chances of winning. Why?

Let's not get confused by the assumptions. To be clear, Monty Hall knows the location of the prize, he always opens a different door from the one you chose, and he will only open a door that does not conceal the prize.

For the purists, we also assume that you prefer Cadillacs to goats. There is a beautiful logical point here and, as the peddler of probability, I really don't want you to miss it.

In the game you will either stick or switch. If you stick with your first choice, you will end up with the Caddy if and only if you initially picked the door concealing the car. If you switch, you will win that beautiful automobile if and only if you initially picked one of the two doors with goats behind them.

If you can accept this logic then you're home and dry, because working out the odds is now as easy as pie - sticking succeeds 1/3 of the time, while switching works 2/3 of the time.

Kamarose was wrong because he fell for the deception - after opening the door, the host is not starting a new 50-50 game. The actions of the host have already stacked the odds in favor of switching.

The mistake is to think that two choices always means a 50-50 chance. If Manchester United play Accrington Stanley in the Cup then, with the greatest respect to proud Stanley, it's more likely that United will progress to the next round.

Still not convinced? You are in good company. The paradox of the Monty Hall Problem has been incredibly powerful, busting the brains of scientists since 1975.

In 1990 the problem and a solution were published in Parade magazine in the US, generating thousands of furious responses from readers, many with distinguished scientific credentials.

Part of the difficulty was that, as usual, there was fault on both sides as the published solution was arguably unclear in stating its assumptions. Subtly changing the assumptions can change the conclusion, and as a result this topic has attracted sustained interest from mathematicians and riddlers alike.

Even Paul Erdos, an eccentric and brilliant Hungarian mathematician and one-time guest lecturer at Manchester, was taken in.

So what happens on our university's open days? We do a Monty Hall flash mob. The students split into hosts and contestants and pair up. While the hosts set up the game, half the contestants are asked to stick and the other half to switch.

The switchers are normally roughly twice as successful. Last time we had 60 pairs in 30 of which the contestants were always stickers and in the other 30 pairs always switchers:

Among the 30 switcher contestants, the Cadillac was won 18 times out of 30 - a strike rate of 60%Among the 30 sticker contestants, there were 11 successes out of 30, a strike rate of about 36%So switching proved to be nearly twice as successful in our rough and ready experiment and I breathed a sigh of relief.

I had calculated beforehand the chances of ending up with egg on my face and the team of 30 stickers beating the 30 switchers. It was a risk worth taking, but one shouldn't play Russian Roulette too often.

Next year we will need something different, perhaps Simpson's Paradox. Imagine that 1% of people have a certain disease.

A diagnostic test has been developed which performs as follows - if you have the disease, the test has a 99% chance of giving the result "positive", while if you do not have the disease, the test has 2% chance of (falsely) giving the result "positive".

A randomly chosen person takes the test. If they get the result "positive", what is the probability that they actually have the disease? The answer, 1/3, is perhaps surprisingly low.


Click here for the article on the web.

Related blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out the related posts below.

MeasureCamp Berlin

MeasureCamp Berlin: A Preview

In preparation for this year's MeasureCamp Berlin, we sat down with Benjamin Bock, communications lead, to discuss what to expect, as well as his thoughts on the industry in general. Here's what he had to say: Can you explain MeasureCamp for people who haven’t been yet? MeasureCamp is an open, free-to-attend analytics 'un-conference' made by analytics professionals for analytics professionals (and everyone who wants to get there) around the globe. In that sense, it’s different to any conference you know of. Our schedule is created on the day of the event, and our speakers are fellow attendees. Listen to talks, give a talk, and discuss topics that really tickle your fancy. What can we expect at MeasureCamp Berlin this year? Let’s begin with what you can’t and never will expect at MeasureCamp Berlin: Sales pitch presentations. We’ve all been there… you are visiting a fancy, expensive conference and all you get is Heads of 'This n’ That' talking about what their team did, what they spent money on and that you should buy Product X to be as Data-driven as them (mind the cynicism). At MeasureCamp you can expect talks and discussion rounds by around 150 fellow experts, who all know the daily adventures of cleaning Data, setting up analytics or debugging tracking code or running mind-bending analysis first hand.  What is your best tip for someone that has never been at MeasureCamp before? Don’t rush it! MeasureCamp is about mingling with the analytics community as much as it is about the talks and discussion rounds. Pick a few talks that really interest you and use the rest of the day to get to know other attendees. Our awesome sponsors are also more than happy to talk to you. What is the best advice you got last year at MeasureCamp? On a personal level, I was able to get some really good advice when it came to data privacy topics. GDPR was still fairly fresh and nobody really knew if what they had done was actually enough to not get into trouble. That’s the kind of advice you only get if you have the chance to talk to other professionals face to face. On another note, what are the most sought-after skills and technologies currently used? I can only speak of my experience here. On a hard skill level and depending on the individual role, you need a solid understanding of web technologies (JavaScript, HTML, CSS) and tag managing systems to be able to implement tracking (plus some knowledge in mobile development when your focus lies on apps). When it comes to analysing and visualising Data, you should understand the tool you are working with and its underlying Data-structures. Being able to retrieve tool-agnostic Data with SQL and running more sophisticated calculations (e.g. with Python) has become more and more important over the last few years. But there are some softer skills, that should not be overlooked as well. As an analytics professional, you should never assume that your knowledge and language are common ground. You need to be a strong communicator, who is able to explain complicated concepts broken down to the absolute basics. In your opinion, what will be the biggest challenge in digital analytics in the next year? Two weeks ago, I would have answered “bringing web and app Data together”. Now that we know Google is working on that topic, it’s still a challenge, but one I am happy to tackle in the coming year. Digital Analytics is constantly changing. What do you expect to be the most talked about topic at MeasureCamp this year? As a Tracking Specialist with a focus on Google products, I’d love to hear some talks about Google Tag Manager Custom Templates. But my top guess is, that the newly released Apps and Web properties beta for Google Analytics will be the talk of the hour. MeasureCamp Berlin is an open and free-to-attend 'un-conference', taking place this year on the 28th of September. The final batch of tickets will be released on the 21st of August at 03:00 PM (CEST). Click here for more information and to get hold of your place. 

Where Tech Meets Tradition

Where Tech Meets Tradition

If you’re lamenting the decline of handmade traditional products, cast your cares aside. There’s a new Sheriff in town and its name is, Tech. Just a generation ago, children would leave the farm or the family business, go to school, and then move on to make their place in the world doing their own thing. Away from family.  Today, the landscape has changed and those who have left are coming home. But this time, they’re bringing technology with them to help make things more efficient and more productive. Is Tech-Assisted Still Handmade? In a word, yes. Artists still make things “from scratch”, except now technologies allow them to not only see their vision in real-time, but their customers, too. Have you ever wondered what the image in your head might look like on paper or in metal? What about the design of prosthetic arms and healthcare devices by 3D printers? You’re still designing, creating.  But just like any new technology, there’s still a learning curve. Even for cutting-edge craftspeople who find that sometimes, the line between craftsmanship and high-tech creativity may be a bit of a blur. Not to mention the expense for either the equipment required or being able to offer art using traditional tools at technology-assisted prices. Somewhere between the two, there is a trade-off. It’s up to the individual to determine where and what that trade-off is. Life in the Creative Economy One of Banksy’s paintings shredded itself upon purchase at an auction recently. AI is making music and writing books. Augmented Reality, Virtual Reality, and Blockchain all have their place in the creative economy from immersive entertainment to efficient manufacturing processes. Each of these touches the way we live now. In a joint study between McKinsey and the World Economic Forum, 'Creative Disruption: The impact of emerging technologies on the creative economy', the organisations broke down the various technologies used in the creative economy and how they’re driving change. For example: AI is being used to distill user preferences when it comes to curating movies and music. The Associated Press has used AI to free up reporters’ time and the Washington Post has created a tool to help it generate up to 70 articles a month, many stories of which they wouldn’t have otherwise dedicated staff.Machine Learning has begun to create original content. Virtual Reality and Augmented Reality have come together as a new medium to help move people to get up, get active, and go play whether it’s a stroll through a virtual art gallery or watching your children play at the playground.  Where else might immersive media play out? Content today could help tell humanitarian stories or offer work-place diversity training. But back to the artisan handicrafts.  Artistry with technology Whilst publishing firms may be looking to use AI to redefine the creative economy, they are not alone. Other artists utilising these technologies include:  SculptorsDigital artistsPaintersJewellery makersBourbon distillers America’s oldest distiller has gotten on the technology bandwagon and while there is no rushing good Bourbon, but you can manage the process more efficiently. They’ve even taken things a step further and have created an app for aficionados to follow along in the process. Talk about crafted and curated for individual tastes and transparency. It may seem almost self-explanatory to note how other artisans are using technology. But what about distilleries? What are they doing? They’re creating efficiency by: Adding IoT sensors for Data Analytics collection Adding RFID tags to their barrels Creating experimental ageing warehouses (AR, anyone?) to refine their craft. Don’t worry, though. These changes won’t affect the spirit itself. After all, according to Mr. Wheatley, Master Distiller, “There’s no way to cheat mother nature or father time.” Ultimately, the idea is to not only understand the history behind the process, but to make it more efficient and repeatable. A way to preserve the processes of the past while using the advances of the present with an eye to the future. If you’re interested in using Data & Analytics to drive creativity, we may have a role for you. Take a look at our latest opportunities or get in touch with one of our expect consultants to find out more. 

RELATED Jobs

Salary

£400 - £450 per day

Location

Manchester, Greater Manchester

Description

A top price comparison site is looking to add a talented Talend Data Engineer to their Manchester team helping them to better understand customer data.

Salary

£50000 - £60000 per annum

Location

London

Description

Are you a commercially minded Insight Analyst with a strong numerical stats background? This is company that places TECH at the heart of their business

Salary

£45000 - £50000 per annum + Bonus + Benefits

Location

Nottingham, Nottinghamshire

Description

We are working with one of the largest and best known Credit Card providers in the UK. They are seeking a Fraud Strategy Analyst.

Salary

£40000 - £55000 per annum

Location

West London, London

Description

Looking to move away from campaign analytics? Want to use your strategic skills to their fullest? Fancy a client-side role? this is for you!

recently viewed jobs