The probability puzzle that makes your head melt

Scott Lazarus our consultant managing the role
Posting date: 1/21/2015 1:38 PM

A reference in a recent Magazine article to the Monty Hall problem - where a contestant has to pick one of three boxes - left readers scratching their heads. Why does this probability scenario hurt everyone's brain so much, asks maths lecturer Dr John Moriarty.

Imagine Deal or No Deal with only three sealed red boxes.

The three cash prizes, one randomly inserted into each box, are 50p, £1 and £10,000. You pick a box, let's say box two, and the dreaded telephone rings.

The Banker tempts you with an offer but this one is unusual. Box three is opened in front of you revealing the £1 prize, and he offers you the chance to change your mind and choose box one. Does switching improve your chances of winning the £10,000?

Each year at my university we hold open days for hordes of keen A-level students. We want to sell them a place on our mathematics degree, and I unashamedly have an ulterior motive - to excite the best students about probability using this problem, usually referred to as the Monty Hall Problem.


This mind-melter was alluded to in an AL Kennedy piece on change this week and dates back to Steve Selvin in 1975 when it was published in the academic journal American Statistician.

It imagines a TV game show not unlike Deal or No Deal in which you choose one of three closed doors and win whatever is behind it.

One door conceals a Cadillac - behind the other two doors are goats. The game show host, Monty Hall (of Let's Make a Deal fame), knows where the Cadillac is and opens one of the doors that you did not choose. You are duly greeted by a goat, and then offered the chance to switch your choice to the other remaining door.

Most people will think that with two choices remaining and one Cadillac, the chances are 50-50.

The most eloquent reasoning I could find is from Emerson Kamarose of San Jose, California (from the Chicago Reader's Straight Dope column in 1991): "As any fool can plainly see, when the game-show host opens a door you did not pick and then gives you a chance to change your pick, he is starting a new game. It makes no difference whether you stay or switch, the odds are 50-50."

But the inconvenient truth here is that it's not 50-50 - in fact, switching doubles your chances of winning. Why?

Let's not get confused by the assumptions. To be clear, Monty Hall knows the location of the prize, he always opens a different door from the one you chose, and he will only open a door that does not conceal the prize.

For the purists, we also assume that you prefer Cadillacs to goats. There is a beautiful logical point here and, as the peddler of probability, I really don't want you to miss it.

In the game you will either stick or switch. If you stick with your first choice, you will end up with the Caddy if and only if you initially picked the door concealing the car. If you switch, you will win that beautiful automobile if and only if you initially picked one of the two doors with goats behind them.

If you can accept this logic then you're home and dry, because working out the odds is now as easy as pie - sticking succeeds 1/3 of the time, while switching works 2/3 of the time.

Kamarose was wrong because he fell for the deception - after opening the door, the host is not starting a new 50-50 game. The actions of the host have already stacked the odds in favor of switching.

The mistake is to think that two choices always means a 50-50 chance. If Manchester United play Accrington Stanley in the Cup then, with the greatest respect to proud Stanley, it's more likely that United will progress to the next round.

Still not convinced? You are in good company. The paradox of the Monty Hall Problem has been incredibly powerful, busting the brains of scientists since 1975.

In 1990 the problem and a solution were published in Parade magazine in the US, generating thousands of furious responses from readers, many with distinguished scientific credentials.

Part of the difficulty was that, as usual, there was fault on both sides as the published solution was arguably unclear in stating its assumptions. Subtly changing the assumptions can change the conclusion, and as a result this topic has attracted sustained interest from mathematicians and riddlers alike.

Even Paul Erdos, an eccentric and brilliant Hungarian mathematician and one-time guest lecturer at Manchester, was taken in.

So what happens on our university's open days? We do a Monty Hall flash mob. The students split into hosts and contestants and pair up. While the hosts set up the game, half the contestants are asked to stick and the other half to switch.

The switchers are normally roughly twice as successful. Last time we had 60 pairs in 30 of which the contestants were always stickers and in the other 30 pairs always switchers:

Among the 30 switcher contestants, the Cadillac was won 18 times out of 30 - a strike rate of 60%Among the 30 sticker contestants, there were 11 successes out of 30, a strike rate of about 36%So switching proved to be nearly twice as successful in our rough and ready experiment and I breathed a sigh of relief.

I had calculated beforehand the chances of ending up with egg on my face and the team of 30 stickers beating the 30 switchers. It was a risk worth taking, but one shouldn't play Russian Roulette too often.

Next year we will need something different, perhaps Simpson's Paradox. Imagine that 1% of people have a certain disease.

A diagnostic test has been developed which performs as follows - if you have the disease, the test has a 99% chance of giving the result "positive", while if you do not have the disease, the test has 2% chance of (falsely) giving the result "positive".

A randomly chosen person takes the test. If they get the result "positive", what is the probability that they actually have the disease? The answer, 1/3, is perhaps surprisingly low.


Click here for the article on the web.

Related blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out the related posts below.

2020: The Year of the Data Engineer

Data Engineers are the architects of Data. They lay the foundation businesses use to collect, gather, store, and make Data usable. Each iteration of the Data as it moves along the pipeline is cleaned and analysed to be used by Data professionals for their reports and Machine Learning models. A ROLE IN HIGH DEMAND Even as businesses reopen, reassess, and for some, remain remote, the demand for Data Engineers is high. Computer applications, Data modelling, prediction modelling, Machine Learning, and more need Data professionals to lay the groundwork to help businesses benefit in today’s Data-driven culture. The word gets thrown around a bit, but when the majority of business has moved online, Data-driven is the name of the game. Having a Data plan, a Data team, and all aligned with your business strategy is imperative to the way business is done today. This type of innovation can offer insight for better business decisions, enhance customer engagement, and improve customer retention without missing a beat.  Without Data Engineers, Data Scientists can’t do their jobs. Understanding the amount of Data, the speed at which is delivered, and its variety need Engineers to create reliable and efficient systems. Like many Data professional jobs, even still in 2020, Data Engineers are in high demand. Yet a skills shortage remains. This has created an emerging field of professionals from other backgrounds who are looking to take on the role of Data Engineer and fill the gap. Whether by necessity or design, these individuals build and manage pipelines, automate projects, and see their projects through to the end result. CAREER OPPORTUNITIES OUTSIDE THE NORM As this growing trend emerges, it has created career opportunities for those with experience outside the normal channels of Data Engineering study. While it might involve individuals from backgrounds such as software Engineering, Databases, or something similarly IT-related, some businesses are upskilling their employees with talent. Rapid growth, reskilling, upskilling, and ever-constant changes still leave businesses with a shortage of Data Engineers to meet the demand. It’s critical to fill the gap for success. According to LinkedIn’s 2020 Emerging Jobs Report, Data Engineering is listed in the top 10 of jobs experiencing growth. THREE STEPS TOWARDS BECOMING A DATA ENGINEER This is a vital role in today’s organisations. So, if you’re in the tech industry and want to take a deeper dive into Data as a Data Engineer, what steps can you take? This is a time like no other. There’s time to assess your goals, take online classes, and get hands on with projects. Though having a base of computer science, mathematics, or business-related degree is always a good start. Be well-versed in such popular programming languages such as SQL, Python, R, Hadoop, Spark, and Amazon Web Services (AWS).Prepare for an entry-level role once you have your bachelor’s degree.Consider additional education to stay ahead of the curve. This can include not only professional certifications, but higher education degrees as well. The more experience, hands-on as well as academic, you have the more in demand you’ll be as a Data Engineer. Data scientists might be the rockstars of Data, but Data Engineers set the stage. As business processes have shifted online, looking for your next job has become more daunting than ever before. If you’re looking for your next opportunity in Data, take a look at our current jobs or get in touch with one of our expert consultants to find out more. 

How To Hire With Video Interviews

Virtual interviewing may have erupted over the last few months but the trends are showing that this is something that is likely to last well beyond the remote reality that many people are facing. Virtual interviewing is not as easy as it seems, in fact we’ve found our clients asking us over and over again for advice on how to run an effective video interview process. With that in mind, we’ve compiled a list of some of our top tips for clients and hiring managers for a successful video interview:  1. DON’T FORGET THE PRE-INTERVIEW PREP Confirm: Just like you would confirm a face to face interview with an email with the right address, instruction of how to get there and what to expect – the same applies for virtual interviews. Ensure to email candidates in advance with a link, information about who they are meeting and, most importantly, what you expect from a dress code. One of candidates biggest areas of concern is usually about what to wear for a virtual interview, so setting this out clearly in an email is a great way to start the process off on the right foot.Do not forget to provide instructions for using the video conferencing platform. Whether it is zoom, skype, google hangouts or another – keep in mind the candidate may not be familiar with your platform of choice.  Test:  Make sure to log onto to the interview early to ensure your camera, microphone and set up works. Be sure to ensure that your image is clear and that the volume is adequate. It is likely that the candidate will do the same and will ensure that the first few minutes of the interview aren’t focused on the technical side of things and ‘can you hear/see me?’.  2. PROVIDE A CLEAR STRUCTURE Opening: A usual face to face interview provides opportunity for warming a candidate up, however this time there is no shaking of hands and asking about commute.Just because you are video interviewing does not mean therefore that icebreakers shouldn’t exist, consider still incorporating an icebreaker to put the candidate at ease.  Ease concerns: One of the biggest concerns that candidates have when video interviewing is that there is a lot more out of their control in comparison to sitting in a meeting room opposite your interviewer. To ease any worries that the candidate might have, and to create a great candidate experience, let them know that background noise is okay and not to panic if the connection drops out. It’s likely that the candidate will have done everything they can to stop both of these from occurring, but ultimately, they could happen and it’s important the candidate knows that this will not negatively affect their outcome.  Set the agenda: Once you are through the icebreaker and have eased concerns, make sure to set an agenda for the interview. Let the candidate know what to expect. For example, introduction, CV run through, competency questions, Q&A and end. End the interview the right way, finish up by telling the candidates about the next steps and the timescales that you expect for that.  3. PREPARE THE QUESTIONS IN ADVANCE Due to the nature of video interviews, you will find the experience quite different to what you were used to. Usually you would have the CV and question sheet in front of you on the table, or on a laptop and the candidate separate to that. This time, you will potentially have all of that information on one screen. Preparing for how to optimise your screen and information therefore is important so that you can focus more on the candidate.  Read up on the candidate: Complete your CV read through and background prior to the interview to ensure that you do not need to rely wholly on the CV to make sense of the candidate’s answers. Don’t try and wing it: Prepare your questions in advance, have the questions in front of you and use them to help you to keep the interview on track and ensure all your questions get answered.  4. BE AWARE THAT EYE CONTACT IS DIFFERENT One of the biggest issues that clients and candidates alike feedback to us is that the concept of eye contact when video interviewing has as slightly different meaning. Having real eye contact in a virtual interview is challenging, because it means that you are going to be looking at the camera and not at the candidate, which takes some adjusting to.  Top Tips: Train yourself to look at the camera when you are talking, as this will give the candidate more of that personal feeling.Avoid the temptation to gape at your image on the screen, or the candidate when  you are speaking. If possible, turn off your picture so that the only image that shows on the screen is that of the candidate – this avoids the very familiar desire to look at oneself.  5. AVOID DIGITAL DISTRACTIONS There’s only so much you can do to stop your child running into the room, or your partner forgetting you’re on an interview and heading to the fridge but you can control the digital interruptions. It is important that you give the candidate your full attention. If your entire process is virtual, these are the sole ways that the candidate has to judge whether this is the right opportunity for them – so remember that this is a key part of their experience. Turn off notifications: Interviewing on a computer means that you are more likely to be distracted by your emails, IM messages, we’d advise turning off your notifications for both emails and IMs and closing all unnecessary tabs. Turn your phone onto airplane mode or DND. Harnham are currently supporting our clients within the Data & Analytics space on running completely remote interview processes for candidates. If you're looking to hire we can help you optimise your process in order to get the best talent then get in touch with one of our expert consultants. 

RELATED Jobs

Salary

US$120000 - US$140000 per annum + Additional Benefits

Location

McLean, Virginia

Description

An aviation industry partner is looking to bring on a Tableau Insights Analyst to join their Virginia office.

Salary

£50000 - £70000 per annum

Location

London

Description

An exciting role working for a technology-driven business who are in a growth stage. You will report directly into the Data Director and have lots of ownership

Salary

£50000 - £70000 per annum

Location

London

Description

This fintech brand received £200 million funding last year!

recently viewed jobs