The probability puzzle that makes your head melt

Scott Lazarus our consultant managing the role
Posting date: 1/21/2015 1:38 PM

A reference in a recent Magazine article to the Monty Hall problem - where a contestant has to pick one of three boxes - left readers scratching their heads. Why does this probability scenario hurt everyone's brain so much, asks maths lecturer Dr John Moriarty.

Imagine Deal or No Deal with only three sealed red boxes.

The three cash prizes, one randomly inserted into each box, are 50p, £1 and £10,000. You pick a box, let's say box two, and the dreaded telephone rings.

The Banker tempts you with an offer but this one is unusual. Box three is opened in front of you revealing the £1 prize, and he offers you the chance to change your mind and choose box one. Does switching improve your chances of winning the £10,000?

Each year at my university we hold open days for hordes of keen A-level students. We want to sell them a place on our mathematics degree, and I unashamedly have an ulterior motive - to excite the best students about probability using this problem, usually referred to as the Monty Hall Problem.


This mind-melter was alluded to in an AL Kennedy piece on change this week and dates back to Steve Selvin in 1975 when it was published in the academic journal American Statistician.

It imagines a TV game show not unlike Deal or No Deal in which you choose one of three closed doors and win whatever is behind it.

One door conceals a Cadillac - behind the other two doors are goats. The game show host, Monty Hall (of Let's Make a Deal fame), knows where the Cadillac is and opens one of the doors that you did not choose. You are duly greeted by a goat, and then offered the chance to switch your choice to the other remaining door.

Most people will think that with two choices remaining and one Cadillac, the chances are 50-50.

The most eloquent reasoning I could find is from Emerson Kamarose of San Jose, California (from the Chicago Reader's Straight Dope column in 1991): "As any fool can plainly see, when the game-show host opens a door you did not pick and then gives you a chance to change your pick, he is starting a new game. It makes no difference whether you stay or switch, the odds are 50-50."

But the inconvenient truth here is that it's not 50-50 - in fact, switching doubles your chances of winning. Why?

Let's not get confused by the assumptions. To be clear, Monty Hall knows the location of the prize, he always opens a different door from the one you chose, and he will only open a door that does not conceal the prize.

For the purists, we also assume that you prefer Cadillacs to goats. There is a beautiful logical point here and, as the peddler of probability, I really don't want you to miss it.

In the game you will either stick or switch. If you stick with your first choice, you will end up with the Caddy if and only if you initially picked the door concealing the car. If you switch, you will win that beautiful automobile if and only if you initially picked one of the two doors with goats behind them.

If you can accept this logic then you're home and dry, because working out the odds is now as easy as pie - sticking succeeds 1/3 of the time, while switching works 2/3 of the time.

Kamarose was wrong because he fell for the deception - after opening the door, the host is not starting a new 50-50 game. The actions of the host have already stacked the odds in favor of switching.

The mistake is to think that two choices always means a 50-50 chance. If Manchester United play Accrington Stanley in the Cup then, with the greatest respect to proud Stanley, it's more likely that United will progress to the next round.

Still not convinced? You are in good company. The paradox of the Monty Hall Problem has been incredibly powerful, busting the brains of scientists since 1975.

In 1990 the problem and a solution were published in Parade magazine in the US, generating thousands of furious responses from readers, many with distinguished scientific credentials.

Part of the difficulty was that, as usual, there was fault on both sides as the published solution was arguably unclear in stating its assumptions. Subtly changing the assumptions can change the conclusion, and as a result this topic has attracted sustained interest from mathematicians and riddlers alike.

Even Paul Erdos, an eccentric and brilliant Hungarian mathematician and one-time guest lecturer at Manchester, was taken in.

So what happens on our university's open days? We do a Monty Hall flash mob. The students split into hosts and contestants and pair up. While the hosts set up the game, half the contestants are asked to stick and the other half to switch.

The switchers are normally roughly twice as successful. Last time we had 60 pairs in 30 of which the contestants were always stickers and in the other 30 pairs always switchers:

Among the 30 switcher contestants, the Cadillac was won 18 times out of 30 - a strike rate of 60%Among the 30 sticker contestants, there were 11 successes out of 30, a strike rate of about 36%So switching proved to be nearly twice as successful in our rough and ready experiment and I breathed a sigh of relief.

I had calculated beforehand the chances of ending up with egg on my face and the team of 30 stickers beating the 30 switchers. It was a risk worth taking, but one shouldn't play Russian Roulette too often.

Next year we will need something different, perhaps Simpson's Paradox. Imagine that 1% of people have a certain disease.

A diagnostic test has been developed which performs as follows - if you have the disease, the test has a 99% chance of giving the result "positive", while if you do not have the disease, the test has 2% chance of (falsely) giving the result "positive".

A randomly chosen person takes the test. If they get the result "positive", what is the probability that they actually have the disease? The answer, 1/3, is perhaps surprisingly low.


Click here for the article on the web.

Related blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out the related posts below.

It Takes Two: Data Architect Meets Data Engineer

Information. Data. The lifeblood of business. Information and data are used interchangeably, gathered, collected, and analysed to create actionable insights for informed business decisions. So, what does that mean exactly? And to that end, how do we know what information or data we need to make those decisions? Enter the Data Architect. The Role of a Data Architect Just like you might hire an architect to sketch out your dreamhouse, the Data Architect is a Data Visionary. They see the full picture and can craft the design and framework creating the blueprint for the Data Engineer, who will ultimately build the digital framework. Data Architects are the puzzle solvers who can take a jumble of puzzle pieces, in this case massive amounts of data, and put everything in order. It’s their job to figure out what’s important and what isn’t based on an organisation's business objectives. Skills for a Data Architect might include: Computer Science degree, or some variation thereof.Plenty of experience working with systems and application development.Extensive knowledge and able to deep dive into Information ManagementIf you’re just starting your Data Architect path, be prepared for years of building your experience in data design, data storage, and Data Management. The Role of a Data Engineer The Data Engineer builds the vision and brings it to life. But they don’t work in a vacuum and are integral to the Data Team working nearly in tandem with the Data Architect. These engineers are building the infrastructure – the pipelines and data lakes. Once exclusive to the software-engineering field, the data engineer’s role has evolved exponentially as data-focused software became an industry standard. Important skills for a Data Engineer might include. Strong developer skills.Understand a host of technologies such as Python, R, Hadoop, and moreCraft projects to show what you can do, not just talk about what you can do – your education isn’t much of a factor when it comes to data engineering. On the job training does it best.Social and communication skills are critical as you map initial designs, and a love of learning keeps everything humming along, even as technology libraries shift, and you have to learn something new. The Major Differences between the Data Architect and Data Engineer RolesAs intertwined as these two roles might seem, there are some crucial differences. Data Architect Crafts concept and visualises frameworkLeads the Data Science teams Data Engineer Builds and maintains the frameworkProvides supporting framework With a focus on Database Management technologies, it can seem as though Data Architect and Data Engineer are interchangeable. And at one time, Data Architects did also take on the Data Engineering role. But the knowledge each has is used differently.  Whether you’re looking to enter the field of Data Engineering, want to move up or over with your years of experience to Data Architect, or are just starting out. Harnham may have a role for you. Check out our current opportunities or get in touch with one of our expert consultants to learn more.  

Trade Analysts Keep Money Flowing on the Field

What if you could manage risk and build a winning team the way Billy Bean does in Moneyball? If you’ve never seen the movie, it’s essentially this. You don’t need the best to win, the players who will cost you the most money or who are the most popular. You need players whose sole skill is to get on base. When it comes to the world of finance, how might you manage risk and find ways to get on base so to speak?  You may want to consider a Trade Analyst. Conversely, if you’re a data professional who’s got a nose for numbers, predictions, and the aptitude to get on base yourself, you may want to consider this as your next role. Not unlike so many Data Analyst jobs, you’re using Data to determine risk as well as deep dive into SWOT (strengths, weakness, obstacles, and threats) for your business. You’ll be managing statistics and pinpointing the best times of the day for optimal trading.  A Key Player in the World of Trade Much like a stockbroker begins when the markets open, so too, does a Trade Analyst. Your mission, should you choose to accept it, is to run point between the stockholders and those for whom they’re buying and selling.  Looking for puzzle solvers with an eye for detail and investigation, this role offers work with people from around the world. And as we continue, or as this year comes to a close, begin to cement our remote working opportunities, the world opens a host of opportunities for this role and many like it. What You Need to Know Buzz words abound in the data space and the classification for Trade Analyst can also be Financial Services Agent. Perhaps FSA is better as it gives a much more concise idea of what the job entails. However, Trader Analyst likens to a version of a Stock Broker who can drill down to the sharpest point what works, what will sell, what won’t, and how to fix what won’t work to what will.  While education is important for this role, the soft skills so in demand will be required here, too. Can you be the calm in the chaos? Does making the sale motivate you? Can you think on your feet? If you answered yes to any of these questions, here are a few education and skills components you’ll need to know. Degree in international business is a good place to start as is a degree in finance, economics, or logisticsAdd in a second language for good measureStrong research skills.Understanding financial trends within and across geographic regionsUnderstanding supply and demandHighly communicative with staff, executives, stakeholders, and the public. Not unlike a language professional who roles easily from a foreign language to English and back again, a Trade Analyst must be able to translate numbers and predictions into the language of persuasive bargaining. Market analysis conducted through such platforms as polls and surveys. This role offers job security for the professional who comes alive in a fast-paced environment within the world of business. Your wallet and bank account may thank you, too.  Going to the Show In baseball, going to the show implies you’re in the major leagues. That you’ll perform on the field of a major league team. You’re officially ‘on stage’. And so, it is with your role, even entry-level, of a Trade Analyst. From the moment you’re in the office and the phone rings to the final closing bell of the exchange, you’re on the field, and playing with the heavy hitters. You’ll identify risk, engage with customers, pay attention to the score, er deliverables and expectations, all the while staying in compliance with regulations.  If you’re looking for a role in Data & Analytics or are interested in finance or international trade analysis, we may have a role for you. Take a look at our latest opportunities or get in touch with one of our expert consultants to learn more.  

RELATED Jobs

Salary

US$135000 - US$150000 per annum

Location

New York

Description

If you're looking to assist fortune 500 companies to build their online brand, this role could be the next step in your career!

Salary

US$130000 - US$160000 per annum

Location

New York

Description

If you consider yourself an expert in modeling and predicting customer behavior, this could be the next step in your career!

Salary

US$100000 - US$120000 per annum

Location

Pittsburgh, Pennsylvania

Description

An exciting start-up is developing autonomous drones for inventory management.

recently viewed jobs