The probability puzzle that makes your head melt



A reference in a recent Magazine article to the Monty Hall problem - where a contestant has to pick one of three boxes - left readers scratching their heads. Why does this probability scenario hurt everyone's brain so much, asks maths lecturer Dr John Moriarty.

Imagine Deal or No Deal with only three sealed red boxes.

The three cash prizes, one randomly inserted into each box, are 50p, £1 and £10,000. You pick a box, let's say box two, and the dreaded telephone rings.

The Banker tempts you with an offer but this one is unusual. Box three is opened in front of you revealing the £1 prize, and he offers you the chance to change your mind and choose box one. Does switching improve your chances of winning the £10,000?

Marcus du Sautoy explains probability and the Monty Hall problem to Alan Davies on Horizon

Each year at my university we hold open days for hordes of keen A-level students. We want to sell them a place on our mathematics degree, and I unashamedly have an ulterior motive - to excite the best students about probability using this problem, usually referred to as the Monty Hall Problem.

This mind-melter was alluded to in an AL Kennedy piece on change this week and dates back to Steve Selvin in 1975 when it was published in the academic journal American Statistician.

It imagines a TV game show not unlike Deal or No Deal in which you choose one of three closed doors and win whatever is behind it.

One door conceals a Cadillac - behind the other two doors are goats. The game show host, Monty Hall (of Let's Make a Deal fame), knows where the Cadillac is and opens one of the doors that you did not choose. You are duly greeted by a goat, and then offered the chance to switch your choice to the other remaining door.

Most people will think that with two choices remaining and one Cadillac, the chances are 50-50.

The most eloquent reasoning I could find is from Emerson Kamarose of San Jose, California (from the Chicago Reader's Straight Dope column in 1991): "As any fool can plainly see, when the game-show host opens a door you did not pick and then gives you a chance to change your pick, he is starting a new game. It makes no difference whether you stay or switch, the odds are 50-50."

But the inconvenient truth here is that it's not 50-50 - in fact, switching doubles your chances of winning. Why?

Let's not get confused by the assumptions. To be clear, Monty Hall knows the location of the prize, he always opens a different door from the one you chose, and he will only open a door that does not conceal the prize.

For the purists, we also assume that you prefer Cadillacs to goats. There is a beautiful logical point here and, as the peddler of probability, I really don't want you to miss it.

In the game you will either stick or switch. If you stick with your first choice, you will end up with the Caddy if and only if you initially picked the door concealing the car. If you switch, you will win that beautiful automobile if and only if you initially picked one of the two doors with goats behind them.

If you can accept this logic then you're home and dry, because working out the odds is now as easy as pie - sticking succeeds 1/3 of the time, while switching works 2/3 of the time.

Kamarose was wrong because he fell for the deception - after opening the door, the host is not starting a new 50-50 game. The actions of the host have already stacked the odds in favor of switching.

The mistake is to think that two choices always means a 50-50 chance. If Manchester United play Accrington Stanley in the Cup then, with the greatest respect to proud Stanley, it's more likely that United will progress to the next round.

Still not convinced? You are in good company. The paradox of the Monty Hall Problem has been incredibly powerful, busting the brains of scientists since 1975.

In 1990 the problem and a solution were published in Parade magazine in the US, generating thousands of furious responses from readers, many with distinguished scientific credentials.

Part of the difficulty was that, as usual, there was fault on both sides as the published solution was arguably unclear in stating its assumptions. Subtly changing the assumptions can change the conclusion, and as a result this topic has attracted sustained interest from mathematicians and riddlers alike.

Even Paul Erdos, an eccentric and brilliant Hungarian mathematician and one-time guest lecturer at Manchester, was taken in.

So what happens on our university's open days? We do a Monty Hall flash mob. The students split into hosts and contestants and pair up. While the hosts set up the game, half the contestants are asked to stick and the other half to switch.

The switchers are normally roughly twice as successful. Last time we had 60 pairs in 30 of which the contestants were always stickers and in the other 30 pairs always switchers:

Among the 30 switcher contestants, the Cadillac was won 18 times out of 30 - a strike rate of 60%
Among the 30 sticker contestants, there were 11 successes out of 30, a strike rate of about 36%
So switching proved to be nearly twice as successful in our rough and ready experiment and I breathed a sigh of relief.

I had calculated beforehand the chances of ending up with egg on my face and the team of 30 stickers beating the 30 switchers. It was a risk worth taking, but one shouldn't play Russian Roulette too often.

Next year we will need something different, perhaps Simpson's Paradox. Imagine that 1% of people have a certain disease.

A diagnostic test has been developed which performs as follows - if you have the disease, the test has a 99% chance of giving the result "positive", while if you do not have the disease, the test has 2% chance of (falsely) giving the result "positive".

A randomly chosen person takes the test. If they get the result "positive", what is the probability that they actually have the disease? The answer, 1/3, is perhaps surprisingly low.


Click here for the article on the web.



<< Click here to see more recent news articles >>

 

 

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our News & Blogs portal or check out our recent posts below.

Leading Remote Teams in the Land of Beer, Barbeque, and Tech

Charlotte, North Carolina known for its beer and barbeque is not the first location to jump to mind when you think digital hub. But here’s the rub, it is. In the Fall of 2019, the city and Microsoft signed a three-year digital alliance. And as the country moved from office locations to work-from-home and remote operations, Charlotte became a prime destination for tech.  Four Skills for Leading in the New Normal Whether you’ve learned to balance work time with virtual schooling or have been working from home for years, there are some skillsets which set leaders apart. The first one may surprise you. Let others lead – According to our most recent salary guide, one of the main reasons people leave their jobs is due to poor management. With the rise of remote working, hierarchy has flattened to a degree as everyone must discipline themselves. Micromanagement becomes almost moot as everyone leans into this learning curve. Lead by example and let others take over the leadership driver’s seat from time-to-time. Balance both Soft and Technical Skills – While technical skills are the backbone of subject matter experience in a Digital Analytics role, it’s the soft skills which can help set you apart. Sure, you’ll want to know the ins and outs of web analytics and optimization, but you’ll also need to have the skills to explain findings and offer recommendations to address client needs. Know When to Pivot – Life throws us curveballs. Consider 2020, for example. Whether you must pivot for survival or simply need to take things in a new direction, knowing when to pivot and how to explain it to your employees is a leadership skill inherent for this new normal. Be Approachable – With open door policies moved online, leaders will want to determine the best way to recreate opportunities for employees that need to talk outside the daily or weekly staff meeting. Having insight into how your team works best, can help you guide them toward success. Programs to know and experience to have often include the technical knowledge you’ll need to ensure your client makes the most informed decisions.  ‘Smart Cities’ of the Future At the beginning of 2020, as the coronavirus came to call, someone joked that in 2020 we’d hoped to have flying cars and smart cities of the future, but instead were being taught how to wash our hands. Perhaps they weren’t far off on the smart cities quip as both Houston, Texas and Charlotte, North Carolina are on the cutting edge of creating these smart cities. Things like Massive Open Online Courses (MOOCs), LinkedIn Learning, even the Entrepreneur Store offers classes and bundles in everything from computer language learning such as C++, R, and Python to Digital Marketing and Graphic Design. But, learning these things and more on a Microsoft campus can catapult students into more jobs and helps guide cities in smart power grids, smart busses, autonomous cars, and the list goes on. Experience in a design agency who works within both the B2C and the B2B verticals helps to expand opportunities exponentially. Remote working opportunities have opened up worlds of collaboration, teamwork, and focus on the next steps into the future. Whether you’re interested in a remote working leadership role in the beachy Carolinas or looking for your first or next role in Big Data, Web Analytics, Marketing & Insight, Life Science Analytics, and more, check out our current vacancies or contact one of our recruitment consultants to learn more.   For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

Career Priorities Are Shifting For The US Data & Analytics Market Place

Positions in Data & Analytics are highly sought after in the US. With a widening talent pool due to the rise of flexible working, shifting skillsets and requirements for more professionals working in data and tech, the sector continues to experience unprecedented growth; regardless of the global pandemic pulling at the very core of the business community. For candidates, the Data & Analytics job market will be more active than ever. In our most recent US Data & Analytics Salary Guide, this was clearly identified. In fact, post-COVID, more respondents were either actively seeking, or willing to move for, a new role than they were in 2019. As businesses look to streamline existing processes and establish new ones, they are more reliant on Data professionals than ever before. Yet, candidates in this market have re-evaluated their needs and the incentives they value, shifting their priorities in the wake of COVID-19. With the impact of COVID-19 felt in almost every part of our day-to-day lives, it wasn’t long until conversation turned to what ‘the new normal’ would look like once the worst of the pandemic had passed. However, for those in the Data & Analytics space, the new normal wasn’t as new as first thought - it had been gradually establishing itself over the past few years. When seeking a new role, respondents prioritized career progression over salary increases, as well as an emphasis on job security and working for a stable and growing business. The needs of the multi-generational workforce in a post-COVID world has never been more dynamic than it is today. It was evidenced that financial ambitions have decreased slightly as a result of the pandemic. Interestingly, ‘a more competitive salary’ was the top reason for seeking a new role. Post-COVID, not only do respondents seek career progression over a salary increase, they also put a significantly greater emphasis on job security and the need to join a stable and growing business.  Specifically, for Data Engineers in the current market space, 80 per cent of professionals would leave their role if the right opportunity came up. This falls in parallel to digital analytics experts, of which 82 per cent would leave their role if the right opportunity came up. This is perhaps unsurprising as the full economic impact of the pandemic has yet to be established. Attracting the best hire for your business is going to be one of the core challenges of our time. Period. The focus for American organizations needs to be on hiring for the needs of tomorrow, rather than for the demands of today. A forward-thinking business will have the critical vision needed in order to keep pace with the market in its current state. Data professionals are in the driving seat; that’s a certainty. Their holistic skillset, understanding and application of core technologies such as SQL, Python, AWS, Kubernetes and Spark position them well when seeking to secure their next role. It is up to organizations to maintain their momentum as we acknowledge that for Data & Analytics, continuing on the trajectory of growth in the ‘new normal’ is something very few outside of this space saw coming. If you're looking for a new opportunity in the world of Data & Analytics or to expand your Data team, we may be able to help. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more: For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.    For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

Navigating the New Normal Now and In the Future

At the beginning of every year and particularly the start of a new decade, we often find ourselves reassessing our priorities. Though this year was no different, it does have the twist of a pandemic. Businesses and individuals were forced to pivot toward a new normal.  Together, we’ve moved quickly and one thing this year’s Salary Guide has shown us is that priorities have changed. In previous years, focus was on higher salaries and it was expected to stay in a position only two or three years before moving on to the next job. But this year, stability is the name of the game. And so, priorities shift. Candidates now want to stay in a role that could lead to career progression.   What’s Changed?  In the Data & Analytics industry, flexible working options have steadily increased, though it was already a way of life for many. Add into the mix, the increased need to work from home for employee safety or in controlled environments for those whose role did not allow for remote working such as those in the life sciences.  Financial considerations were no longer top dog in hiring and retaining top talent. Longevity, career progression, and good management are highly desired. When everyone is online and flexible working options are the new normal, it’s important to have strong leadership.  Four Future Changes to Come   With the world online, working, and learning from home, bandwidth has become a non-renewable resource. Too much traffic. Too many people online could cause issues particularly during prime working and learning hours. Though with everything online, what does it mean to have prime hours now?  Setting Priorities – Just because you’re working from home doesn’t mean you should lock yourself in your office for hours on end. Set aside time for food, family, and fun. Elevate the Home Office - Larger, more stable devices may see a resurgence. The home office is truly just that with the standard desktop and monitor to more easily see information. For those in many industries, two screens really help set the tone. Flexible Shopping Options are Here to Stay – When you’re working and learning from home, time is of the essence. The benefits of delivery, pick-up, and even some more expanded food centers could change the way we eat and gather. Retail is being redesigned for the new normal.Logistics of Social Distancing meet Machine Learning – As we focus on social distancing, mobile applications may shift toward a more logistics focused future using crowdsourcing and Big Data. From contact tracing to food buying, it will be important to have technologies that can keep up with people at all hours of the day and night to deliver goods and services.  In our recently released 2020 Salary Guide we discuss each specialism, what’s working and what isn’t. And how businesses can hire and retain top talent to keep their projects on track and their businesses running smoothly. If you’re interested in Data and Technology, Risk or Digital Analytics, Life Sciences Analytics, Marketing and Insight, Data Science, or Computer Vision, take a look at our current vacancies. If you’d like to learn more, get in touch with one of our expert consultants. For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.    For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

How Digital Analytics & Marketing Help Shape E-Commerce

Since the days of store window displays, the shopping experience has been just that; an experience. So, what happens when you want to recreate the visceral and emotional experience of shopping from within the confines of a computer screen, smartphone, or tablet? Enter Digital and Marketing Analytics. According to a recent report by Adobe, e-commerce sales have seen a 77% jump year-over-year which at any other time in history should have taken 4 to 6 years. The COVID-19 pandemic has helped to escalate and drive these numbers leaving some businesses scrambling and others raking it in. So, what’s the difference? Data professionals. The Role of Data in E-Commerce Whether you buy online and pick up in store or have a product delivered to your door, the role of shopping has irrevocably changed. Ensuring consumers get the personalized experience they’re used to from the days of brick-and-mortar stores, retail stores have turned to Digital and Marketing Analytics to give their customers the shopping experience they’re used to within a different format. Professionals within the UI/UX Design vertical are particularly sought after. Buying habits are changing and competition is fierce. So, how does Data affect e-commerce? Here are 3 examples: Know Your Customer. No longer a lamented visage from yesteryear, knowing your customer is inherent to the survival of an online retailer.  Data professionals bring to life the customer through historical data, demographics, and creates products and services which elicit an emotional response to stop, look around, and buy. And if the cart gets abandoned, follow up email campaigns to jog the memory that you were here and were thinking about buying this or that item. Get Personal. Personalize shopping has evolved into the personal shopping experience. What made someone buy a product during their last visit? Dynamic presentation and emotion-driven verbiage can certainly contribute.  When an online retailer knows what the buyer wants or needs and presents it in a way that resonates. This is the personalization once the domain of sales clerks.  From the Data you enter at checkout to the social media platforms and search engines with information like what you’re looking for, where you’re located, your purchase history and more can sometimes leave the personalized experience out in the cold. Bridging the old personalization with the new is the key ingredient for successful online retailers. Sell Where Your Consumers are Buying. If your customers are on Facebook looking for a product or service, sell to them where they’re looking. Though reviews are still important, even more prevalent is the range of social influencers to help buyers make decisions. Enter social commerce. Layout, design, Data gathering, collecting, and analyzing all have a slightly different flavor within this construct.  Data Professionals in Demand Within the Digital Analytics and Marketing specialism are a variety of Data professionals in demand as retailers are forced online. Businesses need E-Commerce Analysts to help present the new normal for the online retailer. As buy online and pick up in store convenience increases as well as purchases made from devices such as your smartphone, there’s been a rise in demand for Web & Mobile Product Management as well. Digital Transformation is no longer at the doorstep of business, it has crossed the threshold. And rather than focus on one tool or another, the impetus is on the importance of having more than one tool at a candidate’s disposal. What candidates want has changed and evolved as well. Salary and bonuses have dwindled while the demand for experience and professional development have become rallying cries for finding and retaining top talent. For more information In our recently released 2020 Salary Guide we discuss each specialism. What’s working. What isn’t. And how businesses can hire and retain top talent to keep their projects on track and their businesses running smoothly. You can download your copy here.  If you’re interested in Data and Technology, Risk or Digital Analytics, Life Sciences Analytics, Advanced Analytics & Insight, Data Science, or Computer Vision, we invite you to check out our latest jobs. If you’d like to learn more, contact one of our expert consultants: For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

Recently Viewed jobs