The probability puzzle that makes your head melt



A reference in a recent Magazine article to the Monty Hall problem - where a contestant has to pick one of three boxes - left readers scratching their heads. Why does this probability scenario hurt everyone's brain so much, asks maths lecturer Dr John Moriarty.

Imagine Deal or No Deal with only three sealed red boxes.

The three cash prizes, one randomly inserted into each box, are 50p, £1 and £10,000. You pick a box, let's say box two, and the dreaded telephone rings.

The Banker tempts you with an offer but this one is unusual. Box three is opened in front of you revealing the £1 prize, and he offers you the chance to change your mind and choose box one. Does switching improve your chances of winning the £10,000?

Marcus du Sautoy explains probability and the Monty Hall problem to Alan Davies on Horizon

Each year at my university we hold open days for hordes of keen A-level students. We want to sell them a place on our mathematics degree, and I unashamedly have an ulterior motive - to excite the best students about probability using this problem, usually referred to as the Monty Hall Problem.

This mind-melter was alluded to in an AL Kennedy piece on change this week and dates back to Steve Selvin in 1975 when it was published in the academic journal American Statistician.

It imagines a TV game show not unlike Deal or No Deal in which you choose one of three closed doors and win whatever is behind it.

One door conceals a Cadillac - behind the other two doors are goats. The game show host, Monty Hall (of Let's Make a Deal fame), knows where the Cadillac is and opens one of the doors that you did not choose. You are duly greeted by a goat, and then offered the chance to switch your choice to the other remaining door.

Most people will think that with two choices remaining and one Cadillac, the chances are 50-50.

The most eloquent reasoning I could find is from Emerson Kamarose of San Jose, California (from the Chicago Reader's Straight Dope column in 1991): "As any fool can plainly see, when the game-show host opens a door you did not pick and then gives you a chance to change your pick, he is starting a new game. It makes no difference whether you stay or switch, the odds are 50-50."

But the inconvenient truth here is that it's not 50-50 - in fact, switching doubles your chances of winning. Why?

Let's not get confused by the assumptions. To be clear, Monty Hall knows the location of the prize, he always opens a different door from the one you chose, and he will only open a door that does not conceal the prize.

For the purists, we also assume that you prefer Cadillacs to goats. There is a beautiful logical point here and, as the peddler of probability, I really don't want you to miss it.

In the game you will either stick or switch. If you stick with your first choice, you will end up with the Caddy if and only if you initially picked the door concealing the car. If you switch, you will win that beautiful automobile if and only if you initially picked one of the two doors with goats behind them.

If you can accept this logic then you're home and dry, because working out the odds is now as easy as pie - sticking succeeds 1/3 of the time, while switching works 2/3 of the time.

Kamarose was wrong because he fell for the deception - after opening the door, the host is not starting a new 50-50 game. The actions of the host have already stacked the odds in favor of switching.

The mistake is to think that two choices always means a 50-50 chance. If Manchester United play Accrington Stanley in the Cup then, with the greatest respect to proud Stanley, it's more likely that United will progress to the next round.

Still not convinced? You are in good company. The paradox of the Monty Hall Problem has been incredibly powerful, busting the brains of scientists since 1975.

In 1990 the problem and a solution were published in Parade magazine in the US, generating thousands of furious responses from readers, many with distinguished scientific credentials.

Part of the difficulty was that, as usual, there was fault on both sides as the published solution was arguably unclear in stating its assumptions. Subtly changing the assumptions can change the conclusion, and as a result this topic has attracted sustained interest from mathematicians and riddlers alike.

Even Paul Erdos, an eccentric and brilliant Hungarian mathematician and one-time guest lecturer at Manchester, was taken in.

So what happens on our university's open days? We do a Monty Hall flash mob. The students split into hosts and contestants and pair up. While the hosts set up the game, half the contestants are asked to stick and the other half to switch.

The switchers are normally roughly twice as successful. Last time we had 60 pairs in 30 of which the contestants were always stickers and in the other 30 pairs always switchers:

Among the 30 switcher contestants, the Cadillac was won 18 times out of 30 - a strike rate of 60%
Among the 30 sticker contestants, there were 11 successes out of 30, a strike rate of about 36%
So switching proved to be nearly twice as successful in our rough and ready experiment and I breathed a sigh of relief.

I had calculated beforehand the chances of ending up with egg on my face and the team of 30 stickers beating the 30 switchers. It was a risk worth taking, but one shouldn't play Russian Roulette too often.

Next year we will need something different, perhaps Simpson's Paradox. Imagine that 1% of people have a certain disease.

A diagnostic test has been developed which performs as follows - if you have the disease, the test has a 99% chance of giving the result "positive", while if you do not have the disease, the test has 2% chance of (falsely) giving the result "positive".

A randomly chosen person takes the test. If they get the result "positive", what is the probability that they actually have the disease? The answer, 1/3, is perhaps surprisingly low.


Click here for the article on the web.



<< Click here to see more recent news articles >>

 

 

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our News & Blogs portal or check out our recent posts below.

Living the Hybrid Life with Advanced Analytics

In a recent documentary created by Dave Grohl of the Foo Fighters, he interviews Lars Uhlrich, a once outspoken critic of the demand for online music. In the video produced in 2021, Lars has now embraced the online music world. Yet, in 2019, one-quarter of all music sales were vinyl.  Much as we’d love to deep dive into some music, the point here is that what was once either/or has become both.  Brick-and-mortar business have e-commerce channels, e-commerce channels have brick-and-mortar stores (looking at you Amazon bookstores). Stores are now their own fulfillment centers, items and food can be ordered online and picked up curbside, in-store, or via delivery. In other words, businesses are meeting their consumers where and how they want to shop. Would we have gotten to this point as quickly if not for the pandemic? Maybe. But for many retailers and businesses, there are a few Marketing Insights you’ll want to remember to maintain this new hybrid environment.  Customer Experience Should Remain Top of Mind Imagine this. You visit a store to buy something off the shelf. The store is essentially empty, except for signs which say ‘available to order online.’ If everything is only available online, you’re meeting only one segment of your consumer. While the idea is to create a consumer relationship, you need to know who your customer is, their needs and desires, and meet them when, where, and how they shop.  The customer journey should be interwoven between online shopping and the increasing desire for social interaction. Shopping should still be an experience. Here a few things to consider when planning your hybrid customer journey for your business: How well do you know your customers? Does your customer prefer to shop online, then pick up in store? If they entered the store without shopping online, would you have the product they needed on the shelf or at least, in the back rooms? Why or why not?How do you move the customer from in-store shopping to online and back to in-store shopping?If your customer shops fully online, is their data protected? Privacy is a major concern for many consumers and should be considered at every level of business. Who has access to what and why? And if your consumer is fully remote, or unable to shop in-store, how do you provide and maintain the best customer experience all they way to their door? And with these questions in mind, who’s helping your business build and maintain those relationships? Many businesses have remote call centers to handle customer complaints, but what about the beginning of the journey. Enter AI, chatbots, and Advanced Analytics to move your customer and your business forward for the future. It’s important to build your customer relationship across all your shopping channels – on-line, in person, via AI or chatbot or customer service representative – a unified experience is key. Chatbots and AI Learn from Advanced Analytics and NLP There are only so many humans who can answer call demands. Add in all the online orders, delivery dispatches, and fulfillment options, and it can be overwhelming. Using Advanced Analytics, good Data Management, Artificial Intelligence, and Automation can all augment the ultimate customer journey experience. AI using NLP scraped from past conversations can Data mine language samples to interpret, understand, and respond intelligently to consumer calls. Add text-based support and the benefits increase. If you’re business is more service-based than product-related, AI and chatbots can help there, too. Much like the salesclerk when you enter a brick-and-mortar store, the chatbot is your first engagement with a prospective consumer. And its that chatbot or AI representative who will set the stage for their customer journey and experience. Beyond Retail and Back to Basics Business? For those in the office, or working remotely, some businesses have begun offering hybrid options. Come into the office a few days a week and work from home a few days a week. Yet, others are insisting it’s back to business as usual with some additional measures such as sanitation, plexiglass between cubicles, and physical distancing of employees.  Over the past year, it’s been proven employees don’t need to be in the office. But, it’s also important for humans to interact with each other on some social level and that includes meeting in-person. Physical distancing will be important for some time, but human interaction outside of a computer screen is also. So, while there are pros and cons, for now, the hybrid environment for businesses, schools, retailers, and restaurants is here to stay. How businesses choose to meet this mix will be interesting to watch as they incorporate available technologies while also remembering those they’re trying to serve are human. If you’re interested in Big Data & Analytics, Harnham may have a role for you. Check out our current vacancies or contact one of our expert consultants to learn more.  For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.  For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

Mental Health Apps Help Patients Make the Move into Therapy

May is Mental Health Awareness Month. As we continue to untangle ourselves from the cabin-fever isolation of the winter months post-pandemic, many are feeling overwhelmed and mentally burdened. It’s a lot to manage – remote working, virtual schooling, mask mandates, vaccine appointments, and the day-to-day screen time and Zoom meetings. It’s no wonder people are turning, once more to the virtual space for help.  From mindfulness and meditation apps and videos to physician-focused telehealth assistance in the health space, there is digital technology at play. While it may seem as if apps keep people from going to the source and seeking professional help in the mental health space, having an app is helping people make the decision to sit down one on one with someone.  Mind the App: A Note on Market Trends There is a proliferation of mindfulness meditation apps available on Google Play, iOS systems, and on YouTube, just to name a few. In fact, the market is set to grow exponentially over the next five to ten years. And technology companies have been paying attention. Key trends include: Design simplicity and collaboration with subject matter expertsEngagements both personal and professional which lower energy levels leading to disturbed sleep. While millennials search for a more whole (read: work/life) balanced life, it’s reported Generation Z is likely to have the worst mental health issues and desire to seek alternative therapy options.Apps-focused issues include insomnia, anxiety, depression, mindfulness, and whole being self-care.More and more Americans understand mindfulness and meditation are the best resources to help unwind and lessen stressors. Not all apps are created equal, but Headspace and Calm are two of the top contenders and most widely used. Those higher-level products backed by teams to ensure everything runs smoothly are outpacing the proliferation of self-help apps which number upwards of 20,000. Digital Technology and Machine Learning are Moving Things Forward From AI to Digital Technology to Machine Learning, Data professionals are working to ensure mental health apps are not a flash in the pan. Especially for those who may need help the most such as high utilizers. These high utilizers are those who may have multiple issues and check themselves into hospitals most frequently. Just like people worried about robots taking jobs, some therapists may worry these apps could replace in-person therapy treatments. This is unlikely as it may embolden those who need help to seek treatment in the more traditional sense. In fact, some apps could enhance therapy work rather than detract from it. Machine learning tools helped researchers analyze nearly 10,000 patients’ EHR Data over a 2-year period. The algorithm analyzed all the factors at once for a much more efficient breakdown and determination of each factor to best identify patient characteristics. Understanding this information helped researchers determine which factors lead to the disorders for higher utilization. Dangers and Predictions With nearly 20,000 mental health apps on the market, there are probably a fair number which aren’t as useful as they hope or claim to be. And some can be dangerous as was discovered in a 2015 study in which therapies for bipolar disorder were found to be inconsistent with established treatments (Nicholas, J., et al., Journal of Medical Internet Research, 2015). Not to mention ineffectual apps and those which disrupt treatments already in place and established. Mindfulness Apps at Work For happier, healthier workers not burned out from Zoom-fatigue and remote working, business leaders are focusing more and more on mental health. Some are offering mindfulness moments or group yoga classes or meditation opportunities throughout the day for a mental health break. Mental health discussions are now standard practice and efforts to open up more on this front through self-care prioritization and workplace wellness are making their mark. Employee mental health is top of mind as businesses and employees navigate the new normal when it comes to work, education, and the blurring of personal and professional lives as we continue remote practices.  What was once taboo and swept under the rug has been met with Digital Technology, Apps, AI, Machine Learning, and the door is open for discussion. Whether you take a mindfulness minute, write in a journal, meditate, or log in to your app remember mental health awareness is about self-care for the whole you. Need a mental health break? There’s an app for that. If you’re interested in Big Data and Analytics, Harnham may have a role for you. Check out our current vacancies or contact one of our expert consultants to learn more.  For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.  For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

How Computer Vision Can Modernize American Infrastructure

What if we could imagine a city in which there were no traffic bottlenecks or jams? What if we could sail through tolls and park using an app? Could high-speed rail replace cars as our main means of transport commuting to and from work? Would travel apps with our faces, government IDs, and medical records launch us past the long security lines at the airport?  We already use computers to help us navigate our destinations, buy plane or train tickets, and in some cities, park using only our location and a credit card. Already a part of our day-to-day for many, computer vision is moving us along at a rapid rate.  Smart cities are changing the way we move Computer Vision is being used in a variety of ways and is modernizing our transportation infrastructure for a start. Using advanced technology, computers are learning to ‘see and hear’ and make informed decisions much as humans might to make our world more efficient and safer.  Below are just a few places Computer Vision is already in play in our communities: Traffic patternsTraffic signals – connected cameras watch for pedestrians as they monitor traffic flow. These connected cameras also help to optimize flow and reduce congestion.Street lamps – imagine saving electricity by dimming street lamp switches then turning them back on if movement is detected.Tolls – camera captures of your license plate billed to where it's registered and EZ Pass lanes – one pass to move across highways without slowing, stopping, or having to handle cash or coin.GPS warnings – red for stopped traffic, yellow for slowed trafficParking apps - in cities with significant parking issues such as limited space. Imagine using a location-based service parsed into zones and your credit card to not only find a parking spot more easily, but to guarantee your car won’t be towed or mishandled.Clear ID - when traveling both domestically and internationallyHigh-speed rail - in some cities and talks under way for more. Commuter trains are moving beyond New York and Chicago to other cities such as Atlanta, Detroit, and Dallas just to name a few. These are just a few ways cities are being modernized for individuals. But what about on a grander scale? With the increase of online orders and demand for delivery in the last year or two, it’s estimated that over 1 million packages were delivered in New York each day. Extrapolate that to the other 49 states and the amount of traffic on the road that is strictly for deliveries soon overtakes that of individual commuters. Computer Vision is Solving for Delivery Congestion For many downtowns, parking can be a bear. Especially if you have a delivery van or truck. Where do you park to make your deliveries? Remember, you’re on a deadline. All. The. Time. What if a car is parked in your loading zone? What if there’s too much traffic to park? What had once plagued urban roadways was now a bone of contention to deliver products in a timely manner. On the more negative side of these frustrations, parking revenue was virtually eliminated as the larger companies absorbed the costs into their cost of doing business. No one was getting anywhere fast. Enter Computer Vision technology. Once the reason behind the problem was understood, cities could begin to plan for a solution. This technology could be used to help city planners understand curbside activity so they could tailor their plans based on their cities’ needs. With the proper data to make more informed decisions, cities can plan locations for passenger parking, ride-hailing lots, and delivery only zones to make more efficient use of people’s parking needs. Next Steps Toward a Modernized Infrastructure Curb management is just one of the options that America is working on to modernize its infrastructure. Making traffic patterns and flow more efficient is another. Imagine a plan in which pedestrians, drivers, cyclists, commuters, and delivery fleets can safely share the road while also investing in clean energy solutions. Adding in infrastructure plans to also help combat climate change makes it a win-win for everyone. If you’re interested in Big Data & Analytics, Harnham may have a role for you. Check out our current vacancies or contact one of our expert consultants to learn more.  For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.  For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

How Big Data and Risk Analytics Can Help Fight Climate Change

Data is all around us. We use it to calculate our calories and our steps to ensure a healthy body. We use it track our packages and ensure delivery to the right location. We look to it to check the weather for exercise, driving conditions, and in extreme cases, safety preparedness. But, could we use it to fight climate change? Could we use it to reign in swiftly rising temperature changes which could affect our food and ecosystems?  Greener Choices for Greener Products People have more choice than ever before. They also have information at their fingertips and can see at a glance the benefits or the drawbacks of purchases. From how their food is grown to how far their food is delivered to the practices of companies from oil and gas producers to the wearables on their wrist. Climate change and Big Data have been linked, but mostly to determine greenhouse gases and effects of pollutants. But with the rise of consumer advocacy groups, farm-to-table traditions, micro-and macro-farming, and a desire to know more about what we’re putting into our bodies, consumers are dictating greener options from the markets. The Business of Climate Risk Analytics As consumers take note of climate change, companies are merging knowledge of climate change risk into their financial decision making. How will climate change their business practices? How will it be scaled based on how climate science rules inform financial risk assessments not yet developed? The markets need just as much information as consumers when it comes to climate risk. These assessments are intended to businesses determine consequences, responses, and likelihood of the impacts of their actions. Enter climate risk analytics. Climate Risk Analytics uses risk assessment and risk management based on natural disasters and their impact. However, the climate is not in a static state. It’s ever-changing and those changes are often in the extremes with little information related to averages. This complicates risk assessments as do the differences in regional projections. How AI Can Help Big data combined with climate risk analytics is getting an additional boost from artificial intelligence. AI techniques are being used for a variety of situations such as disease tracking, crop optimization, and monitoring everything from our heartbeat to endangered species. Solutions from advances in Deep Learning and Machine Learning could solve global environmental crises while assuaging financial risk with predictive modelling. Yet barriers to effective solutions from AI include cost and regulatory approval. But if these items weren’t an issue? We could determine such vital information as water availability and ecosystem wellbeing. Water and ecosystems aside, AI can help: Track and monitor endangered speciesImprove energy efficienciesOptimize wildlife conservation Fight poaching of endangered speciesTrack mosquito populations to prevent diseaseWarn populations of upcoming storms• Inform agriculture, health, and climate studiesDetermine water, forest, and urbanization changesSome vineyards in California use AI to determine if vines receive enough or too much water. AI’s ability to process large amounts of information quickly are a boon to the ever-changing climate, its risk assessments for businesses, and its benefits to consumers and investors who want to know what businesses are doing to keep everyone safe. In Honor of Earth Day This week we celebrate Earth Day. It’s a day to remember and honor the earth who gives us our air, our food, our animals, plants, fish, and so much more. From Greta Thunberg’s School Strike for Climate to Naomi Klein’s book, The (Burning) Case for a Green New Deal, climate is front and center of our thoughts and our survival. Want to be part of the movement working with Climate Risk Analytics or the effect of Artificial Intelligence in our environment? Harnham may have a role for you. From Big Data & Analytics to the Life Sciences, there’s something for everyone interested in the Data industry. Check out our current vacancies or contact one of our expert consultants to learn more.  For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.  For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

Recently Viewed jobs