The probability puzzle that makes your head melt



A reference in a recent Magazine article to the Monty Hall problem - where a contestant has to pick one of three boxes - left readers scratching their heads. Why does this probability scenario hurt everyone's brain so much, asks maths lecturer Dr John Moriarty.

Imagine Deal or No Deal with only three sealed red boxes.

The three cash prizes, one randomly inserted into each box, are 50p, £1 and £10,000. You pick a box, let's say box two, and the dreaded telephone rings.

The Banker tempts you with an offer but this one is unusual. Box three is opened in front of you revealing the £1 prize, and he offers you the chance to change your mind and choose box one. Does switching improve your chances of winning the £10,000?

Marcus du Sautoy explains probability and the Monty Hall problem to Alan Davies on Horizon

Each year at my university we hold open days for hordes of keen A-level students. We want to sell them a place on our mathematics degree, and I unashamedly have an ulterior motive - to excite the best students about probability using this problem, usually referred to as the Monty Hall Problem.

This mind-melter was alluded to in an AL Kennedy piece on change this week and dates back to Steve Selvin in 1975 when it was published in the academic journal American Statistician.

It imagines a TV game show not unlike Deal or No Deal in which you choose one of three closed doors and win whatever is behind it.

One door conceals a Cadillac - behind the other two doors are goats. The game show host, Monty Hall (of Let's Make a Deal fame), knows where the Cadillac is and opens one of the doors that you did not choose. You are duly greeted by a goat, and then offered the chance to switch your choice to the other remaining door.

Most people will think that with two choices remaining and one Cadillac, the chances are 50-50.

The most eloquent reasoning I could find is from Emerson Kamarose of San Jose, California (from the Chicago Reader's Straight Dope column in 1991): "As any fool can plainly see, when the game-show host opens a door you did not pick and then gives you a chance to change your pick, he is starting a new game. It makes no difference whether you stay or switch, the odds are 50-50."

But the inconvenient truth here is that it's not 50-50 - in fact, switching doubles your chances of winning. Why?

Let's not get confused by the assumptions. To be clear, Monty Hall knows the location of the prize, he always opens a different door from the one you chose, and he will only open a door that does not conceal the prize.

For the purists, we also assume that you prefer Cadillacs to goats. There is a beautiful logical point here and, as the peddler of probability, I really don't want you to miss it.

In the game you will either stick or switch. If you stick with your first choice, you will end up with the Caddy if and only if you initially picked the door concealing the car. If you switch, you will win that beautiful automobile if and only if you initially picked one of the two doors with goats behind them.

If you can accept this logic then you're home and dry, because working out the odds is now as easy as pie - sticking succeeds 1/3 of the time, while switching works 2/3 of the time.

Kamarose was wrong because he fell for the deception - after opening the door, the host is not starting a new 50-50 game. The actions of the host have already stacked the odds in favor of switching.

The mistake is to think that two choices always means a 50-50 chance. If Manchester United play Accrington Stanley in the Cup then, with the greatest respect to proud Stanley, it's more likely that United will progress to the next round.

Still not convinced? You are in good company. The paradox of the Monty Hall Problem has been incredibly powerful, busting the brains of scientists since 1975.

In 1990 the problem and a solution were published in Parade magazine in the US, generating thousands of furious responses from readers, many with distinguished scientific credentials.

Part of the difficulty was that, as usual, there was fault on both sides as the published solution was arguably unclear in stating its assumptions. Subtly changing the assumptions can change the conclusion, and as a result this topic has attracted sustained interest from mathematicians and riddlers alike.

Even Paul Erdos, an eccentric and brilliant Hungarian mathematician and one-time guest lecturer at Manchester, was taken in.

So what happens on our university's open days? We do a Monty Hall flash mob. The students split into hosts and contestants and pair up. While the hosts set up the game, half the contestants are asked to stick and the other half to switch.

The switchers are normally roughly twice as successful. Last time we had 60 pairs in 30 of which the contestants were always stickers and in the other 30 pairs always switchers:

Among the 30 switcher contestants, the Cadillac was won 18 times out of 30 - a strike rate of 60%
Among the 30 sticker contestants, there were 11 successes out of 30, a strike rate of about 36%
So switching proved to be nearly twice as successful in our rough and ready experiment and I breathed a sigh of relief.

I had calculated beforehand the chances of ending up with egg on my face and the team of 30 stickers beating the 30 switchers. It was a risk worth taking, but one shouldn't play Russian Roulette too often.

Next year we will need something different, perhaps Simpson's Paradox. Imagine that 1% of people have a certain disease.

A diagnostic test has been developed which performs as follows - if you have the disease, the test has a 99% chance of giving the result "positive", while if you do not have the disease, the test has 2% chance of (falsely) giving the result "positive".

A randomly chosen person takes the test. If they get the result "positive", what is the probability that they actually have the disease? The answer, 1/3, is perhaps surprisingly low.


Click here for the article on the web.



<< Click here to see more recent news articles >>

 

 

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our News & Blogs portal or check out our recent posts below.

The Science Behind the Sales: Statistical Modelling and Consumer Insights

Black Friday, once reserved for the Friday after Thanksgiving, has a new lease on life. It’s not just one day a year. Now, it can be any day of the year. Why? Customers the world over appreciate the deals and discounts as they get ready for the holidays, and the insights gleaned from previous years help businesses determine what will sell best in the current year.  Add in the rise of mobile marketing in which you can buy anything, anywhere, anytime, and more people working from home than ever before, and you have the recipe for Black Friday every day. Especially once the leaves begin to Fall and the holiday season comes knocking at our doors. Behind Advanced Analytics & Insights within the Data profession, there are a host of professions which together help to create the products and launch them. Who could imagine the science behind the sales? Five Important Roles Behind Advanced Analytics & Insights There’s a reason that Data professionals are no longer siloed and must work across and within departments. Data influences every decision within business today. So, having the right people on your team in the right roles can ensure your business thrives. Marketing Analyst Using a spoke-and-wheel analogy, the Marketing Analyst is the spoke. These are the professionals central to taking the information from their Campaign Analysts, Pricing Analysts, and Statistical Modelling experts to determine what will sell best and what price. In essence, their research and Data helps companies figure out ‘what the market will bear.’ Campaign Analyst Focused campaigns to a target market will find a Campaign Analyst understands consumer behaviors. Once the customer is understood – what they buy, when they buy, why and how they buy – can help analysts measure, review, and justify each campaign to justify ROI.Consumer Insights While the Campaign Analyst is focused on targeted campaigns to specific customers, the Consumer Insights Analyst helps businesses tailor their marketing strategies to meet the needs of those customers. All of them. Whether they’re part of a targeted marketing campaign or if they’re prospective clients ‘window shopping’ when they download a ‘free product or click on an email’. Understanding these consumers helps to convert from passing interest to the purchase of a product or service. Statistical Modelling Analyst Do you love puzzles? A jumble of pieces which need to be put together so everyone can see the full picture?  Statistical Modelling are the jigsaw puzzle solvers of the Marketing department. These professionals pull together data from multiple sources and analyze the information to help form a clear picture of their ideal consumer. Toss in a bit of psychology – why customers do what they do and predict what they might do next – and your executives will understand the best way to distribute funds to grow their business. Pricing Analyst It seems simple enough. Determine how much it cost to produce a product or service and mark it up to make a profit, right? Not exactly. These days, calculating not only what it cost to make a product, any overhead to consider, and how much you need to mark it to make a profit is a complex prospect. Factor in what your competition is doing…on a global scale. Consider the fickle buying power of consumers – some days money flows and some days it’s reigned in tight. How do you determine the best price at the right time to ensure maximum profit while keeping pulse on spending patterns? Pricing Analysts take complex data, study customer spending habits, and conduct analyses on what the math says and what the impact might be. So, back to Black Friday sales. Taking what we know of the roles which help businesses set the tone for their marketing strategies, Black Friday every day makes sense. We’re home more. The lines between work and family life are blurred, but computers and phones are at our fingertips, and everyone delivers what we need right to our door. Why not get a jump on your holiday shopping? And once that’s done, you can focus on what your plans are for the future. Are you looking for a career change or just want to see what the market looks like within the Data professionals industry? There is plenty of information to be collected and analyzed. What role might you fill? If you’re looking for a role in Data & Analytics or are interested in Advanced Analytics & Insights, Harnham may have a role for you. Check out our current vacancies or contact one of our expert consultants to learn more.   For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

Jumpstart Your Job Search Now and in the Years Ahead

COVID-19 may have rocked the world of employment, but it also created a whole new series of opportunities. If you are looking for work, there are a few updates to the rules of the job search. Not rules, really. More preferred guidelines. A Note on the Guidelines of the Job Search Remember when your work experience was honed to two pieces of paper – your resume and cover letter – and you hoped it made it to the hiring manager? Well, there are a lot more direct ways to get there and plenty of opportunities abound to help you stand out from the crowd. It’s not just your educational experience. In fact, today’s hiring is much more about emotional intelligence, collaboration, and how you use what you’ve learned in previous roles. Among the negative notes, there are a host of positive movements for jobseekers with both established and startup firms. And sometimes, even within a legacy firm which has pivoted with the changes of our new world of work. Thread the Needle of Responsibility Google and IBM may be leading the charge to hire without use of a degree, but most businesses still want that piece of paper. How you present it is another matter. Think video, project portfolios, and online forms via application tracking systems (ATS). That’s just to get you in the door. The more important hurdle is understanding the nuances of your role and responsibilities. The list of qualifications and duties has always been part of the job search. Do you fit all the requirements? Can you handle all the responsibilities? Did you read between the lines to understand what the company hopes will find in their ideal candidate to help them meet their business goals and objectives?  Below are some questions you might ask yourself when reading through job descriptions or considering where you’d like to apply: Are you able to not only craft reports, but also see patterns to help you gain insight into what the reports are telling you?From this, can you not only discuss it with your colleagues and teammates, but also across departments, executives, and stakeholders?Can you not only explain the patterns to both technical and non-technical audiences, but do so across multiple projects?What challenges will you face and how will you solve them?How well do you manage your time? Can you step in to lead a team or do you prefer to work on your own? Could you be flexible between the two?Are you able to build relationships both internally and externally – teammates, vendors, executive leaders, department heads, and the board room.Do you have emotional intelligence? Can you take ownership of a project and hold yourself and others accountable?Do you show initiative? Not just in diving into a project, but asking questions. Can you ask objective questions playing devil’s advocate on one side and seeing the possibilities on the other? These are wide open questions to challenge yourself. Some are leadership-centric. Some are simply ‘can I do the job?’ questions. But, ultimately, it’s these kinds of questions which are asked in interview under the purview of seemingly inane questions. They’re meant to make you think and for the hiring manager to see how you think. 3 Surprising Ways to Stand Out in Your Job Search With the rise of remote working, Zoom, and Slack, the video interview and application process has gained ground. It’s quickly become the virtual way to seek, apply for, meet with, hire, and work with prospective and current team members. But, if you’re in the tech space, you can go even deeper than video and you’ve got more than project portfolios to fall back on. Write about your experiences – what you’ve learned, where, why, and how. Think Medium’s Toward Data Science. Don’t forget to hit publish! Not a writer? Follow the blog and make comments.Network – this seems like a ‘no brainer’, but in our somewhat virtual world it can be even easier for those who already comfortable in this environment. Whether it’s via LinkedIn or Twitch, you are building relationships to help you move forward in your job search.Use your skills to help you run and track your job search more efficiently.  If you’re interested in Big Data, Web Analytics, Marketing & Insight, Life Science Analytics, and more, check out our current vacancies or contact one of our expert consultants to learn more.   For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

The Surprising Collaboration of Ada Lovelace, Charles Babbage, and Alan Turing

What do you get when you combine Amelia Earhart with Ada Lovelace? A Data Visualization Engineer ready to work with an aviation industry partner. Reaching new heights and shattering the glass ceiling is the modus operandi for many women, and what better role models than the ladies listed. Creative, free-spirited, pioneering, and well before their time in thoughts and action. Ada Lovelace, now attributed as the first computer programmer saw beyond the automatons of her day. She saw beyond the Berullean language in front of her she was translating.  A poet father and a passion for numbers collided into her thoughts and as we marvel at AI making art, writing stories and music, and winning strategy games, we have one lady to thank. Ada. She might also be called the first Data Visualization Engineer. Don’t you think? Insightful Business Decisions are Key in Collaboration Data professionals are no longer siloed from other departments in business allowing for collaboration between teams. In partnership between both technical and non-technical employees, businesses can be sure they’re teams have a single vision to help realize business objectives and goals. The collaboration between Ada Lovelace and Charles Babbage may not have been business-related, but the ideas are the same. He passed her the document and asked her to translate, she made notes, and those notes have made history. Together they created a vision for The Analytical Machine – it exists only on paper, but it’s design, layout, and potential implementation are realized in ways unimaginable to most 100 years ago.Ada’s mathematical prowess was such that she wrote her notes in easily explainable language.She worked closely with Charles Babbage and wrote in earnest to work with Michael Farraday – she reached out to others in her field, some accepted, others didn’t. How Data Helps Inform the Future Whether you use predictive modeling, machine learning, natural language processing, or some combination of each, the data you collect helps to inform the future. We may often lament the old adage that those who don’t know their history are doomed to repeat it, but history has a shining light as well. Collaboration across the ages. Consider this. Alan Turing, the man who worked in Bletchley Park with the Enigma machine, used the notes he found to help him solve the problem. Those notes belonged to Ada Lovelace. The information she set to paper informed every stage of computer programming leading to what we know today as Artificial Intelligence. Machines that could learn and ‘think,’ not just the automatons of her age which had been ‘programmed to perform.’ The Enchantress of Numbers Known as the Enchantress of Numbers, the pioneering Ada Lovelace shares the spotlight with other pioneering women in the sciences. Think Madame Curie, Joan Clarke, even Hedy Lamarr, and of course Amelia Earhart. They weren’t of the same eras, but each of their contributions have added to what we know as the Science, Technology, Engineering, and Mathematics (STEM). We have a name for it now, but it’s always been around. And the collaborative efforts of women everywhere are growing and increasing diversity and inclusion in many businesses across the world. And at the heart of it all, in the beginning, a surprising and time-defying collaboration began. It set in motion a spark of business intelligence and insight as men and women mentored and partnered for the sake of their vision of the future. Who will be remembered one hundred years from now?  If you’re interested in Big Data, Web Analytics, Marketing & Insight, Life Science Analytics, and more, check out our current vacancies or contact one of our recruitment consultants to learn more.   For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

Computer Vision in Healthcare Beyond Covid-19

2020. It sounds like the name of a futuristic science-fiction movie or TV show, doesn’t it? Maybe it is. And like our favorite sci-fi flicks there are cutting edge changes happening in real time. We’re the characters in this story and the Computer Vision and Artificial Intelligence partnerships in healthcare are moving fast to help us take care of ourselves. When computers can see what we can’t. When AI can help us make more informed decisions. When the two are combined to help doctors and providers work more efficiently to save lives, that’s when the cutting-edge shines. From the collaboration of Johns Hopkins, the CDC, and the WHO mapping out the data to contact traces to medical professionals on the front lines, we’ve been focused on one thing. Saving lives. But, what about the other medical issues that affect us? Heart disease. Cancer. Neurological illnesses.  What if the latest advances in healthcare could help here, too? Five Ways Computer Vision Helps Healthcare Providers Identifies leading causes of medical illnesses in a time-sensitive manner by creating algorithms for image processing, classification, segmentation, and object detection.Develops deep learning models to create neural networks.Collaboration of teams of scientists working together for the advancement of projects and present findings to business leaders, stakeholders, and clients.Allows providers to spend more time with their patients.Optimization of medical diagnoses using deep learning so doctors can spend more time with patients to help see and solve the problem faster. Computer Vision Engineer Meets AI Professional Artificial Intelligence (AI) offers real world answers in healthcare the world needs today. Computer Vision Engineers build the means to which AI helps providers, patients, and leaders make informed decisions. Core requirements for both roles include, but aren’t limited to: Experience in machine learning and deep learning.How to build computer vision algorithms and probability models.Problem-solving skills, creativity, ingenuity, and innovation.Languages like Python, R, Hadoop, Java, and Spark.Be able to see the big picture while at the same time finding the devil in the details. Always striving to improve, to make better, to advance the technology within the industry. The Challenges and the Potential of Technology in Healthcare At the moment, Computer Vision, AI, and other healthcare technology models are localized to individual placements. The next step is to have these technologies ‘speak’ to each other across hospitals, provider’s offices, telehealth applications, and electronic health records management for a more cohesive benefit of care. As this year rounds to a close, we know the vulnerabilities of our healthcare system, and can find solace in the though that technology is bringing it forward at lightning speed. Automation and telehealth appointments have made it a breeze to talk to our doctors and get results faster. We can pay our bills with the click of a button and even carve out a payment plan, if need be. All without leaving our homes. The data now available to us and our providers offers a foundation, a benchmark of information, so our doctors can make more informed decisions. This data goes beyond the individual, it helps set a precedent for not only individuals, but also entire populations, to help us identify future health issues, epidemics, and pandemics.  Stored data is private and stays within its construct of hospital or doctor’s office, but from it we can create models to plan for the future. Want to make your make your mark in the healthcare and tech industry? We may have just the role for you. Check out our current vacancies or get in touch with one of our expert consultants to learn more.   For our West Coast Team, contact us at (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to newyorkinfo@harnham.com.  

Recently Viewed jobs