How Netflix Got Big with Big Data

Femi Akintoye our consultant managing the role
Posting date: 8/24/2018 7:51 AM
There’s little argument that Netflix have changed the game when it comes to how people consume entertainment. Whilst Amazon, Disney and Apple seek to replicate the success of Netflix’s model, they still lead the way with over 130 million subscribers worldwide and have just broken HBO’s 17-year streak as the most nominated ‘network’ at the Emmy’s with an astonishing 112 nominations. 

Having begun life as a subscription-based DVD rental-service created in response to founder Reed Hasting’s frustration with late rental fines, Netflix were one of the first to offer video-streaming as an option for viewing films and TV. Now filled with scores of original programming, the secret to their success lies not just in creativity and innovation, but in Big Data. 

Top Picks From Your Data


When the former CEO of the now-defunct Blockbuster claimed: “Netflix doesn’t really have or do anything that we can’t or don’t already do ourselves”, he made a vital oversight. Whilst Netflix may have offered fewer films and TV shows at the time, they were already busy collecting, and utilising, customer data in a way that hadn’t been done before. This included:

What do people search for?
When do they watch a program?
What device do they watch on?
Do genre preferences vary with device?
When do they stop watching?
What shows are the likely to ‘binge’?
Or even what are the horror films that people find too scary to watch until the end…

Netflix used, and still uses, this information to create recommendations for each user, curating an individual experience based upon personal preferences. This technique has been incredibly successful with over 75% of viewer activity based upon these recommendations.

And they continue to finesse how their collect their data, switching from a five-star rating system to a thumbs up/thumbs down model. Cameron Johnson, Netflix’s Director of Product Innovation had observed: “a difference between what [users] say, and what they do,”. For example, frequently-watched comedies were being awarded three stars, as opposed to occasionally-watched, but ‘more worthy’ documentaries being given five stars. By simplifying the system to a like/dislike set-up, Netflix can provide subscribers with recommendations “more aligned with what people actually play”. 

Stream if you want to go faster


Unlike traditional broadcast mediums, Netflix’s income doesn’t come from advertising, or a pay-per-view service, but subscribers. That means their main ambitions are to generate new subscribers and keep existing ones.

If Netflix has data that tells them users who stream over a specific number of hours of programming are more likely to stay, they can place their focus on ensuring they watch at least that many hours. It’s highly likely that the introduction of the ‘skip-credits’ feature was a result of Netflix realising that this was the time when people were most likely to turn off, when the was an opportunity to encourage them to watch more. 

Perhaps most interestingly of all, Netflix’s Big Data team are helping inform creativity. This ranges from supplying that data that helps personalise trailers for new content based on each subscriber’s preferences, to deciding which shows to commission. Netflix’s data told them that prison-based dramas, shows with strong-female ensembles, and programs with LGBT+ themes and characters were both popular, and shared a lot of audience overlap. With all this information at hand when they commissioned ‘Orange Is The New Black’ for a full series, Netflix could be sure that there was an audience for the show. 

As more and more companies add their own streaming services, including Disney’s expected behemoth, this targeted original content is going to become more and more valuable for Netflix. Fortunately, they’re long-used to changing not just how people watch, but also what they watch. 

Browse Our Collection


If you’re looking to apply your understanding of Big Data to disrupt and revolutionise an industry, we may have a role for you.

Take a look at our current selection of opportunities here

Related blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out the related posts below.

WE HAVE TO TEACH SPECIALISATION, WE CAN’T EXPECT IT: A Q&A WITH VIN VASHISHTA

We recently spoke to Vin Vashishta, a consulting Data Scientist and Strategist who was named one of LinkedIn’s Top Voices in Data Science.  Having started off in the tech world 25 years ago and progressing from web design and hardware installation to Business Intelligence Analytics, Vin found for many years that enterprises were reluctant to adopt AI technologies and embrace the value of Data. In fact, it wasn’t until the beginning of the decade just passed that companies started to think about their Data more strategically and the world of Data Science was born, albeit hesitantly:  “When I first started, it was a lot of experimentation, everyone wanted a proof of concept,” he says. “A lot of work was creating models that could go from whiteboard to production and productise and show their value.” However, it wasn’t until halfway through the decade that he began to see businesses who had adopted Machine Learning move away from experimentation into incorporating it more deeply into their companies, relying more on analytical and optimisation models to make strategic business decisions.  “After that, in about 2017/2018 the maturity changed. It went from being a one off implementation to it being a comprehensive tool within an organisation where we have full lifecycles of model implementation and full models that were full views of the system. The key component of development was allowing users to access a small part of the system to do their job better without having to understand the whole thing. And that’s where we are now. We have this applied Deep Learning and we are seeing, especially this year, attempts to optimise that, make things go faster and make them more repeatable.” But, as we all know, with great power comes great responsibility: “There’s this whole depth we are getting into, the expectations are so much higher, people don’t just expect it to work they expect it to work the way they want it to and in a way they can adopt.” So, with so much expected and required of Data Scientists in 2020, building the right team is more important than ever. However, many businesses, Vin believes, are yet to get their hiring processes right: “A lot of the measures that we use to sort of evaluate employees are fictional – when you say years of experience, it has no correlation to employee outcomes or the quality of employee you get long term. It’s the same thing as college degree, there’s no correlation.” So when Vin is trying to build a highly specialised team, what does he do? “We have to teach specialisation, we can’t expect it. We can’t bring someone in and call them a Data Scientist and hope that they train up. You end up with teams that are exactly the same because they have hired the same people, people who reinforce the bias of what they do, and that is where true leadership needs to come in.” A specialised team made up of individuals who bring their own ideas to the table is more important than ever, particularly as businesses demand more from their Data teams. Gone are the days of one-size-fits-all models. Businesses now want something tailored to them: “Custom models are huge. The “import from…” Machine Learning development from three years ago adds value when it comes to wrangling and doing the Analysis, but when it comes to creating models companies are now expecting it to become a competitive advantage. Companies no longer want the same model that everyone else has, now it has to be differentiating.” These smart, customised models, he adds, will help businesses through the current pandemic. “The best models right now are adapting rather than reacting.”  However, he’s sceptical about the Data Science community becoming too preachy:  “When it comes to COVID-19 one message I want to send to the Machine Learning and Deep Learning community is ‘shut up’. We don’t have the Data! We have so many Data Scientists talking about something that’s very important to get right. If you get it wrong the consequences and the credibility we will lose as a field is enormous.” Indeed, discussions about the lack of quality Data on COVID-19 are widespread at the moment and raise concerns for Vin: “What the last two and a half months has revealed is the danger of bad Data, the danger of assumptions that are hidden in Data that hasn’t been looked over well or wasn’t gathered well and was fed into these models that now aren’t robust. Of course, no model can account for something this drastic, but they should still be performing far better than they are right now.” Despite these concerns, Vin believes any change in the world brings about opportunities for those in the Data and technology space. “What I’ve been trying to do ever since I joined the technology space is figure it out. It’s constantly evolving and it’s constantly changing. That’s really what has driven my journey. I’m always trying to figure out ‘what’s next’ over the next five years, ten years whatever it may be.” If you’re looking for your next Data Science, Machine Learning or Deep Learning role, or want to build out your own highly-specialised team, we may be able to help.  Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.   

The Search For Toilet Paper: A Q&A With The Data Society

We recently spoke Nisha Iyer, Head of Data Science, and Nupur Neti, a Data Scientist from Data Society.  Founded in 2014, Data Society consult and offer tailored Data Science training for businesses and organisations across the US. With an adaptable back-end model, they create training programs that are not only tailored when it comes to content, but also incorporate a company’s own Data to create real-life situations to work with.  However, recently they’ve been looking into another area: toilet paper.  Following mass, ill-informed, stock-piling as countries began to go into lockdown, toilet paper became one of a number of items that were suddenly unavailable. And, with a global pandemic declared, Data Society were one of a number of Data Science organisations who were looking to help anyway they could.  “When this Pandemic hit, we began thinking how could we help?” says Iyer. “There’s a lot of ways Data Scientists could get involved with this but our first thought was about how people were freaking out about toilet paper. That was the base of how we started, as kind of a joke. But then we realised we already had an app in place that could help.” The app in question began life as a project for the World Central Kitchen (WCK), a non-profit who help support communities after natural disasters occur.  With the need to go out and get nutritionally viable supplies upon arriving at a new location, WCK teams needed to know which local grocery stores had the most stock available.  “We were working with World Central Kitchen as a side project. What we built was an app that supposed to help locate resources during disasters. So we already had the base done.” The app in question allows the user to select their location and the products they are after. It then provides information on where you can get each item, and what their nutritional values are, with the aim of improving turnaround time for volunteers.  One of the original Data Scientists, Nupur Neti, explained how they built the platform: “We used a combination of R and Python to build the back-end processing and R Shiny to build the web application. We also included Google APIs that took your location and could find the closest store to you. Then, once you have the product and the sizes, we had an internal ranking algorithm which could rank the products selected based on optimisation, originally were based on nutritional value.”  The team figured that the same technology could help in the current situation, ranking based on stock levels rather than nutritional value. With an updated app, Iyer notes “People won’t have to go miles and stand in lines where they are not socially distancing. They’ll know to visit a local grocery store that does have what they need in stock, that they’ve probably not even thought of before.” However, creating an updated version presented its own challenges. Whereas the WCK app utilised static Data, this version has to rely on real-time Data. Unfortunately this isn’t as easy to come by, as Iyer knows too well:  “When we were building this for the nutrition app we reached out to groceries stores and got some responses for static Data. Now, we know there is real-time Data on stock levels because they’re scanning products in and out. Where is that inventory though? We don’t know.” After putting an article out asking for help finding live Data, crowdsourcing app OurStreets got in touch. They, like Data Society, were looking to help people find groceries in short supply. But, with a robust front and back-end in place, the app already live, and submissions flying in across the States, they were looking for a Data Science team who could make something of their findings.  “We have the opportunity,” says Iyer “to take the conceptual ideas behind our app and work with OurStreets robust framework to create a tool that could be used nationwide.” Before visiting a store, app users select what they are looking for. This allows them to check off what the store has against their expectations, as well as uploading a picture of what is available. They can also report on whether the store is effectively practising social distancing. Neti explains, that this Data holds lots of possibilities for their Data Science team: “Once we take their Data, our system will clean any submitted text using NLP and utilise image recognition on submitted pictures using Deep Learning. This quality Data, paired with the Social Distancing information, will allow us to gain better insights into how and what people are shopping for. We’ll then be able to look at trends, see what people are shopping for and where. Ultimately, it will also allow us to make recommendations as to where people should then go if they are looking for a product.”  In addition to crowdsourced information, Data Society are still keen to get their hands on any real-time Data that supermarkets have to offer. If you know where they could get their hands on it, you can get in touch with their team.  Outside of their current projects, Iyer remains optimistic for the world when it emerges from the current situation: “Things will return to normal. As dark a time as this is, I think it’s going to exemplify why people need to use Artificial Intelligence and Data Science more. If this type of app is publicised during the Coronavirus, maybe more people will understand the power of what Data and Data Science can do and more companies that are slow adaptors will see this and see how it could be helpful to their industry.”   If you want to make the world a better place using Data, we may have a role for you, including a number of remote opportunities. Or, if you’re looking to expand and build out your team with the best minds in Data, get in touch with one of expert consultants who will be able to advise on the best remote and long-term processes. 

RELATED Jobs

Salary

£40000 - £50000 per annum + pension and benefits

Location

London

Description

Great role in a strong BI team of Analysts and Developer and DBA's to work on 3 major migration and Data Warehouse projects

Salary

£95000 - £105000 per annum + + BONUS + BENEFITS

Location

City of London, London

Description

Does technical leadership excite you? Ready to become an expert in streaming? Offering great opps for development, and progression, this role will close fast.

Salary

€80000 - €90000 per annum + BENEFITS

Location

Düsseldorf, Nordrhein-Westfalen

Description

Ein international bekanntes Ecommerce Unternehmen sucht nach einem erfahrenen Data Engineer für die deutsche Niederlassung.

recently viewed jobs