Big data: 4 predictions for 2014

Sandra Namatovu our consultant managing the role
Posting date: 2/27/2014 12:00 AM

Big data was seen as one of the biggest buzzwords of 2013, when companies often used the term inappropriately and in the wrong context. This year, people will finally understand what it means

One could look back at 2013 and consider it the breakthrough year for big data, not in terms of innovation but rather in awareness. The increasing interest in big data meant it received more mainstream attention than ever before. Indeed, the likes of Google, IBM, Facebook and Twitter all acquired companies in the big data space. Documents leaked by Edward Snowden also revealed that intelligence agencies have been collecting big data in the form of metadata and, amongst other things, information from social media profiles for a decade.

And beyond all of that, big data became everyone's most hated buzzword in 2013 after it was inappropriately used everywhere, from boardrooms to conferences. This has led to countless analysts, journalists and readers calling for people to stop talking about big data. A good example could be seen in the Wall Street Journal last week, where a reader wrote in complaining:

A lot of companies talk about it but not many know what it is.

While that's a problem, it leads to my first prediction:

1. In 2014, people will finally start to understand the term big data. Because, as it stands, many do not.

The truth is that we've only really just started to talk about big data and companies aren't going to stop screaming about their latest big data endeavors. In fact, it's only January and the social bookmarking network Pinterest has already acquired image recognition platform VisualGraph. (Why? Pinterest want to understand what users are "pinning" and create better algorithms to help users better connect with their interests).

So let's get 2014 off on the right foot with a definition of big data, from researchers at St Andrews, that's fairly easy to understand:

The storage and analysis of large and/or complex data sets using a series of techniques including, but not limited to: NoSQL, MapReduce and machine learning.

The main elements revolve about volume, velocity and variety. And the word 'big'? If your personal laptop can handle the data on an Excel spreadsheet, it's not big.

Matt Asay, a journalist with ReadWriteWeb, also does a good job in explaining what makes a big data problem (as opposed to more traditional business intelligence).

If you know what questions to ask of your transactional cash register data, which fits nicely into a relational database, you probably don't have a big data problem. If you're storing this same data and also an array of weather, social and other data to try to find trends that might impact sales, you probably do.

2. Consumers will begin to (voluntarily) give up certain elements of privacy for personalization.

We've all heard of cookies – and we know that our actions around the internet affect the adverts that we see on websites and the suggested items we receive on Amazon. This is a concept that we've not only become accustomed to but also accept. After all, if we're going to have information put in front of us, we'd rather that we could relate to it.

But there have been problems in the past. Some websites have taken advantage of customers, for example increasing the prices for a flight that they've previously expressed interest in (consumers might worry that the price will go up even further and therefore decide to buy a ticket).

But as more companies instil big data techniques, customers will cooperate, on the premise that they will benefit. This is likely to follow Tesco's methodology, whereby customers are sent vouchers for goods that they are likely to buy anyway, creating a win-win situation for both parties. Customers, generally, are happy to receive a discount and retailers are pleased customers are coming back (especially if vouchers have an expiry date).

3. Big data-as-a-service will become a big deal

Despite claims from analysts that all businesses will look to hire data scientists, this just isn't going to happen. Firstly, there's a shortfall of data scientists, which goes some way in explaining why companies are retraining existing staff to work with big data) and secondly, not all companies are ready to (nor do they need to) invest in full-time data scientists to analyze and explain their data.

Instead, just as in other areas, I expect a wave of companies hustling to enter the big data-as-a-service space, an idea that began to creep into the latter parts of 2013. This could be anything from small and medium businesses signing up to anything from entire packages of storing, analyzing, explaining and visualizing data to more compact services, which focus on transferring data to cloud-based servers to allow for an accessible way of questioning the data in the future.

4. And finally... remember how Hadoop is an open-source software? Expect a lot more of that.

Hadoop, famously named after a toy elephant, is a well known piece of software to anyone curious about data science and it provides the backbone for many big data systems, allowing businesses to store and analyze masses of data. Most importantly, it's open source, which means that its implementation was inexpensive, allowing many organizations to understand, rather than ignore, the data they were collecting.

Quentin Gallivan, the chief executive of business analytics software firm Pentaho, explained last month that the rise of new open-source software will bring about more innovation and more ways of understand the data. He said:

New open source projects like Hadoop 2.0 and YARN, as the next generation Hadoop resource manager, will make the Hadoop infrastructure more interactive... projects like STORM, a streaming communications protocol, will enable more real-time, on-demand blending of information in the big data ecosystem.


Click here for the article on the web.

Related blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out the related posts below.

Weekly News Digest: 14th - 18th June 2021

This is Harnham’s weekly news digest, the place to come for a quick breakdown of the week’s top news stories from the world of Data & Analytics. Gov.uk: Five signs of a good data quality culture Particularly post-pandemic, we all want to know that our data is fit for purpose. In this article from the Government Data Quality Hub, they look at five ways to ensure that your data's quality is right for your's and your users’ needs. This includes: Everyone is involvedData quality is a commitment, not a taskYou know what works for your organisationYou know why quality mattersYou are proactive not reactive We know that committing to a good data quality culture is a continual process. This core advice allows us to take a step back and think about how you can understand your unique challenges and involve the right people, so you can prevent bad quality data before it damages your work. See more on this here. Analytics Insight: 5 types of artificial intelligence that will shape 2021 and beyond We really like this article from Analytics Insight that explores the future of technology, and specifically the rise in uses of artificial intelligence (AI). AI is often seen to be disruptive as there is an assumption that robots could take over and jobs are wiped out, but it’s more likely that humans and machines will work together to streamline processes across a range of industries. The different types of AI to keep an eye on include: Customised technology providerChoosy algorithmHuman-machine interactionReciprocating machinesTheory of mind We’re always excited to learn more about new technologies, click here to read more on this. KD Nuggets: Five types of thinking for a high performing data scientist In this piece KD Nuggets look at how the way our approach to problem-solving may be guided by your personal skills or the type of problem at hand. As a Data Scientist, appreciating different approaches can help you more effectively model data in the business world and communicate your results to the decision-makers. Whether this is model thinking, systems thinking, agent-based thinking, behavioural thinking, or computational thinking, taking the time to understand your approach will significantly help the way you complete the function of your role. To read the full article, see here.  TechRepublic: These 220+ courses will help you master tech skills and prep for IT certification exams We know that there is a digital skills gap. According to Boston Consulting Group, there will be tens of millions of job vacancies by 2030 that will be hard to fill because not enough workers have the required skills, many of which are in technology. One of the best ways to upgrade your skillset is to complete extra training and qualifications to ensure you’re always learning more about your market and providing yourself with the best opportunities to achieve your next career step. ITU Online has over 200 courses covering cloud deployment, cybersecurity and more. Of course, this isn’t the only way in which you can level up your skills, but it’s a good place to start! To read more about this, click here.  We've loved seeing all the news from Data & Analytics in the past week, it’s a market full of exciting and dynamic opportunities. To learn more about our work in this space, get in touch with us at info@harnham.com.    

How Will Embracing Flexible Working Help The Life Science Sector To Grow?

COVID-19 has drastically changed ways of working in the Life Science industry. Overnight, teams moved online, while new research had to be prioritised. Life Sciences were already moving towards more remote working, and the pandemic has only quickened this shift. There is no doubt these changes have fundamentally changed the Life Science sector and how professionals working in this space operate post-pandemic.  However, uncertainty still remains about the viability of remote working for the sector and there is a divide between those able to work remotely and those who need to go into ‘wet labs’. Is remote working a step too far for Life Sciences? Collaboration  2020 saw an increase in collaboration between professionals working across different areas of Life Sciences. Interestingly, organisations who may usually compete came together to share data and work towards a shared goal. Collaboration is essential in Life Sciences, yet for many, remote working reduces spontaneous teamwork and creativity.  New flexible lab spaces may be the future for Life Sciences though. RUNLABS have recently opened their first fully equipped flexible lab space in Paris for scientists and companies working in Life Sciences. This space hopes to builds on the existing collaborative approach in the industry and encourage further cooperative innovation. Efficiency  Many employees noticed a spike in employee efficiency when working remotely. By eliminating commutes and increasing flexibility, employees were able to be more productive with their time. Remote working also allowed organisations to streamline processes and reduce time spent in meetings.  However, insight from McKinsey highlights that research and development leaders estimate productivity has fallen by between 25 and 75 per cent due to remote working. Those in pharma manufacturing have reported lower levels off efficiency, as well as the potential for lower-quality outputs.  Research The pandemic forced remote trails to become a necessity, and since then, they have increased in popularity. While face-to-face research is still preferrable, remote trials can reduce costs and improve efficiencies. Indeed, on-site monitoring accounts for a significant portion of the costs of bringing a new product to market, yet this is no longer necessary in remote trials.   Not only are remote trials more cost-effective, but they can open research to a wider range of patients and can increase the communication between trial participants. Diversity Flexible working can run a risk to diversity and inclusion though. McKinsey also notes that, ‘when faced with a crisis, leaders often revert to relying on the core team of people they already know and trust. This disproportionately affects women and minorities because they are often not part of that group. Differences in perceptions and experiences of inclusion results in individuals or communities being disenfranchised, which can be devastating to careers and create a two-tiered culture.’ We know that 27 per cent of D&I leaders say their organisation have put all or most of their initiatives that embrace diversity and inclusion on hold because of the pandemic. However, remote work unlocks new hire pools and opens up the workplace to a more diverse workforce. Workers are no longer restricted by their geographical location or personal circumstances. Flexible working is an opportunity for Life Science organisations to harness a wider talent pool and increase their diversity. There is no doubt that Life Science is one of the most cutting-edge sectors globally and the pandemic has only cemented this. COVID-19 has shown the potential for remote working in life sciences, and in-person health care professional access may never return to pre-lockdown levels. But, going forward life sciences need to remember remote working is not practical for everyone nor every role. Organisations will need to consider individual wellbeing and role efficiency as they decide their next step.  If you’re in the world of Data & Analytics and looking to take a step up or find the next member of your team, we can help. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more. 

RELATED Jobs

Salary

£65000 - £75000 per annum + benefits

Location

City of London, London

Description

A disruptive FinTech are looking for a new Lead Analyst - £75,000.

Salary

£60000 - £61000 per annum + Yes

Location

City of London, London

Description

London/Remote

Salary

£60000 - £80000 per annum

Location

London

Description

Exciting opportunity to join a team of elite Data Engineers

recently viewed jobs