Big data: 4 predictions for 2014

Sandra Namatovu our consultant managing the role
Posting date: 2/27/2014 12:00 AM

Big data was seen as one of the biggest buzzwords of 2013, when companies often used the term inappropriately and in the wrong context. This year, people will finally understand what it means

One could look back at 2013 and consider it the breakthrough year for big data, not in terms of innovation but rather in awareness. The increasing interest in big data meant it received more mainstream attention than ever before. Indeed, the likes of Google, IBM, Facebook and Twitter all acquired companies in the big data space. Documents leaked by Edward Snowden also revealed that intelligence agencies have been collecting big data in the form of metadata and, amongst other things, information from social media profiles for a decade.

And beyond all of that, big data became everyone's most hated buzzword in 2013 after it was inappropriately used everywhere, from boardrooms to conferences. This has led to countless analysts, journalists and readers calling for people to stop talking about big data. A good example could be seen in the Wall Street Journal last week, where a reader wrote in complaining:

A lot of companies talk about it but not many know what it is.

While that's a problem, it leads to my first prediction:

1. In 2014, people will finally start to understand the term big data. Because, as it stands, many do not.

The truth is that we've only really just started to talk about big data and companies aren't going to stop screaming about their latest big data endeavors. In fact, it's only January and the social bookmarking network Pinterest has already acquired image recognition platform VisualGraph. (Why? Pinterest want to understand what users are "pinning" and create better algorithms to help users better connect with their interests).

So let's get 2014 off on the right foot with a definition of big data, from researchers at St Andrews, that's fairly easy to understand:

The storage and analysis of large and/or complex data sets using a series of techniques including, but not limited to: NoSQL, MapReduce and machine learning.

The main elements revolve about volume, velocity and variety. And the word 'big'? If your personal laptop can handle the data on an Excel spreadsheet, it's not big.

Matt Asay, a journalist with ReadWriteWeb, also does a good job in explaining what makes a big data problem (as opposed to more traditional business intelligence).

If you know what questions to ask of your transactional cash register data, which fits nicely into a relational database, you probably don't have a big data problem. If you're storing this same data and also an array of weather, social and other data to try to find trends that might impact sales, you probably do.

2. Consumers will begin to (voluntarily) give up certain elements of privacy for personalization.

We've all heard of cookies – and we know that our actions around the internet affect the adverts that we see on websites and the suggested items we receive on Amazon. This is a concept that we've not only become accustomed to but also accept. After all, if we're going to have information put in front of us, we'd rather that we could relate to it.

But there have been problems in the past. Some websites have taken advantage of customers, for example increasing the prices for a flight that they've previously expressed interest in (consumers might worry that the price will go up even further and therefore decide to buy a ticket).

But as more companies instil big data techniques, customers will cooperate, on the premise that they will benefit. This is likely to follow Tesco's methodology, whereby customers are sent vouchers for goods that they are likely to buy anyway, creating a win-win situation for both parties. Customers, generally, are happy to receive a discount and retailers are pleased customers are coming back (especially if vouchers have an expiry date).

3. Big data-as-a-service will become a big deal

Despite claims from analysts that all businesses will look to hire data scientists, this just isn't going to happen. Firstly, there's a shortfall of data scientists, which goes some way in explaining why companies are retraining existing staff to work with big data) and secondly, not all companies are ready to (nor do they need to) invest in full-time data scientists to analyze and explain their data.

Instead, just as in other areas, I expect a wave of companies hustling to enter the big data-as-a-service space, an idea that began to creep into the latter parts of 2013. This could be anything from small and medium businesses signing up to anything from entire packages of storing, analyzing, explaining and visualizing data to more compact services, which focus on transferring data to cloud-based servers to allow for an accessible way of questioning the data in the future.

4. And finally... remember how Hadoop is an open-source software? Expect a lot more of that.

Hadoop, famously named after a toy elephant, is a well known piece of software to anyone curious about data science and it provides the backbone for many big data systems, allowing businesses to store and analyze masses of data. Most importantly, it's open source, which means that its implementation was inexpensive, allowing many organizations to understand, rather than ignore, the data they were collecting.

Quentin Gallivan, the chief executive of business analytics software firm Pentaho, explained last month that the rise of new open-source software will bring about more innovation and more ways of understand the data. He said:

New open source projects like Hadoop 2.0 and YARN, as the next generation Hadoop resource manager, will make the Hadoop infrastructure more interactive... projects like STORM, a streaming communications protocol, will enable more real-time, on-demand blending of information in the big data ecosystem.


Click here for the article on the web.

Related blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out the related posts below.

Weekly News Digest: 1st - 5th March 2021

This is Harnham’s weekly news digest, the place to come for a quick breakdown of the week’s top news stories from the world of Data & Analytics.  Analytics Insight: Top 10 analytics and business intelligence buzzwords in 2021 If you are a fan of both buzzwords and analytics, then look no further, this article from Analytics Insight is for you! The team at the global publication identify and explore the top buzzwords that are being used to define business intelligence and analytics techniques across the industry in 2021. A few of these include: Predictive AnalyticsEmbedded AnalyticsCognitive ComputingData ScienceX Analytics Ultimately, there are a whole host of buzzwords and key terms being used in Data & Analytics at the moment, but professionals should keep up to date with the core (and most influential) technologies and insights in their area of expertise. Find out more about the top 10 buzzwords and view their definitions here. Forbes: Diversity is what you see, Inclusion is what you do Writing for Forbes, Paolo Gaudiano discusses how to really examine the values and culture of an organisation: you need to change the way in which you think about – or approach – understanding the unique contributions of your team. “Being forced to think about recreating an organization from the inside out, i.e., by actually thinking from the point of view of the individual employees and their experiences, helped us to clarify how we can think about—and define—diversity and inclusion.” It is in considering diversity and inclusion in two separate approaches, that an organisation can truly make this a core area of focus. Afterall, Gaudiano highlights that, “Diversity is a measure of how an individual’s personal characteristics differ from those of the normative majority of an organisation; inclusion is the act of ensuring that people’s experiences within an organisation are not impacted negatively as a result of their personal characteristics.” We need to provide more support, education and tools to ensure that our companies can sustain a growing level of diversity, and to enjoy an inclusive environment. Read more on this here. Computer Weekly: It’s now or never for UK fintech, government told The contributors at Computer Weekly have this week reported on how a Treasury-commissioned review of the UK’s future in financial technology (fintech) has told the government that it must urgently introduce effective policies in five key areas if the fintech industry is to continue to thrive. Policy & Regulation To include creating a new regulatory framework for emerging technology.Skills To include retraining and upskilling adults in support of UK fintech.Investment To include introducing a global family of fintech indices.International To include driving international collaboration and delivering an international action plan for fintech.National Connectivity To include accelerating the development and growth of fintech clusters. The review was comprehensive and provides a startling call to action for senior government officials. Read more on the future growth of the UK’s fintech sector here.  AdAge: First-party data strategies for advertisers and publishers in the age of privacy Can brands partnering with publishers discover better Consumer Insights? That’s the question that this insight from AdAge explores, as brands consider shifting their marketing strategies to align with privacy regulations (and maintain or regain consumer trust). Some of the ways brands are responding are by looking at: First-party Data Strategies Marketers need to pinpoint the right customers to target for different messages. One way to do this is by partnering with publishers and other companies that can offer granular consumer insights built from first-party data in a privacy-safe way.Direct Relationships With Publishers By creating ties with publishers, through trusted networks or direct relationships, brands will be operating in a premium environment, thanks to the data and insights the publishers can provide.Creating A New Digital World Marketing’s priorities now need to shift to first-party data strategies and building trusted networks of first-party data owners. This gives brands an opportunity to not only gain more control over their advertising but also to rebuild consumer trust in advertising. To find out more about how brands and marketers are working together in a digital world, take a look at the article in full here. We've loved seeing all the news from Data & Analytics in the past week, it’s a market full of exciting and dynamic opportunities. To learn more about our work in this space, get in touch with us at info@harnham.com.

The Reliability Of Sleep Trackers For Sleep Data

One in three of us regularly suffer from poor sleep. By this we mean not entering the correct stages of the sleep cycle often enough. During the optimum eight hours of slumber, we should be getting per night, the body should enter three different stages of sleep on a cyclical rotation: light, deep and rapid eye movement (REM). The most important stage of this being deep sleep, of which a healthy adult should be entering for around one to two hours.  Unfortunately, it is often the case, for a vast number of reasons, that many adults struggle to wake up feeling refreshed. From absorbing too much blue light from screens before bed, poor dietary habits or increased levels of stress, there are many factors into why good sleep eludes nearly a third of us daily. Over the past year especially, as a direct result of the pandemic, our sleepless nights have become increasingly worse. It seems anxiety related to COVID-19 has spiked our inability to get good rest. What are the dangers of persistent low-quality sleep? Continual restless nights can have profound effects on both our bodies and our minds. It can place immense stress on the immune system, increasing the risk of becoming seriously ill. Other life-threatening diseases also linked with poor sleep include obesity, heart disease and diabetes.  Our mental state can also be incredibly damaged by consistent poor sleep. Not only does our ability to concentrate reduce, but our susceptibility to mental ill-health, such as depression, increases too.  It is no surprise then that, as a global population, our obsession with the amount of sleep we get per night has skyrocketed in the past few years, consequently seeing the boom of sleep tracking technology. From wearable tech such as the Fitbit and Apple Watches, to other bedside devices and bed sensors, the market for sleep trackers is estimated to reach $62bn in 2021 alone. But is this technology a reliable source of data for our sleep patterns? The problems with sleep trackers Wearable technology can only go so far when it comes to measuring our quality of sleep. Watches especially can usually measure aspects of our body such as heart rate and movement – all of which can be used as indicators of restfulness. However, their consistent accuracy is questionable. According to research, sleep trackers are 78 per cent accurate when it comes to identifying whether we are awake or asleep, which is a pretty good statistic for developing technology, however, this drops dramatically to 38 per cent when estimating how long it takes for users to fall asleep. For true accuracy, sleep should be measured through brainwave activity, eye movement, muscle tension, movement and breathing – all of which can only be looked at through a medical polysomnogram.  Additionally, much like many other sources of technology, sleep trackers have become a troublesome culprit for obsessive behaviour. In 2017, scientists coined the term Orthosomnia, the recognition of a real problem many were, and still are, having with become obsessive, to the point of mental ill-health, around tracking sleep. As stated by neurologist, Guy Leschziner; “If you have a device that is telling you, rightly or wrongly, that your sleep is really bad then that is going to increase your anxiety and may well drive more chronic insomnia." However, sleep trackers aren’t all bad. While not a tool to be used for sleep disorder diagnosis, they can be useful gadgets to help rethink our sleep habits to aim for a better night’s sleep.  The positives of sleep trackers While questions around the accuracy of this technology are prominent, trackers, overall, are pretty good when it comes to recording total sleep time. If used as a guide rather than an aid, sleep trackers can help users get into better sleep habits which in turn will undoubtedly improve their quality of sleep.  If the data is showing that users are only achieving five hours of sleep per night, and they are going to bed very late and rising early, then users may be encouraged to practice better sleep hygiene. From removing any blue light from the bedroom space, to taking an hour before bed to engage in less stimulating activities, such as reading, and practicing methods such as mindfulness or meditation to induce relaxation.  Sleep data from trackers can also be a useful tool to begin conversations with health professionals. Someone who regularly finds themselves groggy in the morning, with the notion that their sleep is badly disturbed, may find solace in sleep tracking data and it may give them the confidence to seek relevant help. While this sort of technology and its data will not be the end point for a diagnosis, it may give both the user and their doctor insight into any potential problems or issues they may be having with sleep.  Ultimately, those using sleep trackers shouldn’t be losing sleep over the data they present. Instead, ensure you are taking the analysis provided with a pinch of salt, and explore this in tandem with how you feel in yourself to assess whether you need to make changes to your sleep routine or seek help for a potential sleep disorder. Data is an incredibly important too, but using this in the right way is absolutely critical. If you're looking for a new role to get you out of bed in the morning or to build up your dream data team, we may be able to help. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more. 

RELATED Jobs

Salary

£50000 - £68000 per annum + benefits and bonus

Location

Leeds, West Yorkshire

Description

Do you want to work for a global telecommunications company as part of the data science team?

Salary

£600 - £700 per day

Location

London

Description

Cloud Native DevOps/ MLOPS Contract (Inside IR35) £600 - £700.day Long term contract Kubernetes, AKS, GKE, Seldon

Salary

£50000 - £68000 per annum

Location

Leeds, West Yorkshire

Description

This is your chance to work along side a team of elite Data Scientists as a Senior member of the team!

recently viewed jobs