Big data: 4 predictions for 2014

Sandra Namatovu our consultant managing the role
Posting date: 2/27/2014 12:00 AM

Big data was seen as one of the biggest buzzwords of 2013, when companies often used the term inappropriately and in the wrong context. This year, people will finally understand what it means

One could look back at 2013 and consider it the breakthrough year for big data, not in terms of innovation but rather in awareness. The increasing interest in big data meant it received more mainstream attention than ever before. Indeed, the likes of Google, IBM, Facebook and Twitter all acquired companies in the big data space. Documents leaked by Edward Snowden also revealed that intelligence agencies have been collecting big data in the form of metadata and, amongst other things, information from social media profiles for a decade.

And beyond all of that, big data became everyone's most hated buzzword in 2013 after it was inappropriately used everywhere, from boardrooms to conferences. This has led to countless analysts, journalists and readers calling for people to stop talking about big data. A good example could be seen in the Wall Street Journal last week, where a reader wrote in complaining:

A lot of companies talk about it but not many know what it is.

While that's a problem, it leads to my first prediction:

1. In 2014, people will finally start to understand the term big data. Because, as it stands, many do not.

The truth is that we've only really just started to talk about big data and companies aren't going to stop screaming about their latest big data endeavors. In fact, it's only January and the social bookmarking network Pinterest has already acquired image recognition platform VisualGraph. (Why? Pinterest want to understand what users are "pinning" and create better algorithms to help users better connect with their interests).

So let's get 2014 off on the right foot with a definition of big data, from researchers at St Andrews, that's fairly easy to understand:

The storage and analysis of large and/or complex data sets using a series of techniques including, but not limited to: NoSQL, MapReduce and machine learning.

The main elements revolve about volume, velocity and variety. And the word 'big'? If your personal laptop can handle the data on an Excel spreadsheet, it's not big.

Matt Asay, a journalist with ReadWriteWeb, also does a good job in explaining what makes a big data problem (as opposed to more traditional business intelligence).

If you know what questions to ask of your transactional cash register data, which fits nicely into a relational database, you probably don't have a big data problem. If you're storing this same data and also an array of weather, social and other data to try to find trends that might impact sales, you probably do.

2. Consumers will begin to (voluntarily) give up certain elements of privacy for personalization.

We've all heard of cookies – and we know that our actions around the internet affect the adverts that we see on websites and the suggested items we receive on Amazon. This is a concept that we've not only become accustomed to but also accept. After all, if we're going to have information put in front of us, we'd rather that we could relate to it.

But there have been problems in the past. Some websites have taken advantage of customers, for example increasing the prices for a flight that they've previously expressed interest in (consumers might worry that the price will go up even further and therefore decide to buy a ticket).

But as more companies instil big data techniques, customers will cooperate, on the premise that they will benefit. This is likely to follow Tesco's methodology, whereby customers are sent vouchers for goods that they are likely to buy anyway, creating a win-win situation for both parties. Customers, generally, are happy to receive a discount and retailers are pleased customers are coming back (especially if vouchers have an expiry date).

3. Big data-as-a-service will become a big deal

Despite claims from analysts that all businesses will look to hire data scientists, this just isn't going to happen. Firstly, there's a shortfall of data scientists, which goes some way in explaining why companies are retraining existing staff to work with big data) and secondly, not all companies are ready to (nor do they need to) invest in full-time data scientists to analyze and explain their data.

Instead, just as in other areas, I expect a wave of companies hustling to enter the big data-as-a-service space, an idea that began to creep into the latter parts of 2013. This could be anything from small and medium businesses signing up to anything from entire packages of storing, analyzing, explaining and visualizing data to more compact services, which focus on transferring data to cloud-based servers to allow for an accessible way of questioning the data in the future.

4. And finally... remember how Hadoop is an open-source software? Expect a lot more of that.

Hadoop, famously named after a toy elephant, is a well known piece of software to anyone curious about data science and it provides the backbone for many big data systems, allowing businesses to store and analyze masses of data. Most importantly, it's open source, which means that its implementation was inexpensive, allowing many organizations to understand, rather than ignore, the data they were collecting.

Quentin Gallivan, the chief executive of business analytics software firm Pentaho, explained last month that the rise of new open-source software will bring about more innovation and more ways of understand the data. He said:

New open source projects like Hadoop 2.0 and YARN, as the next generation Hadoop resource manager, will make the Hadoop infrastructure more interactive... projects like STORM, a streaming communications protocol, will enable more real-time, on-demand blending of information in the big data ecosystem.


Click here for the article on the web.

Related blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out the related posts below.

How to Succeed in Self-Service BI

How to Succeed in Self-Service BI

Business Intelligence, along with Business Analytics and Big Data, is one of the terms often associated with decision-making processes in organisations.  However, there is little discussion around the importance of what skills decision makers in your organisation need to use the technology efficiently.  In recent years, the development of user-friendly tools for BI processes, Self-Service BI are increasing. Self-Service BI is an approach to BI where anyone in an organisation can collect and organise data for analysis without the assistance of data specialists. As a result of this, many businesses have invested in comprehensive storage and information processing tools. However, many are beginning to find that they are not able to realise the gains of these investments as they were expecting, may often due to underestimating the difficulties of introducing these systems into the current processes and transforming existing knowledge into actual actions and decisions.  In a worst-case scenario, if left unplanned, Self Service BI can sabotage your successful BI deployment by cutting mass user adoption, impairing query performance, failing to reduce report backlogs, and increasing confusion over the “single truth”. To prevent this from happening, here are our top three tips for ensuring the right implementation of SSBI in your company: UNDERSTAND YOUR USERS’ NEEDS There are three major user areas for analytics tools: strategic, tactical and operational. The strategic users make few, but important decisions. The tactical users make many decisions during a week and need updated information daily. Operational users are often closest to the customer, and this group needs data in its own applications in order to carry out a large number of requests and transactions.  Understanding the different needs of each group is necessary to know what information should be available at each given frequency to help scale the BI solution.  HARNESS THE POWER OF ADVANCED USERS To ensure a successful BI deployment, utilising advanced users is key. Self-service BI is not a one-size fits all approach. Casual users usually don’t have the time to learn the tool and will often reach out to ‘Power Users’ to create what they need. Hence, these users can become the go-to resource for creating ad-hoc views of data. Power Users are the ideal advocates for your business’ self-service BI implementation and should be able to help spur user adoption.  UPGRADE INTERNAL COMPETENCIES  Our final tip for a successful implementation is to communicate the new tool thoroughly to the users.  It is highly unlikely that employees who have not been involved in the actual development project will immediately understand what the tool should be used for, who needs it, and what it should replace. By upgrading internal competencies, you can avoid becoming dependent on external assistance. Establishing a cross-organizational BI competence centre of 5-10 members, who meet regularly to share their experiences will help drives and prioritise future use of the tool. The added benefit of a successful implementation is that it will generate new ideas from users for how the organisation can use data to make better decisions. If you have the skillset to implement Business Intelligence solutions, we may have a role for you.  Take a look at our latest opportunities or get in contact with our team. 

Real Time Pricing - Coming to a store near you

Real Time Pricing - Coming to a store near you

Real-time pricing: coming to a store near you.Personal shopping is on the brink of taking on a whole new meaning. The advancement of mobile technology and the information held on individuals' shopping histories means product prices could soon adapt as shoppers walk up and down their supermarket aisle.Gone are the days of retailers only being able to actively manage the price of a small number of products once a week. Algorithmic pricing and real-time competitive pricing data allows the changing of product prices on the fly.Amazon is at the forefront of such "real-time pricing" initiatives, which have traditionally been the preserve of online-only retailers.However, brick-and-mortar retailers in the US are showing their UK counterparts the limitless possibilities when it comes to dynamic pricing.Independent consumer electronics retailer Abt Electronics pipes competitive pricing data gathered by Dynamite Data into its point-of-sale systems to allow staff to negotiate prices at the point-of-sale, according to Dynamite Data chief executive Diana Schulz.Meanwhile, another one of Dynamite Data’s unnamed clients uses electronic shelf labels and re-prices every product in their stores each morning based on the prices of its rivals.The ability to change prices dynamically is not simply the preserve of all-powerful brands such as Walmart or Target either.Schulz explained that her company has "seen these types of technologies in both large and mid-sized retailers" despite the "investment in technology and competitive data that is typically needed".Commercial sensitivitiesBack in the UK things are not quite as close to a Minority Report-style personalized shopping experience.Even online-only specialists Shop Direct and Ocado claim they do not engage in real-time pricing, while those that do heavily use real-time data to adapt their prices such as the airline brands are reluctant to discuss the issues.EasyJet declined to comment when contacted because of commercial sensitivities around discussing pricing-related issues.Grocers Tesco, Asda and  Sainsbury’s have all claimed they do not engage in real-time pricing, with the latter two both citing the logistical difficulties in aligning such a strategy across their physical stores and online presence.A Sainsbury’s spokesman claims real-time pricing would result in "chaos", while an Asda spokeswoman saying such a strategy would be a "nightmare".Yet, despite such a negative perspective from UK brands, experts are confident real-time pricing will arrive on these shores sooner or later.Simon Spyer, a partner of VCCP data arm Conduit who began his career working on the Sainsbury's Nectar business, believes the UK will begin to see "more and more" of matching rivals’ prices dynamically, particularly in the grocery and electrical sectors.He explained that real-time pricing is likely to affect "anything where the product is largely commoditized" and in instances where the only way retailers can differentiate that product is by "being really keen on price".Electronic labelsAs it stands the major barrier for implementing "real-time pricing" in-store is changing the prices to match the online price, a hurdle that could be removed by the electronic shelf labels being pioneered in the US.Schemes like Tesco Price Promise and Asda Price Guarantee already use real-time data to 'price match'In the UK various retailers have dipped their toes into the water when it comes to electronic shelf-labeling including a Nisa Local store in Shrewsbury that launched a trial in August last year to carry out automatic pricing and timed promotional updates, alongside QR codes and meal deals.Tesco has also experimented with electronic labeling on various occasions with trials in 2006 and 2008, but the retail giant has yet to combine real-time pricing with its electronic labels.Spyer claims "the capability is definitely there both online and offline – it is whether there is a business rationale for investing in it".However, with major UK supermarkets lacking a pressing reason to implement real-time pricing, that investment may be slow in arriving, argues Kaye Coleman, the founder of price consultancy Ripe Strategic.Coleman explains: "The supermarkets already do price matching – it is not so sophisticated but price matching is already happening".Schemes including the Tesco Price Promise, the Asda Price Guarantee and the Sainsbury’s Brand Match currently use real-time data to "price match" by offering money off the next shop.A cynic could argue the supermarkets should knock money off at the till rather than relying on customers to redeem their vouchers at the next shop, but such an action could hit the companies' bottom line.Mobile sophisticationThe growing sophistication of mobile marketing is also likely to revolutionize the way brands approach their price matching."If you can come up with a value proposition where I check-in [on my mobile] when I walk through the store for the first time and that presents me with a personalized experience based on my purchase history then I could see the benefit for a customer and a retailer," said Spyer.The trick for retailers is persuading customers to adopt such behavior, but the offer of being delivered ever-changing personalized price offers and messages in-store is a compelling proposition.Personalization is already a priority for retailers. Sainsbury’s uses anonymized shopping data gathered from the Nectar card to personalize offers.The levels of personalization offered by Sainsbury’s are increasingly complex. If a female customer buys folic acid they will be sent promotions on other pregnancy-related supplements during the pregnancy period and offers on nappies further down the line.UK retailers are sure to keep a close eye on developments over the Atlantic, with Schulz claiming she knows of clients that are piloting technologies that enable in-store personalized discounts.The challenges on the high-street mean there will inevitably be more casualties, but real-time pricing does not have to be the sole preserve of online-only retailers.Innovative ways of manipulating real-time data could be the shot in the arm the high-street retail industry so desperately needs.This article was first published on marketingmagazine.co.ukClick here for the article on the web.

RELATED Jobs

Salary

750000kr - 900000kr per annum

Location

Stockholm

Description

Are you looking for career progression and take on more responsibilities - project ownership or people management then this is the perfect role for you!

Salary

£40000 - £55000 per annum + Competitive Benefits

Location

London

Description

A fast paced role forecasting customer lifetime value and setting pricing and profitability strategies at an exciting lender!

Salary

£85000 - £105000 per annum + bonus + benefits

Location

London

Description

An international bank is looking to bring on-board a proven Manager with skills in SAS and SQL.

Salary

£20000 - £30000 per annum

Location

City of London, London

Description

If you are looking for a fast-paced and social environment with ample learning and progression opportunities, then this is the role for you.

recently viewed jobs