Harnham

Data Engineering and Big Data jobs

What We Do

Processing Big Data has become crucial to countless businesses. Those who decide to pursue a career within Big Data face complex development challenges, too tough for relational database systems. After all, there is no Data Science without Data Engineers.

Whilst businesses may have vast quantities of data at the ready, it holds no real value unless it can be stored, harnessed and utilised. It’s estimated that most UK companies have at least 100 terabytes of data stored which could be used for various purposes around the business, as well as newly accessible hard-to-process sources such as web data, image data, and social media data. 

This is where those who work in Data Engineering become extremely valuable, as they develop Data Platforms that allow this kind of volume to be processed and used by Data Scientists and analysts across the business.

Whether you are a company looking to build a data platform for advanced analytics, or you are a candidate who can build systems and applications to process vast, complex data sources, Harnham are here to help. 

Technical skills: AWS, Google Cloud Platform and Azure, Spark, Hadoop, Java, Scala, Python

Meet the team

Latest Jobs

Solution Architect
Salary

£80000 - £150000 per annum

Location

London

Description

This is a career defining opportunity for either a data scientist or data engineer with extensive Spark and architecture experience.

ADD TO SHORTLIST
Java Data Engineer
Salary

£55000 - £65000 per annum

Location

London

Description

A unique opportunity is available for either a Software Engineer or a Data Engineer to lead the data infrastructure for an energy trading disruptor.

ADD TO SHORTLIST
Senior Big Data Engineer
Salary

£60000 - £80000 per annum

Location

City of London, London

Description

A tech giant based in Central London are looking for a Senior Data Engineer to join as one of the most Senior Engineers on the team.

ADD TO SHORTLIST
Scala Data Engineer
Salary

£60000 - £75000 per annum

Location

City of London, London

Description

A new role has become available to become a Scala Data Engineer in Central London.

ADD TO SHORTLIST
Head of Data Consulting
Salary

£130000 - £170000 per annum

Location

London

Description

Head of Data Consulting role available. This company is looking for someone from a strategic consulting background to lead their new data transformation.

ADD TO SHORTLIST
Senior Machine Learning Engineer
Salary

£90000 - £150000 per annum

Location

London

Description

Up to £100,000 + benefits Lead Software Development for an advanced data science team Work very closely with highly talented colleagues

ADD TO SHORTLIST

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

Why it is hard to build a Big Data team

Increasingly, I speak to managers who are adopting big data tools and developing PoCs to prove how they can make use of them. Just last week I spoke to a data architect who mentioned that if he didn’t get exposure to big data tech sooner rather than later, his current RDBMS skills may become redundant within the next few years. While that is likely an exaggeration, it is certainly an interesting point. Companies that would have never previously had the capability to interpret ‘Big Data’ are now exploring a variety of NoSQL platforms. In particular, the massive performance benefits gained from Spark and real-time/streaming tools have opened up a whole new world beyond just MapReduce. I don’t claim to be a data engineer, but as a recruiter for this sector, what I do is spend all day, every day interacting with big data developers, architects and managers (as well as keeping a close eye on the latest Apache incubator projects). Due to this, I have seen some recurring themes that have become trends when companies look to create and build their big data teams that are coming to the fore. Candidate demand The demand for Big Data professionals is very much a present day issue as the data companies have grand plans for is waiting for the right data developer to use the best tech to extract valuable insights from it. The best candidates receive massive interest, often gain multiple offers from a range of companies. Your business is now no longer just competing with large corporations such as Facebook, Twitter or Yahoo. Startups and SMEs are also vying for the best candidates. Candidates are seeing pay rises twice that of the normal rate, as illustrated in our salary guide. Candidate shortage The number of candidates with hands-on, production level Big Data experience is incredibly limited. We go to great lengths to find the candidates who can add real value to companies. The growth and exciting future for the big data industry has led to increased interest in big data jobs, particularly for those from RDBMS or software. engineering backgrounds. This leaves the industry in a difficult predicament: high demand + low supply = massive competition. There are countless examples of companies that have failed to recruit a Big Data team after a year of looking. Competition to get ahead and stand out Planning - Companies need to have a data road map detailing their future plans. Candidates want to clearly know what they are getting into and what to expect from a job. Innovation - Why get stuck on batch processing? The most exciting positions that candidates love are in data innovations teams, playing with real-time/streaming tech and new languages. Personal development, growth and training – with the data science market experiencing similar growth, many big data engineers are looking for a job that not only offers the chance to work with machine learning and similar fields; but training, mentoring towards clear career progression as standard. Speed – the length of the interview process is often seen as a reflection of the amount of red tape developers have to go through to get a job. The longer and more convoluted the process, the more put off some people may be. Complacency – don’t rest on your laurels, it’s unlikely that you’ll get 10s of CVs through when you are looking to fill a data role, so when you find a candidate you like, move swiftly to show your interest to them as quality candidates don’t come around often. By implementing these small but effective improvements to your recruiting process and how you develop data talent will see you create a team that is a success in this ever more digital analytics landscape. Companies who don’t create and nurture strong, dynamic teams will fall by the wayside. It’s Harnham’s job to help you achieve this goal. Get in touch with us to tell you how. T: (020) 8408 6070 E: info@harnham.com

Ross Whatling

06. June 2017

From Idea to Impact: How Charities Use Data

It’s that time of year again. As the festive season draws near and we pull together wish lists, many of us also begin to think about how we can give back. Given that the UK spent over £7 billion this Black Friday and Cyber Monday weekend, it’s not surprising that the idea of Giving Tuesday is becoming more and more popular.  But with 160,000 registered charities in the UK alone, institutions are turning to data to find new ways to stand out and make a greater impact.  Far from just running quarterly reports, charities are now utilising the insights they gain from data to inform their strategies, improve their services and plan for the future.  IDEAS Given that not every charity is lucky enough to go viral with an Ice Bucket Challenge style video, there is a need to find other ways to stand out in such a crowded market. As such, many are looking to the data they have collected to help create a strategy. Macmillan Cancer Support, one the UK’s biggest charities, wanted to see more success from one of their main fundraisers, ‘The World’s Biggest Coffee Morning’. The event, which sees volunteers hold coffee and cake-fuelled gatherings across the country was revolutionised by data. By engaging with their database and researching what motivated fundraisers, they refocused their marketing around how the occasion could create an opportunity for people to meet up and chat, such as swapping ‘send for your free fundraising pack’ for ‘order your free coffee morning kit’. Whilst these amends may seem superficial, they had a major impact increasing funds raised from £15m to £20m.  Some brands have taken this idea even further, using Data & Analytics tools to engage with potential donors. Homelessness charity Cyrenians’ data told them that there were a number of misconceptions about rough sleepers, including 15% of people believing that they were homeless by choice. To counter this they created an AI chatbot, named Alex, that allowed users to ask questions they may not have been comfortable asking a real person.  Another charity using data tools to counter common misconceptions is Dyslexia Association. Their Moment of Dyslexia campaign saw them utilise facial recognition technology; the longer a person looked at their digital poster, the more jumbled up the words and letters became. By harnessing both insights and the technology made possible by data, they were able to offer an insight into what dyslexia is like for people who previously didn’t understand.  INDIVIDUALS A big issue facing a number of charities is trust. Following a series of recent scandals, the public are more sceptical than ever of how charities are run, and their use of data is no exception. This ‘trust deficit’ has resulted in vast amount of potential donors staying away, with recent research highlighting that only 11% of people are willing to share their data with a charity, even if it means a better service.  Whilst charities with effective Data Governance are able to use their vast amount of data to enhance those business, those who mismanage it are likely to suffer. Following a cyber-attack that exposed the data of over 400,000 donors, the British and Foreign Bible Society were fined £100,000. As hackers were able to enter the network by exploiting a weak password, this serves as a timely reminder that our data needs not only to be clean, but secure.  Financial implications aside, improper data usage can also do irreversible damage to a charity’s reputation. St Mungo’s has faced criticism for passing information about migrant homeless people to the Home Office, putting them at risk of deportation. Whilst they were cleared of any wrongdoing by the ICO, this controversial use of data has had a negative impact on the charity’s image. With a decline in the number of people donating to charity overall, anything that can put people off further is bad news.  IMPACT Whilst there is more demand than ever for charities to share their impact data, there is also more opportunity. With Lord Gus O’Donnell urging charities to make data an ‘organisation-wide priority’, many are going beyond publishing annual reports and fully embracing a culture shift. Youth charity Keyfund have been able to justify how the spend their funds based on their impact data. Having heard concerns from fundraisers regarding whether their leisure projects were effective they looked at the data they had gathered from the 6,000 young people they were helping. What they found was that not only were their leisure projects effective, they had an even more positive impact than their alternatives, particularly for those from the most deprived area. This allowed them to continue to support these programs and even increase funding where necessary. Going one step further are Street League, a charity that use sports programmes to tackle youth unemployment. Rather than share their impact data in quarterly, or even annual, reports they moved to real-time reporting. Interested parties can visit an ‘Online Impact Dashboard’ and see up-to-the-minute data about how the charity’s work is impacting the lives of the people it is trying to help. This not only allows for the most relevant data to be used strategically, but also supports the business holistically, gaining donor both attention and trust. To stand out in the charity sector institutions need to take advantage of data. Not only can this be used to generate campaigns and streamline services but, when used securely and transparently, it can help rebuild trust and offer a competitive edge.  If you want to make the world a better place by harnessing and analysing data, we may have a role for you. Take a look at our latest opportunities or get in touch with one of our expert consultants to see how we can help you. 

Kirsty Garshong

29. November 2018

Recently Viewed jobs