Harnham

Data Engineering and Big Data jobs

What We Do

Processing Big Data has become crucial to countless businesses. Those who decide to pursue a career within Big Data face complex development challenges, too tough for relational database systems. After all, there is no Data Science without Data Engineers.

Whilst businesses may have vast quantities of data at the ready, it holds no real value unless it can be stored, harnessed and utilised. It’s estimated that most UK companies have at least 100 terabytes of data stored which could be used for various purposes around the business, as well as newly accessible hard-to-process sources such as web data, image data, and social media data. 

This is where those who work in Data Engineering become extremely valuable, as they develop Data Platforms that allow this kind of volume to be processed and used by Data Scientists and analysts across the business.

Whether you are a company looking to build a data platform for advanced analytics, or you are a candidate who can build systems and applications to process vast, complex data sources, Harnham are here to help. 

Technical skills: AWS, Google Cloud Platform and Azure, Spark, Hadoop, Java, Scala, Python

Meet the team

Latest Jobs

Principal Data Engineer
Salary

US$140000 - US$200000 per year + Benefits + Bonus

Location

New York

Description

A Series B start-up here in NYC where you will be working on their video platform used to discover the latest events in sports, news, entertainment and music.

ADD TO SHORTLIST
Senior Data Engineering Manager
Salary

£90000 - £110000 per annum

Location

London

Description

Are you looking for a new position within one of the largest retailers in the UK? Then this role is for you!

ADD TO SHORTLIST
SENIOR DATA ENGINEER
Salary

€65000 - €75000 per annum + benefits

Location

Amsterdam, North Holland

Description

One of the fastest growing online (B2C) industry is now looking for a Senior Data Engineer to join their multidisciplinary team at their HQ in Amsterdam.

ADD TO SHORTLIST
INGENIERO SENIOR BIG DATA
Salary

€40000 - €55000 per annum + BENEFICIOS

Location

Madrid

Description

Empresa líder en el sector digital ubicada en Madrid busca incorporar un perfil Senior de Ingeniero con experiencia en entornos Big Data.

ADD TO SHORTLIST
SENIOR DATA SCIENCE ENGINEER (m/w/d)
Salary

€85000 - €110000 per annum + BENEFITS

Location

Berlin

Description

Diese Rolle bietet dir die Möglichkeit als Senior Data Science Engineer an spannenden Projekten teilzunehmen!

ADD TO SHORTLIST
SENIOR BIG DATA ARCHITECT - AVP (m/w/d)
Salary

€110000 - €130000 per annum + BENEFITS

Location

Hamburg

Description

Diese Rolle bietet dir die Möglichkeit als Big Data Architect ein Team aus Spezialisten aufzubauen!

ADD TO SHORTLIST

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

Why it is hard to build a Big Data team

Why it is hard to build a Big Data team

Increasingly, I speak to managers who are adopting big data tools and developing PoCs to prove how they can make use of them. Just last week I spoke to a data architect who mentioned that if he didn’t get exposure to big data tech sooner rather than later, his current RDBMS skills may become redundant within the next few years. While that is likely an exaggeration, it is certainly an interesting point. Companies that would have never previously had the capability to interpret ‘Big Data’ are now exploring a variety of NoSQL platforms. In particular, the massive performance benefits gained from Spark and real-time/streaming tools have opened up a whole new world beyond just MapReduce. I don’t claim to be a data engineer, but as a recruiter for this sector, what I do is spend all day, every day interacting with big data developers, architects and managers (as well as keeping a close eye on the latest Apache incubator projects). Due to this, I have seen some recurring themes that have become trends when companies look to create and build their big data teams that are coming to the fore. Candidate demand The demand for Big Data professionals is very much a present day issue as the data companies have grand plans for is waiting for the right data developer to use the best tech to extract valuable insights from it. The best candidates receive massive interest, often gain multiple offers from a range of companies. Your business is now no longer just competing with large corporations such as Facebook, Twitter or Yahoo. Startups and SMEs are also vying for the best candidates. Candidates are seeing pay rises twice that of the normal rate, as illustrated in our salary guide. Candidate shortage The number of candidates with hands-on, production level Big Data experience is incredibly limited. We go to great lengths to find the candidates who can add real value to companies. The growth and exciting future for the big data industry has led to increased interest in big data jobs, particularly for those from RDBMS or software. engineering backgrounds. This leaves the industry in a difficult predicament: high demand + low supply = massive competition. There are countless examples of companies that have failed to recruit a Big Data team after a year of looking. Competition to get ahead and stand out Planning - Companies need to have a data road map detailing their future plans. Candidates want to clearly know what they are getting into and what to expect from a job. Innovation - Why get stuck on batch processing? The most exciting positions that candidates love are in data innovations teams, playing with real-time/streaming tech and new languages. Personal development, growth and training – with the data science market experiencing similar growth, many big data engineers are looking for a job that not only offers the chance to work with machine learning and similar fields; but training, mentoring towards clear career progression as standard. Speed – the length of the interview process is often seen as a reflection of the amount of red tape developers have to go through to get a job. The longer and more convoluted the process, the more put off some people may be. Complacency – don’t rest on your laurels, it’s unlikely that you’ll get 10s of CVs through when you are looking to fill a data role, so when you find a candidate you like, move swiftly to show your interest to them as quality candidates don’t come around often. By implementing these small but effective improvements to your recruiting process and how you develop data talent will see you create a team that is a success in this ever more digital analytics landscape. Companies who don’t create and nurture strong, dynamic teams will fall by the wayside. It’s Harnham’s job to help you achieve this goal. Get in touch with us to tell you how. T: (020) 8408 6070 E: info@harnham.com

Ross Whatling

Ross Whatling

06. June 2017

Using Data & Analytics To Plan Your Perfect Ski Trip

Using Data & Analytics To Plan Your Perfect Ski Trip

The Ski season may be drawing to a close, but it’s never too early to start planning for next year. Born and raised in the mountains of Austria, I have been skiing all of my life. For me, it’s about freedom, enjoying the views and forgetting about everything else.  But, since I’ve stepped into the world of Data & Analytics, I started to asked myself “what can I learn from my work that I can apply to my skiing”? After having a look around, I began to discover ways in which Data could support my passion. I’ve pulled together some of the most interesting things I’ve discovered and created this handy guide to help you prepare for your next trip. Here’s how you can use data to create the perfect ski trip.  Follow the snow Anyone who has skied before knows about the uncertainty before a trip. Will there be enough snow? Will the weather be good? Which resort is the most suited to my ability? Fortunately, somebody has already pulled this information together for you. Two "web spiders" were built via Scrapy, a Python framework used for data extraction, the first of which extracted ski resort data. The second spider, on the other hand, extracted daily snowfall data for each resort (2009 - present). After collecting Data from more than 600 ski resorts and spitting it into 7 main regions, the spiders were able to form a conclusion. The framework then pulled out key metrics, including the difficulty of runs, meaning that skiers are now able to decide which resort is most suitable for their ability.  As for the weather, onthesnow.com has recorded snowfall data from all major resorts, every year since 2009. We all know that good snow makes any trip better, so the collected data here will help skiers ensure they are prepared for the right weather, or even plan their trip around where the snow will be best.  Optimise your skis Ski manufacturing is a refined and complicated process, with each ski requiring many different materials to be built. Unfortunately, this often results in the best skis running out quickly as supply outspeeds demand.  To help speed up and improve the process, companies are implementing technologies like IBM Cognos* that monitor entire supply chains so that no matter how much demand increases, they have the materials to meet it.   Additionally, since the majority of companies have become more data-driven, production time has been reduced by weeks. Predictions for future demand has also become 50% more accurate, resulting in a drop of 30% idle time on production lines. Skip the Queue Tired of queuing for the ski lift? There’s good news. As they begin to make the most of data, ski resorts are introducing RFID* (Radio Frequency Identification) systems. These involve visitors purchasing cards with RFID chips included, allowing them to skip queues at the lifts as there is no need to check for fake passes. The data can then be utilised for gamification platforms to turn a skier’s time on the slopes into an interactive experience.  The shift towards Big Data not only has advantages for the visitors, but the management are also benefiting. In the past, it has been difficult to analyse skier’s data. Now, with automated and proper data management, the numbers can be crunched seamlessly and marketing campaigns can be directed at how people actually choose to ski.   Carve a Better Technique Skiing isn’t always easy, especially if you haven’t grown up with it. Usually, ski instructors are the solution but, in the age of Data & Analytics, there are other solutions. Jamie Grant and co-founder Pruthvikar Reddy have created an app called Carv 2.0, which allows you to be your own teacher. It works by using a robust insert that fits between the shell of your ski boots and the liner. It then gathers data from 48 pressure sensitive pads, and nine motion sensors.  This data is fed to a connected match-box size tracker unit, sitting on the back of your boots, before being relayed via Bluetooth to the Carv App on your phone. Carv can then measure your speed, acceleration and ski orientation a staggering 300 times a second.  Thanks to a complex set of algorithms this data is then converted into an easy to follow graphic display on your phone’s screen as well as verbal feedback from Carvella. The accuracy of this real-time data could make it a better instructor than any individual person.  Data & Analytics are helping streamline every part of our lives. Whilst the above can’t guarantee a perfect ski trip, they can help us minimise risks and optimize our performance and experience.  If you’re able to use data to improve day-to-day living, we may have a role for you. Take a look at our latest opportunities or get in touch with our expert consultants.  

Marcel Mathis

Marcel Mathis

04. April 2019

Recently Viewed jobs